RFC: OE-Core task rework

RFC: OE-Core task rework

[Paul Eggleton]

Hi all,

There have been a few requests to review the task recipes (i.e. package 
groups) provided by OE-Core, and indeed these have not really been looked at 
seriously since OE-Core was created. Ideally I think we want them to be useful 
to a wide audience and provide useful units of functionality that can be added 
to an image without the person doing the selection having to manually select 
too many individual packages. Imagine presenting the list of tasks to someone 
in a UI for assembling images (such as Hob or Narcissus) and you can start to 
see that we have some work to do in this area.

I know various distros and users of OE-Core have created their own tasks or 
resurrected tasks from OE-Classic, and this is an opportunity to perhaps look 
at bringing some of these (or at least, parts of them) into the core. It is 
true that tasks will often be an expression of distro policy, and we also 
can't have any tasks in OE-Core that refer to packages that don't exist in OE-
Core; thus distros will always be extending the base tasks or adding their own 
- and that's fine. However, with some thought I believe we can come up with a 
set of tasks that are generally useful to most people using OE-Core.

For reference, I've compiled a list on the wiki of the current tasks in OE-
Core with links to the recipes and some notes:

  http://www.openembedded.org/wiki/OE-Core_Task_Rework

Some of the things I think we ought to consider/address as part of this 
exercise:

1) Do we rename "task" to something a little more understandable to the 
uninitiated, such as "package group"? The word "task" is already used in a 
much more natural sense within bitbake as a unit of work. Historically I 
believe we picked up this term from Debian but I'm not aware of significant use 
by other mainstream distributions.

2) Look at the existing tasks and:
 * evaluate their usefulness
 * remove any that are obsolete
 * adjust existing contents if needed
 * look for useful groups of packages that might be added

 We need to pay particular attention to task-core-boot and task-base as these 
are pulled in by default in any image that inherits from core-image.bbclass - 
if these are not generally working for people that are creating their own 
images, we need to change them such that they are.

3) Ensure all task recipes follow reasonable patterns:
 *  Fix recipe DESCRIPTIONs to make sense (and not contain Poky references)
 *  Ensure all individual packages have a decent SUMMARY/DESCRIPTION
 *  Try to make each task have a reasonable name - there are some current uses 
of "core" and "base" that don't actually convey any meaning; LSB specific tasks 
should be named as such, etc.
 *  Make all tasks inherit task.bbclass so that they get proper dbg/dev 
packages and any other common task functionality

4) Consider the usefulness of the existing PACKAGE_GROUP_ structure which 
converts some IMAGE_FEATURES into the addition of tasks to be installed (see 
classes/core-image.bbclass). At the very least we should probably rename this 
if we rename tasks to package groups, and there's a question as to how 
IMAGE_FEATURES scales - should every task be represented in there or only a 
select list? Would we be better off not trying to bring any tasks in via 
IMAGE_FEATURES at all and mostly leave that to control post-processing (e.g. 
package-management, debug-tweaks, etc.)?

Please reply with your thoughts. Once we've come to a consensus on the things 
we want to do (allowing plenty of time for discussion), I'll look at putting 
together a set of patches that makes the appropriate changes.

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [yocto] RFC: OE-Core task rework

[Koen Kooi]

Op 15 aug. 2012, om 11:46 heeft Paul Eggleton <paul.eggleton@linux.intel.com> het volgende geschreven:
> 
> 2) Look at the existing tasks and:
> * evaluate their usefulness
> * remove any that are obsolete
> * adjust existing contents if needed
> * look for useful groups of packages that might be added
> 
> We need to pay particular attention to task-core-boot and task-base as these 
> are pulled in by default in any image that inherits from core-image.bbclass - 
> if these are not generally working for people that are creating their own 
> images, we need to change them such that they are.

Have a look at task-boot and task-basic in meta-oe: https://github.com/openembedded/meta-oe/tree/master/meta-oe/recipes-core/tasks

My pet peeve is the bloat oe-core forces onto qemu* builds in the name of "our testing team needs it!!!". Why am I forced to build an nfs server and distcc when setting MACHINE=qemusomething?

Keep things like that in mind when overhauling the tasks.

regards,

Koen

Re: RFC: OE-Core task rework

[Jack Mitchell]

On 15/08/12 10:46, Paul Eggleton wrote:
> Hi all,
>
> There have been a few requests to review the task recipes (i.e. package
> groups) provided by OE-Core, and indeed these have not really been looked at
> seriously since OE-Core was created. Ideally I think we want them to be useful
> to a wide audience and provide useful units of functionality that can be added
> to an image without the person doing the selection having to manually select
> too many individual packages. Imagine presenting the list of tasks to someone
> in a UI for assembling images (such as Hob or Narcissus) and you can start to
> see that we have some work to do in this area.
>
> I know various distros and users of OE-Core have created their own tasks or
> resurrected tasks from OE-Classic, and this is an opportunity to perhaps look
> at bringing some of these (or at least, parts of them) into the core. It is
> true that tasks will often be an expression of distro policy, and we also
> can't have any tasks in OE-Core that refer to packages that don't exist in OE-
> Core; thus distros will always be extending the base tasks or adding their own
> - and that's fine. However, with some thought I believe we can come up with a
> set of tasks that are generally useful to most people using OE-Core.
>
> For reference, I've compiled a list on the wiki of the current tasks in OE-
> Core with links to the recipes and some notes:
>
>    http://www.openembedded.org/wiki/OE-Core_Task_Rework
>
> Some of the things I think we ought to consider/address as part of this
> exercise:
>
> 1) Do we rename "task" to something a little more understandable to the
> uninitiated, such as "package group"? The word "task" is already used in a
> much more natural sense within bitbake as a unit of work. Historically I
> believe we picked up this term from Debian but I'm not aware of significant use
> by other mainstream distributions.
>

Do the 'tasks' do anything other than specify meta-data groups? If they 
don't include any glue or additional processes then I strongly agree 
that the name should be changed.

-- 

   Jack Mitchell (jack@embed.me.uk)
   Embedded Systems Engineer
   http://www.embed.me.uk

--

Re: RFC: OE-Core task rework

[Phil Blundell]

On Wed, 2012-08-15 at 10:46 +0100, Paul Eggleton wrote:
> 1) Do we rename "task" to something a little more understandable to the 
> uninitiated, such as "package group"? The word "task" is already used in a 
> much more natural sense within bitbake as a unit of work. Historically I 
> believe we picked up this term from Debian but I'm not aware of significant use 
> by other mainstream distributions.

Yeah, I think OE inherited it from Familiar, which in turn got it from
Debian.  But the meaning of the term has drifted slightly through the
generations and, as you say, it is no longer a very accurate reflection
of what the packages in question are doing.

It's never been totally obvious to me that there is much need for
tasks/package group recipes as such in oe-core itself; they're rather
more of a DISTRO policy thing and their presence in the metadata does
obviously have a cost in terms of parse time and memory usage.  It might
perhaps be worth exploring what they're actually being used for in
oe-core and whether those things could be better done in a different way
that doesn't involve having a .bb file for them.

p.

Re: [yocto] RFC: OE-Core task rework

[Paul Eggleton]

On Wednesday 15 August 2012 12:54:17 Koen Kooi wrote:
> Op 15 aug. 2012, om 11:46 heeft Paul Eggleton 
<paul.eggleton@linux.intel.com> het volgende geschreven:
> > 2) Look at the existing tasks and:
> > * evaluate their usefulness
> > * remove any that are obsolete
> > * adjust existing contents if needed
> > * look for useful groups of packages that might be added
> > 
> > We need to pay particular attention to task-core-boot and task-base as
> > these are pulled in by default in any image that inherits from
> > core-image.bbclass - if these are not generally working for people that
> > are creating their own images, we need to change them such that they are.
> 
> Have a look at task-boot and task-basic in meta-oe:
> https://github.com/openembedded/meta-oe/tree/master/meta-oe/recipes-core/ta
> sks

OK, I've taken the opportunity to summarise the tasks in meta-oe on the 
following page:

 http://www.openembedded.org/wiki/Meta-oe_tasks

meta-oe's task-basic indeed contains much less than task-base; it provides an 
alternative (and possibly simpler) means of selecting between ssh or dropbear, 
although it prevents it being an image-level decision if that's something 
people care about. I'm not convinced it's appropriate to unconditionally 
install avahi, htop and cpufrequtils though - those look distro-specific to me.

For meta-oe's task-boot, it doesn't seem all that different from task-core-boot 
with the exception that task-core-boot imposes modutils-initscripts (should 
probably be fixed), and that task-boot mandates that the kernel be installed 
which is not always needed depending on how the bootloader works on whatever 
machine is being built for - although kernel.bbclass seems to indicate that 
currently the machine is supposed to influence this by setting RDEPENDS_kernel-
base which seems a little backwards to me.

I'd like to hear from other users of these tasks (Martin?)

> My pet peeve is the bloat oe-core forces onto qemu* builds in the name of
> "our testing team needs it!!!". Why am I forced to build an nfs server and
> distcc when setting MACHINE=qemusomething?

If you're referring to qemu-config (which depends on distcc, task-core-nfs-
server, oprofileui-server, rsync and bash) it's important to note that it only 
gets enabled for qemu* machines by the poky distro config via POKYQEMUDEPS, so 
this is not being used by OE-Core - in fact nothing in OE-Core refers to qemu-
config at all. I still have an open bug to fix this (#1690), and I agree it's 
egregious, but it's a Poky issue, not an OE-Core one. For the record, the 
reason for including them is not "our testing team needs it!!!" - it was the 
assumption made a long time ago that these things would be generally useful 
for development on emulated targets. Whether or not that assumption is still 
correct is another issue.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RFC: OE-Core task rework

[Paul Eggleton]

On Wednesday 15 August 2012 12:17:40 Jack Mitchell wrote:
> > 1) Do we rename "task" to something a little more understandable to the
> > uninitiated, such as "package group"? The word "task" is already used in a
> > much more natural sense within bitbake as a unit of work. Historically I
> > believe we picked up this term from Debian but I'm not aware of
> > significant use by other mainstream distributions.
> 
> Do the 'tasks' do anything other than specify meta-data groups? If they
> don't include any glue or additional processes then I strongly agree
> that the name should be changed.

They don't, no. Usually they are just lists of packages, although sometimes 
there is some very basic logic to conditionally include/exclude some packages 
based on DISTRO_FEATURES, MACHINE_FEATURES, COMBINED_FEATURES etc. or allow 
substitution of default packages with others using variables.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RFC: OE-Core task rework

[Paul Eggleton]

On Wednesday 15 August 2012 12:25:23 Phil Blundell wrote:
> On Wed, 2012-08-15 at 10:46 +0100, Paul Eggleton wrote:
> > 1) Do we rename "task" to something a little more understandable to the
> > uninitiated, such as "package group"? The word "task" is already used in a
> > much more natural sense within bitbake as a unit of work. Historically I
> > believe we picked up this term from Debian but I'm not aware of
> > significant use by other mainstream distributions.
> 
> Yeah, I think OE inherited it from Familiar, which in turn got it from
> Debian.  But the meaning of the term has drifted slightly through the
> generations and, as you say, it is no longer a very accurate reflection
> of what the packages in question are doing.
> 
> It's never been totally obvious to me that there is much need for
> tasks/package group recipes as such in oe-core itself; they're rather
> more of a DISTRO policy thing and their presence in the metadata does
> obviously have a cost in terms of parse time and memory usage.  It might
> perhaps be worth exploring what they're actually being used for in
> oe-core and whether those things could be better done in a different way
> that doesn't involve having a .bb file for them.

I would say without checking that these days that cost is negligible. Off the 
top of my head, the benefits we get from using recipes for tasks as opposed to 
variables or some similar construct:

 * The tasks can be used and referred to on the target if desired, not just 
when you compose the image (i.e. task packages are produced and thus the 
package manager knows about them).

 * It should be slightly easier to find where tasks are defined within the 
metadata. In practice that's not always the case with the way we have named 
some of the tasks, hopefully that can be improved.

 * Related to the above, they can be customised in a visible manner using 
bbappends in your own layer.

 * If you inherit from task.bbclass you automatically get -dev and -dbg 
complements for the tasks defined by the recipe, which can sometimes be useful.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RFC: OE-Core task rework

[Mark Hatle]

On 8/15/12 8:08 AM, Paul Eggleton wrote:
> On Wednesday 15 August 2012 12:25:23 Phil Blundell wrote:
>> On Wed, 2012-08-15 at 10:46 +0100, Paul Eggleton wrote:
>>> 1) Do we rename "task" to something a little more understandable to the
>>> uninitiated, such as "package group"? The word "task" is already used in a
>>> much more natural sense within bitbake as a unit of work. Historically I
>>> believe we picked up this term from Debian but I'm not aware of
>>> significant use by other mainstream distributions.
>>
>> Yeah, I think OE inherited it from Familiar, which in turn got it from
>> Debian.  But the meaning of the term has drifted slightly through the
>> generations and, as you say, it is no longer a very accurate reflection
>> of what the packages in question are doing.
>>
>> It's never been totally obvious to me that there is much need for
>> tasks/package group recipes as such in oe-core itself; they're rather
>> more of a DISTRO policy thing and their presence in the metadata does
>> obviously have a cost in terms of parse time and memory usage.  It might
>> perhaps be worth exploring what they're actually being used for in
>> oe-core and whether those things could be better done in a different way
>> that doesn't involve having a .bb file for them.
>
> I would say without checking that these days that cost is negligible. Off the
> top of my head, the benefits we get from using recipes for tasks as opposed to
> variables or some similar construct:
>
>   * The tasks can be used and referred to on the target if desired, not just
> when you compose the image (i.e. task packages are produced and thus the
> package manager knows about them).

I think this is a key advantage.  Again, if we think of these tasks as logical 
groups of functionality, it gives an image developer (or installer) the ability 
to say "I need booting, discrete commands, python, perl, and LSB compliance." 
and get a system in the end that -should- work.

The image/installer should always be able to specify individual recipes as well, 
but often inexperienced users won't know that they need two or three recipes 
(that don't have actual dependencies on each other) to get a functionally 
complete answer.

>   * It should be slightly easier to find where tasks are defined within the
> metadata. In practice that's not always the case with the way we have named
> some of the tasks, hopefully that can be improved.
>
>   * Related to the above, they can be customised in a visible manner using
> bbappends in your own layer.
>
>   * If you inherit from task.bbclass you automatically get -dev and -dbg
> complements for the tasks defined by the recipe, which can sometimes be useful.
>
> Cheers,
> Paul
>

Re: RFC: OE-Core task rework

[Chris Larson]

On Wed, Aug 15, 2012 at 11:05 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>   * The tasks can be used and referred to on the target if desired, not
>> just
>> when you compose the image (i.e. task packages are produced and thus the
>> package manager knows about them).
>
>
> I think this is a key advantage.  Again, if we think of these tasks as
> logical groups of functionality, it gives an image developer (or installer)
> the ability to say "I need booting, discrete commands, python, perl, and LSB
> compliance." and get a system in the end that -should- work.
>
> The image/installer should always be able to specify individual recipes as
> well, but often inexperienced users won't know that they need two or three
> recipes (that don't have actual dependencies on each other) to get a
> functionally complete answer.

It seems like you're arguing in favor of the ability to add groups of
packages to an image, which no one disagrees with, and isn't really
relevant to the bit you're quoting. The bit that tasks add that
package groups / image features don't is the ability to add them after
the fact at runtime, not image creation time.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: RFC: OE-Core task rework

[Mark Hatle]

On 8/15/12 2:12 PM, Chris Larson wrote:
> On Wed, Aug 15, 2012 at 11:05 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>>    * The tasks can be used and referred to on the target if desired, not
>>> just
>>> when you compose the image (i.e. task packages are produced and thus the
>>> package manager knows about them).
>>
>>
>> I think this is a key advantage.  Again, if we think of these tasks as
>> logical groups of functionality, it gives an image developer (or installer)
>> the ability to say "I need booting, discrete commands, python, perl, and LSB
>> compliance." and get a system in the end that -should- work.
>>
>> The image/installer should always be able to specify individual recipes as
>> well, but often inexperienced users won't know that they need two or three
>> recipes (that don't have actual dependencies on each other) to get a
>> functionally complete answer.
>
> It seems like you're arguing in favor of the ability to add groups of
> packages to an image, which no one disagrees with, and isn't really
> relevant to the bit you're quoting. The bit that tasks add that
> package groups / image features don't is the ability to add them after
> the fact at runtime, not image creation time.
>

It's both.. cross image creation, and on-target image creation (and updates). 
During runtime, some folks will want to add "features" to the system...  these 
are not what I would consider "image" features, but what we call task recipes today.

I.e. I have a base system, and a vendor says I need LSB compatibility -- 
assuming I have the right distro features set, I should be able to select one or 
more LSB "tasks" and suddenly I'm LSB compliant, without having to know all of 
the individual packages required.

--Mark

Re: [yocto] RFC: OE-Core task rework

[Mark Hatle]

[resend, I had the original mail rejected by the oe list serves as being 
undeliverable.]

On 8/15/12 4:46 AM, Paul Eggleton wrote:
> Hi all,
>
> There have been a few requests to review the task recipes (i.e. package
> groups) provided by OE-Core, and indeed these have not really been looked at
> seriously since OE-Core was created. Ideally I think we want them to be useful
> to a wide audience and provide useful units of functionality that can be added
> to an image without the person doing the selection having to manually select
> too many individual packages. Imagine presenting the list of tasks to someone
> in a UI for assembling images (such as Hob or Narcissus) and you can start to
> see that we have some work to do in this area.
>
> I know various distros and users of OE-Core have created their own tasks or
> resurrected tasks from OE-Classic, and this is an opportunity to perhaps look
> at bringing some of these (or at least, parts of them) into the core. It is
> true that tasks will often be an expression of distro policy, and we also
> can't have any tasks in OE-Core that refer to packages that don't exist in OE-
> Core; thus distros will always be extending the base tasks or adding their own
> - and that's fine. However, with some thought I believe we can come up with a
> set of tasks that are generally useful to most people using OE-Core.
>
> For reference, I've compiled a list on the wiki of the current tasks in OE-
> Core with links to the recipes and some notes:
>
>    http://www.openembedded.org/wiki/OE-Core_Task_Rework
>
> Some of the things I think we ought to consider/address as part of this
> exercise:
>
> 1) Do we rename "task" to something a little more understandable to the
> uninitiated, such as "package group"? The word "task" is already used in a
> much more natural sense within bitbake as a unit of work. Historically I
> believe we picked up this term from Debian but I'm not aware of significant use
> by other mainstream distributions.

"group" or "packagegroup" or "package-group" is my suggestion for the existing 
'task' recipes.  From what I've seen, it should be a rename opportunity -- we 
can even provide/rprovide the old names....

> 2) Look at the existing tasks and:
>   * evaluate their usefulness
>   * remove any that are obsolete
>   * adjust existing contents if needed
>   * look for useful groups of packages that might be added
>
>   We need to pay particular attention to task-core-boot and task-base as these
> are pulled in by default in any image that inherits from core-image.bbclass -
> if these are not generally working for people that are creating their own
> images, we need to change them such that they are.

During the pre-oe-core Yocto Project development, a design was generated to 
roughly group the packages into functional areas.  For the most part, this 
design (as well as legacy elements) still exist.

I think the boot, base and others need to be evaluated for usefulness... but my 
feeling is most of them are close to being correct.

The original design attempted to group things into functionality, so that OE 
could be an "additive" distribution.  I.e., you don't start with a big blob of 
packages and remove them as necessary, but instead you start with small 
functional blocks and put them together to construct a system.  The tasks seemed 
to be the logical blocks, and the contents of these blocks were based on the 
early design.

Very roughly speaking the early design was:

filesystem
|        \
libc    libstdcxx
|           \
small cli    \
(busybox)     \
|              \
basic commands  python -- perl
|                 |         |
initscripts       |         |
multiuser         |         |
system services --+---------+
|
LSB functionality

(Lines indicate implicit dependencies)

The goal was that if you include a specific group, such as "multiuser" support, 
you would get all of the recipes necessary to enable multiuser support on the 
system.  I think this is a reasonable theory to consider when evaluating the 
existing groupings.

> 3) Ensure all task recipes follow reasonable patterns:
>   *  Fix recipe DESCRIPTIONs to make sense (and not contain Poky references)
>   *  Ensure all individual packages have a decent SUMMARY/DESCRIPTION
>   *  Try to make each task have a reasonable name - there are some current uses
> of "core" and "base" that don't actually convey any meaning; LSB specific tasks
> should be named as such, etc.

Yup, there is a logical grouping for the lsb specific tasks, as for others.  The 
naming needs to be made clear as to why it's there, and what it represents. Same 
with the summary and descriptions -- they should list the functionality provided 
by this group of components.

>   *  Make all tasks inherit task.bbclass so that they get proper dbg/dev
> packages and any other common task functionality
>
> 4) Consider the usefulness of the existing PACKAGE_GROUP_ structure which
> converts some IMAGE_FEATURES into the addition of tasks to be installed (see
> classes/core-image.bbclass). At the very least we should probably rename this
> if we rename tasks to package groups, and there's a question as to how
> IMAGE_FEATURES scales - should every task be represented in there or only a
> select list? Would we be better off not trying to bring any tasks in via
> IMAGE_FEATURES at all and mostly leave that to control post-processing (e.g.
> package-management, debug-tweaks, etc.)?

I'd certainly prefer that images were limited to selecting software from task 
(group) recipes only, and not providing their own.  An image should be able to 
change the selection based on an "image feature" or similar configuration, but 
the underlying tasks/groups, recipes, etc should all be 'generic' to a given 
distribution configuration.

I think if you go through the images today, a lot of that stuff is either old, 
or could be simplified by creating appropriate tasks/groups.

--Mark

> Please reply with your thoughts. Once we've come to a consensus on the things
> we want to do (allowing plenty of time for discussion), I'll look at putting
> together a set of patches that makes the appropriate changes.
>
> Thanks,
> Paul
>

Re: [yocto] RFC: OE-Core task rework

[Paul Eggleton]

On Monday 20 August 2012 15:45:07 Mark Hatle wrote:
> > 1) Do we rename "task" to something a little more understandable to the
> > uninitiated, such as "package group"? The word "task" is already used in a
> > much more natural sense within bitbake as a unit of work. Historically I
> > believe we picked up this term from Debian but I'm not aware of
> > significant use by other mainstream distributions.
> 
> "group" or "packagegroup" or "package-group" is my suggestion for the
> existing 'task' recipes.  From what I've seen, it should be a rename
> opportunity -- we can even provide/rprovide the old names....

Indeed, and I think we would for compatibility purposes.
 
> > 2) Look at the existing tasks and:
> >   * evaluate their usefulness
> >   * remove any that are obsolete
> >   * adjust existing contents if needed
> >   * look for useful groups of packages that might be added
> >   
> >   We need to pay particular attention to task-core-boot and task-base as
> >   these> 
> > are pulled in by default in any image that inherits from
> > core-image.bbclass - if these are not generally working for people that
> > are creating their own images, we need to change them such that they are.
> 
> During the pre-oe-core Yocto Project development, a design was generated to
> roughly group the packages into functional areas.  For the most part, this
> design (as well as legacy elements) still exist.
> 
> I think the boot, base and others need to be evaluated for usefulness... but
> my feeling is most of them are close to being correct.

I think so as well. We may be pulling in one or two packages unconditionally 
where they should be optional (e.g. modutils-initscripts?) but most of it is 
pretty sensible.
 
> >   *  Try to make each task have a reasonable name - there are some current
> >   uses of "core" and "base" that don't actually convey any meaning; LSB
> >   specific tasks should be named as such, etc.
> 
> Yup, there is a logical grouping for the lsb specific tasks, as for others. 
> The naming needs to be made clear as to why it's there, and what it
> represents. Same with the summary and descriptions -- they should list the
> functionality provided by this group of components.

The main concern with LSB is that we have something called task-basic, which 
seems to be intended for LSB but does not state as much, and I know at least 
one person has tried to use it and then been a little puzzled as to why rpm 
has been pulled in when ipk packaging has been selected. Just naming some of 
these tasks more appropriately would help avoid such situations. In the 
specific case of task-basic there may be a case for creating a similar but not 
LSB-focused task that pulls in real shell utilities in preference to the light 
versions provided by busybox.

> > 4) Consider the usefulness of the existing PACKAGE_GROUP_ structure which
> > converts some IMAGE_FEATURES into the addition of tasks to be installed
> > (see classes/core-image.bbclass). At the very least we should probably
> > rename this if we rename tasks to package groups, and there's a question
> > as to how IMAGE_FEATURES scales - should every task be represented in
> > there or only a select list? Would we be better off not trying to bring
> > any tasks in via IMAGE_FEATURES at all and mostly leave that to control
> > post-processing (e.g. package-management, debug-tweaks, etc.)?
> 
> I'd certainly prefer that images were limited to selecting software from
> task (group) recipes only, and not providing their own.  An image should be
> able to change the selection based on an "image feature" or similar
> configuration, but the underlying tasks/groups, recipes, etc should all be
> 'generic' to a given distribution configuration.

The first part seems reasonable to me; but I'm still short of an answer as to 
what to do with the existing IMAGE_FEATURES code as mentioned above. At the 
moment aside from the special features such as debug-tweaks, these are just a 
thin layer on top of task recipes which doesn't add very much at all other 
than extra maintenance.

> I think if you go through the images today, a lot of that stuff is either
> old, or could be simplified by creating appropriate tasks/groups.

OK, that's a good point. I have another task on my list to clean up our image 
recipes and that would be a good segue into that.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [yocto] RFC: OE-Core task rework

[Paul Eggleton]

On Tuesday 21 August 2012 09:49:37 Paul Eggleton wrote:
> The main concern with LSB is that we have something called task-basic,
> which  seems to be intended for LSB but does not state as much, and I know
> at least one person has tried to use it and then been a little puzzled as
> to why rpm has been pulled in when ipk packaging has been selected. Just
> naming some of these tasks more appropriately would help avoid such
> situations. In the specific case of task-basic there may be a case for
> creating a similar but not LSB-focused task that pulls in real shell
> utilities in preference to the light versions provided by busybox.

Ugh, sorry. I meant task-core-basic here - not to be confused with task-basic 
in meta-oe [1].

Cheers,
Paul

[1] http://www.openembedded.org/wiki/Meta-oe_tasks

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [yocto] RFC: OE-Core task rework

[Philip Balister]

On 08/21/2012 01:49 AM, Paul Eggleton wrote:
> On Monday 20 August 2012 15:45:07 Mark Hatle wrote:
...

>>
>> Yup, there is a logical grouping for the lsb specific tasks, as for others.
>> The naming needs to be made clear as to why it's there, and what it
>> represents. Same with the summary and descriptions -- they should list the
>> functionality provided by this group of components.
>
> The main concern with LSB is that we have something called task-basic, which
> seems to be intended for LSB but does not state as much, and I know at least
> one person has tried to use it and then been a little puzzled as to why rpm
> has been pulled in when ipk packaging has been selected. Just naming some of
> these tasks more appropriately would help avoid such situations. In the
> specific case of task-basic there may be a case for creating a similar but not
> LSB-focused task that pulls in real shell utilities in preference to the light
> versions provided by busybox.

I was seriously stumped by the presence of rpm in what I felt should 
have been a very basic image. If a task/group is targeted at lsb, it 
needs to have lsb plastered all over the name so the rest of us can 
avoid it :)

Philip

>
>>> 4) Consider the usefulness of the existing PACKAGE_GROUP_ structure which
>>> converts some IMAGE_FEATURES into the addition of tasks to be installed
>>> (see classes/core-image.bbclass). At the very least we should probably
>>> rename this if we rename tasks to package groups, and there's a question
>>> as to how IMAGE_FEATURES scales - should every task be represented in
>>> there or only a select list? Would we be better off not trying to bring
>>> any tasks in via IMAGE_FEATURES at all and mostly leave that to control
>>> post-processing (e.g. package-management, debug-tweaks, etc.)?
>>
>> I'd certainly prefer that images were limited to selecting software from
>> task (group) recipes only, and not providing their own.  An image should be
>> able to change the selection based on an "image feature" or similar
>> configuration, but the underlying tasks/groups, recipes, etc should all be
>> 'generic' to a given distribution configuration.
>
> The first part seems reasonable to me; but I'm still short of an answer as to
> what to do with the existing IMAGE_FEATURES code as mentioned above. At the
> moment aside from the special features such as debug-tweaks, these are just a
> thin layer on top of task recipes which doesn't add very much at all other
> than extra maintenance.
>
>> I think if you go through the images today, a lot of that stuff is either
>> old, or could be simplified by creating appropriate tasks/groups.
>
> OK, that's a good point. I have another task on my list to clean up our image
> recipes and that would be a good segue into that.
>
> Cheers,
> Paul
>

Re: RFC: OE-Core task rework

[Paul Eggleton]

On Wednesday 15 August 2012 10:46:49 Paul Eggleton wrote:
> Hi all,
> 
> There have been a few requests to review the task recipes (i.e. package
> groups) provided by OE-Core, and indeed these have not really been looked at
> seriously since OE-Core was created. Ideally I think we want them to be
> useful to a wide audience and provide useful units of functionality that
> can be added to an image without the person doing the selection having to
> manually select too many individual packages. Imagine presenting the list
> of tasks to someone in a UI for assembling images (such as Hob or
> Narcissus) and you can start to see that we have some work to do in this
> area.
> 
> I know various distros and users of OE-Core have created their own tasks or
> resurrected tasks from OE-Classic, and this is an opportunity to perhaps
> look at bringing some of these (or at least, parts of them) into the core.
> It is true that tasks will often be an expression of distro policy, and we
> also can't have any tasks in OE-Core that refer to packages that don't
> exist in OE- Core; thus distros will always be extending the base tasks or
> adding their own - and that's fine. However, with some thought I believe we
> can come up with a set of tasks that are generally useful to most people
> using OE-Core.
> 
> For reference, I've compiled a list on the wiki of the current tasks in OE-
> Core with links to the recipes and some notes:
> 
>   http://www.openembedded.org/wiki/OE-Core_Task_Rework
> 
> Some of the things I think we ought to consider/address as part of this
> exercise:
> 
> 1) Do we rename "task" to something a little more understandable to the
> uninitiated, such as "package group"? The word "task" is already used in a
> much more natural sense within bitbake as a unit of work. Historically I
> believe we picked up this term from Debian but I'm not aware of significant
> use by other mainstream distributions.
> 
> 2) Look at the existing tasks and:
>  * evaluate their usefulness
>  * remove any that are obsolete
>  * adjust existing contents if needed
>  * look for useful groups of packages that might be added
> 
>  We need to pay particular attention to task-core-boot and task-base as
> these are pulled in by default in any image that inherits from
> core-image.bbclass - if these are not generally working for people that are
> creating their own images, we need to change them such that they are.
> 
> 3) Ensure all task recipes follow reasonable patterns:
>  *  Fix recipe DESCRIPTIONs to make sense (and not contain Poky references)
>  *  Ensure all individual packages have a decent SUMMARY/DESCRIPTION
>  *  Try to make each task have a reasonable name - there are some current
> uses of "core" and "base" that don't actually convey any meaning; LSB
> specific tasks should be named as such, etc.
>  *  Make all tasks inherit task.bbclass so that they get proper dbg/dev
> packages and any other common task functionality
> 
> 4) Consider the usefulness of the existing PACKAGE_GROUP_ structure which
> converts some IMAGE_FEATURES into the addition of tasks to be installed (see
> classes/core-image.bbclass). At the very least we should probably rename
> this if we rename tasks to package groups, and there's a question as to how
> IMAGE_FEATURES scales - should every task be represented in there or only a
> select list? Would we be better off not trying to bring any tasks in via
> IMAGE_FEATURES at all and mostly leave that to control post-processing
> (e.g. package-management, debug-tweaks, etc.)?
> 
> Please reply with your thoughts. Once we've come to a consensus on the
> things we want to do (allowing plenty of time for discussion), I'll look at
> putting together a set of patches that makes the appropriate changes.

So, any further thoughts on this one?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre