MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Darren Hart]

This is something I've run into frequently and have been putting off
trying to resolve. Hopefully the following will help illustrate the path
a BSP developer might go down trying to use the MACHINE_ESSENTIAL* and
MACHINE_EXTRA* variables. If someone with more knowledge on the subject
could fill in the gaps noted below, I'll be happy to test and work with
Scott to improve the documentation.

I am unable to discern from the following text in the reference manual,
which variable I should be using to include a firmware package in the
images for a given BSP.

From
http://www.yoctoproject.org/docs/1.1/poky-ref-manual/poky-ref-manual.html:

"
MACHINE_ESSENTIAL_RDEPENDS

    List of packages required to boot device
MACHINE_ESSENTIAL_RRECOMMENDS

    List of packages required to boot device (usually additional kernel
modules)
MACHINE_EXTRA_RDEPENDS

    List of packages required to use device
MACHINE_EXTRA_RRECOMMNEDS  (<-- typo still present)

    List of packages useful to use device (for example additional kernel
modules)
"

From the above, I don't learn anything about what the impact of using
each of these variables is. Which ones add the packages to the rootfs,
which only build them, do they impact the initrd (one might assume a
package required for boot should be in the initrd). The difference
between RDEPENDS and RRECOMMENDS is unclear. Three of the four use the
term "required" while the other uses "useful" in the description.
Neither semantic split (RDEPENDS vs. RRECOMMENDS, ESSENTIAL vs. EXTRA)
can be understood from the available descriptions.

The firmware I want to add is needed for wireless functionality. This is
likely not required for boot, so I added it to MACHINE_EXTRA_RRECOMMENDS
as it is only needed if the wireless driver is used. This resulted in
the package being built, but not installed. In practice this would mean
that people building the BSP would need to manually add the firmware
package to IMAGE_INSTALL or install it manually later. This strikes me
as unnecessarily complex.

This was also unexpected given the definition of RRECOMMENDS in the
manual, which would have led me to believe the package should have been
installed:

"
RRECOMMENDS

    List of packages which extend usability of the package. Those
packages will be automatically installed but can be removed by user.
"

I wasn't able to find a general meaning for the term "EXTRA" by
searching through the manual.

I then tried the remaining 3 variables, doing the following after
setting each:

$ bitbake -c cleanall task-machine-base task-core-boot
linux-firmware-core-image-sato
$ bitbake -u depexp -g core-image-sato
$ bitbake core-image-sato

I then determined if the linux-firmware package appeared in the list
presented by the depexp UI and whether or not the linux-firmware package
was built and installed. The results follow:

MACHINE_EXTRA_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
# appears in depexp with no rdepends, builds, does not install

#MACHINE_EXTRA_RECOMMENDS += "linux-firmware-iwlwifi-6000g2a-5"
# does not appear in depexp, does not build

#MACHINE_ESSENTIAL_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
# does not appear in depexp, does not build


I've tried searching the sources for these variables, and quickly lose
their scent. Here is what I did find:

task-base.bb:
#
# packages added by machine config
#
RDEPENDS_task-machine-base = "${MACHINE_EXTRA_RDEPENDS}"
RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"

task-core-boot.bb:
RDEPENDS_task-core-boot = "\
    ...
    ${MACHINE_ESSENTIAL_EXTRA_RDEPENDS}"

RRECOMMENDS_task-core-boot = "\
    ${MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS}"


How these should impact the final image is not clear to me.


Thanks,

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Darren Hart]

On 09/21/2011 02:29 PM, Darren Hart wrote:
> This is something I've run into frequently and have been putting off
> trying to resolve. Hopefully the following will help illustrate the path
> a BSP developer might go down trying to use the MACHINE_ESSENTIAL* and
> MACHINE_EXTRA* variables. If someone with more knowledge on the subject
> could fill in the gaps noted below, I'll be happy to test and work with
> Scott to improve the documentation.
> 
> I am unable to discern from the following text in the reference manual,
> which variable I should be using to include a firmware package in the
> images for a given BSP.
> 
> From
> http://www.yoctoproject.org/docs/1.1/poky-ref-manual/poky-ref-manual.html:
> 
> "
> MACHINE_ESSENTIAL_RDEPENDS
> 
>     List of packages required to boot device
> MACHINE_ESSENTIAL_RRECOMMENDS
> 
>     List of packages required to boot device (usually additional kernel
> modules)

As Saul points out, these should be:

MACHINE_ESSENTIAL_EXTRA_*


> MACHINE_EXTRA_RDEPENDS
> 
>     List of packages required to use device
> MACHINE_EXTRA_RRECOMMNEDS  (<-- typo still present)
> 
>     List of packages useful to use device (for example additional kernel
> modules)
> "
> 
> From the above, I don't learn anything about what the impact of using
> each of these variables is. Which ones add the packages to the rootfs,
> which only build them, do they impact the initrd (one might assume a
> package required for boot should be in the initrd). The difference
> between RDEPENDS and RRECOMMENDS is unclear. Three of the four use the
> term "required" while the other uses "useful" in the description.
> Neither semantic split (RDEPENDS vs. RRECOMMENDS, ESSENTIAL vs. EXTRA)
> can be understood from the available descriptions.
> 
> The firmware I want to add is needed for wireless functionality. This is
> likely not required for boot, so I added it to MACHINE_EXTRA_RRECOMMENDS
> as it is only needed if the wireless driver is used. This resulted in
> the package being built, but not installed. In practice this would mean
> that people building the BSP would need to manually add the firmware
> package to IMAGE_INSTALL or install it manually later. This strikes me
> as unnecessarily complex.
> 
> This was also unexpected given the definition of RRECOMMENDS in the
> manual, which would have led me to believe the package should have been
> installed:
> 
> "
> RRECOMMENDS
> 
>     List of packages which extend usability of the package. Those
> packages will be automatically installed but can be removed by user.
> "


Paul E. explained that RRECOMMENDS differs from RDEPENDS in that if a
RRECOMMENDS package is not available (because the providing recipe
doesn't build it - like a custom linux kernel not building a particular
module) the image build will continue and not error out.


> I wasn't able to find a general meaning for the term "EXTRA" by
> searching through the manual.
> 
> I then tried the remaining 3 variables, doing the following after
> setting each:
> 
> $ bitbake -c cleanall task-machine-base task-core-boot
> linux-firmware-core-image-sato
> $ bitbake -u depexp -g core-image-sato
> $ bitbake core-image-sato
> 
> I then determined if the linux-firmware package appeared in the list
> presented by the depexp UI and whether or not the linux-firmware package
> was built and installed. The results follow:
> 
> MACHINE_EXTRA_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
> # appears in depexp with no rdepends, builds, does not install
> 
> #MACHINE_EXTRA_RECOMMENDS += "linux-firmware-iwlwifi-6000g2a-5"
> # does not appear in depexp, does not build
> 
> #MACHINE_ESSENTIAL_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
> # does not appear in depexp, does not build


If I use the appropriate MACHINE_ESSENTIAL_EXTRA_RDEPENDS here, the
package is built, but it is not included in the final image.


> I've tried searching the sources for these variables, and quickly lose
> their scent. Here is what I did find:
> 
> task-base.bb:
> #
> # packages added by machine config
> #
> RDEPENDS_task-machine-base = "${MACHINE_EXTRA_RDEPENDS}"
> RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"
> 
> task-core-boot.bb:
> RDEPENDS_task-core-boot = "\
>     ...
>     ${MACHINE_ESSENTIAL_EXTRA_RDEPENDS}"
> 
> RRECOMMENDS_task-core-boot = "\
>     ${MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS}"
> 
> 
> How these should impact the final image is not clear to me.
> 
> 
> Thanks,
> 


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Richard Purdie]

On Tue, 2011-09-27 at 08:51 -0700, Darren Hart wrote:
> On 09/21/2011 02:29 PM, Darren Hart wrote:
> > This is something I've run into frequently and have been putting off
> > trying to resolve. Hopefully the following will help illustrate the path
> > a BSP developer might go down trying to use the MACHINE_ESSENTIAL* and
> > MACHINE_EXTRA* variables. If someone with more knowledge on the subject
> > could fill in the gaps noted below, I'll be happy to test and work with
> > Scott to improve the documentation.
> > 
> > I am unable to discern from the following text in the reference manual,
> > which variable I should be using to include a firmware package in the
> > images for a given BSP.
> > 
> > From
> > http://www.yoctoproject.org/docs/1.1/poky-ref-manual/poky-ref-manual.html:
> > 
> > "
> > MACHINE_ESSENTIAL_RDEPENDS
> > 
> >     List of packages required to boot device
> > MACHINE_ESSENTIAL_RRECOMMENDS
> > 
> >     List of packages required to boot device (usually additional kernel
> > modules)
> 
> As Saul points out, these should be:
> 
> MACHINE_ESSENTIAL_EXTRA_*
> 
> 
> > MACHINE_EXTRA_RDEPENDS
> > 
> >     List of packages required to use device
> > MACHINE_EXTRA_RRECOMMNEDS  (<-- typo still present)
> > 
> >     List of packages useful to use device (for example additional kernel
> > modules)
> > "
> > 
> > From the above, I don't learn anything about what the impact of using
> > each of these variables is. Which ones add the packages to the rootfs,
> > which only build them, do they impact the initrd (one might assume a
> > package required for boot should be in the initrd). The difference
> > between RDEPENDS and RRECOMMENDS is unclear. Three of the four use the
> > term "required" while the other uses "useful" in the description.
> > Neither semantic split (RDEPENDS vs. RRECOMMENDS, ESSENTIAL vs. EXTRA)
> > can be understood from the available descriptions.
> > 
> > The firmware I want to add is needed for wireless functionality. This is
> > likely not required for boot, so I added it to MACHINE_EXTRA_RRECOMMENDS
> > as it is only needed if the wireless driver is used. This resulted in
> > the package being built, but not installed. In practice this would mean
> > that people building the BSP would need to manually add the firmware
> > package to IMAGE_INSTALL or install it manually later. This strikes me
> > as unnecessarily complex.
> > 
> > This was also unexpected given the definition of RRECOMMENDS in the
> > manual, which would have led me to believe the package should have been
> > installed:
> > 
> > "
> > RRECOMMENDS
> > 
> >     List of packages which extend usability of the package. Those
> > packages will be automatically installed but can be removed by user.
> > "
> 
> 
> Paul E. explained that RRECOMMENDS differs from RDEPENDS in that if a
> RRECOMMENDS package is not available (because the providing recipe
> doesn't build it - like a custom linux kernel not building a particular
> module) the image build will continue and not error out.

Right. Its really there so people can flag modules to include (if
they're built as modules) so that is that difference.

Let me explain what should happen:

ESSENTIAL are ones which are required to get the system to boot and be
of any use. These would be flash/mtd drivers, screen drivers,
keyboard/mouse/touchscreen. 

The EXTRA things are things that are nice to have but are not needed to
boot. Wifi drivers would be an example if we known a given machine
contains some kind of wifi card, the driver would need to be present to
make the system 100% functional.

There is a line in task-base.bb which hints at this:

#
# those ones can be set in machine config to supply packages needed to get machine booting
#
MACHINE_ESSENTIAL_EXTRA_RDEPENDS ?= ""
MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS ?= ""


> > I wasn't able to find a general meaning for the term "EXTRA" by
> > searching through the manual.
> > 
> > I then tried the remaining 3 variables, doing the following after
> > setting each:
> > 
> > $ bitbake -c cleanall task-machine-base task-core-boot
> > linux-firmware-core-image-sato
> > $ bitbake -u depexp -g core-image-sato
> > $ bitbake core-image-sato
> > 
> > I then determined if the linux-firmware package appeared in the list
> > presented by the depexp UI and whether or not the linux-firmware package
> > was built and installed. The results follow:
> > 
> > MACHINE_EXTRA_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
> > # appears in depexp with no rdepends, builds, does not install
> > 
> > #MACHINE_EXTRA_RECOMMENDS += "linux-firmware-iwlwifi-6000g2a-5"
> > # does not appear in depexp, does not build
> > 
> > #MACHINE_ESSENTIAL_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
> > # does not appear in depexp, does not build
> 
> 
> If I use the appropriate MACHINE_ESSENTIAL_EXTRA_RDEPENDS here, the
> package is built, but it is not included in the final image.

We have two different types of image, minimal/basic and the others. The
former is derived from task-core-boot, the others derive from task-base.
The former is a minimal setup of things just needed to boot, task-base
has more dependencies but is more suited to real world use.

> > I've tried searching the sources for these variables, and quickly lose
> > their scent. Here is what I did find:
> > 
> > task-base.bb:
> > #
> > # packages added by machine config
> > #
> > RDEPENDS_task-machine-base = "${MACHINE_EXTRA_RDEPENDS}"
> > RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"
> > 
> > task-core-boot.bb:
> > RDEPENDS_task-core-boot = "\
> >     ...
> >     ${MACHINE_ESSENTIAL_EXTRA_RDEPENDS}"
> > 
> > RRECOMMENDS_task-core-boot = "\
> >     ${MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS}"
> > 
> > 
> > How these should impact the final image is not clear to me.

So if you change MACHINE_EXTRA* you need to rebuild task-base, if you
change MACHINE_ESSENTIAL_EXTRA* you need to rebuild task-core-boot.
Confusing and the checksum code we'll enable soon should make this
automatic.

The task-core-boot is included into task-base by task-distro-base
through DISTRO_EXTRA_RDEPENDS which in meta-yocto is set to
task-core-boot. Both sets of variables should therefore appear in most
images except minimal and basic which should only get the essential
ones.

I hope that at least clarifies what should happen...

Cheers,

Richard

Re: MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Darren Hart]

On 09/27/2011 11:04 AM, Richard Purdie wrote:
> On Tue, 2011-09-27 at 08:51 -0700, Darren Hart wrote:
>> On 09/21/2011 02:29 PM, Darren Hart wrote:
>>> This is something I've run into frequently and have been putting off
>>> trying to resolve. Hopefully the following will help illustrate the path
>>> a BSP developer might go down trying to use the MACHINE_ESSENTIAL* and
>>> MACHINE_EXTRA* variables. If someone with more knowledge on the subject
>>> could fill in the gaps noted below, I'll be happy to test and work with
>>> Scott to improve the documentation.
>>>
>>> I am unable to discern from the following text in the reference manual,
>>> which variable I should be using to include a firmware package in the
>>> images for a given BSP.
>>>
>>> From
>>> http://www.yoctoproject.org/docs/1.1/poky-ref-manual/poky-ref-manual.html:
>>>
>>> "
>>> MACHINE_ESSENTIAL_RDEPENDS
>>>
>>>     List of packages required to boot device
>>> MACHINE_ESSENTIAL_RRECOMMENDS
>>>
>>>     List of packages required to boot device (usually additional kernel
>>> modules)
>>
>> As Saul points out, these should be:
>>
>> MACHINE_ESSENTIAL_EXTRA_*
>>
>>
>>> MACHINE_EXTRA_RDEPENDS
>>>
>>>     List of packages required to use device
>>> MACHINE_EXTRA_RRECOMMNEDS  (<-- typo still present)
>>>
>>>     List of packages useful to use device (for example additional kernel
>>> modules)
>>> "
>>>
>>> From the above, I don't learn anything about what the impact of using
>>> each of these variables is. Which ones add the packages to the rootfs,
>>> which only build them, do they impact the initrd (one might assume a
>>> package required for boot should be in the initrd). The difference
>>> between RDEPENDS and RRECOMMENDS is unclear. Three of the four use the
>>> term "required" while the other uses "useful" in the description.
>>> Neither semantic split (RDEPENDS vs. RRECOMMENDS, ESSENTIAL vs. EXTRA)
>>> can be understood from the available descriptions.
>>>
>>> The firmware I want to add is needed for wireless functionality. This is
>>> likely not required for boot, so I added it to MACHINE_EXTRA_RRECOMMENDS
>>> as it is only needed if the wireless driver is used. This resulted in
>>> the package being built, but not installed. In practice this would mean
>>> that people building the BSP would need to manually add the firmware
>>> package to IMAGE_INSTALL or install it manually later. This strikes me
>>> as unnecessarily complex.
>>>
>>> This was also unexpected given the definition of RRECOMMENDS in the
>>> manual, which would have led me to believe the package should have been
>>> installed:
>>>
>>> "
>>> RRECOMMENDS
>>>
>>>     List of packages which extend usability of the package. Those
>>> packages will be automatically installed but can be removed by user.
>>> "
>>
>>
>> Paul E. explained that RRECOMMENDS differs from RDEPENDS in that if a
>> RRECOMMENDS package is not available (because the providing recipe
>> doesn't build it - like a custom linux kernel not building a particular
>> module) the image build will continue and not error out.
> 
> Right. Its really there so people can flag modules to include (if
> they're built as modules) so that is that difference.
> 
> Let me explain what should happen:
> 
> ESSENTIAL are ones which are required to get the system to boot and be
> of any use. These would be flash/mtd drivers, screen drivers,
> keyboard/mouse/touchscreen. 
> 
> The EXTRA things are things that are nice to have but are not needed to
> boot. Wifi drivers would be an example if we known a given machine
> contains some kind of wifi card, the driver would need to be present to
> make the system 100% functional.
> 
> There is a line in task-base.bb which hints at this:
> 
> #
> # those ones can be set in machine config to supply packages needed to get machine booting
> #
> MACHINE_ESSENTIAL_EXTRA_RDEPENDS ?= ""
> MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS ?= ""
> 
> 
>>> I wasn't able to find a general meaning for the term "EXTRA" by
>>> searching through the manual.
>>>
>>> I then tried the remaining 3 variables, doing the following after
>>> setting each:
>>>
>>> $ bitbake -c cleanall task-machine-base task-core-boot
>>> linux-firmware-core-image-sato
>>> $ bitbake -u depexp -g core-image-sato
>>> $ bitbake core-image-sato
>>>
>>> I then determined if the linux-firmware package appeared in the list
>>> presented by the depexp UI and whether or not the linux-firmware package
>>> was built and installed. The results follow:
>>>
>>> MACHINE_EXTRA_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
>>> # appears in depexp with no rdepends, builds, does not install
>>>
>>> #MACHINE_EXTRA_RECOMMENDS += "linux-firmware-iwlwifi-6000g2a-5"
>>> # does not appear in depexp, does not build
>>>
>>> #MACHINE_ESSENTIAL_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
>>> # does not appear in depexp, does not build
>>
>>
>> If I use the appropriate MACHINE_ESSENTIAL_EXTRA_RDEPENDS here, the
>> package is built, but it is not included in the final image.
> 
> We have two different types of image, minimal/basic and the others. The
> former is derived from task-core-boot, the others derive from task-base.
> The former is a minimal setup of things just needed to boot, task-base
> has more dependencies but is more suited to real world use.
> 
>>> I've tried searching the sources for these variables, and quickly lose
>>> their scent. Here is what I did find:
>>>
>>> task-base.bb:
>>> #
>>> # packages added by machine config
>>> #
>>> RDEPENDS_task-machine-base = "${MACHINE_EXTRA_RDEPENDS}"
>>> RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"
>>>
>>> task-core-boot.bb:
>>> RDEPENDS_task-core-boot = "\
>>>     ...
>>>     ${MACHINE_ESSENTIAL_EXTRA_RDEPENDS}"
>>>
>>> RRECOMMENDS_task-core-boot = "\
>>>     ${MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS}"
>>>
>>>
>>> How these should impact the final image is not clear to me.
> 
> So if you change MACHINE_EXTRA* you need to rebuild task-base, if you
> change MACHINE_ESSENTIAL_EXTRA* you need to rebuild task-core-boot.
> Confusing and the checksum code we'll enable soon should make this
> automatic.
> 
> The task-core-boot is included into task-base by task-distro-base
> through DISTRO_EXTRA_RDEPENDS which in meta-yocto is set to
> task-core-boot. Both sets of variables should therefore appear in most
> images except minimal and basic which should only get the essential
> ones.
> 
> I hope that at least clarifies what should happen...

I pulled master and rebuilt using all 4 options. All 4 resulted in the
firmware packages being built and installed in the rootfs. Either
something changed in master since I tried this last, or my machine was
the victim of a gamma ray event.... sigh.

Nevermind, there is nothing to see here.

Still, I think the discussion of documentation was worth the trouble,
I'll work with Scott to update what we have in the reference manual with
the feedback from Saul, Paul, and RP.

Thanks everyone.


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Rifenbark, Scott M]

I am currently crafting descriptions of these four variables plus the RDEPENDS and RRECOMMENDS variables.  When I get them done I will distribute for discussion. 

Thanks, 
Scott

-----Original Message-----
From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Darren Hart
Sent: Tuesday, September 27, 2011 1:03 PM
To: Richard Purdie
Cc: Yocto Project; Eggleton, Paul; Wold, Saul
Subject: Re: [yocto] MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

On 09/27/2011 11:04 AM, Richard Purdie wrote:
> On Tue, 2011-09-27 at 08:51 -0700, Darren Hart wrote:
>> On 09/21/2011 02:29 PM, Darren Hart wrote:
>>> This is something I've run into frequently and have been putting off
>>> trying to resolve. Hopefully the following will help illustrate the path
>>> a BSP developer might go down trying to use the MACHINE_ESSENTIAL* and
>>> MACHINE_EXTRA* variables. If someone with more knowledge on the subject
>>> could fill in the gaps noted below, I'll be happy to test and work with
>>> Scott to improve the documentation.
>>>
>>> I am unable to discern from the following text in the reference manual,
>>> which variable I should be using to include a firmware package in the
>>> images for a given BSP.
>>>
>>> From
>>> http://www.yoctoproject.org/docs/1.1/poky-ref-manual/poky-ref-manual.html:
>>>
>>> "
>>> MACHINE_ESSENTIAL_RDEPENDS
>>>
>>>     List of packages required to boot device
>>> MACHINE_ESSENTIAL_RRECOMMENDS
>>>
>>>     List of packages required to boot device (usually additional kernel
>>> modules)
>>
>> As Saul points out, these should be:
>>
>> MACHINE_ESSENTIAL_EXTRA_*
>>
>>
>>> MACHINE_EXTRA_RDEPENDS
>>>
>>>     List of packages required to use device
>>> MACHINE_EXTRA_RRECOMMNEDS  (<-- typo still present)
>>>
>>>     List of packages useful to use device (for example additional kernel
>>> modules)
>>> "
>>>
>>> From the above, I don't learn anything about what the impact of using
>>> each of these variables is. Which ones add the packages to the rootfs,
>>> which only build them, do they impact the initrd (one might assume a
>>> package required for boot should be in the initrd). The difference
>>> between RDEPENDS and RRECOMMENDS is unclear. Three of the four use the
>>> term "required" while the other uses "useful" in the description.
>>> Neither semantic split (RDEPENDS vs. RRECOMMENDS, ESSENTIAL vs. EXTRA)
>>> can be understood from the available descriptions.
>>>
>>> The firmware I want to add is needed for wireless functionality. This is
>>> likely not required for boot, so I added it to MACHINE_EXTRA_RRECOMMENDS
>>> as it is only needed if the wireless driver is used. This resulted in
>>> the package being built, but not installed. In practice this would mean
>>> that people building the BSP would need to manually add the firmware
>>> package to IMAGE_INSTALL or install it manually later. This strikes me
>>> as unnecessarily complex.
>>>
>>> This was also unexpected given the definition of RRECOMMENDS in the
>>> manual, which would have led me to believe the package should have been
>>> installed:
>>>
>>> "
>>> RRECOMMENDS
>>>
>>>     List of packages which extend usability of the package. Those
>>> packages will be automatically installed but can be removed by user.
>>> "
>>
>>
>> Paul E. explained that RRECOMMENDS differs from RDEPENDS in that if a
>> RRECOMMENDS package is not available (because the providing recipe
>> doesn't build it - like a custom linux kernel not building a particular
>> module) the image build will continue and not error out.
> 
> Right. Its really there so people can flag modules to include (if
> they're built as modules) so that is that difference.
> 
> Let me explain what should happen:
> 
> ESSENTIAL are ones which are required to get the system to boot and be
> of any use. These would be flash/mtd drivers, screen drivers,
> keyboard/mouse/touchscreen. 
> 
> The EXTRA things are things that are nice to have but are not needed to
> boot. Wifi drivers would be an example if we known a given machine
> contains some kind of wifi card, the driver would need to be present to
> make the system 100% functional.
> 
> There is a line in task-base.bb which hints at this:
> 
> #
> # those ones can be set in machine config to supply packages needed to get machine booting
> #
> MACHINE_ESSENTIAL_EXTRA_RDEPENDS ?= ""
> MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS ?= ""
> 
> 
>>> I wasn't able to find a general meaning for the term "EXTRA" by
>>> searching through the manual.
>>>
>>> I then tried the remaining 3 variables, doing the following after
>>> setting each:
>>>
>>> $ bitbake -c cleanall task-machine-base task-core-boot
>>> linux-firmware-core-image-sato
>>> $ bitbake -u depexp -g core-image-sato
>>> $ bitbake core-image-sato
>>>
>>> I then determined if the linux-firmware package appeared in the list
>>> presented by the depexp UI and whether or not the linux-firmware package
>>> was built and installed. The results follow:
>>>
>>> MACHINE_EXTRA_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
>>> # appears in depexp with no rdepends, builds, does not install
>>>
>>> #MACHINE_EXTRA_RECOMMENDS += "linux-firmware-iwlwifi-6000g2a-5"
>>> # does not appear in depexp, does not build
>>>
>>> #MACHINE_ESSENTIAL_RDEPENDS = "linux-firmware-iwlwifi-6000g2a-5"
>>> # does not appear in depexp, does not build
>>
>>
>> If I use the appropriate MACHINE_ESSENTIAL_EXTRA_RDEPENDS here, the
>> package is built, but it is not included in the final image.
> 
> We have two different types of image, minimal/basic and the others. The
> former is derived from task-core-boot, the others derive from task-base.
> The former is a minimal setup of things just needed to boot, task-base
> has more dependencies but is more suited to real world use.
> 
>>> I've tried searching the sources for these variables, and quickly lose
>>> their scent. Here is what I did find:
>>>
>>> task-base.bb:
>>> #
>>> # packages added by machine config
>>> #
>>> RDEPENDS_task-machine-base = "${MACHINE_EXTRA_RDEPENDS}"
>>> RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"
>>>
>>> task-core-boot.bb:
>>> RDEPENDS_task-core-boot = "\
>>>     ...
>>>     ${MACHINE_ESSENTIAL_EXTRA_RDEPENDS}"
>>>
>>> RRECOMMENDS_task-core-boot = "\
>>>     ${MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS}"
>>>
>>>
>>> How these should impact the final image is not clear to me.
> 
> So if you change MACHINE_EXTRA* you need to rebuild task-base, if you
> change MACHINE_ESSENTIAL_EXTRA* you need to rebuild task-core-boot.
> Confusing and the checksum code we'll enable soon should make this
> automatic.
> 
> The task-core-boot is included into task-base by task-distro-base
> through DISTRO_EXTRA_RDEPENDS which in meta-yocto is set to
> task-core-boot. Both sets of variables should therefore appear in most
> images except minimal and basic which should only get the essential
> ones.
> 
> I hope that at least clarifies what should happen...

I pulled master and rebuilt using all 4 options. All 4 resulted in the
firmware packages being built and installed in the rootfs. Either
something changed in master since I tried this last, or my machine was
the victim of a gamma ray event.... sigh.

Nevermind, there is nothing to see here.

Still, I think the discussion of documentation was worth the trouble,
I'll work with Scott to update what we have in the reference manual with
the feedback from Saul, Paul, and RP.

Thanks everyone.


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Koen Kooi]

Op 27 sep. 2011, om 20:04 heeft Richard Purdie het volgende geschreven:

[..]

> So if you change MACHINE_EXTRA* you need to rebuild task-base, if you
> change MACHINE_ESSENTIAL_EXTRA* you need to rebuild task-core-boot.
> Confusing and the checksum code we'll enable soon should make this
> automatic.
> 
> The task-core-boot is included into task-base by task-distro-base
> through DISTRO_EXTRA_RDEPENDS which in meta-yocto is set to
> task-core-boot. Both sets of variables should therefore appear in most
> images except minimal and basic which should only get the essential
> ones.
> 
> I hope that at least clarifies what should happen...

To combat the confusion I did it slightly different in task-basic:

https://github.com/openembedded/meta-oe/blob/master/meta-oe/recipes-core/tasks/task-basic.bb

Any var besides *_FEATURES you can poke is prefixed with TASK_BASIC:

TASK_BASIC_SSHDAEMON

I would propose that if the EXTRA and ESSENTIAL stuff is only used a single recipe that they should be prefixed to give people a better chance at finding out what it impacts.

Right now you'd need to call it BUMP_PR_OF_TASK_CORE_RECIPE_IF_YOU_CHANGE_ME_MACHINE_ESSENTIAL_*, but that's a different discussion :)

regards,

Koen

Re: MACHINE_EXTRA_* and MACHINE_ESSENTIAL_* documentation

[Koen Kooi]

Op 27 sep. 2011, om 20:04 heeft Richard Purdie het volgende geschreven:

[..]

> So if you change MACHINE_EXTRA* you need to rebuild task-base, if you
> change MACHINE_ESSENTIAL_EXTRA* you need to rebuild task-core-boot.
> Confusing and the checksum code we'll enable soon should make this
> automatic.
> 
> The task-core-boot is included into task-base by task-distro-base
> through DISTRO_EXTRA_RDEPENDS which in meta-yocto is set to
> task-core-boot. Both sets of variables should therefore appear in most
> images except minimal and basic which should only get the essential
> ones.
> 
> I hope that at least clarifies what should happen...

To combat the confusion I did it slightly different in task-basic:

https://github.com/openembedded/meta-oe/blob/master/meta-oe/recipes-core/tasks/task-basic.bb

Any var besides *_FEATURES you can poke is prefixed with TASK_BASIC:

TASK_BASIC_SSHDAEMON

I would propose that if the EXTRA and ESSENTIAL stuff is only used a single recipe that they should be prefixed to give people a better chance at finding out what it impacts.

Right now you'd need to call it BUMP_PR_OF_TASK_CORE_RECIPE_IF_YOU_CHANGE_ME_MACHINE_ESSENTIAL_*, but that's a different discussion :)

regards,

Koen