Removing rootfs from SDK

Removing rootfs from SDK

[John Ernberg]

Hi

We're trying to optimize the SDK generated by bitbake -c populate_sdk.
Currently we're trying to remove the kernel, modules and other 
executables which we have no use for, most of it could be removed using 
IMAGE_INSTALL = "" and IMAGE_FEATURES = "".

Due to us using 2 different kernel module sets, we're using 
IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not 
cleared by the IMAGE_INSTALL = "" setting.

I've tried to do IMAGE_INSTALL_remove using the same variable that we 
use to populate the IMAGE_INSTALL_append, but that doesn't work. I can 
however remove each individual package added by IMAGE_INSTALL_append.
Due to the number of packages added by IMAGE_INSTALL_append this is not 
really feasible.

Is there a way to clear IMAGE_INSTALL_append without doing an 
IMAGE_INSTALL_remove per package? Alternatively clearing it using a 
python loop without needing to know the name of each package.

We're also seeing busybox getting included into the SDK without anything 
showing a dependency on it from running bitbake -g -c populate_sdk.

What could be going on with that?

We're using poky daisy 11 and Yocto 1.6.1.

Thank you in advance.

Best regards // John Ernberg

Re: Removing rootfs from SDK

[Paul Eggleton]

Hi John,

On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
> Currently we're trying to remove the kernel, modules and other
> executables which we have no use for, most of it could be removed using
> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
> 
> Due to us using 2 different kernel module sets, we're using
> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
> cleared by the IMAGE_INSTALL = "" setting.
> 
> I've tried to do IMAGE_INSTALL_remove using the same variable that we
> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
> however remove each individual package added by IMAGE_INSTALL_append.
> Due to the number of packages added by IMAGE_INSTALL_append this is not
> really feasible.
> 
> Is there a way to clear IMAGE_INSTALL_append without doing an
> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
> python loop without needing to know the name of each package.
> 
> We're also seeing busybox getting included into the SDK without anything
> showing a dependency on it from running bitbake -g -c populate_sdk.
> 
> What could be going on with that?

We construct the SDK for an image by getting a list of the packages in the 
image, and then including the -dev and -dbg packages that correspond to those 
in the host portion of the SDK. Thus if your image has busybox in it then you 
will get busybox-dev and busybox-dbg in your SDK.

In the dizzy (1.7) and later releases, there is a 
PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match packages 
that you do not wish to install complementary packages for. You could set this 
in your image recipe. Note that this will affect all complementary package 
processing for the image as well as the SDK (e.g. dev-pkgs in IMAGE_FEATURES 
will also be affected, if you used that).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Removing rootfs from SDK

[John Ernberg]

Hi Paul

I managed to clear out everything set by IMAGE_INSTALL and 
IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
So we no longer package the kernel etc into our SDK.
We do however still package the busybox binaries, when I run:
bitbake -e -c populate_sdk [image] | less
and then search for busybox, I get no results, so according to the 
variables nothing adds busybox to the SDK.
I cannot figure out why busybox would get included, and I don't mean the 
dev and dbg packages here, but the actual binary package.

For this project we're not able to update to dizzy or later at this 
time, we're currently on 1.6.1.
So the PACKAGE_EXCLUDE_COMPLEMENTARY variable is not available to us.

Best regards // John Ernberg

On 2015-06-12 14:43, Paul Eggleton wrote:
> Hi John,
>
> On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
>> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
>> Currently we're trying to remove the kernel, modules and other
>> executables which we have no use for, most of it could be removed using
>> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
>>
>> Due to us using 2 different kernel module sets, we're using
>> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
>> cleared by the IMAGE_INSTALL = "" setting.
>>
>> I've tried to do IMAGE_INSTALL_remove using the same variable that we
>> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
>> however remove each individual package added by IMAGE_INSTALL_append.
>> Due to the number of packages added by IMAGE_INSTALL_append this is not
>> really feasible.
>>
>> Is there a way to clear IMAGE_INSTALL_append without doing an
>> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
>> python loop without needing to know the name of each package.
>>
>> We're also seeing busybox getting included into the SDK without anything
>> showing a dependency on it from running bitbake -g -c populate_sdk.
>>
>> What could be going on with that?
> We construct the SDK for an image by getting a list of the packages in the
> image, and then including the -dev and -dbg packages that correspond to those
> in the host portion of the SDK. Thus if your image has busybox in it then you
> will get busybox-dev and busybox-dbg in your SDK.
>
> In the dizzy (1.7) and later releases, there is a
> PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match packages
> that you do not wish to install complementary packages for. You could set this
> in your image recipe. Note that this will affect all complementary package
> processing for the image as well as the SDK (e.g. dev-pkgs in IMAGE_FEATURES
> will also be affected, if you used that).
>
> Cheers,
> Paul
>

Re: Removing rootfs from SDK

[Paul Eggleton]

On Monday 15 June 2015 05:39:24 John Ernberg wrote:
> On 2015-06-12 14:43, Paul Eggleton wrote:
> > On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
> >> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
> >> Currently we're trying to remove the kernel, modules and other
> >> executables which we have no use for, most of it could be removed using
> >> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
> >> 
> >> Due to us using 2 different kernel module sets, we're using
> >> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
> >> cleared by the IMAGE_INSTALL = "" setting.
> >> 
> >> I've tried to do IMAGE_INSTALL_remove using the same variable that we
> >> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
> >> however remove each individual package added by IMAGE_INSTALL_append.
> >> Due to the number of packages added by IMAGE_INSTALL_append this is not
> >> really feasible.
> >> 
> >> Is there a way to clear IMAGE_INSTALL_append without doing an
> >> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
> >> python loop without needing to know the name of each package.
> >> 
> >> We're also seeing busybox getting included into the SDK without anything
> >> showing a dependency on it from running bitbake -g -c populate_sdk.
> >> 
> >> What could be going on with that?
> > 
> > We construct the SDK for an image by getting a list of the packages in the
> > image, and then including the -dev and -dbg packages that correspond to
> > those in the host portion of the SDK. Thus if your image has busybox in
> > it then you will get busybox-dev and busybox-dbg in your SDK.
> > 
> > In the dizzy (1.7) and later releases, there is a
> > PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match
> > packages that you do not wish to install complementary packages for. You
> > could set this in your image recipe. Note that this will affect all
> > complementary package processing for the image as well as the SDK (e.g.
> > dev-pkgs in IMAGE_FEATURES will also be affected, if you used that).
>
> I managed to clear out everything set by IMAGE_INSTALL and
> IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
> So we no longer package the kernel etc into our SDK.
> We do however still package the busybox binaries, when I run:
> bitbake -e -c populate_sdk [image] | less
> and then search for busybox, I get no results, so according to the
> variables nothing adds busybox to the SDK.
> I cannot figure out why busybox would get included, and I don't mean the
> dev and dbg packages here, but the actual binary package.

That'll be because the -dev package depends on the main package; 
meta/conf/bitbake.conf has the following line which sets up this dependency:

RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"

You could set RDEPENDS_${PN}-dev = "" at the configuration level to disable 
this, or you could do it per recipe with bbappends.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Removing rootfs from SDK

[John Ernberg]

On 2015-06-15 10:59, Paul Eggleton wrote:
> On Monday 15 June 2015 05:39:24 John Ernberg wrote:
>> On 2015-06-12 14:43, Paul Eggleton wrote:
>>> On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
>>>> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
>>>> Currently we're trying to remove the kernel, modules and other
>>>> executables which we have no use for, most of it could be removed using
>>>> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
>>>>
>>>> Due to us using 2 different kernel module sets, we're using
>>>> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
>>>> cleared by the IMAGE_INSTALL = "" setting.
>>>>
>>>> I've tried to do IMAGE_INSTALL_remove using the same variable that we
>>>> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
>>>> however remove each individual package added by IMAGE_INSTALL_append.
>>>> Due to the number of packages added by IMAGE_INSTALL_append this is not
>>>> really feasible.
>>>>
>>>> Is there a way to clear IMAGE_INSTALL_append without doing an
>>>> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
>>>> python loop without needing to know the name of each package.
>>>>
>>>> We're also seeing busybox getting included into the SDK without anything
>>>> showing a dependency on it from running bitbake -g -c populate_sdk.
>>>>
>>>> What could be going on with that?
>>> We construct the SDK for an image by getting a list of the packages in the
>>> image, and then including the -dev and -dbg packages that correspond to
>>> those in the host portion of the SDK. Thus if your image has busybox in
>>> it then you will get busybox-dev and busybox-dbg in your SDK.
>>>
>>> In the dizzy (1.7) and later releases, there is a
>>> PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match
>>> packages that you do not wish to install complementary packages for. You
>>> could set this in your image recipe. Note that this will affect all
>>> complementary package processing for the image as well as the SDK (e.g.
>>> dev-pkgs in IMAGE_FEATURES will also be affected, if you used that).
>> I managed to clear out everything set by IMAGE_INSTALL and
>> IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
>> So we no longer package the kernel etc into our SDK.
>> We do however still package the busybox binaries, when I run:
>> bitbake -e -c populate_sdk [image] | less
>> and then search for busybox, I get no results, so according to the
>> variables nothing adds busybox to the SDK.
>> I cannot figure out why busybox would get included, and I don't mean the
>> dev and dbg packages here, but the actual binary package.
> That'll be because the -dev package depends on the main package;
> meta/conf/bitbake.conf has the following line which sets up this dependency:
>
> RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
>
> You could set RDEPENDS_${PN}-dev = "" at the configuration level to disable
> this, or you could do it per recipe with bbappends.
>
> Cheers,
> Paul
>
Hi Paul

I just tried to apply this, combined with the PACKAGE_INSTALL solution, 
it will not work, and throws a huge build error log. Removing the 
PACKAGE_INSTALL solution and setting only the RDEPENDS solution in the 
local.conf will have no effect on the SDK at all. Busybox is still 
included, and it seems like all other binaries are back as well.

An excerpt of the build log:
ERROR: Cannot get the package dependencies. Command 
'[path-to-yocto-build-dir]/tmp/sysroots/x86_64-linux/usr/bin/rpmresolve 
-t 
[path-to-yocto-build-dir]/tmp/work/[machine-and-sdk]/1.0-r0/sdk/image/opt/[distro]/1.5+snapshot/sysroots/[target-arch]/var/lib/rpm' 
returned 1:
base-files|busybox
base-files|busybox
base-files|busybox
base-files|busybox
base-files|busybox
base-files|busybox
base-files|busybox
base-files|busybox
base-passwd|busybox
base-passwd-dev|libc6-dev [REC]
busybox|libc6
busybox|update-alternatives-opkg
busybox|libc6
busybox|libc6
busybox|libc6
busybox|libc6

And then the log continues similarly for several pages. What could have 
gone wrong?

Best regards // John Ernberg

Re: Removing rootfs from SDK

[Paul Eggleton]

On Tuesday 16 June 2015 14:05:15 John Ernberg wrote:
> On 2015-06-15 10:59, Paul Eggleton wrote:
> > On Monday 15 June 2015 05:39:24 John Ernberg wrote:
> >> On 2015-06-12 14:43, Paul Eggleton wrote:
> >>> On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
> >>>> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
> >>>> Currently we're trying to remove the kernel, modules and other
> >>>> executables which we have no use for, most of it could be removed using
> >>>> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
> >>>> 
> >>>> Due to us using 2 different kernel module sets, we're using
> >>>> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
> >>>> cleared by the IMAGE_INSTALL = "" setting.
> >>>> 
> >>>> I've tried to do IMAGE_INSTALL_remove using the same variable that we
> >>>> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
> >>>> however remove each individual package added by IMAGE_INSTALL_append.
> >>>> Due to the number of packages added by IMAGE_INSTALL_append this is not
> >>>> really feasible.
> >>>> 
> >>>> Is there a way to clear IMAGE_INSTALL_append without doing an
> >>>> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
> >>>> python loop without needing to know the name of each package.
> >>>> 
> >>>> We're also seeing busybox getting included into the SDK without
> >>>> anything
> >>>> showing a dependency on it from running bitbake -g -c populate_sdk.
> >>>> 
> >>>> What could be going on with that?
> >>> 
> >>> We construct the SDK for an image by getting a list of the packages in
> >>> the
> >>> image, and then including the -dev and -dbg packages that correspond to
> >>> those in the host portion of the SDK. Thus if your image has busybox in
> >>> it then you will get busybox-dev and busybox-dbg in your SDK.
> >>> 
> >>> In the dizzy (1.7) and later releases, there is a
> >>> PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match
> >>> packages that you do not wish to install complementary packages for. You
> >>> could set this in your image recipe. Note that this will affect all
> >>> complementary package processing for the image as well as the SDK (e.g.
> >>> dev-pkgs in IMAGE_FEATURES will also be affected, if you used that).
> >> 
> >> I managed to clear out everything set by IMAGE_INSTALL and
> >> IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
> >> So we no longer package the kernel etc into our SDK.
> >> We do however still package the busybox binaries, when I run:
> >> bitbake -e -c populate_sdk [image] | less
> >> and then search for busybox, I get no results, so according to the
> >> variables nothing adds busybox to the SDK.
> >> I cannot figure out why busybox would get included, and I don't mean the
> >> dev and dbg packages here, but the actual binary package.
> > 
> > That'll be because the -dev package depends on the main package;
> > meta/conf/bitbake.conf has the following line which sets up this
> > dependency:
> > 
> > RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
> > 
> > You could set RDEPENDS_${PN}-dev = "" at the configuration level to
> > disable this, or you could do it per recipe with bbappends.
> 
> I just tried to apply this, combined with the PACKAGE_INSTALL solution,
> it will not work, and throws a huge build error log. Removing the
> PACKAGE_INSTALL solution and setting only the RDEPENDS solution in the
> local.conf will have no effect on the SDK at all. Busybox is still
> included, and it seems like all other binaries are back as well.

OK, that is not what I would expect - somehow there is still a dependency on
those files. You may wish to enable buildhistory as described here and look at
the dependency graphs it produces:

http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality

It just occurred to me that if *any* of the packages you include in your
SDK include shell scripts, rpm will pretty much insist that you have a provider
of /bin/sh and that would be busybox, so that might also account for busybox
being in there. I don't immediately know how you would exclude that.

> An excerpt of the build log:
> ERROR: Cannot get the package dependencies. Command
> '[path-to-yocto-build-dir]/tmp/sysroots/x86_64-linux/usr/bin/rpmresolve
> -t
> [path-to-yocto-build-dir]/tmp/work/[machine-and-sdk]/1.0-r0/sdk/image/opt/[d
> istro]/1.5+snapshot/sysroots/[target-arch]/var/lib/rpm' returned 1:
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-files|busybox
> base-passwd|busybox
> base-passwd-dev|libc6-dev [REC]
> busybox|libc6
> busybox|update-alternatives-opkg
> busybox|libc6
> busybox|libc6
> busybox|libc6
> busybox|libc6
> 
> And then the log continues similarly for several pages. What could have
> gone wrong?

I can't tell what could be wrong from just this start of the message -
is there anything useful at the end?

Also what exactly did you set PACKAGE_INSTALL to ?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Removing rootfs from SDK

[John Ernberg]

On 2015-06-16 16:11, Paul Eggleton wrote:
> On Tuesday 16 June 2015 14:05:15 John Ernberg wrote:
>> On 2015-06-15 10:59, Paul Eggleton wrote:
>>> On Monday 15 June 2015 05:39:24 John Ernberg wrote:
>>>> On 2015-06-12 14:43, Paul Eggleton wrote:
>>>>> On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
>>>>>> We're trying to optimize the SDK generated by bitbake -c populate_sdk.
>>>>>> Currently we're trying to remove the kernel, modules and other
>>>>>> executables which we have no use for, most of it could be removed using
>>>>>> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
>>>>>>
>>>>>> Due to us using 2 different kernel module sets, we're using
>>>>>> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are not
>>>>>> cleared by the IMAGE_INSTALL = "" setting.
>>>>>>
>>>>>> I've tried to do IMAGE_INSTALL_remove using the same variable that we
>>>>>> use to populate the IMAGE_INSTALL_append, but that doesn't work. I can
>>>>>> however remove each individual package added by IMAGE_INSTALL_append.
>>>>>> Due to the number of packages added by IMAGE_INSTALL_append this is not
>>>>>> really feasible.
>>>>>>
>>>>>> Is there a way to clear IMAGE_INSTALL_append without doing an
>>>>>> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
>>>>>> python loop without needing to know the name of each package.
>>>>>>
>>>>>> We're also seeing busybox getting included into the SDK without
>>>>>> anything
>>>>>> showing a dependency on it from running bitbake -g -c populate_sdk.
>>>>>>
>>>>>> What could be going on with that?
>>>>> We construct the SDK for an image by getting a list of the packages in
>>>>> the
>>>>> image, and then including the -dev and -dbg packages that correspond to
>>>>> those in the host portion of the SDK. Thus if your image has busybox in
>>>>> it then you will get busybox-dev and busybox-dbg in your SDK.
>>>>>
>>>>> In the dizzy (1.7) and later releases, there is a
>>>>> PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match
>>>>> packages that you do not wish to install complementary packages for. You
>>>>> could set this in your image recipe. Note that this will affect all
>>>>> complementary package processing for the image as well as the SDK (e.g.
>>>>> dev-pkgs in IMAGE_FEATURES will also be affected, if you used that).
>>>> I managed to clear out everything set by IMAGE_INSTALL and
>>>> IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
>>>> So we no longer package the kernel etc into our SDK.
>>>> We do however still package the busybox binaries, when I run:
>>>> bitbake -e -c populate_sdk [image] | less
>>>> and then search for busybox, I get no results, so according to the
>>>> variables nothing adds busybox to the SDK.
>>>> I cannot figure out why busybox would get included, and I don't mean the
>>>> dev and dbg packages here, but the actual binary package.
>>> That'll be because the -dev package depends on the main package;
>>> meta/conf/bitbake.conf has the following line which sets up this
>>> dependency:
>>>
>>> RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
>>>
>>> You could set RDEPENDS_${PN}-dev = "" at the configuration level to
>>> disable this, or you could do it per recipe with bbappends.
>> I just tried to apply this, combined with the PACKAGE_INSTALL solution,
>> it will not work, and throws a huge build error log. Removing the
>> PACKAGE_INSTALL solution and setting only the RDEPENDS solution in the
>> local.conf will have no effect on the SDK at all. Busybox is still
>> included, and it seems like all other binaries are back as well.
> OK, that is not what I would expect - somehow there is still a dependency on
> those files. You may wish to enable buildhistory as described here and look at
> the dependency graphs it produces:
>
> http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality
>
> It just occurred to me that if *any* of the packages you include in your
> SDK include shell scripts, rpm will pretty much insist that you have a provider
> of /bin/sh and that would be busybox, so that might also account for busybox
> being in there. I don't immediately know how you would exclude that.
>
>> An excerpt of the build log:
>> ERROR: Cannot get the package dependencies. Command
>> '[path-to-yocto-build-dir]/tmp/sysroots/x86_64-linux/usr/bin/rpmresolve
>> -t
>> [path-to-yocto-build-dir]/tmp/work/[machine-and-sdk]/1.0-r0/sdk/image/opt/[d
>> istro]/1.5+snapshot/sysroots/[target-arch]/var/lib/rpm' returned 1:
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-files|busybox
>> base-passwd|busybox
>> base-passwd-dev|libc6-dev [REC]
>> busybox|libc6
>> busybox|update-alternatives-opkg
>> busybox|libc6
>> busybox|libc6
>> busybox|libc6
>> busybox|libc6
>>
>> And then the log continues similarly for several pages. What could have
>> gone wrong?
> I can't tell what could be wrong from just this start of the message -
> is there anything useful at the end?
>
> Also what exactly did you set PACKAGE_INSTALL to ?
>
> Cheers,
> Paul
>
Hi Paul

We do have some shell scripts in our SDK, I guess we're pretty much 
destined to keep including busybox as well due to that.
Checking the graphs for populate_sdk doesn't show any dependency on 
Busybox however.

The PACKAGE_INSTALL variable is set to "" in our SDK target, when I 
removed that, there was no build error, but it also left the SDK unaffected.

The error log shows nothing else, just continues in the same way as the 
part I shown (but with new package names).

We will just keep setting the PACKAGE_INSTALL to "", and leave busybox 
in it for now.

Thank you very much for the assistance.

Best regards // John Ernberg

Re: Removing rootfs from SDK

[Paul Eggleton]

On Tuesday 16 June 2015 14:20:35 John Ernberg wrote:
> On 2015-06-16 16:11, Paul Eggleton wrote:
> > On Tuesday 16 June 2015 14:05:15 John Ernberg wrote:
> >> On 2015-06-15 10:59, Paul Eggleton wrote:
> >>> On Monday 15 June 2015 05:39:24 John Ernberg wrote:
> >>>> On 2015-06-12 14:43, Paul Eggleton wrote:
> >>>>> On Thursday 04 June 2015 09:30:49 John Ernberg wrote:
> >>>>>> We're trying to optimize the SDK generated by bitbake -c
> >>>>>> populate_sdk.
> >>>>>> Currently we're trying to remove the kernel, modules and other
> >>>>>> executables which we have no use for, most of it could be removed
> >>>>>> using
> >>>>>> IMAGE_INSTALL = "" and IMAGE_FEATURES = "".
> >>>>>> 
> >>>>>> Due to us using 2 different kernel module sets, we're using
> >>>>>> IMAGE_INSTALL_append_[machine] additions to IMAGE_INSTALL which are
> >>>>>> not
> >>>>>> cleared by the IMAGE_INSTALL = "" setting.
> >>>>>> 
> >>>>>> I've tried to do IMAGE_INSTALL_remove using the same variable that we
> >>>>>> use to populate the IMAGE_INSTALL_append, but that doesn't work. I
> >>>>>> can
> >>>>>> however remove each individual package added by IMAGE_INSTALL_append.
> >>>>>> Due to the number of packages added by IMAGE_INSTALL_append this is
> >>>>>> not
> >>>>>> really feasible.
> >>>>>> 
> >>>>>> Is there a way to clear IMAGE_INSTALL_append without doing an
> >>>>>> IMAGE_INSTALL_remove per package? Alternatively clearing it using a
> >>>>>> python loop without needing to know the name of each package.
> >>>>>> 
> >>>>>> We're also seeing busybox getting included into the SDK without
> >>>>>> anything
> >>>>>> showing a dependency on it from running bitbake -g -c populate_sdk.
> >>>>>> 
> >>>>>> What could be going on with that?
> >>>>> 
> >>>>> We construct the SDK for an image by getting a list of the packages in
> >>>>> the
> >>>>> image, and then including the -dev and -dbg packages that correspond
> >>>>> to
> >>>>> those in the host portion of the SDK. Thus if your image has busybox
> >>>>> in
> >>>>> it then you will get busybox-dev and busybox-dbg in your SDK.
> >>>>> 
> >>>>> In the dizzy (1.7) and later releases, there is a
> >>>>> PACKAGE_EXCLUDE_COMPLEMENTARY variable that can take a regex to match
> >>>>> packages that you do not wish to install complementary packages for.
> >>>>> You
> >>>>> could set this in your image recipe. Note that this will affect all
> >>>>> complementary package processing for the image as well as the SDK
> >>>>> (e.g.
> >>>>> dev-pkgs in IMAGE_FEATURES will also be affected, if you used that).
> >>>> 
> >>>> I managed to clear out everything set by IMAGE_INSTALL and
> >>>> IMAGE_FEATURES by setting PACKAGE_INSTALL = "".
> >>>> So we no longer package the kernel etc into our SDK.
> >>>> We do however still package the busybox binaries, when I run:
> >>>> bitbake -e -c populate_sdk [image] | less
> >>>> and then search for busybox, I get no results, so according to the
> >>>> variables nothing adds busybox to the SDK.
> >>>> I cannot figure out why busybox would get included, and I don't mean
> >>>> the
> >>>> dev and dbg packages here, but the actual binary package.
> >>> 
> >>> That'll be because the -dev package depends on the main package;
> >>> meta/conf/bitbake.conf has the following line which sets up this
> >>> dependency:
> >>> 
> >>> RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
> >>> 
> >>> You could set RDEPENDS_${PN}-dev = "" at the configuration level to
> >>> disable this, or you could do it per recipe with bbappends.
> >> 
> >> I just tried to apply this, combined with the PACKAGE_INSTALL solution,
> >> it will not work, and throws a huge build error log. Removing the
> >> PACKAGE_INSTALL solution and setting only the RDEPENDS solution in the
> >> local.conf will have no effect on the SDK at all. Busybox is still
> >> included, and it seems like all other binaries are back as well.
> > 
> > OK, that is not what I would expect - somehow there is still a dependency
> > on those files. You may wish to enable buildhistory as described here and
> > look at the dependency graphs it produces:
> > 
> > http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#mainta
> > ining-build-output-quality
> > 
> > It just occurred to me that if *any* of the packages you include in your
> > SDK include shell scripts, rpm will pretty much insist that you have a
> > provider of /bin/sh and that would be busybox, so that might also account
> > for busybox being in there. I don't immediately know how you would
> > exclude that.> 
> >> An excerpt of the build log:
> >> ERROR: Cannot get the package dependencies. Command
> >> '[path-to-yocto-build-dir]/tmp/sysroots/x86_64-linux/usr/bin/rpmresolve
> >> -t
> >> [path-to-yocto-build-dir]/tmp/work/[machine-and-sdk]/1.0-r0/sdk/image/opt
> >> /[d istro]/1.5+snapshot/sysroots/[target-arch]/var/lib/rpm' returned 1:
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-files|busybox
> >> base-passwd|busybox
> >> base-passwd-dev|libc6-dev [REC]
> >> busybox|libc6
> >> busybox|update-alternatives-opkg
> >> busybox|libc6
> >> busybox|libc6
> >> busybox|libc6
> >> busybox|libc6
> >> 
> >> And then the log continues similarly for several pages. What could have
> >> gone wrong?
> > 
> > I can't tell what could be wrong from just this start of the message -
> > is there anything useful at the end?
> > 
> > Also what exactly did you set PACKAGE_INSTALL to ?
> 
> We do have some shell scripts in our SDK, I guess we're pretty much
> destined to keep including busybox as well due to that.
> Checking the graphs for populate_sdk doesn't show any dependency on
> Busybox however.

You do mean the graphs produced by buildhistory - not bitbake -g - right?

> The PACKAGE_INSTALL variable is set to "" in our SDK target, when I
> removed that, there was no build error, but it also left the SDK unaffected.

Ah right yes you did say that in one of your earlier replies - sorry.

> The error log shows nothing else, just continues in the same way as the
> part I shown (but with new package names).
> 
> We will just keep setting the PACKAGE_INSTALL to "", and leave busybox
> in it for now.

OK, fair enough.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre