[PATCH 0/1] buildtools-tarball: add nativesdk-libxml2-utils

[PATCH 0/1] buildtools-tarball: add nativesdk-libxml2-utils

[Chen Qi]

*** BLURB HERE ***
The following changes since commit 4a68a44f56c725914cfa721993a2ea8a3dc6ebd5:

  cve-update-db: Catch request.urlopen errors. (2019-07-05 12:00:20 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/buildtools-xmlcatalog
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/buildtools-xmlcatalog

Chen Qi (1):
  buildtools-tarball: add nativesdk-libxml2-utils

 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

-- 
1.9.1

[PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[Chen Qi]

For build-sysroots.bb, the xmlcatalog would not be in its
staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
   IMAGE_INSTALL_append = " btrfs-tools"
   DISTRO_FEATURES_append = " api-documentation"
   INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball
to ensure the existence of xmlcatalog. Also add it
to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 5e93f5c..2f64eae 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
 # Link to git-lfs if present
 HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"
+
 CCACHE ??= ""
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index 91df6f1..d39d6f7 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -25,6 +25,7 @@ TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-texinfo \
     nativesdk-libnss-nis \
     nativesdk-rpcsvc-proto \
+    nativesdk-libxml2-utils \
     "
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
-- 
1.9.1

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[akuster808]

On 7/8/19 12:37 AM, Chen Qi wrote:
> For build-sysroots.bb, the xmlcatalog would not be in its
> staging directory. Causing the following error for eSDK.
>
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function failed: do_build_native_sysroot
>
> The problem could be reproduced by the following steps.
> 1. Add in local.conf:
>    IMAGE_INSTALL_append = " btrfs-tools"
>    DISTRO_FEATURES_append = " api-documentation"
>    INHERIT += "testsdk"
> 2. bitbake core-image-minimal -c populate_sdk_ext
> 3. bitbake core-image-minimal -c testsdkext

should these steps be added to eSDK tests?

- armin
>
> So we add nativesdk-libxml2-utils to buildtools-tarball
> to ensure the existence of xmlcatalog. Also add it
> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/conf/bitbake.conf                       | 3 +++
>  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>  2 files changed, 4 insertions(+)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 5e93f5c..2f64eae 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>  # Link to git-lfs if present
>  HOSTTOOLS_NONFATAL += "git-lfs"
>  
> +# build-sysroot needs xmlcatalog in order for eSDK installation
> +HOSTTOOLS_NONFATAL += "xmlcatalog"
> +
>  CCACHE ??= ""
>  
>  TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
> index 91df6f1..d39d6f7 100644
> --- a/meta/recipes-core/meta/buildtools-tarball.bb
> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
> @@ -25,6 +25,7 @@ TOOLCHAIN_HOST_TASK ?= "\
>      nativesdk-texinfo \
>      nativesdk-libnss-nis \
>      nativesdk-rpcsvc-proto \
> +    nativesdk-libxml2-utils \
>      "
>  
>  MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[Richard Purdie]

On Mon, 2019-07-08 at 08:55 -0700, akuster808 wrote:
> 
> On 7/8/19 12:37 AM, Chen Qi wrote:
> > For build-sysroots.bb, the xmlcatalog would not be in its
> > staging directory. Causing the following error for eSDK.
> > 
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> > 127.
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > failed: do_build_native_sysroot
> > 
> > The problem could be reproduced by the following steps.
> > 1. Add in local.conf:
> >    IMAGE_INSTALL_append = " btrfs-tools"
> >    DISTRO_FEATURES_append = " api-documentation"
> >    INHERIT += "testsdk"
> > 2. bitbake core-image-minimal -c populate_sdk_ext
> > 3. bitbake core-image-minimal -c testsdkext
> 
> should these steps be added to eSDK tests?

api-documentation is only tested in a single build in qa-extras and
esdk is only tested in the standard config.

There are many different variants we don't test, its hard to tell which
ones we should be adding and are worth the extra build times...

Cheers,

Richard

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[Richard Purdie]

On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> For build-sysroots.bb, the xmlcatalog would not be in its
> staging directory. Causing the following error for eSDK.
> 
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> 127.
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> failed: do_build_native_sysroot
> 
> The problem could be reproduced by the following steps.
> 1. Add in local.conf:
>    IMAGE_INSTALL_append = " btrfs-tools"
>    DISTRO_FEATURES_append = " api-documentation"
>    INHERIT += "testsdk"
> 2. bitbake core-image-minimal -c populate_sdk_ext
> 3. bitbake core-image-minimal -c testsdkext
> 
> So we add nativesdk-libxml2-utils to buildtools-tarball
> to ensure the existence of xmlcatalog. Also add it
> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> 
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/conf/bitbake.conf                       | 3 +++
>  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>  2 files changed, 4 insertions(+)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 5e93f5c..2f64eae 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>  # Link to git-lfs if present
>  HOSTTOOLS_NONFATAL += "git-lfs"
>  
> +# build-sysroot needs xmlcatalog in order for eSDK installation
> +HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
grow to contain far too much random stuff which could impact
reproduciblity.

Is there some other way we could fix this? It still feels like a
dependency problem which we chould fix by adding a missing dependency
although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer
about dependencies?

Cheers,

Richard

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[Burton, Ross]

Agreed, making it a non-fatal dependency just means that some eSDKs
break, not all.  There needs to be a better fix.

Ross

On Mon, 8 Jul 2019 at 17:53, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> > For build-sysroots.bb, the xmlcatalog would not be in its
> > staging directory. Causing the following error for eSDK.
> >
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> > 127.
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > failed: do_build_native_sysroot
> >
> > The problem could be reproduced by the following steps.
> > 1. Add in local.conf:
> >    IMAGE_INSTALL_append = " btrfs-tools"
> >    DISTRO_FEATURES_append = " api-documentation"
> >    INHERIT += "testsdk"
> > 2. bitbake core-image-minimal -c populate_sdk_ext
> > 3. bitbake core-image-minimal -c testsdkext
> >
> > So we add nativesdk-libxml2-utils to buildtools-tarball
> > to ensure the existence of xmlcatalog. Also add it
> > to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> >
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > ---
> >  meta/conf/bitbake.conf                       | 3 +++
> >  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
> >  2 files changed, 4 insertions(+)
> >
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 5e93f5c..2f64eae 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
> >  # Link to git-lfs if present
> >  HOSTTOOLS_NONFATAL += "git-lfs"
> >
> > +# build-sysroot needs xmlcatalog in order for eSDK installation
> > +HOSTTOOLS_NONFATAL += "xmlcatalog"
>
> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
> grow to contain far too much random stuff which could impact
> reproduciblity.
>
> Is there some other way we could fix this? It still feels like a
> dependency problem which we chould fix by adding a missing dependency
> although I appreciate its far from being that simple.
>
> I wonder if there is a way we could teach build-sysroots to be cleverer
> about dependencies?
>
> Cheers,
>
> Richard
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[ChenQi]

On 07/09/2019 12:47 AM, Richard Purdie wrote:
> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>> For build-sysroots.bb, the xmlcatalog would not be in its
>> staging directory. Causing the following error for eSDK.
>>
>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>> 127.
>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>> failed: do_build_native_sysroot
>>
>> The problem could be reproduced by the following steps.
>> 1. Add in local.conf:
>>     IMAGE_INSTALL_append = " btrfs-tools"
>>     DISTRO_FEATURES_append = " api-documentation"
>>     INHERIT += "testsdk"
>> 2. bitbake core-image-minimal -c populate_sdk_ext
>> 3. bitbake core-image-minimal -c testsdkext
>>
>> So we add nativesdk-libxml2-utils to buildtools-tarball
>> to ensure the existence of xmlcatalog. Also add it
>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   meta/conf/bitbake.conf                       | 3 +++
>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>   2 files changed, 4 insertions(+)
>>
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 5e93f5c..2f64eae 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>   # Link to git-lfs if present
>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>   
>> +# build-sysroot needs xmlcatalog in order for eSDK installation
>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
> grow to contain far too much random stuff which could impact
> reproduciblity.
>
> Is there some other way we could fix this? It still feels like a
> dependency problem which we chould fix by adding a missing dependency
> although I appreciate its far from being that simple.
>
> I wonder if there is a way we could teach build-sysroots to be cleverer
> about dependencies?

I'll try to implement it and send out a new patch.

Regards,
Chen Qi

> Cheers,
>
> Richard
>
>

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[ChenQi]

On 07/09/2019 02:45 AM, Burton, Ross wrote:
> Agreed, making it a non-fatal dependency just means that some eSDKs
> break, not all.  There needs to be a better fix.

But it's already in buildtools, and eSDK installation process involves 
extracting buildtools and sourcing its script.
Under which circumstances will we break some eSDKs?
Anyway, I'm trying to do something about build-sysroots recipe and will 
send out a new patch.

Regards,
Chen Qi

> Ross
>
> On Mon, 8 Jul 2019 at 17:53, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>> staging directory. Causing the following error for eSDK.
>>>
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>>> 127.
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>> failed: do_build_native_sysroot
>>>
>>> The problem could be reproduced by the following steps.
>>> 1. Add in local.conf:
>>>     IMAGE_INSTALL_append = " btrfs-tools"
>>>     DISTRO_FEATURES_append = " api-documentation"
>>>     INHERIT += "testsdk"
>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>> 3. bitbake core-image-minimal -c testsdkext
>>>
>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>> to ensure the existence of xmlcatalog. Also add it
>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>
>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>> ---
>>>   meta/conf/bitbake.conf                       | 3 +++
>>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>   2 files changed, 4 insertions(+)
>>>
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index 5e93f5c..2f64eae 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>   # Link to git-lfs if present
>>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>>
>>> +# build-sysroot needs xmlcatalog in order for eSDK installation
>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
>> grow to contain far too much random stuff which could impact
>> reproduciblity.
>>
>> Is there some other way we could fix this? It still feels like a
>> dependency problem which we chould fix by adding a missing dependency
>> although I appreciate its far from being that simple.
>>
>> I wonder if there is a way we could teach build-sysroots to be cleverer
>> about dependencies?
>>
>> Cheers,
>>
>> Richard
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[ChenQi]

On 07/09/2019 09:38 AM, ChenQi wrote:
> On 07/09/2019 12:47 AM, Richard Purdie wrote:
>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>> staging directory. Causing the following error for eSDK.
>>>
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>>> 127.
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>> failed: do_build_native_sysroot
>>>
>>> The problem could be reproduced by the following steps.
>>> 1. Add in local.conf:
>>>     IMAGE_INSTALL_append = " btrfs-tools"
>>>     DISTRO_FEATURES_append = " api-documentation"
>>>     INHERIT += "testsdk"
>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>> 3. bitbake core-image-minimal -c testsdkext
>>>
>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>> to ensure the existence of xmlcatalog. Also add it
>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>
>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>> ---
>>>   meta/conf/bitbake.conf                       | 3 +++
>>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>   2 files changed, 4 insertions(+)
>>>
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index 5e93f5c..2f64eae 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>   # Link to git-lfs if present
>>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>>   +# build-sysroot needs xmlcatalog in order for eSDK installation
>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
>> grow to contain far too much random stuff which could impact
>> reproduciblity.
>>
>> Is there some other way we could fix this? It still feels like a
>> dependency problem which we chould fix by adding a missing dependency
>> although I appreciate its far from being that simple.
>>
>> I wonder if there is a way we could teach build-sysroots to be cleverer
>> about dependencies?
>
> I'll try to implement it and send out a new patch.
>

Hi Richard and Ross,

Unfortunately I cannot figure out a good way to implement this.
Could you please help giving me some hints?

Best Regards,
Chen Qi

> Regards,
> Chen Qi
>
>> Cheers,
>>
>> Richard
>>
>>
>

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[Richard Purdie]

On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
> On 07/09/2019 09:38 AM, ChenQi wrote:
> > On 07/09/2019 12:47 AM, Richard Purdie wrote:
> > > On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> > > > For build-sysroots.bb, the xmlcatalog would not be in its
> > > > staging directory. Causing the following error for eSDK.
> > > > 
> > > > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > > > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst
> > > > -
> > > > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit
> > > > status
> > > > 127.
> > > > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > > > failed: do_build_native_sysroot
> > > > 
> > > > The problem could be reproduced by the following steps.
> > > > 1. Add in local.conf:
> > > >     IMAGE_INSTALL_append = " btrfs-tools"
> > > >     DISTRO_FEATURES_append = " api-documentation"
> > > >     INHERIT += "testsdk"
> > > > 2. bitbake core-image-minimal -c populate_sdk_ext
> > > > 3. bitbake core-image-minimal -c testsdkext
> > > > 
> > > > So we add nativesdk-libxml2-utils to buildtools-tarball
> > > > to ensure the existence of xmlcatalog. Also add it
> > > > to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> > > > 
> > > > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > > > ---
> > > >   meta/conf/bitbake.conf                       | 3 +++
> > > >   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
> > > >   2 files changed, 4 insertions(+)
> > > > 
> > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > > > index 5e93f5c..2f64eae 100644
> > > > --- a/meta/conf/bitbake.conf
> > > > +++ b/meta/conf/bitbake.conf
> > > > @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
> > > >   # Link to git-lfs if present
> > > >   HOSTTOOLS_NONFATAL += "git-lfs"
> > > >   +# build-sysroot needs xmlcatalog in order for eSDK
> > > > installation
> > > > +HOSTTOOLS_NONFATAL += "xmlcatalog"
> > > I don't mind the buildtools-tarball change but HOSTTOOLS is
> > > starting to
> > > grow to contain far too much random stuff which could impact
> > > reproduciblity.
> > > 
> > > Is there some other way we could fix this? It still feels like a
> > > dependency problem which we chould fix by adding a missing
> > > dependency
> > > although I appreciate its far from being that simple.
> > > 
> > > I wonder if there is a way we could teach build-sysroots to be
> > > cleverer
> > > about dependencies?
> > 
> > I'll try to implement it and send out a new patch.
> 
> Unfortunately I cannot figure out a good way to implement this.
> Could you please help giving me some hints?

What happens if we add:

${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \

to
`
meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb 

?

Cheers,

Richard

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[ChenQi]

On 07/09/2019 06:18 PM, Richard Purdie wrote:
> On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
>> On 07/09/2019 09:38 AM, ChenQi wrote:
>>> On 07/09/2019 12:47 AM, Richard Purdie wrote:
>>>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>>>> staging directory. Causing the following error for eSDK.
>>>>>
>>>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst
>>>>> -
>>>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit
>>>>> status
>>>>> 127.
>>>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>>>> failed: do_build_native_sysroot
>>>>>
>>>>> The problem could be reproduced by the following steps.
>>>>> 1. Add in local.conf:
>>>>>      IMAGE_INSTALL_append = " btrfs-tools"
>>>>>      DISTRO_FEATURES_append = " api-documentation"
>>>>>      INHERIT += "testsdk"
>>>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>>>> 3. bitbake core-image-minimal -c testsdkext
>>>>>
>>>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>>>> to ensure the existence of xmlcatalog. Also add it
>>>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>>>
>>>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>>>> ---
>>>>>    meta/conf/bitbake.conf                       | 3 +++
>>>>>    meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>>>    2 files changed, 4 insertions(+)
>>>>>
>>>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>>>> index 5e93f5c..2f64eae 100644
>>>>> --- a/meta/conf/bitbake.conf
>>>>> +++ b/meta/conf/bitbake.conf
>>>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>>>    # Link to git-lfs if present
>>>>>    HOSTTOOLS_NONFATAL += "git-lfs"
>>>>>    +# build-sysroot needs xmlcatalog in order for eSDK
>>>>> installation
>>>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>>>> I don't mind the buildtools-tarball change but HOSTTOOLS is
>>>> starting to
>>>> grow to contain far too much random stuff which could impact
>>>> reproduciblity.
>>>>
>>>> Is there some other way we could fix this? It still feels like a
>>>> dependency problem which we chould fix by adding a missing
>>>> dependency
>>>> although I appreciate its far from being that simple.
>>>>
>>>> I wonder if there is a way we could teach build-sysroots to be
>>>> cleverer
>>>> about dependencies?
>>> I'll try to implement it and send out a new patch.
>> Unfortunately I cannot figure out a good way to implement this.
>> Could you please help giving me some hints?
> What happens if we add:
>
> ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \
>
> to
> `
> meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
>
> ?

This does not work. Using nativesdk-libxml2-utils does not work too.
Both buildtools and eSDK do not use this packagegroup.

I've sent out a V2 of this patch, with change to put the 
'libxml2-native' dependency in xmlcatalog.bbclass.
In this way, we can ensure that the xmlcatalog is from our recipe. In 
normal build, it's from libxml2-native; in build-sysroots in eSDK, it's 
from nativesdk-libxml2-utils. Do you think this is acceptable?

Best Regards,
Chen Qi

> Cheers,
>
> Richard
>
>

Re: [PATCH 1/1] buildtools-tarball: add nativesdk-libxml2-utils

[richard.purdie]

On Wed, 2019-07-10 at 11:06 +0800, ChenQi wrote:
> On 07/09/2019 06:18 PM, Richard Purdie wrote:
> > On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
> > What happens if we add:
> > 
> > ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation',
> > 'nativesdk-libxml2', '', d)} \
> > 
> > to
> > `
> > meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
> > 
> > ?
> 
> This does not work. Using nativesdk-libxml2-utils does not work too.
> Both buildtools and eSDK do not use this packagegroup.

Ok, I'll need to spend more time looking at the problem then :(.

> I've sent out a V2 of this patch, with change to put the 
> 'libxml2-native' dependency in xmlcatalog.bbclass.
> In this way, we can ensure that the xmlcatalog is from our recipe.
> In normal build, it's from libxml2-native; in build-sysroots in eSDK,
> it's from nativesdk-libxml2-utils. Do you think this is acceptable?

The class change is fine and I'll queue that.

We are not adding xmlcatalog to HOSTTOOLS_NONFATAL however, that is
simply not an option or the right way to fix this.

Cheers,

Richard