[PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Ioan-Adrian Ratiu]

do_kernel_configme calls merge_config.sh (installed in the sysroot by
the kern-tools-native recipe) which calls make to fill in any missing
symbols from the resulting merged config.

This errors out on my system because of sysroot poisoning [1]. Here is
a partial output from my .kernel-meta/cfg/merge_config_build.log (this
file is created in do_kernel_configme() while callig merge_config.sh):

make[1]: Entering directory '/media/adi/ssd/nilrt-master/build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
  HOSTCC  scripts/basic/fixdep
/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error: sys/types.h: No such file or directory
compilation terminated.
make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/x64/kernel-source/scripts/basic/Makefile:22: scripts/basic/x86_64-nilrt-linux-fixdep] Error 1

This issue is hard to debug because merge_config.sh does NOT check the
error output of its make call (this is added in the second patch) and
even though make errors out, the build continues as if nothing happened
and compiles a kernel with garbage configs (the .config generated by
do_kernel_configme is empty and gets filled later) which doesn't boot.

Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
uses it to correctly compile & fill all missing kernel config options.

[1] http://lists.openembedded.org/pipermail/openembedded-core/2014-October/098253.html

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
---
 meta/classes/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 8650e55..4397a9d 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -249,7 +249,7 @@ do_kernel_configme() {
 		bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
 	fi
 
-	ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
+	CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}"	ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
 	if [ $? -ne 0 ]; then
 		bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
 	fi
-- 
2.9.3

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 3263 bytes --]

On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
wrote:

> do_kernel_configme calls merge_config.sh (installed in the sysroot by
> the kern-tools-native recipe) which calls make to fill in any missing
> symbols from the resulting merged config.


That's not what it does ... but that isn't important.


>
> This errors out on my system because of sysroot poisoning [1]. Here is
> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
> file is created in do_kernel_configme() while callig merge_config.sh):
>
> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>   HOSTCC  scripts/basic/fixdep
> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
> sys/types.h: No such file or directory
> compilation terminated.
> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> x64/kernel-source/scripts/basic/Makefile:22: scripts/basic/x86_64-nilrt-linux-fixdep]
> Error 1
>

This just means that we are missing a dependency. Everything that
merge_config
needs should be already in place before it runs, i.e. do_kernel_metadata
should
have already run and built any host tools that it needs.

How are you managing to trigger this error ? I've done plenty of builds,
and haven't
seen this one before.

Bruce


>
> This issue is hard to debug because merge_config.sh does NOT check the
> error output of its make call (this is added in the second patch) and
> even though make errors out, the build continues as if nothing happened
> and compiles a kernel with garbage configs (the .config generated by
> do_kernel_configme is empty and gets filled later) which doesn't boot.
>
> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
> uses it to correctly compile & fill all missing kernel config options.
>
> [1] http://lists.openembedded.org/pipermail/openembedded-core/
> 2014-October/098253.html
>
> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> ---
>  meta/classes/kernel-yocto.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel-yocto.bbclass
> b/meta/classes/kernel-yocto.bbclass
> index 8650e55..4397a9d 100644
> --- a/meta/classes/kernel-yocto.bbclass
> +++ b/meta/classes/kernel-yocto.bbclass
> @@ -249,7 +249,7 @@ do_kernel_configme() {
>                 bbfatal_log "Could not find configuration queue
> (${meta_dir}/config.queue)"
>         fi
>
> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} >
> ${meta_dir}/cfg/merge_config_build.log 2>&1
> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
> merge_config.sh -O ${B} ${config_flags} ${configs} >
> ${meta_dir}/cfg/merge_config_build.log 2>&1
>         if [ $? -ne 0 ]; then
>                 bbfatal_log "Could not configure
> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>         fi
> --
> 2.9.3
>
>


-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 4545 bytes --]

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Ioan-Adrian Ratiu]

On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> wrote:
>
>> do_kernel_configme calls merge_config.sh (installed in the sysroot by
>> the kern-tools-native recipe) which calls make to fill in any missing
>> symbols from the resulting merged config.
>
>
> That's not what it does ... but that isn't important.

Can you please explain in a simple sentence what it does?

>
>
>>
>> This errors out on my system because of sysroot poisoning [1]. Here is
>> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
>> file is created in do_kernel_configme() while callig merge_config.sh):
>>
>> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
>> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
>> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>>   HOSTCC  scripts/basic/fixdep
>> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
>> sys/types.h: No such file or directory
>> compilation terminated.
>> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> x64/kernel-source/scripts/basic/Makefile:22: scripts/basic/x86_64-nilrt-linux-fixdep]
>> Error 1
>>
>
> This just means that we are missing a dependency. Everything that
> merge_config
> needs should be already in place before it runs, i.e. do_kernel_metadata
> should
> have already run and built any host tools that it needs.

This is not a missing dependency. The problem is the make command called
in merge_configs fails silently because of a misconfigured sysroot. The
correct sysroot is in the TOOLCHAIN_OPTIONS variable and it does not get
passed to the make call in merge_configs.

>
> How are you managing to trigger this error ? I've done plenty of builds,
> and haven't
> seen this one before.

Every one of my builds error out in that make call from merge_config
without this change because make does not search in the correct sysroot.

>
> Bruce
>
>
>>
>> This issue is hard to debug because merge_config.sh does NOT check the
>> error output of its make call (this is added in the second patch) and
>> even though make errors out, the build continues as if nothing happened
>> and compiles a kernel with garbage configs (the .config generated by
>> do_kernel_configme is empty and gets filled later) which doesn't boot.
>>
>> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
>> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
>> uses it to correctly compile & fill all missing kernel config options.
>>
>> [1] http://lists.openembedded.org/pipermail/openembedded-core/
>> 2014-October/098253.html
>>
>> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> ---
>>  meta/classes/kernel-yocto.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/kernel-yocto.bbclass
>> b/meta/classes/kernel-yocto.bbclass
>> index 8650e55..4397a9d 100644
>> --- a/meta/classes/kernel-yocto.bbclass
>> +++ b/meta/classes/kernel-yocto.bbclass
>> @@ -249,7 +249,7 @@ do_kernel_configme() {
>>                 bbfatal_log "Could not find configuration queue
>> (${meta_dir}/config.queue)"
>>         fi
>>
>> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} >
>> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
>> merge_config.sh -O ${B} ${config_flags} ${configs} >
>> ${meta_dir}/cfg/merge_config_build.log 2>&1
>>         if [ $? -ne 0 ]; then
>>                 bbfatal_log "Could not configure
>> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>>         fi
>> --
>> 2.9.3
>>
>>
>
>
> -- 
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 5084 bytes --]

On Mon, Aug 29, 2016 at 9:14 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
wrote:

> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > wrote:
> >
> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
> >> the kern-tools-native recipe) which calls make to fill in any missing
> >> symbols from the resulting merged config.
> >
> >
> > That's not what it does ... but that isn't important.
>
> Can you please explain in a simple sentence what it does?
>

It concatenates/merges configuration fragments.

The need to run make, or fill in any additional symbols is the optional
behaviour
of the script and isn't the essential service of what it provides. Since
for the
most part, it does very little if well specified configs and defconfigs are
passed.


>
> >
> >
> >>
> >> This errors out on my system because of sysroot poisoning [1]. Here is
> >> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
> >> file is created in do_kernel_configme() while callig merge_config.sh):
> >>
> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
> >>   HOSTCC  scripts/basic/fixdep
> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
> >> sys/types.h: No such file or directory
> >> compilation terminated.
> >> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> x64/kernel-source/scripts/basic/Makefile:22:
> scripts/basic/x86_64-nilrt-linux-fixdep]
> >> Error 1
> >>
> >
> > This just means that we are missing a dependency. Everything that
> > merge_config
> > needs should be already in place before it runs, i.e. do_kernel_metadata
> > should
> > have already run and built any host tools that it needs.
>
> This is not a missing dependency. The problem is the make command called
> in merge_configs fails silently because of a misconfigured sysroot. The
> correct sysroot is in the TOOLCHAIN_OPTIONS variable and it does not get
> passed to the make call in merge_configs.
>

That's my definition of a missing dependency. The host tools should already
exist before
merge_config ever runs. It shouldn't need to build anything.


>
> >
> > How are you managing to trigger this error ? I've done plenty of builds,
> > and haven't
> > seen this one before.
>
> Every one of my builds error out in that make call from merge_config
> without this change because make does not search in the correct sysroot.
>
>
So there's something drastically different. I've done hundreds of builds
before
and after my changes to the kern tools.

I need to understand what is different.

What's your configuration ? Are you building linux-yocto, or something else
?
(so I can run the same build here.)

Bruce


> >
> > Bruce
> >
> >
> >>
> >> This issue is hard to debug because merge_config.sh does NOT check the
> >> error output of its make call (this is added in the second patch) and
> >> even though make errors out, the build continues as if nothing happened
> >> and compiles a kernel with garbage configs (the .config generated by
> >> do_kernel_configme is empty and gets filled later) which doesn't boot.
> >>
> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
> >> uses it to correctly compile & fill all missing kernel config options.
> >>
> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
> >> 2014-October/098253.html
> >>
> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> >> ---
> >>  meta/classes/kernel-yocto.bbclass | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/meta/classes/kernel-yocto.bbclass
> >> b/meta/classes/kernel-yocto.bbclass
> >> index 8650e55..4397a9d 100644
> >> --- a/meta/classes/kernel-yocto.bbclass
> >> +++ b/meta/classes/kernel-yocto.bbclass
> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
> >>                 bbfatal_log "Could not find configuration queue
> >> (${meta_dir}/config.queue)"
> >>         fi
> >>
> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
> >
> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >>         if [ $? -ne 0 ]; then
> >>                 bbfatal_log "Could not configure
> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
> >>         fi
> >> --
> >> 2.9.3
> >>
> >>
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> > at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 7344 bytes --]

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Ioan-Adrian Ratiu]

On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Mon, Aug 29, 2016 at 9:14 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> wrote:
>
>> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> > wrote:
>> >
>> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
>> >> the kern-tools-native recipe) which calls make to fill in any missing
>> >> symbols from the resulting merged config.
>> >
>> >
>> > That's not what it does ... but that isn't important.
>>
>> Can you please explain in a simple sentence what it does?
>>
>
> It concatenates/merges configuration fragments.
>
> The need to run make, or fill in any additional symbols is the optional
> behaviour
> of the script and isn't the essential service of what it provides. Since
> for the
> most part, it does very little if well specified configs and defconfigs are
> passed.

Even if it is an optional behaviour, this is exactly what make does, so
I don't know how I mistated what it does (it fills in missing symbols).
This is exactly what I said.

The important issue is that it can fail and if it can fail then we need
to check its return code. Isn't this true?



>
>
>>
>> >
>> >
>> >>
>> >> This errors out on my system because of sysroot poisoning [1]. Here is
>> >> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
>> >> file is created in do_kernel_configme() while callig merge_config.sh):
>> >>
>> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
>> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
>> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>> >>   HOSTCC  scripts/basic/fixdep
>> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
>> >> sys/types.h: No such file or directory
>> >> compilation terminated.
>> >> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/Makefile:22:
>> scripts/basic/x86_64-nilrt-linux-fixdep]
>> >> Error 1
>> >>
>> >
>> > This just means that we are missing a dependency. Everything that
>> > merge_config
>> > needs should be already in place before it runs, i.e. do_kernel_metadata
>> > should
>> > have already run and built any host tools that it needs.
>>
>> This is not a missing dependency. The problem is the make command called
>> in merge_configs fails silently because of a misconfigured sysroot. The
>> correct sysroot is in the TOOLCHAIN_OPTIONS variable and it does not get
>> passed to the make call in merge_configs.
>>
>
> That's my definition of a missing dependency. The host tools should already
> exist before
> merge_config ever runs. It shouldn't need to build anything.
>
>
>>
>> >
>> > How are you managing to trigger this error ? I've done plenty of builds,
>> > and haven't
>> > seen this one before.
>>
>> Every one of my builds error out in that make call from merge_config
>> without this change because make does not search in the correct sysroot.
>>
>>
> So there's something drastically different. I've done hundreds of builds
> before
> and after my changes to the kern tools.
>
> I need to understand what is different.
>
> What's your configuration ? Are you building linux-yocto, or something else
> ?
> (so I can run the same build here.)
>
> Bruce
>
>
>> >
>> > Bruce
>> >
>> >
>> >>
>> >> This issue is hard to debug because merge_config.sh does NOT check the
>> >> error output of its make call (this is added in the second patch) and
>> >> even though make errors out, the build continues as if nothing happened
>> >> and compiles a kernel with garbage configs (the .config generated by
>> >> do_kernel_configme is empty and gets filled later) which doesn't boot.
>> >>
>> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
>> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
>> >> uses it to correctly compile & fill all missing kernel config options.
>> >>
>> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
>> >> 2014-October/098253.html
>> >>
>> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> >> ---
>> >>  meta/classes/kernel-yocto.bbclass | 2 +-
>> >>  1 file changed, 1 insertion(+), 1 deletion(-)
>> >>
>> >> diff --git a/meta/classes/kernel-yocto.bbclass
>> >> b/meta/classes/kernel-yocto.bbclass
>> >> index 8650e55..4397a9d 100644
>> >> --- a/meta/classes/kernel-yocto.bbclass
>> >> +++ b/meta/classes/kernel-yocto.bbclass
>> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
>> >>                 bbfatal_log "Could not find configuration queue
>> >> (${meta_dir}/config.queue)"
>> >>         fi
>> >>
>> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
>> >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
>> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >>         if [ $? -ne 0 ]; then
>> >>                 bbfatal_log "Could not configure
>> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>> >>         fi
>> >> --
>> >> 2.9.3
>> >>
>> >>
>> >
>> >
>> > --
>> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
>> > at its end"
>>
>
>
>
> -- 
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 6693 bytes --]

On Mon, Aug 29, 2016 at 9:51 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
wrote:

> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > On Mon, Aug 29, 2016 at 9:14 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > wrote:
> >
> >> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <
> adrian.ratiu@ni.com>
> >> > wrote:
> >> >
> >> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
> >> >> the kern-tools-native recipe) which calls make to fill in any missing
> >> >> symbols from the resulting merged config.
> >> >
> >> >
> >> > That's not what it does ... but that isn't important.
> >>
> >> Can you please explain in a simple sentence what it does?
> >>
> >
> > It concatenates/merges configuration fragments.
> >
> > The need to run make, or fill in any additional symbols is the optional
> > behaviour
> > of the script and isn't the essential service of what it provides. Since
> > for the
> > most part, it does very little if well specified configs and defconfigs
> are
> > passed.
>
> Even if it is an optional behaviour, this is exactly what make does, so
> I don't know how I mistated what it does (it fills in missing symbols).
> This is exactly what I said.
>

I've been working with merge_config for a long time (it is based on parts
of scripts
that I wrote/developed), so lets just say that I have a more precise
definition of
what it does. The extra filling in of symbols is peripheral functionality.


> The important issue is that it can fail and if it can fail then we need
> to check its return code. Isn't this true?
>

I didn't say it wasn't, but I'm not in the habit of merging patches for
errors that
I can't reproduce when I run hundreds/thousands of builds with the tools and
have never seen it. I need to understand the root cause, and see it myself.

Bruce


>
>
>
> >
> >
> >>
> >> >
> >> >
> >> >>
> >> >> This errors out on my system because of sysroot poisoning [1]. Here
> is
> >> >> a partial output from my .kernel-meta/cfg/merge_config_build.log
> (this
> >> >> file is created in do_kernel_configme() while callig
> merge_config.sh):
> >> >>
> >> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
> >> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
> >> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
> >> >>   HOSTCC  scripts/basic/fixdep
> >> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
> >> >> sys/types.h: No such file or directory
> >> >> compilation terminated.
> >> >> make[2]: *** [/media/adi/ssd/nilrt-master/
> build/tmp-glibc/work-shared/
> >> >> x64/kernel-source/scripts/basic/Makefile:22:
> >> scripts/basic/x86_64-nilrt-linux-fixdep]
> >> >> Error 1
> >> >>
> >> >
> >> > This just means that we are missing a dependency. Everything that
> >> > merge_config
> >> > needs should be already in place before it runs, i.e.
> do_kernel_metadata
> >> > should
> >> > have already run and built any host tools that it needs.
> >>
> >> This is not a missing dependency. The problem is the make command called
> >> in merge_configs fails silently because of a misconfigured sysroot. The
> >> correct sysroot is in the TOOLCHAIN_OPTIONS variable and it does not get
> >> passed to the make call in merge_configs.
> >>
> >
> > That's my definition of a missing dependency. The host tools should
> already
> > exist before
> > merge_config ever runs. It shouldn't need to build anything.
> >
> >
> >>
> >> >
> >> > How are you managing to trigger this error ? I've done plenty of
> builds,
> >> > and haven't
> >> > seen this one before.
> >>
> >> Every one of my builds error out in that make call from merge_config
> >> without this change because make does not search in the correct sysroot.
> >>
> >>
> > So there's something drastically different. I've done hundreds of builds
> > before
> > and after my changes to the kern tools.
> >
> > I need to understand what is different.
> >
> > What's your configuration ? Are you building linux-yocto, or something
> else
> > ?
> > (so I can run the same build here.)
> >
> > Bruce
> >
> >
> >> >
> >> > Bruce
> >> >
> >> >
> >> >>
> >> >> This issue is hard to debug because merge_config.sh does NOT check
> the
> >> >> error output of its make call (this is added in the second patch) and
> >> >> even though make errors out, the build continues as if nothing
> happened
> >> >> and compiles a kernel with garbage configs (the .config generated by
> >> >> do_kernel_configme is empty and gets filled later) which doesn't
> boot.
> >> >>
> >> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
> >> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and
> make
> >> >> uses it to correctly compile & fill all missing kernel config
> options.
> >> >>
> >> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
> >> >> 2014-October/098253.html
> >> >>
> >> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> >> >> ---
> >> >>  meta/classes/kernel-yocto.bbclass | 2 +-
> >> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >> >>
> >> >> diff --git a/meta/classes/kernel-yocto.bbclass
> >> >> b/meta/classes/kernel-yocto.bbclass
> >> >> index 8650e55..4397a9d 100644
> >> >> --- a/meta/classes/kernel-yocto.bbclass
> >> >> +++ b/meta/classes/kernel-yocto.bbclass
> >> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
> >> >>                 bbfatal_log "Could not find configuration queue
> >> >> (${meta_dir}/config.queue)"
> >> >>         fi
> >> >>
> >> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags}
> ${configs}
> >> >
> >> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
> >> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
> >> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> >>         if [ $? -ne 0 ]; then
> >> >>                 bbfatal_log "Could not configure
> >> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
> >> >>         fi
> >> >> --
> >> >> 2.9.3
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee
> >> > at its end"
> >>
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> > at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 9635 bytes --]

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Ioan-Adrian Ratiu]

On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> wrote:
>
>> do_kernel_configme calls merge_config.sh (installed in the sysroot by
>> the kern-tools-native recipe) which calls make to fill in any missing
>> symbols from the resulting merged config.
>
>
> That's not what it does ... but that isn't important.
>
>
>>
>> This errors out on my system because of sysroot poisoning [1]. Here is
>> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
>> file is created in do_kernel_configme() while callig merge_config.sh):
>>
>> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
>> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
>> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>>   HOSTCC  scripts/basic/fixdep
>> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
>> sys/types.h: No such file or directory
>> compilation terminated.
>> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> x64/kernel-source/scripts/basic/Makefile:22: scripts/basic/x86_64-nilrt-linux-fixdep]
>> Error 1
>>
>
> This just means that we are missing a dependency. Everything that
> merge_config
> needs should be already in place before it runs, i.e. do_kernel_metadata
> should
> have already run and built any host tools that it needs.
>
> How are you managing to trigger this error ? I've done plenty of builds,
> and haven't
> seen this one before.

Maybe some context is in order for how I build the kernel. I'm using a
custom kernel recipe which inherits from linux-yocto.inc. My kernel
tree contains a defconfig which is passed to OE in my kernel recipe
using KBUILD_DEFCONFIG and KCONFIG_MODE="--alldefconfig".

I have no configuration fragments.

As expected I need the merge_configs script to automatically fill in any
missing CONFIG symbols before building the kernel and this is where it
fails because the sysroot is poisoned since 2014.

>
> Bruce
>
>
>>
>> This issue is hard to debug because merge_config.sh does NOT check the
>> error output of its make call (this is added in the second patch) and
>> even though make errors out, the build continues as if nothing happened
>> and compiles a kernel with garbage configs (the .config generated by
>> do_kernel_configme is empty and gets filled later) which doesn't boot.
>>
>> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
>> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
>> uses it to correctly compile & fill all missing kernel config options.
>>
>> [1] http://lists.openembedded.org/pipermail/openembedded-core/
>> 2014-October/098253.html
>>
>> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> ---
>>  meta/classes/kernel-yocto.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/kernel-yocto.bbclass
>> b/meta/classes/kernel-yocto.bbclass
>> index 8650e55..4397a9d 100644
>> --- a/meta/classes/kernel-yocto.bbclass
>> +++ b/meta/classes/kernel-yocto.bbclass
>> @@ -249,7 +249,7 @@ do_kernel_configme() {
>>                 bbfatal_log "Could not find configuration queue
>> (${meta_dir}/config.queue)"
>>         fi
>>
>> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} >
>> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
>> merge_config.sh -O ${B} ${config_flags} ${configs} >
>> ${meta_dir}/cfg/merge_config_build.log 2>&1
>>         if [ $? -ne 0 ]; then
>>                 bbfatal_log "Could not configure
>> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>>         fi
>> --
>> 2.9.3
>>
>>
>
>
> -- 
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 4670 bytes --]

On Mon, Aug 29, 2016 at 10:05 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
wrote:

> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> > wrote:
> >
> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
> >> the kern-tools-native recipe) which calls make to fill in any missing
> >> symbols from the resulting merged config.
> >
> >
> > That's not what it does ... but that isn't important.
> >
> >
> >>
> >> This errors out on my system because of sysroot poisoning [1]. Here is
> >> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
> >> file is created in do_kernel_configme() while callig merge_config.sh):
> >>
> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
> >>   HOSTCC  scripts/basic/fixdep
> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
> >> sys/types.h: No such file or directory
> >> compilation terminated.
> >> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> x64/kernel-source/scripts/basic/Makefile:22:
> scripts/basic/x86_64-nilrt-linux-fixdep]
> >> Error 1
> >>
> >
> > This just means that we are missing a dependency. Everything that
> > merge_config
> > needs should be already in place before it runs, i.e. do_kernel_metadata
> > should
> > have already run and built any host tools that it needs.
> >
> > How are you managing to trigger this error ? I've done plenty of builds,
> > and haven't
> > seen this one before.
>
> Maybe some context is in order for how I build the kernel. I'm using a
> custom kernel recipe which inherits from linux-yocto.inc. My kernel
> tree contains a defconfig which is passed to OE in my kernel recipe
> using KBUILD_DEFCONFIG and KCONFIG_MODE="--alldefconfig".
>

And that would be the difference, which is what I was looking for. The tools
are always built for the linux-yocto path, hence why I can never reproduce
the issue.

I can take the kernel-yocto patch and queue it, but I'm going to leave
merge_config
untouched, since that change can go via the kernel path.

Bruce


>
> I have no configuration fragments.
>
> As expected I need the merge_configs script to automatically fill in any
> missing CONFIG symbols before building the kernel and this is where it
> fails because the sysroot is poisoned since 2014.
>
> >
> > Bruce
> >
> >
> >>
> >> This issue is hard to debug because merge_config.sh does NOT check the
> >> error output of its make call (this is added in the second patch) and
> >> even though make errors out, the build continues as if nothing happened
> >> and compiles a kernel with garbage configs (the .config generated by
> >> do_kernel_configme is empty and gets filled later) which doesn't boot.
> >>
> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
> >> uses it to correctly compile & fill all missing kernel config options.
> >>
> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
> >> 2014-October/098253.html
> >>
> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> >> ---
> >>  meta/classes/kernel-yocto.bbclass | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/meta/classes/kernel-yocto.bbclass
> >> b/meta/classes/kernel-yocto.bbclass
> >> index 8650e55..4397a9d 100644
> >> --- a/meta/classes/kernel-yocto.bbclass
> >> +++ b/meta/classes/kernel-yocto.bbclass
> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
> >>                 bbfatal_log "Could not find configuration queue
> >> (${meta_dir}/config.queue)"
> >>         fi
> >>
> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
> >
> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >>         if [ $? -ne 0 ]; then
> >>                 bbfatal_log "Could not configure
> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
> >>         fi
> >> --
> >> 2.9.3
> >>
> >>
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> > at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 6556 bytes --]

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 5332 bytes --]

On Mon, Aug 29, 2016 at 10:46 AM, Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

>
>
> On Mon, Aug 29, 2016 at 10:05 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> wrote:
>
>> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com
>> >
>> > wrote:
>> >
>> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
>> >> the kern-tools-native recipe) which calls make to fill in any missing
>> >> symbols from the resulting merged config.
>> >
>> >
>> > That's not what it does ... but that isn't important.
>> >
>> >
>> >>
>> >> This errors out on my system because of sysroot poisoning [1]. Here is
>> >> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
>> >> file is created in do_kernel_configme() while callig merge_config.sh):
>> >>
>> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
>> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
>> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>> >>   HOSTCC  scripts/basic/fixdep
>> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
>> >> sys/types.h: No such file or directory
>> >> compilation terminated.
>> >> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/Makefile:22:
>> scripts/basic/x86_64-nilrt-linux-fixdep]
>> >> Error 1
>> >>
>> >
>> > This just means that we are missing a dependency. Everything that
>> > merge_config
>> > needs should be already in place before it runs, i.e. do_kernel_metadata
>> > should
>> > have already run and built any host tools that it needs.
>> >
>> > How are you managing to trigger this error ? I've done plenty of builds,
>> > and haven't
>> > seen this one before.
>>
>> Maybe some context is in order for how I build the kernel. I'm using a
>> custom kernel recipe which inherits from linux-yocto.inc. My kernel
>> tree contains a defconfig which is passed to OE in my kernel recipe
>> using KBUILD_DEFCONFIG and KCONFIG_MODE="--alldefconfig".
>>
>
> And that would be the difference, which is what I was looking for. The
> tools
> are always built for the linux-yocto path, hence why I can never reproduce
> the issue.
>
> I can take the kernel-yocto patch and queue it, but I'm going to leave
> merge_config
> untouched, since that change can go via the kernel path.
>

And you can see where I've queued this in my poky-contrib zedd/kernel
branch.
It will go out with my queue the introduced 4.8 as a new kernel. I tweaked
the
commit log a bit, but is is otherwise unchanged.

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Cheers,

Bruce


>
> Bruce
>
>
>>
>> I have no configuration fragments.
>>
>> As expected I need the merge_configs script to automatically fill in any
>> missing CONFIG symbols before building the kernel and this is where it
>> fails because the sysroot is poisoned since 2014.
>>
>> >
>> > Bruce
>> >
>> >
>> >>
>> >> This issue is hard to debug because merge_config.sh does NOT check the
>> >> error output of its make call (this is added in the second patch) and
>> >> even though make errors out, the build continues as if nothing happened
>> >> and compiles a kernel with garbage configs (the .config generated by
>> >> do_kernel_configme is empty and gets filled later) which doesn't boot.
>> >>
>> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
>> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
>> >> uses it to correctly compile & fill all missing kernel config options.
>> >>
>> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
>> >> 2014-October/098253.html
>> >>
>> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> >> ---
>> >>  meta/classes/kernel-yocto.bbclass | 2 +-
>> >>  1 file changed, 1 insertion(+), 1 deletion(-)
>> >>
>> >> diff --git a/meta/classes/kernel-yocto.bbclass
>> >> b/meta/classes/kernel-yocto.bbclass
>> >> index 8650e55..4397a9d 100644
>> >> --- a/meta/classes/kernel-yocto.bbclass
>> >> +++ b/meta/classes/kernel-yocto.bbclass
>> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
>> >>                 bbfatal_log "Could not find configuration queue
>> >> (${meta_dir}/config.queue)"
>> >>         fi
>> >>
>> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags}
>> ${configs} >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
>> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >>         if [ $? -ne 0 ]; then
>> >>                 bbfatal_log "Could not configure
>> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>> >>         fi
>> >> --
>> >> 2.9.3
>> >>
>> >>
>> >
>> >
>> > --
>> > "Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee
>> > at its end"
>>
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 8294 bytes --]

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Ioan-Adrian Ratiu]

On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Mon, Aug 29, 2016 at 10:05 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> wrote:
>
>> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> > wrote:
>> >
>> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
>> >> the kern-tools-native recipe) which calls make to fill in any missing
>> >> symbols from the resulting merged config.
>> >
>> >
>> > That's not what it does ... but that isn't important.
>> >
>> >
>> >>
>> >> This errors out on my system because of sysroot poisoning [1]. Here is
>> >> a partial output from my .kernel-meta/cfg/merge_config_build.log (this
>> >> file is created in do_kernel_configme() while callig merge_config.sh):
>> >>
>> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
>> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
>> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
>> >>   HOSTCC  scripts/basic/fixdep
>> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
>> >> sys/types.h: No such file or directory
>> >> compilation terminated.
>> >> make[2]: *** [/media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
>> >> x64/kernel-source/scripts/basic/Makefile:22:
>> scripts/basic/x86_64-nilrt-linux-fixdep]
>> >> Error 1
>> >>
>> >
>> > This just means that we are missing a dependency. Everything that
>> > merge_config
>> > needs should be already in place before it runs, i.e. do_kernel_metadata
>> > should
>> > have already run and built any host tools that it needs.
>> >
>> > How are you managing to trigger this error ? I've done plenty of builds,
>> > and haven't
>> > seen this one before.
>>
>> Maybe some context is in order for how I build the kernel. I'm using a
>> custom kernel recipe which inherits from linux-yocto.inc. My kernel
>> tree contains a defconfig which is passed to OE in my kernel recipe
>> using KBUILD_DEFCONFIG and KCONFIG_MODE="--alldefconfig".
>>
>
> And that would be the difference, which is what I was looking for. The tools
> are always built for the linux-yocto path, hence why I can never reproduce
> the issue.
>
> I can take the kernel-yocto patch and queue it, but I'm going to leave
> merge_config
> untouched, since that change can go via the kernel path.

Thank you.

Can you please explain a little what you mean by "go via the kernel path"?

>
> Bruce
>
>
>>
>> I have no configuration fragments.
>>
>> As expected I need the merge_configs script to automatically fill in any
>> missing CONFIG symbols before building the kernel and this is where it
>> fails because the sysroot is poisoned since 2014.
>>
>> >
>> > Bruce
>> >
>> >
>> >>
>> >> This issue is hard to debug because merge_config.sh does NOT check the
>> >> error output of its make call (this is added in the second patch) and
>> >> even though make errors out, the build continues as if nothing happened
>> >> and compiles a kernel with garbage configs (the .config generated by
>> >> do_kernel_configme is empty and gets filled later) which doesn't boot.
>> >>
>> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
>> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and make
>> >> uses it to correctly compile & fill all missing kernel config options.
>> >>
>> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
>> >> 2014-October/098253.html
>> >>
>> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
>> >> ---
>> >>  meta/classes/kernel-yocto.bbclass | 2 +-
>> >>  1 file changed, 1 insertion(+), 1 deletion(-)
>> >>
>> >> diff --git a/meta/classes/kernel-yocto.bbclass
>> >> b/meta/classes/kernel-yocto.bbclass
>> >> index 8650e55..4397a9d 100644
>> >> --- a/meta/classes/kernel-yocto.bbclass
>> >> +++ b/meta/classes/kernel-yocto.bbclass
>> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
>> >>                 bbfatal_log "Could not find configuration queue
>> >> (${meta_dir}/config.queue)"
>> >>         fi
>> >>
>> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
>> >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
>> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
>> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
>> >>         if [ $? -ne 0 ]; then
>> >>                 bbfatal_log "Could not configure
>> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
>> >>         fi
>> >> --
>> >> 2.9.3
>> >>
>> >>
>> >
>> >
>> > --
>> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
>> > at its end"
>>
>
>
>
> -- 
> "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end"

Re: [PATCH] kernel-yocto: do_kernel_configme: Fix silent sysroot poisoning error

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 6075 bytes --]

On Mon, Aug 29, 2016 at 11:34 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
wrote:

> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > On Mon, Aug 29, 2016 at 10:05 AM, Ioan-Adrian Ratiu <adrian.ratiu@ni.com
> >
> > wrote:
> >
> >> On Mon, 29 Aug 2016, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >> > On Mon, Aug 29, 2016 at 5:19 AM, Ioan-Adrian Ratiu <
> adrian.ratiu@ni.com>
> >> > wrote:
> >> >
> >> >> do_kernel_configme calls merge_config.sh (installed in the sysroot by
> >> >> the kern-tools-native recipe) which calls make to fill in any missing
> >> >> symbols from the resulting merged config.
> >> >
> >> >
> >> > That's not what it does ... but that isn't important.
> >> >
> >> >
> >> >>
> >> >> This errors out on my system because of sysroot poisoning [1]. Here
> is
> >> >> a partial output from my .kernel-meta/cfg/merge_config_build.log
> (this
> >> >> file is created in do_kernel_configme() while callig
> merge_config.sh):
> >> >>
> >> >> make[1]: Entering directory '/media/adi/ssd/nilrt-master/
> >> >> build/tmp-glibc/work/x64-nilrt-linux/linux-nilrt/4.1+
> >> >> gitAUTOINC+a7e53ecc27-r0/linux-x64-standard-build'
> >> >>   HOSTCC  scripts/basic/fixdep
> >> >> /media/adi/ssd/nilrt-master/build/tmp-glibc/work-shared/
> >> >> x64/kernel-source/scripts/basic/fixdep.c:106:23: fatal error:
> >> >> sys/types.h: No such file or directory
> >> >> compilation terminated.
> >> >> make[2]: *** [/media/adi/ssd/nilrt-master/
> build/tmp-glibc/work-shared/
> >> >> x64/kernel-source/scripts/basic/Makefile:22:
> >> scripts/basic/x86_64-nilrt-linux-fixdep]
> >> >> Error 1
> >> >>
> >> >
> >> > This just means that we are missing a dependency. Everything that
> >> > merge_config
> >> > needs should be already in place before it runs, i.e.
> do_kernel_metadata
> >> > should
> >> > have already run and built any host tools that it needs.
> >> >
> >> > How are you managing to trigger this error ? I've done plenty of
> builds,
> >> > and haven't
> >> > seen this one before.
> >>
> >> Maybe some context is in order for how I build the kernel. I'm using a
> >> custom kernel recipe which inherits from linux-yocto.inc. My kernel
> >> tree contains a defconfig which is passed to OE in my kernel recipe
> >> using KBUILD_DEFCONFIG and KCONFIG_MODE="--alldefconfig".
> >>
> >
> > And that would be the difference, which is what I was looking for. The
> tools
> > are always built for the linux-yocto path, hence why I can never
> reproduce
> > the issue.
> >
> > I can take the kernel-yocto patch and queue it, but I'm going to leave
> > merge_config
> > untouched, since that change can go via the kernel path.
>
> Thank you.
>
> Can you please explain a little what you mean by "go via the kernel path"?
>


I keep a copy of merge_config.sh in the kern-tools repo, because I don't
want to
add a dependency on the full kernel source tree to get that single tool.

I also used to carry changes to the script in the kern-tools repo. But with
my recent
updates, I'm able to use it as-is.

So unless a bug is critical (and in this case, we can get by with me
carrying the
toolchain definition patch to kernel-yocto), the right place to submit
patches to
merge_config.sh is via the linux kernel mailing list. That way everyone
gets to
take advantage of the changes.

Cheers,

Bruce


>
> >
> > Bruce
> >
> >
> >>
> >> I have no configuration fragments.
> >>
> >> As expected I need the merge_configs script to automatically fill in any
> >> missing CONFIG symbols before building the kernel and this is where it
> >> fails because the sysroot is poisoned since 2014.
> >>
> >> >
> >> > Bruce
> >> >
> >> >
> >> >>
> >> >> This issue is hard to debug because merge_config.sh does NOT check
> the
> >> >> error output of its make call (this is added in the second patch) and
> >> >> even though make errors out, the build continues as if nothing
> happened
> >> >> and compiles a kernel with garbage configs (the .config generated by
> >> >> do_kernel_configme is empty and gets filled later) which doesn't
> boot.
> >> >>
> >> >> Adding $TOOLCHAIN_OPTIONS to $CFLAGS before calling merge_configs.sh
> >> >> fixes the error because $TOOLCHAIN_OPTIONS defines the sysroot and
> make
> >> >> uses it to correctly compile & fill all missing kernel config
> options.
> >> >>
> >> >> [1] http://lists.openembedded.org/pipermail/openembedded-core/
> >> >> 2014-October/098253.html
> >> >>
> >> >> Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
> >> >> ---
> >> >>  meta/classes/kernel-yocto.bbclass | 2 +-
> >> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >> >>
> >> >> diff --git a/meta/classes/kernel-yocto.bbclass
> >> >> b/meta/classes/kernel-yocto.bbclass
> >> >> index 8650e55..4397a9d 100644
> >> >> --- a/meta/classes/kernel-yocto.bbclass
> >> >> +++ b/meta/classes/kernel-yocto.bbclass
> >> >> @@ -249,7 +249,7 @@ do_kernel_configme() {
> >> >>                 bbfatal_log "Could not find configuration queue
> >> >> (${meta_dir}/config.queue)"
> >> >>         fi
> >> >>
> >> >> -       ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags}
> ${configs}
> >> >
> >> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> >> +       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" ARCH=${ARCH}
> >> >> merge_config.sh -O ${B} ${config_flags} ${configs} >
> >> >> ${meta_dir}/cfg/merge_config_build.log 2>&1
> >> >>         if [ $? -ne 0 ]; then
> >> >>                 bbfatal_log "Could not configure
> >> >> ${KMACHINE}-${LINUX_KERNEL_TYPE}"
> >> >>         fi
> >> >> --
> >> >> 2.9.3
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee
> >> > at its end"
> >>
> >
> >
> >
> > --
> > "Thou shalt not follow the NULL pointer, for chaos and madness await thee
> > at its end"
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"

[-- Attachment #2: Type: text/html, Size: 8815 bytes --]