Help diagnosing a build failure involving ncurses, gettext, and eglibc

Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Darren Hart]

I ran into various issues trying to add mtd-utils to an image today. As
I worked through the issues, I found after fixing a couple things, I was
more guessing and grasping at straws than anything else. I'll recount
what I did, and if anyone can offer a better approach to what I did that
would have resulted in a more deterministic result, I'd appreciate it
very much.

I copied meta-tiny to a new layer. I disabled uclibc (so I'm building
with eglibc). I added mtd-utils from OE and used the DEPENDS line using
util-linux (and not util-linux-ng which we don't have in yocto). This
resulted in ncurses failing the qa configure with host contamination
with the widec headers. I first attempted to enable widechar support in
my DISTRO_FEATURES_LIBC, and rebuild eglibc. This didn't solve the
problem. This turns out to be due to ncurses configuring in widechar
support regardless of whether or not you plan to use it (it does a widec
and a narrowc configure and only builds and installs widec if you set
ENABLE_WIDEC="true". I set that to false and hacked out the widec config
step which allowed ncurses to build. I'll send a proper fix for this
tomorrow.

The next failure was with gettext compilation. Several errors about
wchar related functions being redefined in incompatible ways. I thought
this might be related to DISTRO_FEATURES_LIBC changes I made, so I
commented them all out and rebuilt eglibc so it should be using the
default Poky distro configuration for eglibc (which I assumed gettext
was known to compile with). Gettext would still not compile, claiming
the redefined functions were first defined in /usr/include/wchar.h.
Confused, thinking I had just rebuilt eglibc, I did another cleanall of
eglibc and gettext and a rebuild. Same result.

Not trusting the sanity of my tmp tree at this point, I deleted tmp and
pseudodone and rebuilt the image. Everything succeeded and my final
image with mtd-utils included was built.

I tried to capture my complete log, but screen froze on me while trying
to paste the buffer into a log :( So besides writing a bitbake wrapper
to ensure I record ALL my bitbake sessions for reference, what would you
recommend I change in the above process? What could I have done to
either avoid deleting tmp or to have identified a possible bug that
required me to delete tmp?

Thanks,

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Darren Hart]

On 11/02/2011 05:53 PM, Darren Hart wrote:
...
> I tried to capture my complete log, but screen froze on me while trying
> to paste the buffer into a log :( So besides writing a bitbake wrapper
> to ensure I record ALL my bitbake sessions for reference,

I came up with the following to ensure I have a log of every bitbake
command I run along with some useful stats. Feel free to use it or flame it:


#!/bin/bash
TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
if [ -z "$LOG" ]; then
	echo "ERROR: failed to create log file"
	exit 1
fi

(
	echo "Start: $TIMESTAMP"
	echo "========================================"
	/usr/bin/time 2>&1 -v bitbake $@
	echo "========================================"
	echo "End: $(date -u '+%Y%m%d%H%M%S')"
) | tee $LOG

echo "Logfile: $LOG"


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[McClintock Matthew-B29882]

On Thu, Nov 3, 2011 at 12:12 AM, Darren Hart <dvhart@linux.intel.com> wrote:
> I came up with the following to ensure I have a log of every bitbake
> command I run along with some useful stats. Feel free to use it or flame it:
>
>
> #!/bin/bash
> TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
> LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
> if [ -z "$LOG" ]; then
>        echo "ERROR: failed to create log file"
>        exit 1
> fi
>
> (
>        echo "Start: $TIMESTAMP"
>        echo "========================================"
>        /usr/bin/time 2>&1 -v bitbake $@
>        echo "========================================"
>        echo "End: $(date -u '+%Y%m%d%H%M%S')"
> ) | tee $LOG
>
> echo "Logfile: $LOG"

It would be nice if we had this as a selectable option in the bitbake
wrapper somehow and it saved off logs to tmp/bitbake/logs/ or
something appropriate.

-M

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Darren Hart]

On 11/02/2011 10:16 PM, McClintock Matthew-B29882 wrote:
> On Thu, Nov 3, 2011 at 12:12 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>> I came up with the following to ensure I have a log of every bitbake
>> command I run along with some useful stats. Feel free to use it or flame it:
>>
>>
>> #!/bin/bash
>> TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
>> LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
>> if [ -z "$LOG" ]; then
>>        echo "ERROR: failed to create log file"
>>        exit 1
>> fi
>>
>> (
>>        echo "Start: $TIMESTAMP"
>>        echo "========================================"
>>        /usr/bin/time 2>&1 -v bitbake $@
>>        echo "========================================"
>>        echo "End: $(date -u '+%Y%m%d%H%M%S')"
>> ) | tee $LOG
>>
>> echo "Logfile: $LOG"
> 
> It would be nice if we had this as a selectable option in the bitbake
> wrapper somehow and it saved off logs to tmp/bitbake/logs/ or
> something appropriate.

I was having similar thoughts just after having pressed send :)

Given how annoying it is to not have the log when you forget to capture
it, I think this might actually make a reasonable default. The important
bits are of course already recorded in
tmp/blah/blah/blah/temp/log.blah.PID... but the above serves almost like
an index into the individual files. I'm all for logging it by default -
as well as collecting summary stats. I don't know if the above is the
best way to go about it - but a functional equivalent would be nice.

For all I know something like this already exists and I just haven't
stumbled upon it yet.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Jack Mitchell]

On 03/11/2011 05:21, Darren Hart wrote:
>
> On 11/02/2011 10:16 PM, McClintock Matthew-B29882 wrote:
>> On Thu, Nov 3, 2011 at 12:12 AM, Darren Hart<dvhart@linux.intel.com>  wrote:
>>> I came up with the following to ensure I have a log of every bitbake
>>> command I run along with some useful stats. Feel free to use it or flame it:
>>>
>>>
>>> #!/bin/bash
>>> TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
>>> LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
>>> if [ -z "$LOG" ]; then
>>>         echo "ERROR: failed to create log file"
>>>         exit 1
>>> fi
>>>
>>> (
>>>         echo "Start: $TIMESTAMP"
>>>         echo "========================================"
>>>         /usr/bin/time 2>&1 -v bitbake $@
>>>         echo "========================================"
>>>         echo "End: $(date -u '+%Y%m%d%H%M%S')"
>>> ) | tee $LOG
>>>
>>> echo "Logfile: $LOG"
>> It would be nice if we had this as a selectable option in the bitbake
>> wrapper somehow and it saved off logs to tmp/bitbake/logs/ or
>> something appropriate.
> I was having similar thoughts just after having pressed send :)
>
> Given how annoying it is to not have the log when you forget to capture
> it, I think this might actually make a reasonable default. The important
> bits are of course already recorded in
> tmp/blah/blah/blah/temp/log.blah.PID... but the above serves almost like
> an index into the individual files. I'm all for logging it by default -
> as well as collecting summary stats. I don't know if the above is the
> best way to go about it - but a functional equivalent would be nice.
>
> For all I know something like this already exists and I just haven't
> stumbled upon it yet.
>

I agree, something which actually logs what you are doing and how it was 
done rather than just the output/error logging we have at the moment 
would be a great addition. It would make it much easier to track errors 
and help pin down bugs - especially if you have taken a long winded 
route to get to a particular point.

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Joshua Lock]

On 03/11/11 02:36, Jack Mitchell wrote:
> On 03/11/2011 05:21, Darren Hart wrote:
>>
>> On 11/02/2011 10:16 PM, McClintock Matthew-B29882 wrote:
>>> On Thu, Nov 3, 2011 at 12:12 AM, Darren Hart<dvhart@linux.intel.com> 
>>> wrote:
>>>> I came up with the following to ensure I have a log of every bitbake
>>>> command I run along with some useful stats. Feel free to use it or
>>>> flame it:
>>>>
>>>>
>>>> #!/bin/bash
>>>> TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
>>>> LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
>>>> if [ -z "$LOG" ]; then
>>>>         echo "ERROR: failed to create log file"
>>>>         exit 1
>>>> fi
>>>>
>>>> (
>>>>         echo "Start: $TIMESTAMP"
>>>>         echo "========================================"
>>>>         /usr/bin/time 2>&1 -v bitbake $@
>>>>         echo "========================================"
>>>>         echo "End: $(date -u '+%Y%m%d%H%M%S')"
>>>> ) | tee $LOG
>>>>
>>>> echo "Logfile: $LOG"
>>> It would be nice if we had this as a selectable option in the bitbake
>>> wrapper somehow and it saved off logs to tmp/bitbake/logs/ or
>>> something appropriate.
>> I was having similar thoughts just after having pressed send :)
>>
>> Given how annoying it is to not have the log when you forget to capture
>> it, I think this might actually make a reasonable default. The important
>> bits are of course already recorded in
>> tmp/blah/blah/blah/temp/log.blah.PID... but the above serves almost like
>> an index into the individual files. I'm all for logging it by default -
>> as well as collecting summary stats. I don't know if the above is the
>> best way to go about it - but a functional equivalent would be nice.
>>
>> For all I know something like this already exists and I just haven't
>> stumbled upon it yet.
>>
> 
> I agree, something which actually logs what you are doing and how it was
> done rather than just the output/error logging we have at the moment
> would be a great addition. It would make it much easier to track errors
> and help pin down bugs - especially if you have taken a long winded
> route to get to a particular point.

Sounds like people would find this feature useful, can someone please
file a bug report?

Cheers,
Joshua
-- 
Joshua Lock
        Yocto Project "Johannes factotum"
        Intel Open Source Technology Centre

Re: Help diagnosing a build failure involving ncurses, gettext, and eglibc

[Darren Hart]

On 11/03/2011 11:47 AM, Joshua Lock wrote:
> On 03/11/11 02:36, Jack Mitchell wrote:
>> On 03/11/2011 05:21, Darren Hart wrote:
>>>
>>> On 11/02/2011 10:16 PM, McClintock Matthew-B29882 wrote:
>>>> On Thu, Nov 3, 2011 at 12:12 AM, Darren Hart<dvhart@linux.intel.com> 
>>>> wrote:
>>>>> I came up with the following to ensure I have a log of every bitbake
>>>>> command I run along with some useful stats. Feel free to use it or
>>>>> flame it:
>>>>>
>>>>>
>>>>> #!/bin/bash
>>>>> TIMESTAMP=$(date -u "+%Y%m%d%H%M%S")
>>>>> LOG=$(mktemp --suffix=".log" bb-$TIMESTAMP-XXX)
>>>>> if [ -z "$LOG" ]; then
>>>>>         echo "ERROR: failed to create log file"
>>>>>         exit 1
>>>>> fi
>>>>>
>>>>> (
>>>>>         echo "Start: $TIMESTAMP"
>>>>>         echo "========================================"
>>>>>         /usr/bin/time 2>&1 -v bitbake $@
>>>>>         echo "========================================"
>>>>>         echo "End: $(date -u '+%Y%m%d%H%M%S')"
>>>>> ) | tee $LOG
>>>>>
>>>>> echo "Logfile: $LOG"
>>>> It would be nice if we had this as a selectable option in the bitbake
>>>> wrapper somehow and it saved off logs to tmp/bitbake/logs/ or
>>>> something appropriate.
>>> I was having similar thoughts just after having pressed send :)
>>>
>>> Given how annoying it is to not have the log when you forget to capture
>>> it, I think this might actually make a reasonable default. The important
>>> bits are of course already recorded in
>>> tmp/blah/blah/blah/temp/log.blah.PID... but the above serves almost like
>>> an index into the individual files. I'm all for logging it by default -
>>> as well as collecting summary stats. I don't know if the above is the
>>> best way to go about it - but a functional equivalent would be nice.
>>>
>>> For all I know something like this already exists and I just haven't
>>> stumbled upon it yet.
>>>
>>
>> I agree, something which actually logs what you are doing and how it was
>> done rather than just the output/error logging we have at the moment
>> would be a great addition. It would make it much easier to track errors
>> and help pin down bugs - especially if you have taken a long winded
>> route to get to a particular point.
> 
> Sounds like people would find this feature useful, can someone please
> file a bug report?
> 

http://bugzilla.yoctoproject.org/show_bug.cgi?id=1771

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel