Recent build issues.

Recent build issues.

[Flanagan, Elizabeth]

Recent build issues:

- There seem to be a few issues going on with sanity testing:

First, I've made a change to how sanity tests are run on the autobuilder that should speed them up some. bitbake @ -c qemuimagetest creates a rootfs. I've switched this to bitbake @ -c qemuimagetest_standalone, so this shouldn't create a rootfs and should use the one already existing. We should see some performance improvement with this. I thought this had been included in my pull with BB 0.8.3 but I must have not included it.

Next,  the following bugs have been making it difficult to track down build failures:

Sanity Test Boot test returns invalid results.
http://bugzilla.pokylinux.org/show_bug.cgi?id=611

Sanity tests do not always kill QEMU once timeout is reached
http://bugzilla.pokylinux.org/show_bug.cgi?id=595

There are also sanity test failures in a few images:

qemuarm poky-image-sato
qemuarm poky-image-minimal
qemumips poky-image-sato

mips seems to have run poky-image-minimal ok, but with Bug #611, I can't be sure that is the case and I doubt it's actually the case. My inclination is that there is an issue with arm and mips builds that should be looked at.

- PPC is broken.

PPC is having serious issues that seem to revolve around linux-yocto-2.6.37. Bruce I believe is working on this. Can anyone give an update around where this stands?

I'm running nightly-incremental and nightly now to pull in some changes made this morning and to prepare for weekend delivery to QA. I'll send out those build results on Sunday (or whenever the build finishes.)
-b

Re: Recent build issues.

[Richard Purdie]

On Thu, 2010-12-23 at 14:08 -0800, Flanagan, Elizabeth wrote:
> Next,  the following bugs have been making it difficult to track down build failures:
> 
> Sanity Test Boot test returns invalid results.
> http://bugzilla.pokylinux.org/show_bug.cgi?id=611

Fixes for #611, #612 and some other issues I found are merged into
master. Sanity tests are now working locally for me at least.

> Sanity tests do not always kill QEMU once timeout is reached
> http://bugzilla.pokylinux.org/show_bug.cgi?id=595

This is not fixed yet and we need to sort that as its hurting the
autobuilder.

> There are also sanity test failures in a few images:
> 
> qemuarm poky-image-sato
> qemuarm poky-image-minimal
> qemumips poky-image-sato
> 
> mips seems to have run poky-image-minimal ok, but with Bug #611, I
> can't be sure that is the case and I doubt it's actually the case. My
> inclination is that there is an issue with arm and mips builds that
> should be looked at.

Also, qemux86 was broken until a fix I pushed a few minutes ago (kernel
was missing IDE support and wouldn't boot in qemu).

It looks like the new version of opkg we've upgraded to fixing certain
meta-toolchain corruption issues has some other new characteristics such
as refusing to install overlapping files. This is good but its exposed
bugs (and why isn't rpm showing this?).

There is currently an issue with that when building meta-toolchain and
poky-image-lsb that I know of. Patches to fix these bugs welcome (and
bugs need filing).

> - PPC is broken.
> 
> PPC is having serious issues that seem to revolve around
> linux-yocto-2.6.37. Bruce I believe is working on this. Can anyone
> give an update around where this stands?

Bruce pushed a patch to disable -Os usage. If this isn't showing up in
the next round of autobuilds, we'll have to figure out why it isn't
getting applied.

Cheers,

Richard

Re: Recent build issues.

[Mark Hatle]

On 12/23/10 6:22 PM, Richard Purdie wrote:
> On Thu, 2010-12-23 at 14:08 -0800, Flanagan, Elizabeth wrote:
...

>> mips seems to have run poky-image-minimal ok, but with Bug #611, I
>> can't be sure that is the case and I doubt it's actually the case. My
>> inclination is that there is an issue with arm and mips builds that
>> should be looked at.
> 
> Also, qemux86 was broken until a fix I pushed a few minutes ago (kernel
> was missing IDE support and wouldn't boot in qemu).
> 
> It looks like the new version of opkg we've upgraded to fixing certain
> meta-toolchain corruption issues has some other new characteristics such
> as refusing to install overlapping files. This is good but its exposed
> bugs (and why isn't rpm showing this?).

There are two cases in which RPM allows "overlapping" files.

The first is that the files are identical.  I.e. same md5sum/same sha1.  There
are reasons for overlapping files in most distros, so they are handled
gracefully.  (To me not supporting identical files in multiple packages is a
defect.. because of the second reason...)

The second is two files of different ELF types.  This can occur when you are on
a multilib system.  You have duplicate packages (one 32-bit, one 64-bit) to be
installed at the same time.  They may contain duplicate files, as well as endian
specific files (libraries and binaries).  In the case of overlapping binaries
RPM will check and use a configured policy to determine which one is installed
and which is ignored.  (On IA32 systems, generally 64-bit is installed, and
32-bit is ignored.)  Goal is to allow for both 64-bit and 32-bit libraries, all
common data files and the "preferred" binary format to be installed onto a system.

> There is currently an issue with that when building meta-toolchain and
> poky-image-lsb that I know of. Patches to fix these bugs welcome (and
> bugs need filing).
> 

...

Re: Recent build issues.

[Xu, Jiajun]

> On Thu, 2010-12-23 at 14:08 -0800, Flanagan, Elizabeth wrote:
>> Next,  the following bugs have been making it difficult to track down
>> build failures:
>> 
>> Sanity Test Boot test returns invalid results.
>> http://bugzilla.pokylinux.org/show_bug.cgi?id=611
> 
> Fixes for #611, #612 and some other issues I found are merged into master.
> Sanity tests are now working locally for me at least.
> 
>> Sanity tests do not always kill QEMU once timeout is reached
>> http://bugzilla.pokylinux.org/show_bug.cgi?id=595
> 

I tried to reproduce the bug before, but never seen such issue on my local machine. Meanwhile, when I was checking the log on autobuilder, I found QEMU was actually killed. I will add some check code and report error when qemu is not killed.

> This is not fixed yet and we need to sort that as its hurting the autobuilder.
> 
>> There are also sanity test failures in a few images:
>> 
>> qemuarm poky-image-sato
>> qemuarm poky-image-minimal
>> qemumips poky-image-sato
>> 
>> mips seems to have run poky-image-minimal ok, but with Bug #611, I
>> can't be sure that is the case and I doubt it's actually the case.
>> My inclination is that there is an issue with arm and mips builds
>> that should be looked at.
> 
> Also, qemux86 was broken until a fix I pushed a few minutes ago
> (kernel was missing IDE support and wouldn't boot in qemu).
> 
> It looks like the new version of opkg we've upgraded to fixing certain
> meta-toolchain corruption issues has some other new characteristics
> such as refusing to install overlapping files. This is good but its
> exposed bugs (and why isn't rpm showing this?).
> 
> There is currently an issue with that when building meta-toolchain and
> poky-image-lsb that I know of. Patches to fix these bugs welcome (and
> bugs need filing).
> 
>> - PPC is broken.
>> 
>> PPC is having serious issues that seem to revolve around
>> linux-yocto-2.6.37. Bruce I believe is working on this. Can anyone
>> give an update around where this stands?
> 
> Bruce pushed a patch to disable -Os usage. If this isn't showing up in
> the next round of autobuilds, we'll have to figure out why it isn't getting applied.
> 
> Cheers,
> 
> Richard
> 
> 
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Best Regards,
Jiajun

Re: Recent build issues.

[Bruce Ashfield]

On 10-12-23 05:08 PM, Flanagan, Elizabeth wrote:
> Recent build issues:
>
> - There seem to be a few issues going on with sanity testing:
>
> First, I've made a change to how sanity tests are run on the autobuilder that should speed them up some. bitbake @ -c qemuimagetest creates a rootfs. I've switched this to bitbake @ -c qemuimagetest_standalone, so this shouldn't create a rootfs and should use the one already existing. We should see some performance improvement with this. I thought this had been included in my pull with BB 0.8.3 but I must have not included it.
>
> Next,  the following bugs have been making it difficult to track down build failures:
>
> Sanity Test Boot test returns invalid results.
> http://bugzilla.pokylinux.org/show_bug.cgi?id=611
>
> Sanity tests do not always kill QEMU once timeout is reached
> http://bugzilla.pokylinux.org/show_bug.cgi?id=595
>
> There are also sanity test failures in a few images:
>
> qemuarm poky-image-sato
> qemuarm poky-image-minimal
> qemumips poky-image-sato
>
> mips seems to have run poky-image-minimal ok, but with Bug #611, I can't be sure that is the case and I doubt it's actually the case. My inclination is that there is an issue with arm and mips builds that should be looked at.
>
> - PPC is broken.
>
> PPC is having serious issues that seem to revolve around linux-yocto-2.6.37. Bruce I believe is working on this. Can anyone give an update around where this stands?

Were we talking build or boot for this one ? The slow boot is only
being seen during auto-tests, and I've never reproduced it here.

I'm looking for someone who has time to debug this by tweaking
the kernel configuration, but as for what we produce for qemuppc
the configuration is sane and boots fine. Does anyone who can
reproduce this have time to look into it a bit more, or get me
access to a reproducing environment ?

Bruce

>
> I'm running nightly-incremental and nightly now to pull in some changes made this morning and to prepare for weekend delivery to QA. I'll send out those build results on Sunday (or whenever the build finishes.)
> -b
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto