Re: State of OE World, 2019-03-16

[Martin Jansa <martin.jansa@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6476 bytes --]

On Wed, Mar 27, 2019 at 05:17:54PM +0100, Martin Jansa wrote:
> On Sun, Mar 17, 2019 at 08:26:37AM -0700, Khem Raj wrote:
> > http://www.openembedded.org/wiki/Bitbake_World_Status
> > 
> > == Failed tasks 2019-03-16 ==
> > 
> > INFO: jenkins-job.sh-1.8.45 Complete log available at
> > http://logs.nslu2-linux.org/buildlogs/oe/world/warrior/log.report.20190317_082308.log
> > 
> >     * sources/openembedded-core/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.58.3.bb:do_compile
> 
> I was hit by this one as well on some arm targets.
> 
> http://logs.nslu2-linux.org/buildlogs/oe/world/warrior/log.world.qemuarm.20190321_215508.log/bitbake.log
> 
> shows that it's actually from qemu-arm segfault inside g-ir-scanner-qemuwrapper
> 
> | qemu: uncaught target signal 11 (Segmentation fault) - core dumped
> | /home/jenkins/oe/world/yoe/build/tmpfs/work/armv7vet2hf-neon-yoe-linux-gnueabi/gobject-introspection/1.58.3-r0/build/g-ir-scanner-qemuwrapper: line 6:  1959 Segmentation fault      (core dumped) PSEUDO_UNLOAD=1 qemu-arm -r 3.2.0 -L /home/jenkins/oe/world/yoe/build/tmpfs/work/armv7vet2hf-neon-yoe-linux-gnueabi/gobject-introspection/1.58.3-r0/recipe-sysroot -E LD_LIBRARY_PATH=$GIR_EXTRA_LIBS_PATH:.libs:/home/jenkins/oe/world/yoe/build/tmpfs/work/armv7vet2hf-neon-yoe-linux-gnueabi/gobject-introspection/1.58.3-r0/recipe-sysroot//usr/lib:/home/jenkins/oe/world/yoe/build/tmpfs/work/armv7vet2hf-neon-yoe-linux-gnueabi/gobject-introspection/1.58.3-r0/recipe-sysroot//lib "$@"
> 
> The interesting part is that this happens only when glibc is built with gold enabled.
> 
> I've put a bit more details in temporary work-around here:
> https://github.com/shr-distribution/meta-smartphone/commit/8f06eb355ead85464b1a1bbaa82584504df15743
> 
> I'm comparing qemuarm glibc build with bfd and gold to see if there is some
> significant difference which might cause this and I plan to flash some small
> image on my targets to see if libc is broken in runtime as well or only when
> running inside qemu-arm.
> 
> Is anyone else seeing this as well?

Checking older world status on the wiki shows that this issue was
introduced somewhere around christmas

This is last world build before the issue:
http://logs.nslu2-linux.org/buildlogs/oe/world/warrior/log.report.20181219_112425.log
...
== Tested changes (not included in master yet) - openembedded-core ==
latest upstream commit: 
14c291e1fb gcc-runtime: Add missing libc dependency
not included in master yet: 
7e2ab991fa python/python3: use cc_basename to replace CC for checking compiler
ced915026d python-native: fix one do_populate_sysroot warning
aee47f3e82 netbase: add entry to /etc/hosts according to /etc/hostname
a09e2db43a sstate: add support for caching shared workdir tasks
c104a34166 grub2: Fix passing null to printf formats
6f364ff8c4 gnupg: Upgrade to 2.2.12 release
0224fec86b glibc: Upgrade towards 2.29 release
2486349782 gcc-9.0: Add recipes for upcoming gcc 9.0 release in mid-2019
b3ab29bdbb gcc-runtime: Drop building libmpx

and this is the first which reported qemu-arm segfault:
== Tested changes (not included in master yet) - openembedded-core ==
latest upstream commit: 
95659bed3f populate_sdk_ext.bbclass: Include site.conf in parsing for contents for local.conf
not included in master yet: 
0c9db0ae7d python/python3: use cc_basename to replace CC for checking compiler
bb3eb6bc41 python-native: fix one do_populate_sysroot warning
08a205f872 netbase: add entry to /etc/hosts according to /etc/hostname
b81c8650d0 sstate: add support for caching shared workdir tasks
702be42dc9 glibc: Remove site_config and glibc-initial
fc230822d8 gcc: Drop gcc-cross-initial and use gcc-cross instead
8652df3a0d gcc: Drop the -initial versions of the compiler
0c2f6dfa2f recipes: Drop virtual/libc-for-gcc
c64531600d newlib: Move away from gcc-initial dependency
0d5fb4428f libssp: Remove dependency on gcc-initial
7dd8829d3e musl: Move away from gcc-initial dependency
9abdb4d6ca tcmode-default: Drop pinnings for gcc-initial based recipes
e02724e8f8 base.bbclass, classextend.py: Drop catering to gcc-initial
7756cb7d17 oeqa/concurrencytest: fix for locating meta-selftest
f4c68c39e7 crosssdk/cross-canadian: Set LIBCOVERRIDE correctly
6fc9bb5698 glibc: Enable --with-default-link
d4cb282bc6 gcc-9.0: Add recipes for upcoming gcc 9.0 release in mid-2019
6efe313b46 glibc: Upgrade towards 2.29 release
dac5dc5067 grub2: Fix passing null to printf formats
bb36d9dc8c gnupg: Upgrade to 2.2.12 release
81059437a5 gnupg: Do not apply -Woverride-init guard for gcc >= 9
6fa05e5ace nfs-utils: Disable format-overflow as Werror
8fab7b23ef musl: Update to latest master
a5641cac42 libxcrypt: Upgrade to 4.4.2
c90ec03d07 gdb: Backport a fix to build with gcc9
1adae92573 valgrind: Fix x86_64 build failure with gcc9
38e9e09104 ffmpeg: Backport patch to fix build with gcc9
4e1fa57743 binutils: Upgrade to latest on 2.31 release branch
5cce276d4a binutils: Upgrade to upcoming 2.32 release

and the changes introduced in oe-core between these are:
docker-shr @ ~/openembedded-core $ git log --oneline 14c291e1fb..95659bed3f
95659bed3f populate_sdk_ext.bbclass: Include site.conf in parsing for contents for local.conf
085530e8ff i2c-tools: upgrade 4.0 -> 4.1
673c2dcf10 common-licenses: update Libpng license text
099aecfaa3 libpng: upgrade 1.6.35 -> 1.6.36
6574bb7523 harfbuzz: upgrade 2.1.3 -> 2.2.0
8286e9d79d sqlite3: upgrade 3.25.3 -> 3.26.0
a33170ff9a libatomic-ops: upgrade 7.6.6 -> 7.6.8
6a10ff0bcd gsettings-desktop-schemas: upgrade 3.28.0 -> 3.28.1
dd2c4d9d85 eudev: upgrade 3.2.5 -> 3.2.7
0860216a81 icecc: patchelf is needed by icecc-create-env
d2fcaeb153 icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
4455c46bf7 icecc: Syntax error meant that we weren't waiting for tarball generation
fa6bae80bc icecc: Trivial simplification
6d6788a3ea icecc: readlink -f on the recipe-sysroot gcc/g++
1ee90ec89a libxcrypt: Fix nativesdk build with gcc9
547174fc83 gcc-runtime: Drop building libmpx

It's possible that it's not caused by one of these oe-core changes, but one of the pending
yoe changes (like those 2 binutils upgrades) which were merged to oe-core in some form a bit
later.

I'll verify if the issue is reproducible with oe-core 95659bed3f and continue the bisect from there.

Regards,
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 201 bytes --]