Re: [PATCH 0/9] kernel/linux-yocto: consolidated pull request

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Wed, 2023-05-31 at 19:24 -0400, Bruce Ashfield wrote:
> On Wed, May 31, 2023 at 6:22 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > On Wed, 2023-05-31 at 18:18 -0400, Bruce Ashfield wrote:
> > > On Wed, May 31, 2023 at 6:03 PM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > > 
> > > > Hi Bruce,
> > > > 
> > > > On Wed, 2023-05-31 at 14:48 -0400, bruce.ashfield@gmail.com wrote:
> > > > > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> > > > > 
> > > > > Richard,
> > > > > 
> > > > > Here's my consolidated queue. I've had some of these in testing for quite
> > > > > a while, as I didn't want to send them right before release or before my
> > > > > vacation last week.
> > > > > 
> > > > > Most are -stable updates to 6.1 and finally bumping the -dev recipe to
> > > > > show that we've been testing 6.4 for quite some time.
> > > > > 
> > > > > The make-mod-scripts rmwork fix was discussed at lenght on the list, and
> > > > > has been tested by myself and people seeing the issues. But with all the
> > > > > races and issues we've seen in that area for years, I'll keep a close eye
> > > > > out for AB issues.
> > > > > 
> > > > > I've factored things a bit around enabling PAHOLE, I don't expect it
> > > > > to cause issues since it isn't on by default .. but you never know with
> > > > > these sorts of changes.
> > > > > 
> > > > > I also have a bump for the refernece BSPs (sent to the appropriate
> > > > > list shortly) and a documentation commit for the PAHOLE tweaks (also
> > > > > sent to that list shortly, once we agree on the this change).
> > > > 
> > > > Thanks for the updates!
> > > > 
> > > > I put these in for testing (along with some other changes) and there
> > > > was an issue, probably the same thing showing up but I've not dug into
> > > > it:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7159
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/61/builds/7197
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/104/builds/6093/steps/11/logs/stdio
> > > > 
> > > 
> > > That's odd:
> > > 
> > > ERROR: core-image-sato-sdk-1.0-r0 do_testimage: File
> > > /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/deploy/images/qemux86/core-image-sato-sdk-qemux86.testdata.json
> > > not found ([Errno 2] No such file or directory:
> > > '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/deploy/images/qemux86/core-image-sato-sdk-qemux86.testdata.json').
> > 
> > That is the QA test failing due to an earlier failure to build the
> > image. The earlier failure looks like:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7159/steps/11/logs/stdio
> > 
> > Log data follows:
> > > DEBUG: Executing python function extend_recipe_sysroot
> > > NOTE: Direct dependencies are ['/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-core/glibc/glibc_2.37.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/binutils/binutils-cross_2.40.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/gcc/gcc-cross_13.1.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/gcc/gcc-runtime_13.1.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-connectivity/openssl/openssl_3.1.1.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-core/util-linux/util-linux_2.38.1.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-core/zlib/zlib_1.2.13.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/bison/bison_3.8.2.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/elfutils/elfutils_0.189.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-extended/xz/xz_5.4.3.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-kernel/kmod/kmod_30.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-support/gmp/gmp_6.2.1.bb:do_populate_sysroot', 'virtual:native:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/meta/recipes-support/libmpc/libmpc_1.3.1.bb:do_populate_sysroot']
> > > NOTE: Installed into sysroot: ['kern-tools-native', 'util-linux-native', 'kmod-native', 'libcap-ng-native', 'libpcre2-native']
> > > NOTE: Skipping as already exists in sysroot: ['glibc', 'binutils-cross-i686', 'gcc-cross-i686', 'gcc-runtime', 'quilt-native', 'openssl-native', 'zlib-native', 'bison-native', 'elfutils-native', 'patch-native', 'pkgconfig-native', 'pseudo-native', 'bc-native', 'xz-native', 'gmp-native', 'libmpc-native', 'linux-libc-headers', 'libgcc', 'gnu-config-native', 'flex-native', 'texinfo-dummy-native', 'libtool-native', 'mpfr-native', 'zstd-native', 'perl-native', 'gettext-minimal-native', 'ncurses-native', 'util-linux-libuuid-native', 'libmicrohttpd-native', 'libarchive-native', 'curl-native', 'sqlite3-native', 'readline-native', 'attr-native', 'python3-native', 'm4-native', 'gdbm-native', 'make-native', 'gnutls-native', 'libgcrypt-native', 'bzip2-native', 'libedit-native', 'libffi-native', 'libtirpc-native', 'expat-native', 'libnsl2-native', 'nettle-native', 'libidn2-native', 'libtasn1-native', 'libunistring-native', 'libcap-native', 'libgpg-error-native', 'cmake-native']
> > > DEBUG: sed -e 's:^[^/]*/:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/:g' /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/sysroots-components/x86_64/libpcre2-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/fifo.169218:g'
> > > DEBUG: Python function extend_recipe_sysroot finished
> > > DEBUG: Executing shell function do_configure
> > > NOTE: make HOSTCC=gcc  -isystem/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -L/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -L/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,-rpath,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 -static HOSTCPP=gcc  -E HOSTCXX=g++  -isystem/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -L/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -L/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,-rpath,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath,/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 -static CROSS_COMPILE=i686-poky-linux- CC=i686-poky-linux-gcc  -fuse-ld=bfd -fcanon-prefix-map  -fmacro-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0=/usr/src/debug/make-mod-scripts/1.0-r0  -fdebug-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0=/usr/src/debug/make-mod-scripts/1.0-r0  -fmacro-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0=/usr/src/debug/make-mod-scripts/1.0-r0  -fdebug-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0=/usr/src/debug/make-mod-scripts/1.0-r0  -fdebug-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot=  -fmacro-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot=  -fdebug-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native=  -fdebug-prefix-map=/home/pokybui
> > d/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source=/usr/src/kernel -fdebug-prefix-map=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-build-artifacts=/usr/src/kernel LD=i686-poky-linux-ld.bfd  AR=i686-poky-linux-ar  OBJCOPY=i686-poky-linux-objcopy  HOSTPKG_CONFIG=pkg-config --static CRYPTO_LIBS=-L/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/lib -lcrypto -ldl -pthread -C /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source O=/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-build-artifacts prepare
> > > make: Entering directory '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source'
> > >   SYNC    include/config/auto.conf
> > >   GEN     Makefile
> > >   HOSTCC  scripts/basic/fixdep
> > > /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/hosttools/ld: cannot find -lc
> > > collect2: error: ld returned 1 exit status
> > > make[3]: *** [/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source/scripts/Makefile.host:111: scripts/basic/fixdep] Error 1
> > > make[2]: *** [/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source/Makefile:640: scripts_basic] Error 2
> > > /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source/Makefile:748: include/config/auto.conf: No such file or directory
> > > make[1]: *** [/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source/Makefile:805: include/config/auto.conf] Error 2
> > > make[1]: *** [include/config/auto.conf] Deleting file 'include/generated/rustc_cfg'
> > > make[1]: *** [include/config/auto.conf] Deleting file 'include/generated/autoconf.h'
> > > make: *** [Makefile:238: __sub-make] Error 2
> > > make: Leaving directory '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work-shared/qemux86/kernel-source'
> > > ERROR: oe_runmake failed
> > > WARNING: /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218:195 exit 1 from 'exit 1'
> > > WARNING: Backtrace (BB generated script):
> > >       #1: bbfatal_log, /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218, line 195
> > >       #2: die, /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218, line 179
> > >       #3: oe_runmake, /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218, line 174
> > >       #4: do_configure, /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218, line 164
> > >       #5: main, /home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/make-mod-scripts/1.0-r0/temp/run.do_configure.169218, line 208
> > NOTE: recipe make-mod-scripts-1.0-r0: task do_configure: Failed
> > 
> > > It shouldn't be the kernel updates, so I'm assuming something with
> > > make-mod-scripts. I'll have a look at it on Thursday.
> > 
> > Yes, it is make-mod-scripts, maybe missing static libraries?
> 
> I was able to build qemux86 on my ubuntu builder, and from the message
> it is looking for -lc and not finding it.
> 
> It looks like that was alma linux, so no static c library ?
> 
> I don't have access to anything alma based to test, but I guess a
> dependency on a static native glibc might help ?

If I understand what is being attempted here, what is being asked for
is a static library from the distro the build is running on. We don't
build a native glibc (or gcc).

It is easy to say "just tell people to install that" but it isn't quite
that simple. The work needed off the top of my head is roughly:

* add sanity tests to check it is present and error if not
* documentation updates to tell people to install it
* updating all the autobuilder workers to have it
* update buildtools tarball to contain it
* release a new buildtools
* ensure everything in automated testing uses the new buildtools
release

so I do have to ask if we really need to add this dependency?

Cheers,

Richard