* [PATCH] kernel.bbclass: Set pkg-config variables for building modules @ 2024-02-22 20:28 Munehisa Kamata 2024-02-22 21:38 ` [OE-core] " Bruce Ashfield 0 siblings, 1 reply; 7+ messages in thread From: Munehisa Kamata @ 2024-02-22 20:28 UTC (permalink / raw) To: openembedded-core; +Cc: Munehisa Kamata The pkg-config workaround has been applied for kernel image building, but not for module building. On x86, this can trigger rebuilding of objtool unnecessary at do_compile_kernelmodules task. If the kernel is built with CONFIG_DEBUG_INFO_BTF=y, the task even rebuilds vmlinux and leads to inconsistent build-id between bzImage and vmlinux, and also BTF mismatches between the bzImage and modules. To avoid the inconsistency, apply the same workaround when building modules. For kernels 5.19+, simply set HOSTPKG_CONFIG in the make command line. Signed-off-by: Munehisa Kamata <kamatam@amazon.com> --- meta/classes-recipe/kernel.bbclass | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index a76aaee5ba..db4461e551 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' KERNEL_ALT_IMAGETYPE ??= "" @@ -356,9 +358,6 @@ kernel_do_compile() { export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" export PKG_CONFIG_SYSROOT_DIR="" - # for newer kernels (5.19+) there's a dedicated variable - export HOSTPKG_CONFIG="pkg-config-native" - if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not # be set.... @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install do_compile_kernelmodules() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE + + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" + export PKG_CONFIG_SYSROOT_DIR="" + if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not # be set.... -- 2.34.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [OE-core] [PATCH] kernel.bbclass: Set pkg-config variables for building modules 2024-02-22 20:28 [PATCH] kernel.bbclass: Set pkg-config variables for building modules Munehisa Kamata @ 2024-02-22 21:38 ` Bruce Ashfield 2024-02-23 8:05 ` [PATCH v2] " Munehisa Kamata 0 siblings, 1 reply; 7+ messages in thread From: Bruce Ashfield @ 2024-02-22 21:38 UTC (permalink / raw) To: kamatam; +Cc: openembedded-core On Thu, Feb 22, 2024 at 3:28 PM Munehisa Kamata via lists.openembedded.org <kamatam=amazon.com@lists.openembedded.org> wrote: > > The pkg-config workaround has been applied for kernel image building, but > not for module building. On x86, this can trigger rebuilding of objtool > unnecessary at do_compile_kernelmodules task. If the kernel is built with > CONFIG_DEBUG_INFO_BTF=y, the task even rebuilds vmlinux and leads to > inconsistent build-id between bzImage and vmlinux, and also BTF mismatches > between the bzImage and modules. I don't question the change, as it is what I had to do for the mail kernel build. What I would like to see logged is a set of steps that managed to trigger the issue. Since the module builds are dependent on the main kernel, everything that they need should be available and already timestamped to not be required, so the missing pkconfig shouldn't trigger much. That way we'll know if there's a gap somewhere in the dependencies, or another race condition and we can test for it. Bruce > > To avoid the inconsistency, apply the same workaround when building > modules. For kernels 5.19+, simply set HOSTPKG_CONFIG in the make command > line. > > Signed-off-by: Munehisa Kamata <kamatam@amazon.com> > --- > meta/classes-recipe/kernel.bbclass | 12 +++++++++--- > 1 file changed, 9 insertions(+), 3 deletions(-) > > diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass > index a76aaee5ba..db4461e551 100644 > --- a/meta/classes-recipe/kernel.bbclass > +++ b/meta/classes-recipe/kernel.bbclass > @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" > EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' > EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' > EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' > +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules > +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' > > KERNEL_ALT_IMAGETYPE ??= "" > > @@ -356,9 +358,6 @@ kernel_do_compile() { > export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > export PKG_CONFIG_SYSROOT_DIR="" > > - # for newer kernels (5.19+) there's a dedicated variable > - export HOSTPKG_CONFIG="pkg-config-native" > - > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > # be set.... > @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install > > do_compile_kernelmodules() { > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE > + > + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) > + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" > + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" > + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > + export PKG_CONFIG_SYSROOT_DIR="" > + > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > # be set.... > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#196041): https://lists.openembedded.org/g/openembedded-core/message/196041 > Mute This Topic: https://lists.openembedded.org/mt/104516377/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2] kernel.bbclass: Set pkg-config variables for building modules 2024-02-22 21:38 ` [OE-core] " Bruce Ashfield @ 2024-02-23 8:05 ` Munehisa Kamata 2024-02-23 14:09 ` Bruce Ashfield 0 siblings, 1 reply; 7+ messages in thread From: Munehisa Kamata @ 2024-02-23 8:05 UTC (permalink / raw) To: bruce.ashfield; +Cc: kamatam, openembedded-core The pkg-config workaround has been applied for kernel image building, but not for module building. So pkg-config variables are different between do_compile and do_compile_kernelmodules tasks. It may unnecessary trigger rebuilding of a few host tools at the later task. Especially when CONFIG_DEBUG_INFO_BTF is enabled in the kernel, it may even trigger rebuilding vmlinux at do_compile_kernelmodules due to the rebuilt host tools such as certs/extract-cert or objtool (on x86). This eventually creates an inconsistent set of kernel binaries. Here is the repro steps: - Check out nanbield on x86 - The unexpected rebuild happens on kirkstone or possibly earlier - Ensure that pahole is available (e.g. via meta-oe) - Set KERNEL_DEBUG to "True" to properly set up PAHOLE e.g. $ export KERNEL_DEBUG="True" $ export BB_ENV_PASSTHROUGH_ADDITIONS="${BB_ENV_PASSTHROUGH_ADDITIONS} KERNEL_DEBUG" - Enable CONFIG_DEBUG_INFO_BTF=y e.g. $ bitbake -c menuconfig virtual/kernel -> Kernel hacking -> Compile-time checks and compiler options -> Generate BTF typeinfo - Build the kernel e.g. $ bitbake virtual/kernel The BTF information in the resulting bzImage and kernel modules are inconsistent, because the module's BTF information is generated using the "second" vmlinux that doesn't have the identical BTF to the "first" vmlinux. These modules can't be loaded at runtime due to the BTF mismatch. This also leads to a build-id mismatch between the installed bzImage and vmlinux since the bzImage is created from the first vmlinux, but the installed vmlinux is the second one. $ eu-readelf -n tmp/work/qemux86_64-poky-linux/linux-yocto/6.5.13+git/image/boot/{bzImage*,vmlinux*} | grep "Build ID" Build ID: 4a0d62ee7fef0244950f0f604253729875bea493 Build ID: fb99b3d91399dbe42bf67ddee59e0f5a0c7f74d9 To avoid the unexpected rebuilding that results in such inconsistency, set the same pkg-config variables when building kernel and modules. For kernel 5.19 and above, simply set the HOSTPKG_CONFIG in the make command line. Signed-off-by: Munehisa Kamata <kamatam@amazon.com> --- v1 -> v2: Rewrote the commit message with the repro steps meta/classes-recipe/kernel.bbclass | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index a76aaee5ba..db4461e551 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' KERNEL_ALT_IMAGETYPE ??= "" @@ -356,9 +358,6 @@ kernel_do_compile() { export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" export PKG_CONFIG_SYSROOT_DIR="" - # for newer kernels (5.19+) there's a dedicated variable - export HOSTPKG_CONFIG="pkg-config-native" - if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not # be set.... @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install do_compile_kernelmodules() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE + + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" + export PKG_CONFIG_SYSROOT_DIR="" + if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not # be set.... -- 2.34.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2] kernel.bbclass: Set pkg-config variables for building modules 2024-02-23 8:05 ` [PATCH v2] " Munehisa Kamata @ 2024-02-23 14:09 ` Bruce Ashfield 2024-02-24 7:21 ` Munehisa Kamata 0 siblings, 1 reply; 7+ messages in thread From: Bruce Ashfield @ 2024-02-23 14:09 UTC (permalink / raw) To: Munehisa Kamata; +Cc: openembedded-core On Fri, Feb 23, 2024 at 3:05 AM Munehisa Kamata <kamatam@amazon.com> wrote: > > The pkg-config workaround has been applied for kernel image building, but > not for module building. So pkg-config variables are different between > do_compile and do_compile_kernelmodules tasks. It may unnecessary trigger > rebuilding of a few host tools at the later task. > > Especially when CONFIG_DEBUG_INFO_BTF is enabled in the kernel, it may even > trigger rebuilding vmlinux at do_compile_kernelmodules due to the rebuilt > host tools such as certs/extract-cert or objtool (on x86). This eventually > creates an inconsistent set of kernel binaries. > > Here is the repro steps: > > - Check out nanbield on x86 > - The unexpected rebuild happens on kirkstone or possibly earlier > > - Ensure that pahole is available (e.g. via meta-oe) > > - Set KERNEL_DEBUG to "True" to properly set up PAHOLE > e.g. > $ export KERNEL_DEBUG="True" > $ export BB_ENV_PASSTHROUGH_ADDITIONS="${BB_ENV_PASSTHROUGH_ADDITIONS} KERNEL_DEBUG" > > - Enable CONFIG_DEBUG_INFO_BTF=y > e.g. > $ bitbake -c menuconfig virtual/kernel > -> Kernel hacking > -> Compile-time checks and compiler options > -> Generate BTF typeinfo > > - Build the kernel > e.g. > $ bitbake virtual/kernel > > The BTF information in the resulting bzImage and kernel modules are > inconsistent, because the module's BTF information is generated using the > "second" vmlinux that doesn't have the identical BTF to the "first" vmlinux. > These modules can't be loaded at runtime due to the BTF mismatch. > > This also leads to a build-id mismatch between the installed bzImage and > vmlinux since the bzImage is created from the first vmlinux, but the > installed vmlinux is the second one. > > $ eu-readelf -n tmp/work/qemux86_64-poky-linux/linux-yocto/6.5.13+git/image/boot/{bzImage*,vmlinux*} | grep "Build ID" > Build ID: 4a0d62ee7fef0244950f0f604253729875bea493 > Build ID: fb99b3d91399dbe42bf67ddee59e0f5a0c7f74d9 > > To avoid the unexpected rebuilding that results in such inconsistency, set > the same pkg-config variables when building kernel and modules. For kernel > 5.19 and above, simply set the HOSTPKG_CONFIG in the make command line. That is indeed not a simple workflow! In the past, we've always had the existing packageconfig results picked up and used by later stages of the kernel build which prevented things like this from happening. Have you figured out exactly which packageconfig is triggering the rebuild, and had a look to see if something change such that the existing results aren't used ? All that being said, rather than repeating the exports, it would be better to have them in a common place, since eventually everything but the HOSTPKG_CONFIG definition will be dropped. Have you tried a more class-global export of the values ? We have a few of those, but admittedly, we have way more exports that are duplicated between do_compile and do_compile_kernelmodules(), so the duplicated exports aren't a big issue. Bruce > > Signed-off-by: Munehisa Kamata <kamatam@amazon.com> > --- > > v1 -> v2: Rewrote the commit message with the repro steps > > meta/classes-recipe/kernel.bbclass | 12 +++++++++--- > 1 file changed, 9 insertions(+), 3 deletions(-) > > diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass > index a76aaee5ba..db4461e551 100644 > --- a/meta/classes-recipe/kernel.bbclass > +++ b/meta/classes-recipe/kernel.bbclass > @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" > EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' > EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' > EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' > +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules > +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' > > KERNEL_ALT_IMAGETYPE ??= "" > > @@ -356,9 +358,6 @@ kernel_do_compile() { > export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > export PKG_CONFIG_SYSROOT_DIR="" > > - # for newer kernels (5.19+) there's a dedicated variable > - export HOSTPKG_CONFIG="pkg-config-native" > - > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > # be set.... > @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install > > do_compile_kernelmodules() { > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE > + > + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) > + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" > + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" > + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > + export PKG_CONFIG_SYSROOT_DIR="" > + > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > # be set.... > -- > 2.34.1 > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] kernel.bbclass: Set pkg-config variables for building modules 2024-02-23 14:09 ` Bruce Ashfield @ 2024-02-24 7:21 ` Munehisa Kamata 2024-02-25 14:54 ` Bruce Ashfield 0 siblings, 1 reply; 7+ messages in thread From: Munehisa Kamata @ 2024-02-24 7:21 UTC (permalink / raw) To: bruce.ashfield; +Cc: kamatam, openembedded-core Hi Bruce, > That is indeed not a simple workflow! > > In the past, we've always had the existing packageconfig results picked up and > used by later stages of the kernel build which prevented things like this from > happening. > > Have you figured out exactly which packageconfig is triggering the rebuild, and > had a look to see if something change such that the existing results > aren't used ? The missing of libcrypto.pc and libelf.pc triggers the rebuild of certs/extract-cert and objtool respectively. When PKG_CONFIG_DIR is set to recipe-sysroot at module build time, it points to a non-existent directory and therefore pkg-config can't find .pc for the requested library whereas they are present in the recipe-sysroot-native, then it causes make to rebuild the target. Although I admit that I don't fully understand how make particularly decides to trigger the rebuild in this case. > All that being said, rather than repeating the exports, it would be > better to have > them in a common place, since eventually everything but the HOSTPKG_CONFIG > definition will be dropped. > > Have you tried a more class-global export of the values ? We have a > few of those, > but admittedly, we have way more exports that are duplicated between do_compile > and do_compile_kernelmodules(), so the duplicated exports aren't a big issue. Yes, I actually tried it first and it worked at lesat for simple kernel building. But I wasn't entirely sure if it was safe enough for the ecosystem and came up with the change that is hopefully less impact. That said, if you think it's fine, I'm happy to submit v3 with the class-global export. It would be better indeed. Thanks, Munehisa > Bruce > > > > > Signed-off-by: Munehisa Kamata <kamatam@amazon.com> > > --- > > > > v1 -> v2: Rewrote the commit message with the repro steps > > > > meta/classes-recipe/kernel.bbclass | 12 +++++++++--- > > 1 file changed, 9 insertions(+), 3 deletions(-) > > > > diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass > > index a76aaee5ba..db4461e551 100644 > > --- a/meta/classes-recipe/kernel.bbclass > > +++ b/meta/classes-recipe/kernel.bbclass > > @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" > > EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' > > EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' > > EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' > > +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules > > +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' > > > > KERNEL_ALT_IMAGETYPE ??= "" > > > > @@ -356,9 +358,6 @@ kernel_do_compile() { > > export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > export PKG_CONFIG_SYSROOT_DIR="" > > > > - # for newer kernels (5.19+) there's a dedicated variable > > - export HOSTPKG_CONFIG="pkg-config-native" > > - > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > # be set.... > > @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install > > > > do_compile_kernelmodules() { > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE > > + > > + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) > > + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" > > + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" > > + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > + export PKG_CONFIG_SYSROOT_DIR="" > > + > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > # be set.... > > -- > > 2.34.1 > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] kernel.bbclass: Set pkg-config variables for building modules 2024-02-24 7:21 ` Munehisa Kamata @ 2024-02-25 14:54 ` Bruce Ashfield 2024-02-26 23:50 ` Munehisa Kamata 0 siblings, 1 reply; 7+ messages in thread From: Bruce Ashfield @ 2024-02-25 14:54 UTC (permalink / raw) To: Munehisa Kamata; +Cc: openembedded-core On Sat, Feb 24, 2024 at 2:21 AM Munehisa Kamata <kamatam@amazon.com> wrote: > > Hi Bruce, > > > That is indeed not a simple workflow! > > > > In the past, we've always had the existing packageconfig results picked up and > > used by later stages of the kernel build which prevented things like this from > > happening. > > > > Have you figured out exactly which packageconfig is triggering the rebuild, and > > had a look to see if something change such that the existing results > > aren't used ? > > The missing of libcrypto.pc and libelf.pc triggers the rebuild of > certs/extract-cert and objtool respectively. When PKG_CONFIG_DIR is set to > recipe-sysroot at module build time, it points to a non-existent directory > and therefore pkg-config can't find .pc for the requested library whereas > they are present in the recipe-sysroot-native, then it causes make to > rebuild the target. > > Although I admit that I don't fully understand how make particularly > decides to trigger the rebuild in this case. Fair enough. I can't say that I always understand how the kernel's Makefiles are triggering difficult builds without significant hacking up of the Makefiles, and it isn't worth doing for this. At a minimum, it is good to know that the .pc files are being consulted so we do have something reproducible between builds when everything is full defined. > > > All that being said, rather than repeating the exports, it would be > > better to have > > them in a common place, since eventually everything but the HOSTPKG_CONFIG > > definition will be dropped. > > > > Have you tried a more class-global export of the values ? We have a > > few of those, > > but admittedly, we have way more exports that are duplicated between do_compile > > and do_compile_kernelmodules(), so the duplicated exports aren't a big issue. > > Yes, I actually tried it first and it worked at lesat for simple kernel > building. But I wasn't entirely sure if it was safe enough for the > ecosystem and came up with the change that is hopefully less impact. That > said, if you think it's fine, I'm happy to submit v3 with the class-global > export. It would be better indeed. > It is something that should be safe, since really, whenever we build anything in the kernel having a consistent environment should be in place. The kernel modules build has expanded over the years and more of the kernel tools and common build components are coming into play, so things we didn't previously need .. are now needed in the modules compilation. I had recommended that this go into test on Friday via IRC, but there's no harm in doing a cleaned up v3 with a class global set of exports. We can always apply it post release if it is decided there is extra risk .. but at least it will be a reference to how we should start consolidating some of the exports. Bruce > > Thanks, > Munehisa > > > Bruce > > > > > > > > Signed-off-by: Munehisa Kamata <kamatam@amazon.com> > > > --- > > > > > > v1 -> v2: Rewrote the commit message with the repro steps > > > > > > meta/classes-recipe/kernel.bbclass | 12 +++++++++--- > > > 1 file changed, 9 insertions(+), 3 deletions(-) > > > > > > diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass > > > index a76aaee5ba..db4461e551 100644 > > > --- a/meta/classes-recipe/kernel.bbclass > > > +++ b/meta/classes-recipe/kernel.bbclass > > > @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" > > > EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' > > > EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' > > > EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' > > > +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules > > > +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' > > > > > > KERNEL_ALT_IMAGETYPE ??= "" > > > > > > @@ -356,9 +358,6 @@ kernel_do_compile() { > > > export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > > export PKG_CONFIG_SYSROOT_DIR="" > > > > > > - # for newer kernels (5.19+) there's a dedicated variable > > > - export HOSTPKG_CONFIG="pkg-config-native" > > > - > > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > > # be set.... > > > @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install > > > > > > do_compile_kernelmodules() { > > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE > > > + > > > + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) > > > + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" > > > + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" > > > + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > > + export PKG_CONFIG_SYSROOT_DIR="" > > > + > > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > > # be set.... > > > -- > > > 2.34.1 > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] kernel.bbclass: Set pkg-config variables for building modules 2024-02-25 14:54 ` Bruce Ashfield @ 2024-02-26 23:50 ` Munehisa Kamata 0 siblings, 0 replies; 7+ messages in thread From: Munehisa Kamata @ 2024-02-26 23:50 UTC (permalink / raw) To: bruce.ashfield; +Cc: kamatam, openembedded-core Hi Bruce, On Sun, 2024-02-25 06:54:58 -0800, Bruce Ashfield wrote: > > On Sat, Feb 24, 2024 at 2:21 AM Munehisa Kamata <kamatam@amazon.com> wrote: > > > > Hi Bruce, > > > > > That is indeed not a simple workflow! > > > > > > In the past, we've always had the existing packageconfig results picked up and > > > used by later stages of the kernel build which prevented things like this from > > > happening. > > > > > > Have you figured out exactly which packageconfig is triggering the rebuild, and > > > had a look to see if something change such that the existing results > > > aren't used ? > > > > The missing of libcrypto.pc and libelf.pc triggers the rebuild of > > certs/extract-cert and objtool respectively. When PKG_CONFIG_DIR is set to > > recipe-sysroot at module build time, it points to a non-existent directory > > and therefore pkg-config can't find .pc for the requested library whereas > > they are present in the recipe-sysroot-native, then it causes make to > > rebuild the target. > > > > Although I admit that I don't fully understand how make particularly > > decides to trigger the rebuild in this case. > > Fair enough. I can't say that I always understand how the kernel's > Makefiles are triggering difficult builds without significant hacking > up of the Makefiles, and it isn't worth doing for this. > > At a minimum, it is good to know that the .pc files are being consulted > so we do have something reproducible between builds when everything > is full defined. > > > > > > All that being said, rather than repeating the exports, it would be > > > better to have > > > them in a common place, since eventually everything but the HOSTPKG_CONFIG > > > definition will be dropped. > > > > > > Have you tried a more class-global export of the values ? We have a > > > few of those, > > > but admittedly, we have way more exports that are duplicated between do_compile > > > and do_compile_kernelmodules(), so the duplicated exports aren't a big issue. > > > > Yes, I actually tried it first and it worked at lesat for simple kernel > > building. But I wasn't entirely sure if it was safe enough for the > > ecosystem and came up with the change that is hopefully less impact. That > > said, if you think it's fine, I'm happy to submit v3 with the class-global > > export. It would be better indeed. > > > > It is something that should be safe, since really, whenever we build > anything in the kernel having a consistent environment should be in > place. The kernel modules build has expanded over the years and > more of the kernel tools and common build components are coming > into play, so things we didn't previously need .. are now needed in > the modules compilation. > > I had recommended that this go into test on Friday via IRC, but > there's no harm in doing a cleaned up v3 with a class global set > of exports. We can always apply it post release if it is decided there > is extra risk .. but at least it will be a reference to how we should > start consolidating some of the exports. Since v2 has been already merged, I sent out v3 as just a new patch on the top of the lastet master branch. Thanks, Munehisa > Bruce > > > > > Thanks, > > Munehisa > > > > > Bruce > > > > > > > > > > > Signed-off-by: Munehisa Kamata <kamatam@amazon.com> > > > > --- > > > > > > > > v1 -> v2: Rewrote the commit message with the repro steps > > > > > > > > meta/classes-recipe/kernel.bbclass | 12 +++++++++--- > > > > 1 file changed, 9 insertions(+), 3 deletions(-) > > > > > > > > diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass > > > > index a76aaee5ba..db4461e551 100644 > > > > --- a/meta/classes-recipe/kernel.bbclass > > > > +++ b/meta/classes-recipe/kernel.bbclass > > > > @@ -239,6 +239,8 @@ KERNEL_EXTRA_ARGS ?= "" > > > > EXTRA_OEMAKE += ' CC="${KERNEL_CC}" LD="${KERNEL_LD}" OBJCOPY="${KERNEL_OBJCOPY}" STRIP="${KERNEL_STRIP}"' > > > > EXTRA_OEMAKE += ' HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOSTLDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}"' > > > > EXTRA_OEMAKE += ' HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}"' > > > > +# Only for newer kernels (5.19+), native pkg-config variables are set for older kernels when building kernel and modules > > > > +EXTRA_OEMAKE += ' HOSTPKG_CONFIG="pkg-config-native"' > > > > > > > > KERNEL_ALT_IMAGETYPE ??= "" > > > > > > > > @@ -356,9 +358,6 @@ kernel_do_compile() { > > > > export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > > > export PKG_CONFIG_SYSROOT_DIR="" > > > > > > > > - # for newer kernels (5.19+) there's a dedicated variable > > > > - export HOSTPKG_CONFIG="pkg-config-native" > > > > - > > > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > > > # be set.... > > > > @@ -408,6 +407,13 @@ addtask transform_kernel after do_compile before do_install > > > > > > > > do_compile_kernelmodules() { > > > > unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE > > > > + > > > > + # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native) > > > > + export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig" > > > > + export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig" > > > > + export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR" > > > > + export PKG_CONFIG_SYSROOT_DIR="" > > > > + > > > > if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then > > > > # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not > > > > # be set.... > > > > -- > > > > 2.34.1 > > > > > > > > > > > > > -- > > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > > thee at its end > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-02-26 23:50 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-02-22 20:28 [PATCH] kernel.bbclass: Set pkg-config variables for building modules Munehisa Kamata 2024-02-22 21:38 ` [OE-core] " Bruce Ashfield 2024-02-23 8:05 ` [PATCH v2] " Munehisa Kamata 2024-02-23 14:09 ` Bruce Ashfield 2024-02-24 7:21 ` Munehisa Kamata 2024-02-25 14:54 ` Bruce Ashfield 2024-02-26 23:50 ` Munehisa Kamata
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox