Re: [OE-core] [kirkstone][PATCH 3/3] kernel: make LOCALVERSION consistent between recipes

[Ryan Eatmon <reatmon@ti.com>]

On 2/22/2024 4:46 PM, Ryan Eatmon via lists.openembedded.org wrote:
> 
> 
> On 2/22/2024 3:56 PM, Bruce Ashfield wrote:
>> On Thu, Feb 22, 2024 at 4:51 PM Ryan Eatmon via lists.openembedded.org
>> <reatmon=ti.com@lists.openembedded.org> wrote:
>>>
>>>
>>>
>>> On 2/22/2024 4:35 AM, Andreas Helbech Kleist wrote:
>>>> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>>>>
>>>> The initial fix for localversion setting in 6.3+ broke older
>>>> recipes and also broke recipes setting localversion in a kernel
>>>> recipe, as make-mod-scripts (and other locations) can trigger
>>>> a regeneration of files and don't have access to the variable.
>>>>
>>>> Moving the setting of this variable to the global namespace
>>>> doesn't make sense, so we follow the example of the kernel-abiversion
>>>> and save a kernel-localversion to the build artifacts.
>>>>
>>>> Recipes that may regenerate scripts/dynamic files, must
>>>> depend on the do_shared_workedir of the kernel and use the helper
>>>> function to read the file storing the localversion.
>>>>
>>>> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
>>>> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
>>>>
>>>> cherry-picked from master b378eec156998eea55ba61e59103cb34fab0d07c
>>>>
>>>> Signed-off-by: Andreas Helbech Kleist <andreaskleist@gmail.com>
>>>
>>> I'm not sure what is going on with our meta-ti testing.  This set of
>>> changes causes the same issue we were seeing before.  Plus, now that I'm
>>> looking into it deeper, I'm seeing a breakage in master related to these
>>> same patches on there.  Not sure why the changes were not caught in our
>>> testing on master back when the patches were accepted there...  Clearly
>>> I need some better checks on the images as part of our QC process...
>>>
>>> I need to figure out how to fix the issue on master and see if the same
>>> change can be applied to these patches on kirkstone.  I'll try and get
>>> all this done as quickly as I can.
>>
>> Please make sure to copy me directly, both as the maintainer of the
>> parts of core in question and the author of the patches.
>>
>> We've had no reports of issues since any of these went in, and they
>> aren't exactly a recent change. Which obviously means there's
>> potentially something different in the environment or the kernel
>> recipes.
>>
>> If you get a set of reproducing steps with linux-yocto and core, I
>> can lend a hand with any fixes.
>>
>> Bruce
> 
> This may not be an issue with these patches per se.  It may be an issue 
> with our kernel recipes in meta-ti.  But since kirkstone is our primary 
> release vector right now, it is breaking us to accept these patches onto 
> kirkstone.  I fully acknowledge that this might solely a TI issue.  And 
> now that we are aware of the issue we are going to work very very hard 
> to find a solution as quickly as we can.
> 
> As soon as we have an understanding of how to recreate the issue outside 
> of our recipe we will definitely share with this thread.
> 

Ok.  My initial thinking that there was a problem was incorrect.  I 
looked at the file system and found our value for KERNEL_LOCALVERSION 
was included in the modules directory name twice.

/lib/modules/6.1.69-gcb84067eaf83-gcb84067eaf83

That was the symptom that caused the issue with our request for the 
revert last week.  However, with this full patch series the problem 
caused by the double entry from the last series is not a problem because 
the double entry is everywhere in the kernel, so it all works together. 
It's just goofy.

So with that.  This bigger series can be accepted into Kirkstone.

Acked-by: Ryan Eatmon <reatmon@ti.com>



Now for the second issue, and maybe this was known that it would occur, 
if we set KERNEL_LOCALVERSION it is placed into both the .scmversion 
file AND the LOCALVERSION variable.  The setlocalversion script in the 
6.1 kernel we are currently pointing at for our production includes both 
in the version string, which causes the entire kernel to be named named 
with a redundant string.

/lib/modules/6.1.69-gcb84067eaf83-gcb84067eaf83

I guess we can live with this.  It is unnecessary, but not incorrect. 
It is cleaned up with the next LTS kernel 6.6.  This is just the hazard 
of only having a single class to handle kernel issues that needs to span 
multiple kernel versions that are changing things up in big ways.

The only way I can see to fix it would be to create some new control 
variables that recipes can set to control the above behavior and then 
set defaults for them.

But we can look into patches to add that if needed.

Apologies for taking so much time in testing this series.  We are just 
about to make a release and things that break the kernel cause us a lot 
anxiety.


>>>
>>>
>>>
>>>> ---
>>>>    meta/classes/kernel-arch.bbclass                   |  8 --------
>>>>    meta/classes/kernel.bbclass                        | 14 
>>>> ++++++++++++++
>>>>    meta/classes/kernelsrc.bbclass                     |  1 +
>>>>    meta/classes/linux-kernel-base.bbclass             | 11 +++++++++++
>>>>    meta/classes/module-base.bbclass                   |  1 +
>>>>    .../make-mod-scripts/make-mod-scripts_1.0.bb       |  3 +++
>>>>    6 files changed, 30 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/meta/classes/kernel-arch.bbclass 
>>>> b/meta/classes/kernel-arch.bbclass
>>>> index 0a79dea0af..62c8211621 100644
>>>> --- a/meta/classes/kernel-arch.bbclass
>>>> +++ b/meta/classes/kernel-arch.bbclass
>>>> @@ -65,11 +65,3 @@ KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc 
>>>> ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DE
>>>>    KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}"
>>>>    KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}"
>>>>    TOOLCHAIN ?= "gcc"
>>>> -
>>>> -# 6.3+ requires the variable LOCALVERSION to be set to not get a 
>>>> "+" in
>>>> -# the local version. Having it empty means nothing will be added, 
>>>> and any
>>>> -# value will be appended to the local kernel version. This replaces 
>>>> the
>>>> -# use of .scmversion file for setting a localversion without using
>>>> -# the CONFIG_LOCALVERSION option.
>>>> -KERNEL_LOCALVERSION ??= ""
>>>> -export LOCALVERSION ?= "${KERNEL_LOCALVERSION}"
>>>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>>>> index 940f1a3cf4..96e41b5192 100644
>>>> --- a/meta/classes/kernel.bbclass
>>>> +++ b/meta/classes/kernel.bbclass
>>>> @@ -541,6 +541,7 @@ do_shared_workdir () {
>>>>        #
>>>>
>>>>        echo "${KERNEL_VERSION}" > 
>>>> $kerneldir/${KERNEL_PACKAGE_NAME}-abiversion
>>>> +     echo "${KERNEL_LOCALVERSION}" > 
>>>> $kerneldir/${KERNEL_PACKAGE_NAME}-localversion
>>>>
>>>>        # Copy files required for module builds
>>>>        cp System.map $kerneldir/System.map-${KERNEL_VERSION}
>>>> @@ -630,6 +631,19 @@ python check_oldest_kernel() {
>>>>    check_oldest_kernel[vardepsexclude] += "OLDEST_KERNEL 
>>>> KERNEL_VERSION"
>>>>    do_configure[prefuncs] += "check_oldest_kernel"
>>>>
>>>> +KERNEL_LOCALVERSION ??= ""
>>>> +
>>>> +# 6.3+ requires the variable LOCALVERSION to be set to not get a 
>>>> "+" in
>>>> +# the local version. Having it empty means nothing will be added, 
>>>> and any
>>>> +# value will be appended to the local kernel version. This replaces 
>>>> the
>>>> +# use of .scmversion file for setting a localversion without using
>>>> +# the CONFIG_LOCALVERSION option.
>>>> +#
>>>> +# Note: This class saves the value of localversion to a file
>>>> +# so other recipes like make-mod-scripts can restore it via the
>>>> +# helper function get_kernellocalversion_file
>>>> +export LOCALVERSION="${KERNEL_LOCALVERSION}"
>>>> +
>>>>    kernel_do_configure() {
>>>>        # fixes extra + in /lib/modules/2.6.37+
>>>>        # $ scripts/setlocalversion . => +
>>>> diff --git a/meta/classes/kernelsrc.bbclass 
>>>> b/meta/classes/kernelsrc.bbclass
>>>> index a951ba3325..a79bf18b09 100644
>>>> --- a/meta/classes/kernelsrc.bbclass
>>>> +++ b/meta/classes/kernelsrc.bbclass
>>>> @@ -5,6 +5,7 @@ do_patch[depends] += "virtual/kernel:do_shared_workdir"
>>>>    do_patch[noexec] = "1"
>>>>    do_package[depends] += "virtual/kernel:do_populate_sysroot"
>>>>    KERNEL_VERSION = 
>>>> "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}"
>>>> +LOCAL_VERSION = 
>>>> "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}"
>>>>
>>>>    inherit linux-kernel-base
>>>>
>>>> diff --git a/meta/classes/linux-kernel-base.bbclass 
>>>> b/meta/classes/linux-kernel-base.bbclass
>>>> index 73a6fe36d9..0e2a4a4abe 100644
>>>> --- a/meta/classes/linux-kernel-base.bbclass
>>>> +++ b/meta/classes/linux-kernel-base.bbclass
>>>> @@ -33,6 +33,17 @@ def get_kernelversion_file(p):
>>>>        except IOError:
>>>>            return None
>>>>
>>>> +def get_kernellocalversion_file(p):
>>>> +    fn = p + '/kernel-localversion'
>>>> +
>>>> +    try:
>>>> +        with open(fn, 'r') as f:
>>>> +            return f.readlines()[0].strip()
>>>> +    except IOError:
>>>> +        return ""
>>>> +
>>>> +    return ""
>>>> +
>>>>    def linux_module_packages(s, d):
>>>>        suffix = ""
>>>>        return " ".join(map(lambda s: "kernel-module-%s%s" % 
>>>> (s.lower().replace('_', '-').replace('@', '+'), suffix), s.split()))
>>>> diff --git a/meta/classes/module-base.bbclass 
>>>> b/meta/classes/module-base.bbclass
>>>> index 27bd69ff33..5b2fde8144 100644
>>>> --- a/meta/classes/module-base.bbclass
>>>> +++ b/meta/classes/module-base.bbclass
>>>> @@ -14,6 +14,7 @@ export CROSS_COMPILE = "${TARGET_PREFIX}"
>>>>    export KBUILD_OUTPUT = "${STAGING_KERNEL_BUILDDIR}"
>>>>
>>>>    export KERNEL_VERSION = 
>>>> "${@oe.utils.read_file('${STAGING_KERNEL_BUILDDIR}/kernel-abiversion')}"
>>>> +export LOCALVERSION = 
>>>> "${@oe.utils.read_file('${STAGING_KERNEL_BUILDDIR}/kernel-localversion')}"
>>>>    KERNEL_OBJECT_SUFFIX = ".ko"
>>>>
>>>>    # kernel modules are generally machine specific
>>>> diff --git 
>>>> a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb 
>>>> b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
>>>> index f6f47cfff5..8727d003f9 100644
>>>> --- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
>>>> +++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
>>>> @@ -21,6 +21,9 @@ DEPENDS += "gmp-native"
>>>>    EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} 
>>>> ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
>>>>    EXTRA_OEMAKE += " HOSTCXX="${BUILD_CXX} ${BUILD_CXXFLAGS} 
>>>> ${BUILD_LDFLAGS}" CROSS_COMPILE=${TARGET_PREFIX}"
>>>>
>>>> +KERNEL_LOCALVERSION = 
>>>> "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}"
>>>> +export LOCALVERSION="${KERNEL_LOCALVERSION}"
>>>> +
>>>>    # Build some host tools under work-shared.  CC, LD, and AR are 
>>>> probably
>>>>    # not used, but this is the historical way of invoking "make 
>>>> scripts".
>>>>    #
>>>
>>> -- 
>>> Ryan Eatmon                reatmon@ti.com
>>> -----------------------------------------
>>> Texas Instruments, Inc.  -  LCPD  -  MGTS
>>>
>>>
>>>
>>
>>
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#196045): https://lists.openembedded.org/g/openembedded-core/message/196045
> Mute This Topic: https://lists.openembedded.org/mt/104505983/6551054
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [reatmon@ti.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

-- 
Ryan Eatmon                reatmon@ti.com
-----------------------------------------
Texas Instruments, Inc.  -  LCPD  -  MGTS