[RFC PATCH 0/3] Add a mechanism to check runtime and recipe version mismatch

[RFC PATCH 0/3] Add a mechanism to check runtime and recipe version mismatch

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

This RFC add a mechanims to check runtime and recipe version mismatch.
For example, if some tool says its runtime version is 1.0, but its recipe version is 1.1,
we can now detect it at build time.

Let me explain the most important thing in this cover letter: why.

Briefly speaking, there are two major benefits:
1) Make things look better.
2) Detect potential problems.

See more detailed explanations below.

When people use some software they care about, they would type "--version" or something similar
to check its version. And if the output does not match the package version that is installed, or
there's some "bad" words in output such as "unknown", "dirty", "dev", they would raise questions in
their minds: Does the vendor really know what it's building? Am I using stable versions? If they can't
even get versions right, how can I trust them on other things?

So here comes the first reason on adding such a mechanim: we can detect version mismatch, we can then
fix them. This makes things look better. Patch 3/3 is an example fix.

Another reason, which might be a more important reason, is that it helps us defect potential problems.
First of all, version information sometimes matters more than just showing user some info. It affects
runtime behavior. A typical example is k8s. If k8s is built with correct version info, it will try to
pull major.minor version compatible images and do its work. But if it's built with a wrong version,
either init fails or it pulls some incompatible images. Secondly, a version mismatch indicates some
potential error. Use sassc an an example. It got a version mismatch because it's not using the header
file generated under build directory. Instead, it uses the default one with the same name in source
directory. Luckily, that header file only contains version info. But this case demonstrates that
a version mismatch actually indicates some error.

During development, I found the errors that version mismatch indicates fall into the following categories:
1) The upstream project made a mistake.
   e.g.,
   libuser got a mismatch because upstream project forgets to update configure.ac/configure files.
2) The recipe made a mistake.
   e.g.,
   python3-pyyaml-include_2.2.bb got a mismatch because it's srcrev actually points to v2.2a1 tag instead
   of the v2.2 tag.
3) Some corner case will raise a mismatch that normal build does not see.
   e.g.,
   If we build libsass from git shallow tarball, we got a mismatch. The "Version" field in .pc file is
   7073 which is wrong. In a normal build, the "Version" field in the .pc file is 3.6.6 which is correct.

With this mechanism added, we can detect such problems and fix them in time.

Hope to hear your opinions.

Regards,
Qi


--- Result for oe-core world build (11 mismatches) ---
WARNING: python3-unittest-automake-output-0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['0.1'] do not match recipe version 0.2
WARNING: time-1.9-r0 do_package_check_version_mismatch: Possible runtime versions ['UNKNOWN'] do not match recipe version 1.9
WARNING: pinentry-1.3.1-r0 do_package_check_version_mismatch: Possible runtime versions ['1.3.1-unknown'] do not match recipe version 1.3.1
WARNING: gnupg-2.5.5-r0 do_package_check_version_mismatch: Possible runtime versions ['1.11.0-unknown', '1.6.7-unknown', '2.5.5-unknown'] do not match recipe version 2.5.5
WARNING: dpkg-1.22.11-r0 do_package_check_version_mismatch: Possible runtime versions ['1.22.11-dirty'] do not match recipe version 1.22.11
WARNING: libgpg-error-1.51-r0 do_package_check_version_mismatch: Possible runtime versions ['1.51-unknown'] do not match recipe version 1.51
WARNING: sassc-3.6.2-r0 do_package_check_version_mismatch: Possible runtime versions ['1.1.1', '3.5'] do not match recipe version 3.6.2
WARNING: libassuan-3.0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['3.0.2-unknown'] do not match recipe version 3.0.2
WARNING: libksba-1.6.7-r0 do_package_check_version_mismatch: Possible runtime versions ['1.6.7-unknown'] do not match recipe version 1.6.7
WARNING: libgcrypt-1.11.0-r0 do_package_check_version_mismatch: Possible runtime versions ['1.11.0-unknown'] do not match recipe version 1.11.0
WARNING: glslang-1_1.3.296.0-r0 do_package_check_version_mismatch: Possible runtime versions ['0.97', '15.0.0', '3.20', '4.60'] do not match recipe version 1.3.296.0
----------------------------------------


The following changes since commit 6610cad12a062592956257961a790ec6a3012b8b:

  bitbake: data_smart: Ensure module dependency changes invalidate the base config cache (2025-03-13 21:21:04 +0000)

are available in the Git repository at:

  https://github.com/ChenQi1989/poky version-check
  https://github.com/ChenQi1989/poky/tree/version-check

Chen Qi (3):
  version-check.conf: add mechanism for checking version mismatch
  mc: fix buildpath QA regarding unzip
  time: fix runtime version from UNKNOWN to 1.9

 meta/classes/check-version-mismatch.bbclass   | 399 ++++++++++++++++++
 meta/{classes-recipe => classes}/qemu.bbclass |   0
 meta/conf/version-check.conf                  |  14 +
 meta/recipes-extended/mc/mc_4.8.33.bb         |   1 +
 meta/recipes-extended/time/time_1.9.bb        |   6 +
 5 files changed, 420 insertions(+)
 create mode 100644 meta/classes/check-version-mismatch.bbclass
 rename meta/{classes-recipe => classes}/qemu.bbclass (100%)
 create mode 100644 meta/conf/version-check.conf

-- 
2.34.1

[RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

Add a mechanism to check mismatch between runtime version and build time version.

To use, add the following line to local.conf:

  include conf/version-check.conf

The basic idea is to use qemu to run executables at build time, extract
possible versions, and check if there's a mismatch found.

Python meta data and .pc files are also checked for quick match. This
is because such info are also easy to be checked by users.

check-version-mismatch.bbclass is the class that does the actual work.
A new variable, CHECK_VERSION_PV, is introduced. It defaults to ${PKGV},
but also allows override. This allows us to handle special cases in each
layer.

version-check.conf is the configuration file that makes this functionality
easier to use and draws some baseline. It contains some override settings
for some recipes. With these overrides, all recipes in oe-core are handled
well. All warnings are valid warnings.

Note that 'ps' is added to HOSTTOOLS in version-check.conf. This is because
we need 'ps' to find stale processes and then clean them.

The warnings are like below:

  WARNING: time-1.9-r0 do_package_check_version_mismatch: Possible runtime versions ['UNKNOWN'] do not match recipe version 1.9
  WARNING: python3-unittest-automake-output-0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['0.1'] do not match recipe version 0.2
  WARNING: pinentry-1.3.1-r0 do_package_check_version_mismatch: Possible runtime versions ['1.3.1-unknown'] do not match recipe version 1.3.1
  ...

There will be a data directory containing all details: tmp/check-version-mismatch.
This directory contains detailed data for each recipe that is built.
If users don't want it, they can set DEBUG_VERSION_MISMATCH_CHECK to 0.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/check-version-mismatch.bbclass   | 399 ++++++++++++++++++
 meta/{classes-recipe => classes}/qemu.bbclass |   0
 meta/conf/version-check.conf                  |  14 +
 3 files changed, 413 insertions(+)
 create mode 100644 meta/classes/check-version-mismatch.bbclass
 rename meta/{classes-recipe => classes}/qemu.bbclass (100%)
 create mode 100644 meta/conf/version-check.conf

diff --git a/meta/classes/check-version-mismatch.bbclass b/meta/classes/check-version-mismatch.bbclass
new file mode 100644
index 0000000000..fac24ad45b
--- /dev/null
+++ b/meta/classes/check-version-mismatch.bbclass
@@ -0,0 +1,399 @@
+inherit qemu
+
+ENABLE_VERSION_MISMATCH_CHECK ?= "${@'1' if bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', True, False, d) else '0'}"
+DEBUG_VERSION_MISMATCH_CHECK ?= "1"
+CHECK_VERSION_PV ?= ""
+
+DEPENDS:append:class-target = "${@' qemu-native' if bb.utils.to_boolean(d.getVar('ENABLE_VERSION_MISMATCH_CHECK')) else ''}"
+
+QEMU_EXEC ?= "${@qemu_wrapper_cmdline(d, '${STAGING_DIR_HOST}', ['${STAGING_DIR_HOST}${libdir}','${STAGING_DIR_HOST}${base_libdir}', '${PKGD}${libdir}', '${PKGD}${base_libdir}'])}"
+
+python do_package_check_version_mismatch() {
+    import re
+    import subprocess
+    import shutil
+    import signal
+
+    classes_skip = ["nopackage", "image", "native", "cross", "crosssdk", "cross-canadian"]
+    for cs in classes_skip:
+        if bb.data.inherits_class(cs, d):
+            bb.note(f"Skip do_package_check_version_mismatch as {cs} is inherited.")
+            return
+
+    if not bb.utils.to_boolean(d.getVar('ENABLE_VERSION_MISMATCH_CHECK')):
+        bb.note("Skip do_package_check_version_mismatch as ENABLE_VERSION_MISMATCH_CHECK is disabled.")
+        return
+
+    __regexp_version_broad_match__ = re.compile(r"(?:\s|^|-|_|/|=| go|\()" +
+                                                r"(?P<version>v?[0-9][0-9.][0-9+.\-_~\(\)]*?|UNKNOWN)" +
+                                                r"(?:-release|-stable|)" +
+                                                r"(?P<extra>[+\-]unknown|[+\-]dirty|[+\-]rc?\d{1,3}|\+cargo-[0-9.]+|" +
+                                                r"[a-z]|-?[pP][0-9]{1,3}|-beta[^\s]*|-alpha[^\s]*|)" +
+                                                r"(?P<extra2>[+\-]dev|)" +
+                                                r"(?:,|:|\.|\)|)" +
+                                                r"(?=\s|$)"
+                                                )
+    __regexp_exclude_year__ = re.compile(r"^(19|20)[0-9]{2}$")
+    __regexp_single_number_ending_with_dot__ = re.compile(r"^\d\.$")
+
+    def is_shared_library(filepath):
+        return re.match(r'.*\.so(\.\d+)*$', filepath) is not None
+
+    def get_possible_versions(output_contents, full_cmd=None, max_lines=None):
+        #
+        # Algorithm:
+        #   1. Check version line by line.
+        #   2. Skip some lines which we know that do not contain version information, e.g., License, Copyright.
+        #   3. Do broad match, finding all possible versions.
+        #   4. If there's a version found by any match, do exclude match (e.g., exclude years)
+        #   5. If there's a valid version, do stripping and converting and then add to possible_versions.
+        #   6. Return possible_versions
+        #
+        possible_versions = []
+        content_lines = output_contents.split("\n")
+        if max_lines:
+            content_lines = content_lines[0:max_lines]
+        if full_cmd:
+            base_cmd = os.path.basename(full_cmd)
+        __regex_help_format__ = re.compile(r"-[^\s].*")
+        for line in content_lines:
+            line = line.strip()
+            # skip help lines
+            if __regex_help_format__.match(line):
+                continue
+            # avoid command itself affecting output
+            if full_cmd:
+                if line.startswith(base_cmd):
+                    line = line[len(base_cmd):]
+                elif line.startswith(full_cmd):
+                    line = line[len(full_cmd):]
+            # skip specific lines
+            skip_keywords_start = ["Copyright", "License"]
+            skip_line = False
+            for sks in skip_keywords_start:
+                if line.startswith(sks):
+                    skip_line = True
+                    break
+            if skip_line:
+                continue
+
+            # try broad match
+            for match in __regexp_version_broad_match__.finditer(line):
+                version = match.group("version")
+                #print(f"version = {version}")
+                # do exclude match
+                exclude_match = __regexp_exclude_year__.match(version)
+                if exclude_match:
+                    continue
+                exclude_match = __regexp_single_number_ending_with_dot__.match(version)
+                if exclude_match:
+                    continue
+                # do some stripping and converting
+                if version.startswith("("):
+                    version = version[1:-1]
+                if version.startswith("v"):
+                    version = version[1:]
+                if version.endswith(")") and "(" not in version:
+                    version = version[:-1]
+                # handle extra version info
+                version = version + match.group("extra") + match.group("extra2")
+                possible_versions.append(version)
+        return possible_versions
+
+    def is_version_mismatch(rvs, pv):
+        got_match = False
+        if pv.startswith("git"):
+            return False
+        if "-pre" in pv:
+            pv = pv.split("-pre")[0]
+        if pv.startswith("v"):
+            pv = pv[1:]
+        for rv in rvs:
+            if rv == pv:
+                got_match = True
+                break
+            pv = pv.split("+git")[0]
+            # handle % character in pv which means matching any chars
+            if '%' in pv:
+                escaped_pv = re.escape(pv)
+                regex_pattern = escaped_pv.replace('%', '.*')
+                regex_pattern = f'^{regex_pattern}$'
+                if re.fullmatch(regex_pattern, rv):
+                    got_match = True
+                    break
+                else:
+                    continue
+            # handle cases such as 2.36.0-r0 v.s. 2.36.0
+            if "-r" in rv:
+                rv = rv.split("-r")[0]
+            chars_to_replace = ["-", "+", "_", "~"]
+            # convert to use "." as the version seperator
+            for cr in chars_to_replace:
+                rv = rv.replace(cr, ".")
+                pv = pv.replace(cr, ".")
+            if rv == pv:
+                got_match = True
+                break
+            # handle case such as 5.2.37(1) v.s. 5.2.37
+            if "(" in rv:
+                rv = rv.split("(")[0]
+                if rv == pv:
+                    got_match = True
+                    break
+            # handle case such as 4.4.3p1
+            if "p" in pv and "p" in rv.lower():
+                pv = pv.lower().replace(".p", "p")
+                rv = rv.lower().replace(".p", "p")
+                if pv == rv:
+                    got_match = True
+                    break
+            # handle cases such as 6.00 v.s. 6.0
+            if rv.startswith(pv):
+                if rv == pv + "0" or rv == pv + ".0":
+                    got_match = True
+                    break
+            elif pv.startswith(rv):
+                if pv == rv + "0" or pv == rv + ".0":
+                    got_match = True
+                    break
+            # handle cases such as 21306 v.s. 2.13.6
+            if "." in pv and not "." in rv:
+                pv_components = pv.split(".")
+                if rv.startswith(pv_components[0]):
+                    pv_num = 0
+                    for i in range(0, len(pv_components)):
+                        pv_num = pv_num * 100 + int(pv_components[i])
+                    if pv_num == int(rv):
+                        got_match = True
+                        break
+        if got_match:
+            return False
+        else:
+            return True
+
+    # helper function to get PKGV, useful for recipes such as perf
+    def get_pkgv(pn):
+        pkgdestwork = d.getVar("PKGDESTWORK")
+        recipe_data_fn = pkgdestwork + "/" + pn
+        pn_data = oe.packagedata.read_pkgdatafile(recipe_data_fn)
+        if not "PACKAGES" in pn_data:
+            return d.getVar("PV")
+        packages = pn_data["PACKAGES"].split()
+        for pkg in packages:
+            pkg_fn = pkgdestwork + "/runtime/" + pkg
+            pkg_data = oe.packagedata.read_pkgdatafile(pkg_fn)
+            if "PKGV" in pkg_data:
+                return pkg_data["PKGV"]
+
+    #
+    # traverse PKGD, find executables and run them to get runtime version information and compare it with recipe version information
+    #
+    enable_debug = bb.utils.to_boolean(d.getVar("DEBUG_VERSION_MISMATCH_CHECK"))
+    pkgd = d.getVar("PKGD")
+    pn = d.getVar("PN")
+    pv = d.getVar("CHECK_VERSION_PV")
+    if not pv:
+        pv = get_pkgv(pn)
+    qemu_exec = d.getVar("QEMU_EXEC").strip()
+    executables = []
+    possible_versions_all = []
+    data_lines = []
+
+    if enable_debug:
+        debug_directory = d.getVar("TMPDIR") + "/check-version-mismatch"
+        debug_data_file = debug_directory + "/" + pn
+        os.makedirs(debug_directory, exist_ok=True)
+        data_lines.append("pv: %s\n" % pv)
+
+    got_quick_match_result = False
+    # handle python3-xxx recipes quickly
+    __regex_python_module_version__ = re.compile(r"(?:^|.*:)Version: (?P<version>.*)$")
+    if "python3-" in pn:
+        version_check_cmd = "find %s -name 'METADATA' | xargs grep '^Version: '" % pkgd
+        try:
+            output = subprocess.check_output(version_check_cmd, shell=True).decode("utf-8")
+            data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+            data_lines.append("output:\n'''\n%s'''\n" % output)
+            possible_versions = []
+            for line in output.split("\n"):
+                match = __regex_python_module_version__.match(line)
+                if match:
+                    possible_versions.append(match.group("version"))
+            possible_versions = sorted(set(possible_versions))
+            data_lines.append("possible versions: %s\n" % possible_versions)
+            if is_version_mismatch(possible_versions, pv):
+                data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+                bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions, pv))
+            else:
+                data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+            got_quick_match_result = True
+        except:
+            data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+            data_lines.append("result: RUN_FAILED\n\n")
+    if got_quick_match_result:
+        if enable_debug:
+            with open(debug_data_file, "w") as f:
+                f.writelines(data_lines)
+        return
+
+    # handle .pc files
+    version_check_cmd = "find %s -name '*.pc' | xargs grep -i version" % pkgd
+    try:
+        output = subprocess.check_output(version_check_cmd, shell=True).decode("utf-8")
+        data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+        data_lines.append("output:\n'''\n%s'''\n" % output)
+        possible_versions = get_possible_versions(output)
+        possible_versions = sorted(set(possible_versions))
+        data_lines.append("possible versions: %s\n" % possible_versions)
+        if is_version_mismatch(possible_versions, pv):
+            if pn.startswith("lib"):
+                data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+                bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions, pv))
+                got_quick_match_result = True
+            else:
+                data_lines.append("result: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+        else:
+            data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+            got_quick_match_result = True
+    except:
+        data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+        data_lines.append("result: RUN_FAILED\n\n")
+    if got_quick_match_result:
+        if enable_debug:
+            with open(debug_data_file, "w") as f:
+                f.writelines(data_lines)
+        return
+
+    skipped_directories = [".debug", "ptest", "installed-tests", "tests", "test", "__pycache__", "testcases"]
+    pkgd_libdir = pkgd + d.getVar("libdir")
+    pkgd_base_libdir = pkgd + d.getVar("base_libdir")
+    extra_exec_libdirs = []
+    for root, dirs, files in os.walk(pkgd):
+        for dname in dirs:
+            fdir = os.path.join(root, dname)
+            if os.path.isdir(fdir) and fdir != pkgd_libdir and fdir != pkgd_base_libdir:
+                if fdir.startswith(pkgd_libdir) or fdir.startswith(pkgd_base_libdir):
+                    for sd in skipped_directories:
+                        if fdir.endswith("/" + sd) or ("/" + sd + "/") in fdir:
+                            break
+                    else:
+                        extra_exec_libdirs.append(fdir)
+        for fname in files:
+            fpath = os.path.join(root, fname)
+            if os.path.isfile(fpath) and os.access(fpath, os.X_OK):
+                for sd in skipped_directories:
+                    if ("/" + sd + "/") in fpath:
+                        break
+                else:
+                    if is_shared_library(fpath):
+                        # we don't check shared libraries
+                        continue
+                    else:
+                        executables.append(fpath)
+    if enable_debug:
+        data_lines.append("executables: %s\n" % executables)
+
+    found_match = False
+    some_cmd_succeed = False
+    if not executables:
+        bb.debug(1, "No executable found for %s" % pn)
+        data_lines.append("FINAL RESULT: NO_EXECUTABLE_FOUND\n\n")
+    else:
+        # first we extend qemu_exec to include library path if needed
+        if extra_exec_libdirs:
+            qemu_exec += ":" + ":".join(extra_exec_libdirs)
+        for fexec in executables:
+            for version_option in ["--version", "-V", "-v", "--help"]:
+                version_check_cmd_full = "%s %s %s" % (qemu_exec, fexec, version_option)
+                version_check_cmd = version_check_cmd_full
+                #version_check_cmd = "%s %s" % (os.path.relpath(fexec, pkgd), version_option)
+                
+                try:
+                    cwd_temp = d.getVar("TMPDIR") + "/check-version-mismatch/cwd-temp/" + pn
+                    os.makedirs(cwd_temp, exist_ok=True)
+                    # avoid pseudo to manage any file we create
+                    sp_env = os.environ.copy()
+                    sp_env["PSEUDO_UNLOAD"] = "1"
+                    output = subprocess.check_output(version_check_cmd_full,
+                                                     shell=True,
+                                                     stderr=subprocess.STDOUT,
+                                                     cwd=cwd_temp,
+                                                     timeout=10,
+                                                     env=sp_env).decode("utf-8")
+                    some_cmd_succeed = True
+                    data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+                    data_lines.append("output:\n'''\n%s'''\n" % output)
+                    if version_option == "--help":
+                        max_lines = 5
+                    else:
+                        max_lines = None
+                    possible_versions = get_possible_versions(output, full_cmd=fexec, max_lines=max_lines)
+                    if "." in pv:
+                        possible_versions = [item for item in possible_versions if "." in item or item == "UNKNOWN"]
+                    data_lines.append("possible versions: %s\n" % possible_versions)
+                    if not possible_versions:
+                        data_lines.append("result: NO_RUNTIME_VERSION_FOUND\n\n")
+                        continue
+                    possible_versions_all.extend(possible_versions)
+                    possible_versions_all = sorted(set(possible_versions_all))
+                    if is_version_mismatch(possible_versions, pv):
+                        data_lines.append("result: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+                    else:
+                        found_match = True
+                        data_lines.append("result: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv))
+                        break
+                except:
+                    data_lines.append("version_check_cmd: %s\n" % version_check_cmd)
+                    data_lines.append("result: RUN_FAILED\n\n")
+                finally:
+                    shutil.rmtree(cwd_temp)
+            if found_match:
+                break
+    if executables:
+        if found_match:
+            data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n" % (possible_versions_all, pv))
+        elif len(possible_versions_all) == 0:
+            if some_cmd_succeed:
+                bb.debug(1, "No valid runtime version found")
+                data_lines.append("FINAL RESULT: NO_VALID_RUNTIME_VERSION_FOUND\n")
+            else:
+                bb.debug(1, "All version check command failed")
+                data_lines.append("FINAL RESULT: RUN_FAILED\n")
+        else:
+            bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions_all, pv))
+            data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n" % (possible_versions_all, pv))
+
+    if enable_debug:
+        with open(debug_data_file, "w") as f:
+            f.writelines(data_lines)
+
+    # clean up stale processes
+    process_name_common_prefix = "%s %s" % (' '.join(qemu_exec.split()[1:]), pkgd)
+    find_stale_process_cmd = "ps -e -o pid,args | grep -v grep | grep -F '%s'" % process_name_common_prefix
+    try:
+        stale_process_output = subprocess.check_output(find_stale_process_cmd, shell=True).decode("utf-8")
+        stale_process_pids = []
+        for line in stale_process_output.split("\n"):
+            line = line.strip()
+            if not line:
+                continue
+            pid = line.split()[0]
+            stale_process_pids.append(pid)
+        for pid in stale_process_pids:
+            os.kill(int(pid), signal.SIGKILL)
+    except Exception as e:
+        bb.debug(1, "No stale process")
+}
+
+addtask do_package_check_version_mismatch after do_package before do_build
+
+do_build[rdeptask] += "do_package_check_version_mismatch"
+do_rootfs[recrdeptask] += "do_package_check_version_mismatch"
+
+SSTATETASKS += "do_package_check_version_mismatch"
+do_package_check_version_mismatch[sstate-inputdirs] = ""
+do_package_check_version_mismatch[sstate-outputdirs] = ""
+python do_package_check_version_mismatch_setscene () {
+    sstate_setscene(d)
+}
+addtask do_package_check_version_mismatch_setscene
diff --git a/meta/classes-recipe/qemu.bbclass b/meta/classes/qemu.bbclass
similarity index 100%
rename from meta/classes-recipe/qemu.bbclass
rename to meta/classes/qemu.bbclass
diff --git a/meta/conf/version-check.conf b/meta/conf/version-check.conf
new file mode 100644
index 0000000000..154bc4a637
--- /dev/null
+++ b/meta/conf/version-check.conf
@@ -0,0 +1,14 @@
+INHERIT += "check-version-mismatch"
+# we need ps command to clean stale processes
+HOSTTOOLS += "ps"
+
+# Special cases that need to be handled.
+# % has the same meaning as in bbappend files, that is, match any chars.
+CHECK_VERSION_PV:pn-rust-llvm = "${LLVM_RELEASE}"
+CHECK_VERSION_PV:pn-igt-gpu-tools = "${PV}-${PV}"
+CHECK_VERSION_PV:pn-vim = "${@'.'.join(d.getVar('PV').split('.')[:-1])}"
+CHECK_VERSION_PV:pn-vim-tiny = "${@'.'.join(d.getVar('PV').split('.')[:-1])}"
+CHECK_VERSION_PV:pn-ncurses = "${PV}.%"
+CHECK_VERSION_PV:pn-alsa-tools = "%"
+CHECK_VERSION_PV:pn-gst-examples = "%"
+CHECK_VERSION_PV:pn-libedit = "${@d.getVar('PV').split('-')[1]}"
-- 
2.34.1

[RFC PATCH 2/3] mc: fix buildpath QA regarding unzip

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

When unzip-native is somehow introduced indirectly into the dependency
chain, then we get buildpath QA issue. Pass a similar value as the ZIP
to fix this issue.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/recipes-extended/mc/mc_4.8.33.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-extended/mc/mc_4.8.33.bb b/meta/recipes-extended/mc/mc_4.8.33.bb
index ec50cfc7ce..fd18edf7be 100644
--- a/meta/recipes-extended/mc/mc_4.8.33.bb
+++ b/meta/recipes-extended/mc/mc_4.8.33.bb
@@ -32,6 +32,7 @@ CACHED_CONFIGUREVARS += "ac_cv_path_PERL_FOR_BUILD='/usr/bin/env perl'"
 CACHED_CONFIGUREVARS += "ac_cv_path_PYTHON='/usr/bin/env python'"
 CACHED_CONFIGUREVARS += "ac_cv_path_GREP='/usr/bin/env grep'"
 CACHED_CONFIGUREVARS += "ac_cv_path_ZIP='/usr/bin/zip'"
+CACHED_CONFIGUREVARS += "ac_cv_path_UNZIP='/usr/bin/unzip'"
 CACHED_CONFIGUREVARS += "mc_cv_have_zipinfo=yes"
 
 do_install:append () {
-- 
2.34.1

[RFC PATCH 3/3] time: fix runtime version from UNKNOWN to 1.9

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

The runtime version of time is determined by:
build-aux/git-version-gen .tarball-version

But there's no .tarball-version in the tarball. And time's last
commit dates back to 20210105, it's very unlikely for them to
generate a new tarball. So we add this file manually with ${PV}
as the content.

Before the fix, time --version:
time (GNU Time) UNKNOWN

After the fix, time --version:
time (GNU Time) 1.9

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/recipes-extended/time/time_1.9.bb | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/recipes-extended/time/time_1.9.bb b/meta/recipes-extended/time/time_1.9.bb
index 8364210e61..1872a409f7 100644
--- a/meta/recipes-extended/time/time_1.9.bb
+++ b/meta/recipes-extended/time/time_1.9.bb
@@ -21,3 +21,9 @@ SRC_URI[md5sum] = "d2356e0fe1c0b85285d83c6b2ad51b5f"
 SRC_URI[sha256sum] = "fbacf0c81e62429df3e33bda4cee38756604f18e01d977338e23306a3e3b521e"
 
 inherit autotools
+
+# We don't want our version to be "UNKNOWN", and time repo's latest commits dates back
+# to 20210105. It's very unlikely it'll generate a new tarball with a .tarball-version included.
+do_configure:prepend () {
+    echo ${PV} > ${S}/.tarball-version
+}
-- 
2.34.1

Re: [OE-core] [RFC PATCH 3/3] time: fix runtime version from UNKNOWN to 1.9

[Ross Burton]

On 27 Mar 2025, at 06:21, Chen Qi via lists.openembedded.org <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> 
> From: Chen Qi <Qi.Chen@windriver.com>
> 
> The runtime version of time is determined by:
> build-aux/git-version-gen .tarball-version
> 
> But there's no .tarball-version in the tarball. And time's last
> commit dates back to 20210105, it's very unlikely for them to
> generate a new tarball. So we add this file manually with ${PV}
> as the content.

The upstream git repo has had activity since the 1.9 release so there _might_ be a new release in the future.

Can you link to the submitted fix for this (https://lists.gnu.org/archive/html/bug-time/2021-01/msg00000.html) and only write to .tarball-version if it doesn’t exist, so we don’t overwrite a file if it appears in a future release.

Ross

Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch

[Alexander Kanavin]

On Thu, 27 Mar 2025 at 07:22, Chen Qi via lists.openembedded.org
<Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> The basic idea is to use qemu to run executables at build time, extract
> possible versions, and check if there's a mismatch found.

Should this check be in its own layer for now? If it's in core, there
should be evidence that it's broadly useful and used (and no one's
asked for it before as far as I'm aware), and it should be tested for
correct behaviour and against regressions. I don't feel it meets these
criteria at the moment.

Git fetcher has recently gained the ability to match up source
revisions with tags specified in recipes, and will error out if SRCREV
is set wrongly. We just need to update recipes to specify tags (with
${PV} in them somewhere, so they remain valid on version updates). So
sending in those updates would be most welcome.

Alex

Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch

[Richard Purdie]

On Thu, 2025-03-27 at 12:22 +0100, Alexander Kanavin via
lists.openembedded.org wrote:
> On Thu, 27 Mar 2025 at 07:22, Chen Qi via lists.openembedded.org
> <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> > The basic idea is to use qemu to run executables at build time,
> > extract
> > possible versions, and check if there's a mismatch found.
> 
> Should this check be in its own layer for now? If it's in core, there
> should be evidence that it's broadly useful and used (and no one's
> asked for it before as far as I'm aware), and it should be tested for
> correct behaviour and against regressions. I don't feel it meets
> these
> criteria at the moment.
> 
> Git fetcher has recently gained the ability to match up source
> revisions with tags specified in recipes, and will error out if
> SRCREV
> is set wrongly. We just need to update recipes to specify tags (with
> ${PV} in them somewhere, so they remain valid on version updates). So
> sending in those updates would be most welcome.

The new SRCREV vs tag code was my first thought when seeing this patch
too. 

It is a nice idea to check the version information and having something
somewhere to do it is nice but I'm not sure maintaining this in OE-Core
is the right thing to do for it...

Cheers,

Richard

Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch

[ChenQi]

On 3/27/25 19:25, Richard Purdie wrote:
> On Thu, 2025-03-27 at 12:22 +0100, Alexander Kanavin via
> lists.openembedded.org wrote:
>> On Thu, 27 Mar 2025 at 07:22, Chen Qi via lists.openembedded.org
>> <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>>> The basic idea is to use qemu to run executables at build time,
>>> extract
>>> possible versions, and check if there's a mismatch found.
>> Should this check be in its own layer for now? If it's in core, there
>> should be evidence that it's broadly useful and used (and no one's
>> asked for it before as far as I'm aware), and it should be tested for
>> correct behaviour and against regressions. I don't feel it meets
>> these
>> criteria at the moment.
>>
>> Git fetcher has recently gained the ability to match up source
>> revisions with tags specified in recipes, and will error out if
>> SRCREV
>> is set wrongly. We just need to update recipes to specify tags (with
>> ${PV} in them somewhere, so they remain valid on version updates). So
>> sending in those updates would be most welcome.
> The new SRCREV vs tag code was my first thought when seeing this patch
> too.
>
> It is a nice idea to check the version information and having something
> somewhere to do it is nice but I'm not sure maintaining this in OE-Core
> is the right thing to do for it...
>
> Cheers,
>
> Richard

Got it. I'll maintain it in its own layer.

Patch 2/3 and patch 3/3 are fixes. Please consider merging them. I'll 
send out V2 according to Ross's advice.

I personally think this version check feature is useful for layer 
maintainers because it will detect a few problems I mentioned in the 
cover letter. So if people want this feature in the future and the 
decision is changed at that time, I can send out a new RFC with oeqa 
test cases added.

Regards,
Qi

Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch

[Quentin Schulz]

Hi Chen,

On 3/28/25 3:42 AM, Chen Qi via lists.openembedded.org wrote:
> On 3/27/25 19:25, Richard Purdie wrote:
>> On Thu, 2025-03-27 at 12:22 +0100, Alexander Kanavin via
>> lists.openembedded.org wrote:
>>> On Thu, 27 Mar 2025 at 07:22, Chen Qi via lists.openembedded.org
>>> <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>>>> The basic idea is to use qemu to run executables at build time,
>>>> extract
>>>> possible versions, and check if there's a mismatch found.
>>> Should this check be in its own layer for now? If it's in core, there
>>> should be evidence that it's broadly useful and used (and no one's
>>> asked for it before as far as I'm aware), and it should be tested for
>>> correct behaviour and against regressions. I don't feel it meets
>>> these
>>> criteria at the moment.
>>>
>>> Git fetcher has recently gained the ability to match up source
>>> revisions with tags specified in recipes, and will error out if
>>> SRCREV
>>> is set wrongly. We just need to update recipes to specify tags (with
>>> ${PV} in them somewhere, so they remain valid on version updates). So
>>> sending in those updates would be most welcome.
>> The new SRCREV vs tag code was my first thought when seeing this patch
>> too.
>>
>> It is a nice idea to check the version information and having something
>> somewhere to do it is nice but I'm not sure maintaining this in OE-Core
>> is the right thing to do for it...
>>
>> Cheers,
>>
>> Richard
> 
> Got it. I'll maintain it in its own layer.
> 
> Patch 2/3 and patch 3/3 are fixes. Please consider merging them. I'll 
> send out V2 according to Ross's advice.
> 
> I personally think this version check feature is useful for layer 
> maintainers because it will detect a few problems I mentioned in the 
> cover letter. So if people want this feature in the future and the 
> decision is changed at that time, I can send out a new RFC with oeqa 
> test cases added.
> 

If the CVE reporting tool is using PV, having a PV different from the 
actual version we're building is an issue. So running this tool 
regularly would be nice to make sure we're building what we say we are 
building?

Maybe we would want to run this on the autobuilder every now and then to 
catch issues before releases for examples? If that was the case, in 
which layer would we want to store this?

Cheers,
Quentin