[PATCH 0/4] New script to detect obsolete variables

[PATCH 0/4] New script to detect obsolete variables

[Antonin Godard]

First patch of this series proposes a new script that tries to detect
variables missing from OE-Core, meta-yocto, and bitbake. It is simple in
design as it just uses 'git grep' for that, plus a few exceptions.

The remaining patches remove variables that have been detected by the
script.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Antonin Godard (4):
      tools/obsolete-variables: add script
      docs-wide: fix USERMOD_PARAMS -> USERMOD_PARAM
      ref-manual/variables.rst: remove CVSDIR
      ref-manual/variables.rst: drop documentation for FIT_KERNEL_COMP_ALG_EXTENSION

 documentation/migration-guides/migration-6.1.rst   |   4 +-
 .../migration-guides/release-notes-6.1.rst         |   2 +-
 documentation/ref-manual/variables.rst             |  16 +--
 documentation/tools/obsolete-variables             | 142 +++++++++++++++++++++
 4 files changed, 147 insertions(+), 17 deletions(-)
---
base-commit: 9955b0f099b6b42a9750f0d544944a2d8e5a39b2
change-id: 20260821-missing-variables-c654b70b96d4

[PATCH 1/4] tools/obsolete-variables: add script

[Antonin Godard]

Add a script that outputs missing variables from
OE-Core/meta-yocto/BitBake, i.e. for which grepping returned nothing.

There are a few exceptions, which are listed in VAR_EXCEPTIONS.

The script currently returns:

  ERROR: Variable CVSDIR not found anywhere
  ERROR: Variable FIT_KERNEL_COMP_ALG_EXTENSION not found anywhere
  ERROR: Variable USERMOD_PARAMS not found anywhere

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/tools/obsolete-variables | 142 +++++++++++++++++++++++++++++++++
 1 file changed, 142 insertions(+)

diff --git a/documentation/tools/obsolete-variables b/documentation/tools/obsolete-variables
new file mode 100755
index 000000000..bc97cbb71
--- /dev/null
+++ b/documentation/tools/obsolete-variables
@@ -0,0 +1,142 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: MIT
+#
+# Author: Antonin Godard <antonin.godard@bootlin.com>
+#
+# Copyright (C) 2026 Bootlin
+#
+
+import argparse
+import logging
+import subprocess
+import sys
+
+from pathlib import Path
+from sphinx.util.inventory import InventoryFile
+from typing import List
+
+
+DOCS_DIR = Path(__file__).parent.parent
+# False positives:
+# - variables we know exist but have a specific syntax
+# - we keep documentation for it here, already saying it is obsolete
+VAR_EXCEPTIONS = (
+    "CONFLICT_IMAGE_FEATURES",
+    "CONFLICT_TUNE_FEATURES",
+    "FEATURE_PACKAGES",
+    "LAYERRECOMMENDS",
+    "REQUIRED_IMAGE_FEATURES",
+    "VIRTUAL-RUNTIME",
+    "module_autoload",
+    "module_conf",
+)
+ERR_MSG = "Variable %s not found in OE-Core, meta-yocto, or BitBake"
+
+
+def parse_arguments() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Use the Sphinx's inventory to "
+                                     "check for variables not present in "
+                                     "OE-Core/meta-yocto/BitBake")
+
+    parser.add_argument("--debug",
+                        action="store_true",
+                        help="Print debug messages")
+
+    parser.add_argument("--yocto-docs-inv",
+                        type=Path,
+                        default=DOCS_DIR / "_build/html/objects.inv",
+                        help="Input yocto-docs inventory file")
+
+    parser.add_argument("--bitbake-inv",
+                        type=Path,
+                        default=DOCS_DIR / "_build/doctrees/__intersphinx_cache__/bitbake_objects.inv",
+                        help="Input bitbake inventory file")
+
+    parser.add_argument("oecore_dir",
+                        type=Path,
+                        help="Path to openembedded-core")
+
+    parser.add_argument("meta_yocto_dir",
+                        type=Path,
+                        help="Path to meta-yocto")
+
+    parser.add_argument("bitbake_dir",
+                        type=Path,
+                        help="Path to bitbake")
+
+    return parser.parse_args()
+
+
+def var_exists_in(var: str, gitdir: Path) -> bool:
+    """
+    In gitdir, check if a grepping for "<var>" return something (then return
+    True, False otherwise).
+
+    Special case where in OE-Core we can have:
+        BB_RENAMED_VARIABLES[<var>] = "..."
+    Then exclude that.
+    """
+    cmd = [
+        "git", "-C", gitdir, "grep", "--extended-regexp", fr"\<{var}\>",
+    ]
+    _out = ""
+    try:
+        _out = subprocess.check_output(cmd, encoding="utf-8")
+    except subprocess.CalledProcessError:
+        pass
+
+    out = ""
+    for line in _out.splitlines():
+        if not line.startswith(f"BB_RENAMED_VARIABLES[{var}]"):
+            out += f"{line}\n"
+
+    if out:
+        logging.debug(f"{var} found in {gitdir.name}:\n{out}")
+        return True
+
+    return False
+
+
+def var_exists(var: str, repos: List[Path]):
+    return any(var_exists_in(var, d) for d in repos)
+
+
+def check_inventory(inv_p: Path, uri: str, repos: List[Path]) -> int:
+    exit_code = 0
+    inv = InventoryFile.loads(inv_p.read_bytes(), uri="")
+    for entry, inv_item in sorted(inv.data["std:term"].items()):
+        if inv_item.uri.startswith(uri) \
+                and entry not in VAR_EXCEPTIONS \
+                and not var_exists(entry, repos):
+            exit_code = 1
+            logging.error(ERR_MSG % entry)
+    return exit_code
+
+
+def main():
+    exit_code = 0
+    args = parse_arguments()
+
+    if args.debug:
+        logging.basicConfig(format="%(levelname)s: %(message)s", level=logging.DEBUG)
+    else:
+        logging.basicConfig(format="%(levelname)s: %(message)s")
+
+    if not (args.yocto_docs_inv.exists() and args.bitbake_inv.exists()):
+        logging.error(f"yocto-docs and bitbake inventories not found at {args.yocto_docs_inv}/"
+                      f"{args.bitbake_inv}. Build the documentation or use the "
+                      "--yocto-docs-inv/--bitbake-inv options")
+        sys.exit(1)
+
+    exit_code = check_inventory(args.yocto_docs_inv, "ref-manual/variables.html#term-",
+                                (args.oecore_dir, args.meta_yocto_dir, args.bitbake_dir))
+
+    exit_code = check_inventory(args.bitbake_inv, "bitbake-user-manual/bitbake-user-manual-ref-variables.html#term-",
+                                (args.oecore_dir, args.meta_yocto_dir, args.bitbake_dir))
+
+    sys.exit(exit_code)
+
+
+if __name__ == "__main__":
+    main()

-- 
2.55.0

Re: [docs] [PATCH 1/4] tools/obsolete-variables: add script

[Quentin Schulz]

Hi Antonin,

On 8/27/26 9:46 AM, Antonin Godard via lists.yoctoproject.org wrote:
> Add a script that outputs missing variables from
> OE-Core/meta-yocto/BitBake, i.e. for which grepping returned nothing.
> 
> There are a few exceptions, which are listed in VAR_EXCEPTIONS.
> 
> The script currently returns:
> 
>    ERROR: Variable CVSDIR not found anywhere
>    ERROR: Variable FIT_KERNEL_COMP_ALG_EXTENSION not found anywhere
>    ERROR: Variable USERMOD_PARAMS not found anywhere
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>   documentation/tools/obsolete-variables | 142 +++++++++++++++++++++++++++++++++
>   1 file changed, 142 insertions(+)
> 
> diff --git a/documentation/tools/obsolete-variables b/documentation/tools/obsolete-variables
> new file mode 100755
> index 000000000..bc97cbb71
> --- /dev/null
> +++ b/documentation/tools/obsolete-variables
> @@ -0,0 +1,142 @@
> +#!/usr/bin/env python3
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +# Author: Antonin Godard <antonin.godard@bootlin.com>
> +#
> +# Copyright (C) 2026 Bootlin
> +#
> +
> +import argparse
> +import logging
> +import subprocess
> +import sys
> +
> +from pathlib import Path
> +from sphinx.util.inventory import InventoryFile

I would recommend splitting modules that aren't from core Python into a 
separate section. pathlib and typing are python core modules, but 
sphinx.util... isn't.

> +from typing import List
> +
> +
> +DOCS_DIR = Path(__file__).parent.parent
> +# False positives:
> +# - variables we know exist but have a specific syntax
> +# - we keep documentation for it here, already saying it is obsolete
> +VAR_EXCEPTIONS = (
> +    "CONFLICT_IMAGE_FEATURES",
> +    "CONFLICT_TUNE_FEATURES",
> +    "FEATURE_PACKAGES",
> +    "LAYERRECOMMENDS",
> +    "REQUIRED_IMAGE_FEATURES",
> +    "VIRTUAL-RUNTIME",
> +    "module_autoload",
> +    "module_conf",
> +)
> +ERR_MSG = "Variable %s not found in OE-Core, meta-yocto, or BitBake"
> +
> +
> +def parse_arguments() -> argparse.Namespace:
> +    parser = argparse.ArgumentParser(description="Use the Sphinx's inventory to "
> +                                     "check for variables not present in "
> +                                     "OE-Core/meta-yocto/BitBake")
> +
> +    parser.add_argument("--debug",
> +                        action="store_true",
> +                        help="Print debug messages")
> +
> +    parser.add_argument("--yocto-docs-inv",
> +                        type=Path,
> +                        default=DOCS_DIR / "_build/html/objects.inv",
> +                        help="Input yocto-docs inventory file")
> +
> +    parser.add_argument("--bitbake-inv",
> +                        type=Path,
> +                        default=DOCS_DIR / "_build/doctrees/__intersphinx_cache__/bitbake_objects.inv",
> +                        help="Input bitbake inventory file")

Consider printing the default values with %(default)s, c.f. 
https://docs.python.org/3/library/argparse.html#help.

> +
> +    parser.add_argument("oecore_dir",
> +                        type=Path,
> +                        help="Path to openembedded-core")
> +
> +    parser.add_argument("meta_yocto_dir",
> +                        type=Path,
> +                        help="Path to meta-yocto")
> +
> +    parser.add_argument("bitbake_dir",
> +                        type=Path,
> +                        help="Path to bitbake")
> +

Please specify those need to be *git* repositories (otherwise you cannot 
run git-grep on them).

> +    return parser.parse_args()
> +
> +
> +def var_exists_in(var: str, gitdir: Path) -> bool:
> +    """
> +    In gitdir, check if a grepping for "<var>" return something (then return
> +    True, False otherwise).
> +
> +    Special case where in OE-Core we can have:
> +        BB_RENAMED_VARIABLES[<var>] = "..."
> +    Then exclude that.
> +    """
> +    cmd = [
> +        "git", "-C", gitdir, "grep", "--extended-regexp", fr"\<{var}\>",
> +    ]
> +    _out = ""
> +    try:
> +        _out = subprocess.check_output(cmd, encoding="utf-8")
> +    except subprocess.CalledProcessError:
> +        pass
> +
> +    out = ""
> +    for line in _out.splitlines():
> +        if not line.startswith(f"BB_RENAMED_VARIABLES[{var}]"):
> +            out += f"{line}\n"
> +
> +    if out:
> +        logging.debug(f"{var} found in {gitdir.name}:\n{out}")
> +        return True
> +

Can't you simplify this to

git grep --extended-regexp -e '\<{var}\>' --and --not -e 
'BB_RENAMED_VARIABLES\[{var}\]'

?

If this works for you, please consider adding --max-count=1 so that it 
exits as soon as possible if there's at least one match. You probably 
also want to pass --quiet as we won't care about the output anymore.

> +    return False
> +
> +
> +def var_exists(var: str, repos: List[Path]):
> +    return any(var_exists_in(var, d) for d in repos)
> +
> +
> +def check_inventory(inv_p: Path, uri: str, repos: List[Path]) -> int:
> +    exit_code = 0
> +    inv = InventoryFile.loads(inv_p.read_bytes(), uri="")
> +    for entry, inv_item in sorted(inv.data["std:term"].items()):
> +        if inv_item.uri.startswith(uri) \
> +                and entry not in VAR_EXCEPTIONS \
> +                and not var_exists(entry, repos):

Typo here, it should be var_exists_in().

> +            exit_code = 1
> +            logging.error(ERR_MSG % entry)

Does it make sense to have a variable (far from where it's used) for 
something that's used only once?

> +    return exit_code
> +
> +
> +def main():
> +    exit_code = 0
> +    args = parse_arguments()
> +
> +    if args.debug:
> +        logging.basicConfig(format="%(levelname)s: %(message)s", level=logging.DEBUG)
> +    else:
> +        logging.basicConfig(format="%(levelname)s: %(message)s")
> +
> +    if not (args.yocto_docs_inv.exists() and args.bitbake_inv.exists()):
> +        logging.error(f"yocto-docs and bitbake inventories not found at {args.yocto_docs_inv}/"
> +                      f"{args.bitbake_inv}. Build the documentation or use the "
> +                      "--yocto-docs-inv/--bitbake-inv options")
> +        sys.exit(1)
> +
> +    exit_code = check_inventory(args.yocto_docs_inv, "ref-manual/variables.html#term-",
> +                                (args.oecore_dir, args.meta_yocto_dir, args.bitbake_dir))
> +

This exit_code is never tested. I'm assuming you may want to do += for 
the next one (or store the result in another variable).

> +    exit_code = check_inventory(args.bitbake_inv, "bitbake-user-manual/bitbake-user-manual-ref-variables.html#term-",
> +                                (args.oecore_dir, args.meta_yocto_dir, args.bitbake_dir))
> +
> +    sys.exit(exit_code)
> +
> +
> +if __name__ == "__main__":
> +    main()
> 
Cheers,
Quentin

[PATCH 2/4] docs-wide: fix USERMOD_PARAMS -> USERMOD_PARAM

[Antonin Godard]

There is a typo in the documentation: the variable is named
USERMOD_PARAM, not USERMOD_PARAMS.

Fixes: 85eeb215978a ("ref-manual/variables.rst: document the USERMOD_PARAMS variable")
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/migration-guides/migration-6.1.rst     | 4 ++--
 documentation/migration-guides/release-notes-6.1.rst | 2 +-
 documentation/ref-manual/variables.rst               | 4 ++--
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/documentation/migration-guides/migration-6.1.rst b/documentation/migration-guides/migration-6.1.rst
index 025b8f709..f0e0ce14c 100644
--- a/documentation/migration-guides/migration-6.1.rst
+++ b/documentation/migration-guides/migration-6.1.rst
@@ -36,7 +36,7 @@ See :ref:`all supported distributions <system-requirements-supported-distros>`.
 
 .. _ref-migration-6-1-groupmems:
 
-useradd: replace :term:`GROUPMEMS_PARAM` assignments to :term:`USERMOD_PARAMS`
+useradd: replace :term:`GROUPMEMS_PARAM` assignments to :term:`USERMOD_PARAM`
 ------------------------------------------------------------------------------
 
 The ``groupmems`` command is removed from the ``shadow`` recipe starting from
@@ -44,7 +44,7 @@ version 4.20. The same functionality provided by ``groupmems`` can be achieved
 with the ``usermod`` command.
 
 Assignments made to the :term:`GROUPMEMS_PARAM` variable can be converted to use
-:term:`USERMOD_PARAMS`, by replacing::
+:term:`USERMOD_PARAM`, by replacing::
 
    GROUPMEMS_PARAM:${PN} = "--add user --group group1; \
                             --add user --group group2"
diff --git a/documentation/migration-guides/release-notes-6.1.rst b/documentation/migration-guides/release-notes-6.1.rst
index 021af5d57..fc7fd279e 100644
--- a/documentation/migration-guides/release-notes-6.1.rst
+++ b/documentation/migration-guides/release-notes-6.1.rst
@@ -99,7 +99,7 @@ New Features / Enhancements in |yocto-ver|
       several recipes to replace ``${nonarch_base_libdir}/firmware``
       (:oecore_rev:`2b75c7ba5e3aa6fc57d7b4afe59e2277b4d87de1`)
 
-   -  :ref:`ref-classes-useradd`: Add support for the :term:`USERMOD_PARAMS`
+   -  :ref:`ref-classes-useradd`: Add support for the :term:`USERMOD_PARAM`
       variable, acting as a replacement of the :term:`GROUPMEMS_PARAM` variable
       (:oecore_rev:`b8da733ab12c64503a353d5ceb2eb63fed95d851`,
       :oecore_rev:`cec67e24ac94554e092f8ab18b42e09b4feba77e`)
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index ae9cbfddf..a806f907e 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -3956,7 +3956,7 @@ system and gives an overview of their function and contents.
       ``groupadd``, see :manpage:`groupadd(8)`.
 
    :term:`GROUPMEMS_PARAM`
-      Deprecated in favor of :term:`USERMOD_PARAMS`. See
+      Deprecated in favor of :term:`USERMOD_PARAM`. See
       :ref:`ref-migration-6-1-groupmems` for more information.
 
    :term:`GRUB_GFXSERIAL`
@@ -12599,7 +12599,7 @@ system and gives an overview of their function and contents.
       Additionally, you should also set the
       :term:`USERADD_ERROR_DYNAMIC` variable.
 
-   :term:`USERMOD_PARAMS`
+   :term:`USERMOD_PARAM`
       When a recipe inherits the :ref:`ref-classes-useradd` class, this variable
       specifies for a package what parameters should be passed to the ``usermod``
       command if you wish to modify a user when the package is installed.

-- 
2.55.0

Re: [PATCH 2/4] docs-wide: fix USERMOD_PARAMS -> USERMOD_PARAM

[Antonin Godard]

Applied, thanks!

[2/4] docs-wide: fix USERMOD_PARAMS -> USERMOD_PARAM
      commit: 93ac6175a6d2c478c783c6a2a40548e9b5e49559

[PATCH 3/4] ref-manual/variables.rst: remove CVSDIR

[Antonin Godard]

The CVS fetcher was removed with 5427500e4f23 ("fetch: Drop CVS fetcher
module") in BitBake. Remove documentation for this obsolete variable.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/variables.rst | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index a806f907e..d1bbc437f 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -2110,10 +2110,6 @@ system and gives an overview of their function and contents.
 
           CVE_VERSION = "2.39"
 
-   :term:`CVSDIR`
-      The directory in which files checked out under the CVS system are
-      stored.
-
    :term:`CXX`
       The minimal command and arguments used to run the C++ compiler.
 

-- 
2.55.0

[PATCH 4/4] ref-manual/variables.rst: drop documentation for FIT_KERNEL_COMP_ALG_EXTENSION

[Antonin Godard]

Variable was dropped from OE-Core with 46bb48b42be3 ("kernel-uboot: drop
FIT_KERNEL_COMP_ALG_EXTENSION"). Remove the documentation for this
obsolete variable.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/variables.rst | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index d1bbc437f..b7846585e 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -3454,17 +3454,9 @@ system and gives an overview of their function and contents.
    :term:`FIT_KERNEL_COMP_ALG`
       The compression algorithm to use for the kernel image inside the FIT Image.
       At present, the only supported values are "gzip" (default), "lzo" or "none".
-      If you set this variable to anything other than "none" you may also need
-      to set :term:`FIT_KERNEL_COMP_ALG_EXTENSION`.
 
       This variable is used in the :ref:`ref-classes-kernel-uboot` class.
 
-   :term:`FIT_KERNEL_COMP_ALG_EXTENSION`
-      File extension corresponding to :term:`FIT_KERNEL_COMP_ALG`. The default
-      value is set ".gz" by the :ref:`ref-classes-kernel-uboot` class. If you
-      set :term:`FIT_KERNEL_COMP_ALG` to "lzo", you may want to set this
-      variable to ".lzo".
-
    :term:`FIT_KERNEL_SIGN_ENABLE`
       This variable is used by the :ref:`ref-classes-kernel-fit-image` class
       to enable or disable signing of the FIT image.

-- 
2.55.0