Re: [docs] [PATCH v2] doc: fix the switchers menu

[Quentin Schulz <quentin.schulz@cherry.de>]

Hi Antonin,

On 2/4/26 11:03 AM, Antonin Godard via lists.yoctoproject.org wrote:
> Fix the switchers.js script of Bitbake to show what are the supported
> versions and be able to switch between them.
> 
> The default landing page is the stable branch and shows the BitBake
> version along with the corresponding Yocto Project codename. This
> hopefully makes it easier to remember the correspondance between the
> BitBake version and the Yocto Project version.
> 
> This works thanks to a set_versions.py script, which is largely inspired
> from yocto-docs. It reads the tags from the repository and tries to
> guess the currently checked out version of BitBake on which we are.
> 
> The "obsolete" warning is now also shown when browsing outdated manuals,
> meaning any version not part of activereleases in set_versions.py.
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
> Note: This will be accompanied by a patch to adapt the Autobuilder to
> generate this file for each release.
> ---
> Changes in v2:
> - Remove environment information grabbing mechanism, making this fully
>    standalone.
> - Link to v1: https://lore.kernel.org/r/20250915-fix-switchers-js-v1-0-523ef53fe802@bootlin.com
> ---
>   doc/.gitignore                    |   2 +
>   doc/Makefile                      |   3 +-
>   doc/bitbake.yaml.in               |   1 +
>   doc/conf.py                       |  10 +-
>   doc/set_versions.py               | 155 ++++++++++++++++++++++++
>   doc/sphinx-static/switchers.js    | 233 -----------------------------------
>   doc/sphinx-static/switchers.js.in | 247 ++++++++++++++++++++++++++++++++++++++
>   7 files changed, 416 insertions(+), 235 deletions(-)
> 
> diff --git a/doc/.gitignore b/doc/.gitignore
> index 69fa449dd96..40ebe76c088 100644
> --- a/doc/.gitignore
> +++ b/doc/.gitignore
> @@ -1 +1,3 @@
>   _build/
> +sphinx-static/switchers.js
> +bitbake.yaml
> diff --git a/doc/Makefile b/doc/Makefile
> index 996f01b7d5c..cdb054d3a52 100644
> --- a/doc/Makefile
> +++ b/doc/Makefile
> @@ -27,9 +27,10 @@ publish: Makefile html singlehtml
>   	sed -i -e 's@index.html#@singleindex.html#@g' $(BUILDDIR)/$(DESTDIR)/singleindex.html
>   
>   clean:
> -	@rm -rf $(BUILDDIR)
> +	@rm -rf $(BUILDDIR) sphinx-static/switchers.js bitbake.yaml
>   
>   # Catch-all target: route all unknown targets to Sphinx using the new
>   # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
>   %: Makefile
> +	$(SOURCEDIR)/set_versions.py
>   	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
> diff --git a/doc/bitbake.yaml.in b/doc/bitbake.yaml.in
> new file mode 100644
> index 00000000000..4085582df60
> --- /dev/null
> +++ b/doc/bitbake.yaml.in
> @@ -0,0 +1 @@
> +DOCCONF_VERSION : "dev"
> diff --git a/doc/conf.py b/doc/conf.py
> index bce386624e2..5b28808c590 100644
> --- a/doc/conf.py
> +++ b/doc/conf.py
> @@ -14,17 +14,25 @@
>   # import sys
>   # sys.path.insert(0, os.path.abspath('.'))
>   
> +import os

I don't see it being used?

>   import sys
>   import datetime
> +import yaml
>   
>   from pathlib import Path
>   
> -current_version = "dev"
> +current_version = 'dev'

Unnecessary noise.

> +
> +with open("bitbake.yaml") as data:
> +    buff = data.read()
> +    subst_vars = yaml.safe_load(buff)
> +    current_version = subst_vars["DOCCONF_VERSION"]
>   
>   # String used in sidebar
>   version = 'Version: ' + current_version
>   if current_version == 'dev':
>       version = 'Version: Current Development'
> +

Ditto.

>   # Version seen in documentation_options.js and hence in js switchers code
>   release = current_version
>   
> diff --git a/doc/set_versions.py b/doc/set_versions.py
> new file mode 100755
> index 00000000000..63fba2fe948
> --- /dev/null
> +++ b/doc/set_versions.py
> @@ -0,0 +1,155 @@
> +#!/usr/bin/env python3
> +#
> +# This is a minimal version of the set_versions.py from yocto-docs,
> +# use to replace VERSIONS_PLACEHOLDER in switchers.js.in by a list defined below
> +# with BITBAKE_ACTIVE_RELEASES in the environment.
> +#

Where?

> +# When the documentation is built with the autobuilder, the versions are
> +# calculated based on the info found in set_versions.py from yocto-docs.
> +#
> +# Copyright Linux Foundation
> +# Author: Antonin Godard <antonin.godard@bootlin.com>
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +import collections
> +import itertools
> +import re
> +import os
> +import subprocess
> +import sys
> +

Sort alphabetically please.

> +from packaging.version import Version
> +from typing import Tuple
> +
> +devbranch = "2.18"
> +ltsseries = ["2.8", "2.0"]
> +activereleases = ["2.16"] + ltsseries
> +
> +yocto_mapping = {
> +    "2.18": "wrynose",
> +    "2.16": "whinlatter",
> +    "2.12": "walnascar",
> +    "2.10": "styhead",
> +    "2.8": "scarthgap",
> +    "2.6": "nanbield",
> +    "2.4": "mickledore",
> +    "2.2": "langdale",
> +    "2.0": "kirkstone",
> +    "1.52": "honister",
> +    "1.50": "hardknott",
> +    "1.48": "gatesgarth",
> +    "1.46": "dunfell",
> +}
> +
> +bbver_re = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+$")
> +ourversion = None
> +
> +# Test that we are building from a Git repository
> +try:
> +    subprocess.run(["git", "rev-parse", "--is-inside-work-tree"],
> +                   stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
> +except subprocess.CalledProcessError:
> +    sys.exit("Building bitbake's documentation must be done from its Git repository.\n"
> +             "Clone the repository with the following command:\n"
> +             "git clone https://git.openembedded.org/bitbake ")
> +
> +# Test tags exist and inform the user to fetch if not
> +try:
> +    subprocess.run(["git", "show", f"{ltsseries[0]}.0"],
> +                   stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
> +except subprocess.CalledProcessError:
> +    sys.exit("Please run 'git fetch --tags' before building the documentation")
> +
> +
> +# Try and figure out what we are
> +tags = subprocess.run(["git", "tag", "--points-at", "HEAD"],
> +                      stdout=subprocess.PIPE, stderr=subprocess.PIPE,
> +                      universal_newlines=True).stdout
> +for t in tags.split():
> +    if re.match(bbver_re, t):
> +        ourversion = t

We can break here as we can only match one X.Y.Z tag for a given commit 
hash.

> +
> +if ourversion:
> +    # We're a tagged release
> +    components = ourversion.split(".")
> +    ourseries = f"{components[0]}.{components[1]}"
> +else:
> +    # We're floating on a branch
> +    branch = subprocess.run(["git", "branch", "--show-current"],
> +                            stdout=subprocess.PIPE, stderr=subprocess.PIPE,
> +                            universal_newlines=True).stdout.strip()
> +
> +    if branch == "" or branch not in list(yocto_mapping.keys()) + ["master", "master-next"]:
> +        # We're not on a known release branch so we have to guess. Compare the
> +        # numbers of commits from each release branch and assume the smallest
> +        # number of commits is the one we're based off
> +        possible_branch = None
> +        branch_count = 0
> +        for b in itertools.chain(yocto_mapping.keys(), ["master"]):
> +            result = subprocess.run(["git", "log", "--format=oneline", "HEAD..origin/" + b],
> +                                    stdout=subprocess.PIPE, stderr=subprocess.PIPE,
> +                                    universal_newlines=True)
> +            if result.returncode == 0:
> +                count = result.stdout.count('\n')
> +                if not possible_branch or count < branch_count:
> +                    print("Branch %s has count %s" % (b, count))
> +                    possible_branch = b
> +                    branch_count = count
> +        if possible_branch:
> +            branch = possible_branch
> +        else:
> +            branch = "master"
> +        print("Nearest release branch estimated to be %s" % branch)
> +
> +    if branch == "master":
> +        ourversion = "dev"
> +        ourseries = devbranch
> +    elif branch == "master-next":
> +        ourversion = "next"
> +        ourseries = devbranch
> +    else:
> +        ourversion = branch
> +        head_commit = subprocess.run(["git", "rev-parse", "--short", "HEAD"],
> +                                     stdout=subprocess.PIPE, stderr=subprocess.PIPE,
> +                                     universal_newlines=True).stdout.strip()
> +        branch_commit = subprocess.run(["git", "rev-parse", "--short", branch],
> +                                        stdout=subprocess.PIPE, stderr=subprocess.PIPE,
> +                                        universal_newlines=True).stdout.strip()
> +        if head_commit != branch_commit:
> +            ourversion += f" ({head_commit})"
> +        ourseries = branch
> +
> +print("Version calculated to be %s" % ourversion)
> +print("Series calculated to be %s" % ourseries)
> +
> +replacements = {
> +    "DOCCONF_VERSION": ourversion,
> +}
> +
> +if os.path.exists("bitbake.yaml.in"):
> +    with open("bitbake.yaml.in", "r") as r, open("bitbake.yaml", "w") as w:
> +        lines = r.readlines()
> +        for line in lines:
> +            data = line.split(":")
> +            k = data[0].strip()
> +            if k in replacements:
> +                w.write("%s : \"%s\"\n" % (k, replacements[k]))
> +            else:
> +                w.write(line)
> +
> +    print("bitbake.yaml generated from bitbake.yaml.in")
> +
> +with open("sphinx-static/switchers.js.in", "r") as r, \
> +     open("sphinx-static/switchers.js", "w") as w:
> +    lines = r.readlines()
> +    for line in lines:
> +        if "VERSIONS_PLACEHOLDER" in line:
> +            if ourversion != "dev":
> +                w.write(f"    'dev': 'Unstable (dev)',\n")
> +            for series in activereleases:
> +                w.write(f"    '{series}': '{series} ({yocto_mapping[series]})',\n")

Why is this different from yp-docs? We don't handle showing the current 
series (ourseries)/version (ourversion) if it's outdated anymore?

To answer Richard on the other thread, we use YAML because JSON doesn't 
allow comments. Also, because we used to want multiline variables (see 
https://yaml-multiline.info/) which aren't supported in JSON, but those 
are gone since 8d993022c2ae ("docs: use literalinclude for system 
requirements"). Not sure we can get rid of it for yp-docs as we only 
override *some* variables in this file by set_versions.py so the 
autobuilder won't replace poky.yaml.in, and the YAML is consumed by 
sphinx/yocto-vars.py.

If we don't plan on having comments in this file *ever*, then the YAML 
dependency isn't necessary.

Another option is INI and use configparser, c.f. 
https://docs.python.org/3/library/configparser.html

It seems that INI supports the ':' as key-value delimiter as well as 
multiline strings (but not the way we used it for YAML!), so it could be 
a drop-in replacement. We could even simply write to the file with 
configparser.write() (though that will drop the in-file comments).

Cheers,
Quentin