[PATCH RFC v2 2/2] tools: add script for building documentation inside containers

[Quentin Schulz <foss+yocto@0leil.net>]

From: Quentin Schulz <quentin.schulz@cherry.de>

This adds a script for building a container and building the
documentation within that new container image.

The openSUSE instructions now require a --non-interactive flag otherwise
they fail to run. Sadly there doesn't seem to be a way to have this in
an environment variable à-la DEBIAN_FRONTEND=noninteractive.

Somehow tzdata package in Ubuntu doesn't respect
DEBIAN_FRONTEND=noninteractive hence why the timezone needs to be set by
hand.

Note that only instructions for Debian Bookworm (12), Ubuntu 22.04 and
24.04, and Fedora 38, 39 and 40, currently work.

Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
---
 documentation/tools/Containerfile.almalinux        |   1 +
 documentation/tools/Containerfile.apt              |  26 +++++
 documentation/tools/Containerfile.debian           |   1 +
 documentation/tools/Containerfile.dnf              |  25 +++++
 documentation/tools/Containerfile.fedora           |   1 +
 documentation/tools/Containerfile.ubuntu           |   1 +
 documentation/tools/Containerfile.zypper           |  24 +++++
 documentation/tools/build-docs-container           | 113 +++++++++++++++++++++
 .../tools/host_packages_scripts/opensuse_docs.sh   |   2 +-
 .../host_packages_scripts/opensuse_docs_pdf.sh     |   2 +-
 .../host_packages_scripts/opensuse_essential.sh    |   2 +-
 11 files changed, 195 insertions(+), 3 deletions(-)

diff --git a/documentation/tools/Containerfile.almalinux b/documentation/tools/Containerfile.almalinux
new file mode 120000
index 0000000000000000000000000000000000000000..7237e9b99f4132957c0ad5b11fa6cefe9daaec74
--- /dev/null
+++ b/documentation/tools/Containerfile.almalinux
@@ -0,0 +1 @@
+Containerfile.dnf
\ No newline at end of file
diff --git a/documentation/tools/Containerfile.apt b/documentation/tools/Containerfile.apt
new file mode 100644
index 0000000000000000000000000000000000000000..5e30b65eb8f81a7764c00eb1078dabaf59b64517
--- /dev/null
+++ b/documentation/tools/Containerfile.apt
@@ -0,0 +1,26 @@
+ARG ARG_FROM=debian:12 # default value to avoid warning
+FROM $ARG_FROM
+
+ARG DOCS=ubuntu_docs.sh
+ARG DOCS_PDF=ubuntu_docs_pdf.sh
+
+ENV DEBIAN_FRONTEND=noninteractive
+ARG TZ=Europe/Vienna
+
+# relative to the location of the dockerfile
+COPY --chmod=777 ${DOCS} /temp/host_packages_docs.sh
+COPY --chmod=777 ${DOCS_PDF} /temp/host_packages_docs_pdf.sh
+
+RUN ln -fs "/usr/share/zoneinfo/$TZ" /etc/localtime \
+ && apt-get update \
+ && apt-get install -y sudo \
+ && yes | /temp/host_packages_docs.sh \
+ && yes | /temp/host_packages_docs_pdf.sh \
+ && apt-get --yes autoremove \
+ && apt-get clean \
+ && rm -rf /temp
+
+RUN git config --global --add safe.directory /docs
+
+ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"]
+CMD ["publish"]
diff --git a/documentation/tools/Containerfile.debian b/documentation/tools/Containerfile.debian
new file mode 120000
index 0000000000000000000000000000000000000000..5a7a425197afc2928802fcad5f34699b1ad3348a
--- /dev/null
+++ b/documentation/tools/Containerfile.debian
@@ -0,0 +1 @@
+Containerfile.apt
\ No newline at end of file
diff --git a/documentation/tools/Containerfile.dnf b/documentation/tools/Containerfile.dnf
new file mode 100644
index 0000000000000000000000000000000000000000..3dae74445585961a6f2d29b8acde09f2456dd886
--- /dev/null
+++ b/documentation/tools/Containerfile.dnf
@@ -0,0 +1,25 @@
+ARG ARG_FROM=fedora:40 # default value to avoid warning
+FROM $ARG_FROM
+
+ARG DOCS=fedora_docs.sh
+ARG DOCS_PDF=fedora_docs_pdf.sh
+ARG PIP3=pip3_docs.sh
+
+# relative to the location of the dockerfile
+COPY --chmod=777 ${DOCS} /temp/host_packages_docs.sh
+COPY --chmod=777 ${DOCS_PDF} /temp/host_packages_docs_pdf.sh
+COPY --chmod=777 ${PIP3} /temp/pip3_docs.sh
+
+RUN dnf update -y \
+ && dnf install -y sudo \
+ && yes | /temp/host_packages_docs.sh \
+ && yes | /temp/host_packages_docs_pdf.sh \
+ && yes | /temp/pip3_docs.sh \
+ && dnf autoremove -y \
+ && dnf clean all -y \
+ && rm -rf /temp
+
+RUN git config --global --add safe.directory /docs
+
+ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"]
+CMD ["publish"]
diff --git a/documentation/tools/Containerfile.fedora b/documentation/tools/Containerfile.fedora
new file mode 120000
index 0000000000000000000000000000000000000000..7237e9b99f4132957c0ad5b11fa6cefe9daaec74
--- /dev/null
+++ b/documentation/tools/Containerfile.fedora
@@ -0,0 +1 @@
+Containerfile.dnf
\ No newline at end of file
diff --git a/documentation/tools/Containerfile.ubuntu b/documentation/tools/Containerfile.ubuntu
new file mode 120000
index 0000000000000000000000000000000000000000..5a7a425197afc2928802fcad5f34699b1ad3348a
--- /dev/null
+++ b/documentation/tools/Containerfile.ubuntu
@@ -0,0 +1 @@
+Containerfile.apt
\ No newline at end of file
diff --git a/documentation/tools/Containerfile.zypper b/documentation/tools/Containerfile.zypper
new file mode 100644
index 0000000000000000000000000000000000000000..bcda5261ae98e8df16a3bcef17a7bf204033da99
--- /dev/null
+++ b/documentation/tools/Containerfile.zypper
@@ -0,0 +1,24 @@
+ARG ARG_FROM=opensuse/leap:15.4 # default value to avoid warning
+FROM $ARG_FROM
+
+ARG DOCS=opensuse_docs.sh
+ARG DOCS_PDF=opensuse_docs_pdf.sh
+ARG PIP3=pip3_docs.sh
+
+# relative to the location of the dockerfile
+COPY --chmod=777 ${DOCS} /temp/host_packages_docs.sh
+COPY --chmod=777 ${DOCS_PDF} /temp/host_packages_docs_pdf.sh
+COPY --chmod=777 ${PIP3} /temp/pip3_docs.sh
+
+RUN zypper update -y \
+ && zypper install -y sudo \
+ && yes | /temp/host_packages_docs.sh \
+ && yes | /temp/host_packages_docs_pdf.sh \
+ && yes | /temp/pip3_docs.sh \
+ && zypper clean --all \
+ && rm -rf /temp
+
+RUN git config --global --add safe.directory /docs
+
+ENTRYPOINT ["/usr/bin/env", "make", "-C", "documentation/"]
+CMD ["publish"]
diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container
new file mode 100755
index 0000000000000000000000000000000000000000..f382af75a7e89cfcc8a6080fdbc65fb68ff9ea8d
--- /dev/null
+++ b/documentation/tools/build-docs-container
@@ -0,0 +1,113 @@
+#!/usr/bin/env bash
+#
+# Build a container ready to build the documentation be reading the dependencies
+# listed in shell scripts in documentation/tools/host_packages_scripts, and
+# start a documentation build in this container.
+#
+# Usage:
+#
+#   ./documentation/tools/build-docs-container <image> [<make target>]
+#
+# e.g.:
+#
+#   ./documentation/tools/build-docs-container ubuntu:24.04 html
+#
+# Will build the docs in an Ubuntu 24.04 container in html.
+#
+# The container engine can be selected by exporting CONTAINERCMD in the
+# environment. The default is docker, but podman can also be used.
+
+set -eu -o pipefail
+
+SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)
+CONTAINERCMD=${CONTAINERCMD:-docker}
+DOCS_DIR="$SCRIPT_DIR/../.."
+SH_DIR="$SCRIPT_DIR/host_packages_scripts"
+
+main ()
+{
+  local image="$1"
+  shift
+
+  OCI=$(which "$CONTAINERCMD")
+
+  # docker build doesn't accept 2 colons, so "sanitize" the name
+  local sanitized_dockername
+  sanitized_dockername=$(echo "$image" | tr ':.' '-')
+
+  local version
+  version=$(echo "$image" | awk -F: '{print $NF}')
+
+  case $image in
+    almalinux*)
+      containerfile=Containerfile.almalinux
+      docs=almalinux_docs.sh
+      docs_pdf=almalinux_docs_pdf.sh
+      pip3=pip3_docs.sh
+      ;;
+    debian*)
+      containerfile=Containerfile.debian
+      docs=ubuntu_docs.sh
+      docs_pdf=ubuntu_docs_pdf.sh
+      ;;
+    fedora*)
+      containerfile=Containerfile.fedora
+      docs=fedora_docs.sh
+      docs_pdf=fedora_docs_pdf.sh
+      pip3=pip3_docs.sh
+      ;;
+    opensuse* | leap*)
+      image=opensuse/leap:$version
+      containerfile=Containerfile.zypper
+      docs=opensuse_docs.sh
+      docs_pdf=opensuse_docs_pdf.sh
+      pip3=pip3_docs.sh
+      ;;
+    ubuntu*)
+      containerfile=Containerfile.ubuntu
+      docs=ubuntu_docs.sh
+      docs_pdf=ubuntu_docs_pdf.sh
+      ;;
+    *)
+      echo "$image not supported"
+      exit 1
+      ;;
+  esac
+
+  $OCI build \
+    --tag "yocto-docs-$sanitized_dockername:latest" \
+    --build-arg ARG_FROM="docker.io/$image" \
+    --build-arg DOCS="$docs" \
+    --build-arg DOCS_PDF="$docs_pdf" \
+    --build-arg PIP3="${pip3:-}" \
+    --file "$SCRIPT_DIR/$containerfile" \
+    "$SH_DIR/"
+
+  local -a args_run=(
+    --rm
+    --interactive
+    --tty
+    --volume="$DOCS_DIR:/docs:rw"
+    --workdir=/docs
+    --security-opt label=disable
+  )
+
+  if [ "$OCI" = "docker" ]; then
+    args_run+=(
+      --user="$(id -u)":"$(id -g)"
+    )
+  elif [ "$OCI" = "podman" ]; then
+    # we need net access to fetch bitbake terms
+    args_run+=(
+      --cap-add=NET_RAW
+      --userns=keep-id
+    )
+  fi
+
+  $OCI run \
+    "${args_run[@]}" \
+    "yocto-docs-$sanitized_dockername" \
+    "$@"
+}
+
+main "$@"
diff --git a/documentation/tools/host_packages_scripts/opensuse_docs.sh b/documentation/tools/host_packages_scripts/opensuse_docs.sh
index 7d36eb0f99668168dc4fed44d71181338bdc59a3..896ad52b5042a9679555a30e582fa32076c3a3e7 100644
--- a/documentation/tools/host_packages_scripts/opensuse_docs.sh
+++ b/documentation/tools/host_packages_scripts/opensuse_docs.sh
@@ -1 +1 @@
-sudo zypper install git glibc-i18ndata make python3-pip rsvg-convert which
+sudo zypper --non-interactive install git glibc-i18ndata make python3-pip rsvg-convert which
diff --git a/documentation/tools/host_packages_scripts/opensuse_docs_pdf.sh b/documentation/tools/host_packages_scripts/opensuse_docs_pdf.sh
index ee9f07886ce1c0eac8e1d1118cda8ddc496f0139..f8a3b800d9790c0434f0c3f577ea1916a7fe1b28 100644
--- a/documentation/tools/host_packages_scripts/opensuse_docs_pdf.sh
+++ b/documentation/tools/host_packages_scripts/opensuse_docs_pdf.sh
@@ -1 +1 @@
-sudo zypper install 'texlive-collection-lang*' texlive-collection-fontsextra texlive-collection-fontsrecommended texlive-collection-latex texlive-collection-latexextra texlive-collection-latexrecommended texlive-collection-xetex texlive-fncychap texlive-gnu-freefont texlive-latexmk texlive-tex-gyre texlive-xetex
+sudo zypper --non-interactive install 'texlive-collection-lang*' texlive-collection-fontsextra texlive-collection-fontsrecommended texlive-collection-latex texlive-collection-latexextra texlive-collection-latexrecommended texlive-collection-xetex texlive-fncychap texlive-gnu-freefont texlive-latexmk texlive-tex-gyre texlive-xetex
diff --git a/documentation/tools/host_packages_scripts/opensuse_essential.sh b/documentation/tools/host_packages_scripts/opensuse_essential.sh
index a784f4a5dc05f3734177f08ed97663a04d20a978..19e85f4a736532b890fc55492721cf345c2b4083 100644
--- a/documentation/tools/host_packages_scripts/opensuse_essential.sh
+++ b/documentation/tools/host_packages_scripts/opensuse_essential.sh
@@ -1,2 +1,2 @@
-sudo zypper install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 lz4 make makeinfo patch python python-curses python-xml python3 python3-Jinja2 python3-curses python3-pexpect python3-pip rpcgen socat tar wget which xz zstd
+sudo zypper --non-interactive install bzip2 chrpath diffstat gcc gcc-c++ git gzip hostname libacl1 lz4 make makeinfo patch python python-curses python-xml python3 python3-Jinja2 python3-curses python3-pexpect python3-pip rpcgen socat tar wget which xz zstd
 sudo pip3 install GitPython

-- 
2.47.1