[yocto-autobuilder-helper] [PATCH 1/2] scripts/run-docs-build: Add latexmk to tex install

[yocto-autobuilder-helper] [PATCH 1/2] scripts/run-docs-build: Add latexmk to tex install

[Richard Purdie]

Tweak the tex install command to include latexmk which is needed
by the docs build process.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 scripts/run-docs-build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 4ba759c..bf1d261 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -57,7 +57,7 @@ if [ -e /srv/autobuilder/valkyrie.yocto.io/ ]; then
         zcat < install-tl-unx.tar.gz | tar xf -
         cd install-tl-*/
         ./install-tl --scheme=small --texdir=${textooldir} --no-interaction
-        PATH=$PATH:${textooldir}/bin/x86_64-linux tlmgr install titlesec varwidth tabulary needspace upquote framed capt-of wrapfig fncychap gnu-freefont ctex
+        PATH=$PATH:${textooldir}/bin/x86_64-linux tlmgr install titlesec varwidth tabulary needspace upquote framed capt-of wrapfig fncychap gnu-freefont ctex latexmk
         popd
     fi
     PATH=$PATH:${textooldir}/bin/x86_64-linux

[yocto-autobuilder-helper] [PATCH 2/2] scripts/run-docs-build: Add support for multiple buildtools and add modified 5.1 option

[Richard Purdie]

Add a modified 5.1 buildtools tarball for docs builds which has support for svg -> png
conversion using rsvg-convert.

Older releases need older docs buildtools. Add support for this in the build
script for docs, installing both and choosing depending on how old the main
release branch is. For now I've put the changeover at scarthgap, it may be
possible to make it older. I know kirkstone fails with the new tools.

Bitbake doesn't need this since it's docs work fine with both versions.

The actual docs build is done in a subshell so we can choose the environment.
We may need to improve the selection logic in future since some branches may
start to build with the newer tools after backports.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 scripts/run-docs-build | 42 ++++++++++++++++++++++++++++++++++--------
 1 file changed, 34 insertions(+), 8 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index bf1d261..9945d6b 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -25,22 +25,36 @@ outputdir=$builddir/output
 scriptdir="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
 PUBLISH=${PUBLISH:-1}
 sharedir=`jq -r '.["BASE_SHAREDDIR"]' < $HOME/config-local.json`
-buildtools_url=${buildtools_url:-https://downloads.yoctoproject.org/releases/yocto/yocto-4.1.2/buildtools/x86_64-buildtools-docs-nativesdk-standalone-4.1.2.sh}
+# Modified 5.1 tools image to include rsvg, switch to next released version
+buildtools_url=${buildtools_url:-https://downloads.yoctoproject.org/releases/yocto/yocto-4.1.2/buildtools/x86_64-buildtools-docs-nativesdk-standalone-5.1-modified.sh}
+buildtools_url_old=${buildtools_url_old:-https://downloads.yoctoproject.org/releases/yocto/yocto-4.1.2/buildtools/x86_64-buildtools-docs-nativesdk-standalone-4.1.2.sh}
 docbookarchive_url=${docbookarchive_url:-https://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-archives-20201105.tar.xz}
 
+
 buildtools_name=`basename $buildtools_url`
 buildtools_localpath=${sharedir}/cluster-downloads-cache/${buildtools_name}
-
 if [ ! -e ${buildtools_localpath} ]; then
     mkdir -p `dirname ${buildtools_localpath}`
     wget ${buildtools_url} -O ${buildtools_localpath}
     chmod a+x ${buildtools_localpath}
 fi
-
 cd $builddir
 mkdir buildtools
 ${buildtools_localpath} -y -d $builddir/buildtools
-. $builddir/buildtools/environment-setup*
+
+
+buildtools_name_old=`basename $buildtools_url_old`
+buildtools_localpath_old=${sharedir}/cluster-downloads-cache/${buildtools_name_old}
+if [ ! -e ${buildtools_localpath_old} ]; then
+    mkdir -p `dirname ${buildtools_localpath_old}`
+    wget ${buildtools_url_old} -O ${buildtools_localpath_old}
+    chmod a+x ${buildtools_localpath_old}
+fi
+
+cd $builddir
+mkdir buildtools-old
+${buildtools_localpath_old} -y -d $builddir/buildtools-old
+
 
 # To build the latexpdf output we need Xetex (which can handle international characters)
 # Rather than require it on all autobuilder workers, along with dependencies, we install
@@ -99,8 +113,12 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
     git reset --hard
     git clean -ffdx
     git checkout origin/master releases.rst
-    make clean
-    SPHINXOPTS="-j auto" make publish
+
+    (
+        . $builddir/buildtools/environment-setup*
+        make clean
+        SPHINXOPTS="-j auto" make publish
+    )
 
     if [ "$branch" = "master-next" ]; then
         branch="next"
@@ -182,8 +200,16 @@ for branch in "$ypdocsbranch" dunfell $(git branch --remote --contains "$first_s
         esac
     fi
 
-    make clean
-    SPHINXOPTS="-j auto" make publish
+    (
+        # Anything older than scarthgap needs the older buildtools
+        if git merge-base --is-ancestor 0cdc0afd3332459d30cfc8f4c2e62bdcc23f5ed5 HEAD; then
+            . $builddir/buildtools/environment-setup*
+        else
+            . $builddir/buildtools-old/environment-setup*
+        fi
+        make clean
+        SPHINXOPTS="-j auto" make publish
+    )
 
     # Strip yocto- from tag names
     branch=$(echo "$branch" | sed 's/yocto-//')

Re: [yocto-patches] [yocto-autobuilder-helper] [PATCH 2/2] scripts/run-docs-build: Add support for multiple buildtools and add modified 5.1 option

[Richard Purdie]

On Thu, 2024-10-31 at 14:15 +0000, Richard Purdie via
lists.yoctoproject.org wrote:
> Add a modified 5.1 buildtools tarball for docs builds which has
> support for svg -> png
> conversion using rsvg-convert.
> 
> Older releases need older docs buildtools. Add support for this in
> the build
> script for docs, installing both and choosing depending on how old
> the main
> release branch is. For now I've put the changeover at scarthgap, it
> may be
> possible to make it older. I know kirkstone fails with the new tools.
> 
> Bitbake doesn't need this since it's docs work fine with both
> versions.
> 
> The actual docs build is done in a subshell so we can choose the
> environment.
> We may need to improve the selection logic in future since some
> branches may
> start to build with the newer tools after backports.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Just to be clear, the new docs buildtools includes the latest sphinx so
this lets us start moving to new spinx versions while at the same time
being able to build our older docs. We've wanted to do that for a
while.

Cheers,

Richard