[PATCH v2 1/3] conf.py: add linkcheck builder exclusions for frequent links

[Antonin Godard <antonin.godard@bootlin.com>]

The linkcheck builder can be used to scout for broken links. By looking at
the output of:

  grep -E -r --no-filename -o 'href="http.://[^/"]+' | sort | uniq -c | sort -nr

from the HTML output directory, exclude links that are too frequent.

Remove docs.yoctoproject.org links are those should already be
validated when building the documentation, and add a LINKCHECK_NOT_NICE
env variable that can be set to check for those links anyway.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/README  | 14 ++++++++++++++
 documentation/conf.py | 17 +++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/documentation/README b/documentation/README
index 326930932..39835f014 100644
--- a/documentation/README
+++ b/documentation/README
@@ -163,6 +163,20 @@ or directories:
   $ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
   $ make sphinx-lint SPHINXLINTDOCS=<dir>
 
+Checking for broken links in the Yocto Project documentation
+============================================================
+
+To scout for broken links, the "linkcheck" builder from Sphinx can be used with
+the following command:
+
+  $ make linkcheck
+
+The builder is already configured in conf.py to exclude the links that are too
+frequent in the documentation. You can enable linkcheck for these links by
+setting the LINKCHECK_NOT_NICE environment variable to "1":
+
+  $ LINKCHECK_NOT_NICE=1 make linkcheck
+
 Sphinx theme and CSS customization
 ==================================
 
diff --git a/documentation/conf.py b/documentation/conf.py
index 7b201ebd6..8b50656bf 100644
--- a/documentation/conf.py
+++ b/documentation/conf.py
@@ -143,6 +143,23 @@ suppress_warnings = ['epub.unknown_project_files']
 # sphinx-copybutton configuration
 copybutton_prompt_text = "$ "
 
+# Don't check self-references to yocto-docs since they are already
+# checked when building.
+linkcheck_ignore = [r'https?://docs\.yoctoproject\.org.*']
+
+# When using the linkcheck builder, ignore the following links which are too
+# frequent in the docs, unless the LINKCHECK_NOT_NICE environment variable is set
+# to 1.
+if os.environ.get('LINKCHECK_NOT_NICE') != "1":
+    linkcheck_ignore.extend([
+        r'https?://nvd\.nist\.gov.*',
+        r'https?://git\.yoctoproject\.org.*',
+        r'https?://git\.openembedded\.org.*',
+        r'https?://downloads\.yoctoproject\.org.*',
+        r'https?://mirrors\.kernel\.org.*',
+        r'https?://mirrors\.edge\.kernel\.org.*',
+    ])
+
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for

-- 
2.55.0