[PATCH 00/10] meson: wire up missing HTML documentation

[PATCH 00/10] meson: wire up missing HTML documentation

[Patrick Steinhardt]

Hi,

this patch series wires up missing HTML-based documentation with Meson.
This includes a couple of missing manpages, the user manual as well as
the random set of articles that we have. It also starts to generate the
indices for API docs and howtos so that the result is a complete set of
HTML docs, same as with our Makefile. It also fixes a couple of smaller
issues I found while working on the series.

Notably missing yet is an integration with CI as well as sanity checks
for any kind of missing docs in Meson. I'll work on this in a separate
patch series once the initial CI integration as well as this patch
series here have landed.

Further missing is the generation of both info pages and a user manual
PDF. I couldn't find any users of these anywhere in downstream distros,
so I decided to not care for now until somebody complains.

The series is built on top of caacdb5dfd (The fifteenth batch,
2024-12-10) with ps/build at 904339edbd (Introduce support for the Meson
build system, 2024-12-06) merged into it.

Thanks!

Patrick

---
Patrick Steinhardt (10):
      meson: wire up support for AsciiDoctor
      meson: properly wire up dependencies for our docs
      meson: fix generation of merge tools
      meson: generate HTML pages for all man page categories
      Documentation: inline user-manual.conf
      meson: generate user manual
      Documentation: refactor "api-index.sh" for out-of-tree builds
      Documentation: refactor "howto-index.sh" for out-of-tree builds
      meson: generate articles
      meson: install static files for HTML documentation

 Documentation/Makefile                   |   8 +-
 Documentation/asciidoc.conf.in           |  10 ++
 Documentation/{ => howto}/howto-index.sh |   2 +-
 Documentation/howto/meson.build          |  62 +++++++++
 Documentation/meson.build                | 221 +++++++++++++++++++++++++------
 Documentation/technical/api-index.sh     |  19 ++-
 Documentation/technical/meson.build      |  66 +++++++++
 Documentation/user-manual.conf           |  11 --
 meson_options.txt                        |   2 +
 9 files changed, 344 insertions(+), 57 deletions(-)


---
base-commit: 0b8924716a9b7975cb21e464917bb475de842a27
change-id: 20241212-b4-pks-meson-docs-2634bf3e7764

[PATCH 01/10] meson: wire up support for AsciiDoctor

[Patrick Steinhardt]

While our Makefile supports both Asciidoc and AsciiDoctor, our Meson
build instructions only support the former. Wire up support for the
latter, as well.

Our Makefile always favors Asciidoc, but Meson will automatically figure
out which of both to use based on whether they are installed or not. To
keep compatibility with our Makefile it favors Asciidoc over Asciidoctor
in case both are available.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 107 ++++++++++++++++++++++++++++++++++------------
 meson_options.txt         |   2 +
 2 files changed, 82 insertions(+), 27 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index f2426ccaa30c29bd60b850eb0a9a4ab77c66a629..d62b0846d3f8ebc412f5fa9f775f037a3656093a 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -204,28 +204,85 @@ manpages = {
   'gitworkflows.txt' : 7,
 }
 
-asciidoc = find_program('asciidoc')
-git = find_program('git', required: false)
-xmlto = find_program('xmlto')
+docs_backend = get_option('docs_backend')
+if docs_backend == 'auto'
+  if find_program('asciidoc', required: false).found()
+    docs_backend = 'asciidoc'
+  elif find_program('asciidoctor', required: false).found()
+    docs_backend = 'asciidoctor'
+  else
+    error('Neither asciidoc nor asciidoctor were found.')
+  endif
+endif
 
-asciidoc_conf = custom_target(
-  command: [
-    shell,
-    meson.project_source_root() / 'GIT-VERSION-GEN',
-    meson.project_source_root(),
-    '@INPUT@',
-    '@OUTPUT@',
-  ],
-  input: meson.current_source_dir() / 'asciidoc.conf.in',
-  output: 'asciidoc.conf',
-  depends: [git_version_file],
-)
+if docs_backend == 'asciidoc'
+  asciidoc = find_program('asciidoc', required: true)
+  asciidoc_html = 'xhtml11'
+  asciidoc_docbook = 'docbook'
+  xmlto_extra = [ ]
 
-asciidoc_common_options = [
-  asciidoc,
-  '--conf-file=' + asciidoc_conf.full_path(),
-  '--attribute=build_dir=' + meson.current_build_dir(),
-]
+  asciidoc_conf = custom_target(
+    command: [
+      shell,
+      meson.project_source_root() / 'GIT-VERSION-GEN',
+      meson.project_source_root(),
+      '@INPUT@',
+      '@OUTPUT@',
+    ],
+    input: meson.current_source_dir() / 'asciidoc.conf.in',
+    output: 'asciidoc.conf',
+    depends: [git_version_file],
+  )
+
+  asciidoc_common_options = [
+    asciidoc,
+    '--conf-file=' + asciidoc_conf.full_path(),
+    '--attribute=build_dir=' + meson.current_build_dir(),
+  ]
+
+  documentation_deps = [
+    asciidoc_conf,
+  ]
+elif docs_backend == 'asciidoctor'
+  asciidoctor = find_program('asciidoctor', required: true)
+  asciidoc_html = 'xhtml5'
+  asciidoc_docbook = 'docbook5'
+  xmlto_extra = [
+    '--skip-validation',
+    '-x', meson.current_source_dir() / 'manpage.xsl',
+  ]
+
+  asciidoctor_extensions = custom_target(
+    command: [
+      shell,
+      meson.project_source_root() / 'GIT-VERSION-GEN',
+      meson.project_source_root(),
+      '@INPUT@',
+      '@OUTPUT@',
+    ],
+    input: meson.current_source_dir() / 'asciidoctor-extensions.rb.in',
+    output: 'asciidoctor-extensions.rb',
+    depends: [git_version_file],
+  )
+
+  asciidoc_common_options = [
+    asciidoctor,
+    '--attribute', 'compat-mode',
+    '--attribute', 'tabsize=8',
+    '--attribute', 'litdd=&#x2d;&#x2d;',
+    '--attribute', 'docinfo=shared',
+    '--attribute', 'build_dir=' + meson.current_build_dir(),
+    '--load-path', meson.current_build_dir(),
+    '--require', 'asciidoctor-extensions',
+  ]
+
+  documentation_deps = [
+    asciidoctor_extensions,
+  ]
+endif
+
+git = find_program('git', required: false)
+xmlto = find_program('xmlto')
 
 cmd_lists = [
   'cmds-ancillaryinterrogators.txt',
@@ -242,10 +299,6 @@ cmd_lists = [
   'cmds-foreignscminterface.txt',
 ]
 
-documentation_deps = [
-  asciidoc_conf,
-]
-
 documentation_deps += custom_target(
   command: [
     perl,
@@ -277,7 +330,7 @@ foreach manpage, category : manpages
   if get_option('docs').contains('man')
     manpage_xml_target = custom_target(
       command: asciidoc_common_options + [
-        '--backend=docbook',
+        '--backend=' + asciidoc_docbook,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
         meson.current_source_dir() / manpage,
@@ -300,7 +353,7 @@ foreach manpage, category : manpages
         manpage_xml_target,
         '-o',
         meson.current_build_dir(),
-      ],
+      ] + xmlto_extra,
       output: manpage_path,
       install: true,
       install_dir: get_option('mandir') / 'man' + category.to_string(),
@@ -310,7 +363,7 @@ foreach manpage, category : manpages
   if get_option('docs').contains('html') and category == 1
     custom_target(
       command: asciidoc_common_options + [
-        '--backend=xhtml11',
+        '--backend=' + asciidoc_html,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
         meson.current_source_dir() / manpage,
diff --git a/meson_options.txt b/meson_options.txt
index 32a72139bae870745d9131cc9086a4594826be91..0d8ba28de6da7d0ed2ac4bdef7efa967509ec898 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -73,6 +73,8 @@ option('docs', type: 'array', choices: ['man', 'html'], value: [],
   description: 'Which documenattion formats to build and install.')
 option('default_help_format', type: 'combo', choices: ['man', 'html'], value: 'man',
   description: 'Default format used when executing git-help(1).')
+option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto',
+  description: 'Which backend to use to generate documentation.')
 
 # Testing.
 option('tests', type: 'boolean', value: true,

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 02/10] meson: properly wire up dependencies for our docs

[Patrick Steinhardt]

A couple of Meson documentation targets use `meson.current_source_dir()`
to resolve inputs. This has the downside that it does not automagically
make Meson track these inputs as a dependency. After all, string
arguments really can be anything, even if they happen to match an actual
filesystem path.

Adapt these build targets to instead use inputs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index d62b0846d3f8ebc412f5fa9f775f037a3656093a..d23ed82795026e511379ff1e77355d2ec33ec499 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -229,7 +229,7 @@ if docs_backend == 'asciidoc'
       '@INPUT@',
       '@OUTPUT@',
     ],
-    input: meson.current_source_dir() / 'asciidoc.conf.in',
+    input: 'asciidoc.conf.in',
     output: 'asciidoc.conf',
     depends: [git_version_file],
   )
@@ -260,7 +260,7 @@ elif docs_backend == 'asciidoctor'
       '@INPUT@',
       '@OUTPUT@',
     ],
-    input: meson.current_source_dir() / 'asciidoctor-extensions.rb.in',
+    input: 'asciidoctor-extensions.rb.in',
     output: 'asciidoctor-extensions.rb',
     depends: [git_version_file],
   )
@@ -302,10 +302,11 @@ cmd_lists = [
 documentation_deps += custom_target(
   command: [
     perl,
-    meson.current_source_dir() / 'cmd-list.perl',
+    '@INPUT@',
     meson.project_source_root(),
     meson.current_build_dir(),
   ] + cmd_lists,
+  input: 'cmd-list.perl',
   output: cmd_lists
 )
 
@@ -313,7 +314,7 @@ foreach mode : [ 'diff', 'merge' ]
   documentation_deps += custom_target(
     command: [
       shell,
-      meson.current_source_dir() / 'generate-mergetool-list.sh',
+      '@INPUT@',
       '..',
       'diff',
       '@OUTPUT@'
@@ -322,6 +323,7 @@ foreach mode : [ 'diff', 'merge' ]
       'MERGE_TOOLS_DIR=' + meson.project_source_root() / 'mergetools',
       'TOOL_MODE=' + mode,
     ],
+    input: 'generate-mergetool-list.sh',
     output: 'mergetools-' + mode + '.txt',
   )
 endforeach
@@ -333,9 +335,10 @@ foreach manpage, category : manpages
         '--backend=' + asciidoc_docbook,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
-        meson.current_source_dir() / manpage,
+        '@INPUT@',
       ],
       depends: documentation_deps,
+      input: manpage,
       output: fs.stem(manpage) + '.xml',
     )
 
@@ -343,10 +346,8 @@ foreach manpage, category : manpages
     manpage_target = custom_target(
       command: [
         xmlto,
-        '-m',
-        meson.current_source_dir() / 'manpage-normal.xsl',
-        '-m',
-        meson.current_source_dir() / 'manpage-bold-literal.xsl',
+        '-m', '@INPUT0@',
+        '-m', '@INPUT1@',
         '--stringparam',
         'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'),
         'man',
@@ -354,6 +355,10 @@ foreach manpage, category : manpages
         '-o',
         meson.current_build_dir(),
       ] + xmlto_extra,
+      input: [
+        'manpage-normal.xsl',
+        'manpage-bold-literal.xsl',
+      ],
       output: manpage_path,
       install: true,
       install_dir: get_option('mandir') / 'man' + category.to_string(),
@@ -366,9 +371,10 @@ foreach manpage, category : manpages
         '--backend=' + asciidoc_html,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
-        meson.current_source_dir() / manpage,
+        '@INPUT@',
       ],
       depends: documentation_deps,
+      input: manpage,
       output: fs.stem(manpage) + '.html',
       install: true,
       install_dir: get_option('datadir') / 'doc/git-doc',

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 03/10] meson: fix generation of merge tools

[Patrick Steinhardt]

Our buildsystems generate a list of diff and merge tools that ultimately
end up in our documentation. And while Meson does wire up the logic, it
tries to use the TOOL_MODE environment variable to set up the mode. This
is wrong though: the mode is set via an argument that we have fixed to
'diff' mode by accident.

Fix this such that merge tools are properly generated.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index d23ed82795026e511379ff1e77355d2ec33ec499..0d8b58145274c7854fe3fd91de469fe9d1e0bb6f 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -316,12 +316,11 @@ foreach mode : [ 'diff', 'merge' ]
       shell,
       '@INPUT@',
       '..',
-      'diff',
+      mode,
       '@OUTPUT@'
     ],
     env: [
       'MERGE_TOOLS_DIR=' + meson.project_source_root() / 'mergetools',
-      'TOOL_MODE=' + mode,
     ],
     input: 'generate-mergetool-list.sh',
     output: 'mergetools-' + mode + '.txt',

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 04/10] meson: generate HTML pages for all man page categories

[Patrick Steinhardt]

When generating HTML pages for our man pages we only generate them for
category 1 in MEson, which are the pages corresponding to our built-in
commands. I cannot tell why I added this filter though: our Makefile
installs all man pages, so a Meson-based build misses out on many of
them.

Fix this by removing the filter.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 0d8b58145274c7854fe3fd91de469fe9d1e0bb6f..d36b2b0d8e7795d0520976c1e54a2f90b332cacb 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -364,7 +364,7 @@ foreach manpage, category : manpages
     )
   endif
 
-  if get_option('docs').contains('html') and category == 1
+  if get_option('docs').contains('html')
     custom_target(
       command: asciidoc_common_options + [
         '--backend=' + asciidoc_html,

-- 
2.47.1.668.gf74b3f243a.dirty

Re: [PATCH 04/10] meson: generate HTML pages for all man page categories

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> When generating HTML pages for our man pages we only generate them for
> category 1 in MEson, which are the pages corresponding to our built-in

The tiniest nit: I don't think you intended to spell Meson with a
capital E.

-- 
Toon

Re: [PATCH 04/10] meson: generate HTML pages for all man page categories

[Patrick Steinhardt]

On Mon, Dec 23, 2024 at 12:52:25PM +0100, Toon Claes wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > When generating HTML pages for our man pages we only generate them for
> > category 1 in MEson, which are the pages corresponding to our built-in
> 
> The tiniest nit: I don't think you intended to spell Meson with a
> capital E.

Indeed, fixed now.

Patrick

[PATCH 05/10] Documentation: inline user-manual.conf

[Patrick Steinhardt]

When generating our user manual we set up a bit of extra configuration
compared to our normal configuration. This is done by having an extra
"user-manual.conf" file that Asciidoc seems to pull in automatically due
to matching filenames with "user-manual.txt". This dependency is quite
hidden though and thus easy to miss. Furthermore, it seems that Asciidoc
does not know to pull it in for out-of-tree builds where we use relative
paths.

The setup in AsciiDoctor is somewhat different: instead of having two
sets of configuration, we condition the use of manual-specific configs
based on whether the document type is "book". And as we only build our
user manual with that type this is sufficient.

Use the same trick for our user manual by inlining the configuration
into "asciidoc.conf.in" and making it conditional on whether or not
"doctype-book" is defined.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile         |  2 +-
 Documentation/asciidoc.conf.in | 10 ++++++++++
 Documentation/user-manual.conf | 11 -----------
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 3392e1ce7ebc540784912476847380d9c1775ac8..31c17f7d655e1dcbdde315115609b798363e7328 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -367,7 +367,7 @@ manpage-cmd = $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
 %.xml : %.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_XML) -d manpage -o $@ $<
 
-user-manual.xml: user-manual.txt user-manual.conf $(ASCIIDOC_DEPS)
+user-manual.xml: user-manual.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_XML) -d book -o $@ $<
 
 technical/api-index.txt: technical/api-index-skel.txt \
diff --git a/Documentation/asciidoc.conf.in b/Documentation/asciidoc.conf.in
index dbe36a52eabfabef59e31d3be6518549e4f90206..83ddbb76f65d7a041e4e787a81e19ff1db1d9d55 100644
--- a/Documentation/asciidoc.conf.in
+++ b/Documentation/asciidoc.conf.in
@@ -25,12 +25,22 @@ manmanual='Git Manual'
 mansource='Git @GIT_VERSION@'
 revdate='@GIT_DATE@'
 
+ifdef::doctype-book[]
+[titles]
+	underlines="__","==","--","~~","^^"
+endif::doctype-book[]
+
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]
+ifndef::doctype-book[]
 {0%{target}}
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+endif::doctype-book[]
+ifdef::doctype-book[]
+<ulink url="{target}.html">{target}{0?({0})}</ulink>
+endif::doctype-book[]
 
 [literal-inlinemacro]
 {eval:re.sub(r'(&lt;[-a-zA-Z0-9.]+&gt;)', r'<emphasis>\1</emphasis>', re.sub(r'([\[\s|()>]|^|\]|&gt;)(\.?([-a-zA-Z0-9:+=~@,\/_^\$]+\.?)+)',r'\1<literal>\2</literal>', re.sub(r'(\.\.\.?)([^\]$.])', r'<literal>\1</literal>\2', macros.passthroughs[int(attrs['passtext'][1:-1])] if attrs['passtext'][1:-1].isnumeric() else attrs['passtext'][1:-1])))}
diff --git a/Documentation/user-manual.conf b/Documentation/user-manual.conf
deleted file mode 100644
index 0148f126dcdf6aca15a5560fb5b122b85b022461..0000000000000000000000000000000000000000
--- a/Documentation/user-manual.conf
+++ /dev/null
@@ -1,11 +0,0 @@
-[titles]
-	underlines="__","==","--","~~","^^"
-
-[attributes]
-caret=^
-startsb=&#91;
-endsb=&#93;
-tilde=&#126;
-
-[linkgit-inlinemacro]
-<ulink url="{target}.html">{target}{0?({0})}</ulink>

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 06/10] meson: generate user manual

[Patrick Steinhardt]

Our documentation contains a user manual that gives people a short
introduction to Git. Our Makefile knows to generate the manual into
three different formats: an HTML page, a PDF and an info page. The Meson
build instructions don't yet generate any of these.

While wiring up all these formats I hit a couple of road blocks with how
we generate our info pages. Even though I eventually resolved these, it
made me question whether anybody actually uses info pages in the first
place. Checking through a couple of downstream consumers I couldn't find
a single user of either the info pages nor of our PDF manual in Arch
Linux, Debian, Fedora, Ubuntu, FreeBSD or OpenBSDFedora. So it's rather
safe to assume that there aren't really any users out there, and thus
the added complexity does not seem worth it.

Wire up support for building the user manual in HTML format and
conciously skip over the other two formats. This is basically a form of
silent deprecation: if people out there use the other two formats they
will eventually complain about them missing in Meson, which means we can
wire them up at a later point. If they don't we can phase out these
formats eventually.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index d36b2b0d8e7795d0520976c1e54a2f90b332cacb..1fdc6a61eb04707d7c4b7aabb412b32ddc517dc7 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -380,3 +380,35 @@ foreach manpage, category : manpages
     )
   endif
 endforeach
+
+if get_option('docs').contains('html')
+  xsltproc = find_program('xsltproc')
+
+  user_manual_xml = custom_target(
+    command: asciidoc_common_options + [
+      '--backend=' + asciidoc_docbook,
+      '--doctype=book',
+      '--out-file=@OUTPUT@',
+      '@INPUT@',
+    ],
+    input: 'user-manual.txt',
+    output: 'user-manual.xml',
+    depends: documentation_deps,
+  )
+
+  custom_target(
+    command: [
+      xsltproc,
+      '--xinclude',
+      '--stringparam', 'html.stylesheet', 'docbook-xsl.css',
+      '--param', 'generate.consistent.ids', '1',
+      '--output', '@OUTPUT@',
+      '@INPUT@',
+      user_manual_xml,
+    ],
+    input: 'docbook.xsl',
+    output: 'user-manual.html',
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+endif

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 07/10] Documentation: refactor "api-index.sh" for out-of-tree builds

[Patrick Steinhardt]

The "api-index.sh" script generates an index of API-related
documentation. The script does not handle out-of-tree builds and thus
cannot be used easily by Meson.

Refactor it to be independent of locations by both accepting a source
directory where the API docs live as well as a path to an output file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile               |  2 +-
 Documentation/technical/api-index.sh | 19 +++++++++++++++----
 2 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 31c17f7d655e1dcbdde315115609b798363e7328..44f68e7a53843dc5ea24085d5f48b592d34aec41 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -372,7 +372,7 @@ user-manual.xml: user-manual.txt $(ASCIIDOC_DEPS)
 
 technical/api-index.txt: technical/api-index-skel.txt \
 	technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
-	$(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh
+	$(QUIET_GEN)'$(SHELL_PATH_SQ)' technical/api-index.sh ./technical ./technical/api-index.txt
 
 technical/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
 $(patsubst %,%.html,$(API_DOCS) technical/api-index $(TECH_DOCS)): %.html : %.txt \
diff --git a/Documentation/technical/api-index.sh b/Documentation/technical/api-index.sh
index 9c3f4131b8586408acd81d1e60912b51688575ed..296488557434b7fff60ab25f4246a4dc270729c0 100755
--- a/Documentation/technical/api-index.sh
+++ b/Documentation/technical/api-index.sh
@@ -1,6 +1,17 @@
 #!/bin/sh
 
+if test $# -ne 2
+then
+	echo >&2 "USAGE: $0 <SOURCE_DIR> <OUTPUT>"
+	exit 1
+fi
+
+SOURCE_DIR="$1"
+OUTPUT="$2"
+
 (
+	cd "$SOURCE_DIR"
+
 	c=////////////////////////////////////////////////////////////////
 	skel=api-index-skel.txt
 	sed -e '/^\/\/ table of contents begin/q' "$skel"
@@ -18,11 +29,11 @@
 	done
 	echo "$c"
 	sed -n -e '/^\/\/ table of contents end/,$p' "$skel"
-) >api-index.txt+
+) >"$OUTPUT"+
 
-if test -f api-index.txt && cmp api-index.txt api-index.txt+ >/dev/null
+if test -f "$OUTPUT" && cmp "$OUTPUT" "$OUTPUT"+ >/dev/null
 then
-	rm -f api-index.txt+
+	rm -f "$OUTPUT"+
 else
-	mv api-index.txt+ api-index.txt
+	mv "$OUTPUT"+ "$OUTPUT"
 fi

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 08/10] Documentation: refactor "howto-index.sh" for out-of-tree builds

[Patrick Steinhardt]

The "howto-index.sh" is used to generate an index of our how-to docs. It
receives as input the paths to these documents, which would typically be
relative to the "Documentation/" directory in Makefile-based builds. In
an out-of-tree build though it will get relative that may be rooted
somewhere else entirely.

The file paths do end up in the generated index, and the expectation is
that they should always start with "howto/". But for out-of-tree builds
we would populate it with the paths relative to the build directory,
which is wrong.

Fix the issue by using `$(basename "$file")` to generate the path. While
at it, move the script into "howto/" to align it with the location of
the comparable "api-index.sh" script.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile                   | 4 ++--
 Documentation/{ => howto}/howto-index.sh | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 44f68e7a53843dc5ea24085d5f48b592d34aec41..388b5ffef99f696948042ad2bed87d573fbd4e95 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -416,8 +416,8 @@ gitman.info: gitman.texi
 $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml
 	$(QUIET_DB2TEXI)$(DOCBOOK2X_TEXI) --to-stdout $*.xml >$@
 
-howto-index.txt: howto-index.sh $(HOWTO_TXT)
-	$(QUIET_GEN)'$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(HOWTO_TXT)) >$@
+howto-index.txt: howto/howto-index.sh $(HOWTO_TXT)
+	$(QUIET_GEN)'$(SHELL_PATH_SQ)' ./howto/howto-index.sh $(sort $(HOWTO_TXT)) >$@
 
 $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_HTML) $*.txt
diff --git a/Documentation/howto-index.sh b/Documentation/howto/howto-index.sh
similarity index 92%
rename from Documentation/howto-index.sh
rename to Documentation/howto/howto-index.sh
index 167b363668b8b53d752d5971798d3ca26c8f7f1f..eecd123a93607998e8b4eb8511f4165973f9d93e 100755
--- a/Documentation/howto-index.sh
+++ b/Documentation/howto/howto-index.sh
@@ -48,7 +48,7 @@ do
 		file="$txt"
 	fi
 
-	echo "* link:$file[$title] $from
+	echo "* link:howto/$(basename "$file")[$title] $from
 $abstract
 
 "

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 09/10] meson: generate articles

[Patrick Steinhardt]

While the Meson build system already knows to generate man pages and our
user manual, it does not yet generate the random assortment of articles
that we have. Plug this gap.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/howto/meson.build     | 62 ++++++++++++++++++++++++++++++++++
 Documentation/meson.build           | 36 ++++++++++++++++++++
 Documentation/technical/meson.build | 66 +++++++++++++++++++++++++++++++++++++
 3 files changed, 164 insertions(+)

diff --git a/Documentation/howto/meson.build b/Documentation/howto/meson.build
new file mode 100644
index 0000000000000000000000000000000000000000..c023c104161e61ca0399b5390d59e20343746621
--- /dev/null
+++ b/Documentation/howto/meson.build
@@ -0,0 +1,62 @@
+howto_sources = [
+  'coordinate-embargoed-releases.txt',
+  'keep-canonical-history-correct.txt',
+  'maintain-git.txt',
+  'new-command.txt',
+  'rebase-from-internal-branch.txt',
+  'rebuild-from-update-hook.txt',
+  'recover-corrupted-blob-object.txt',
+  'recover-corrupted-object-harder.txt',
+  'revert-a-faulty-merge.txt',
+  'revert-branch-rebase.txt',
+  'separating-topic-branches.txt',
+  'setup-git-server-over-http.txt',
+  'update-hook-example.txt',
+  'use-git-daemon.txt',
+  'using-merge-subtree.txt',
+  'using-signed-tag-in-pull-request.txt',
+]
+
+howto_index = custom_target(
+  command: [
+    shell,
+    meson.current_source_dir() / 'howto-index.sh',
+    '@INPUT@',
+  ],
+  env: script_environment,
+  capture: true,
+  input: howto_sources,
+  output: 'howto-index.txt',
+)
+
+custom_target(
+  command: asciidoc_html_options,
+  input: howto_index,
+  output: 'howto-index.html',
+  depends: documentation_deps,
+  install: true,
+  install_dir: get_option('datadir') / 'doc/git-doc',
+)
+
+foreach howto : howto_sources
+  howto_stripped = custom_target(
+    command: [
+      find_program('sed'),
+      '-e',
+      '1,/^$/d',
+      '@INPUT@',
+    ],
+    input: howto,
+    output: fs.stem(howto) + '.stripped',
+    capture: true,
+  )
+
+  custom_target(
+    command: asciidoc_html_options,
+    input: howto_stripped,
+    output: fs.stem(howto_stripped.full_path()) + '.html',
+    depends: documentation_deps,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc/howto',
+  )
+endforeach
diff --git a/Documentation/meson.build b/Documentation/meson.build
index 1fdc6a61eb04707d7c4b7aabb412b32ddc517dc7..1dd84af2d6bcf3214cfa1ec78c00359f64040fca 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -411,4 +411,40 @@ if get_option('docs').contains('html')
     install: true,
     install_dir: get_option('datadir') / 'doc/git-doc',
   )
+
+  articles = [
+    'DecisionMaking.txt',
+    'MyFirstContribution.txt',
+    'MyFirstObjectWalk.txt',
+    'ReviewingGuidelines.txt',
+    'SubmittingPatches',
+    'ToolsForGit.txt',
+    'git-bisect-lk2009.txt',
+    'git-tools.txt',
+  ]
+
+  foreach article : articles
+    custom_target(
+      command: asciidoc_common_options + [
+        '--backend=' + asciidoc_html,
+        '--out-file=@OUTPUT@',
+        '@INPUT@',
+      ],
+      input: article,
+      output: fs.stem(article) + '.html',
+      depends: documentation_deps,
+      install: true,
+      install_dir: get_option('datadir') / 'doc/git-doc',
+    )
+  endforeach
+
+  asciidoc_html_options = asciidoc_common_options + [
+    '--backend=' + asciidoc_html,
+    '--out-file=@OUTPUT@',
+    '--attribute', 'git-relative-html-prefix=../',
+    '@INPUT@',
+  ]
+
+  subdir('howto')
+  subdir('technical')
 endif
diff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build
new file mode 100644
index 0000000000000000000000000000000000000000..21dfb8b5c9d93490f16c24bab1d49631e0395571
--- /dev/null
+++ b/Documentation/technical/meson.build
@@ -0,0 +1,66 @@
+api_docs = [
+  'api-error-handling.txt',
+  'api-merge.txt',
+  'api-parse-options.txt',
+  'api-simple-ipc.txt',
+  'api-trace2.txt',
+]
+
+articles = [
+  'bitmap-format.txt',
+  'build-systems.txt',
+  'bundle-uri.txt',
+  'commit-graph.txt',
+  'directory-rename-detection.txt',
+  'hash-function-transition.txt',
+  'long-running-process-protocol.txt',
+  'multi-pack-index.txt',
+  'packfile-uri.txt',
+  'pack-heuristics.txt',
+  'parallel-checkout.txt',
+  'partial-clone.txt',
+  'platform-support.txt',
+  'racy-git.txt',
+  'reftable.txt',
+  'remembering-renames.txt',
+  'repository-version.txt',
+  'rerere.txt',
+  'scalar.txt',
+  'send-pack-pipeline.txt',
+  'shallow.txt',
+  'sparse-checkout.txt',
+  'sparse-index.txt',
+  'trivial-merge.txt',
+  'unit-tests.txt',
+]
+
+api_index = custom_target(
+  command: [
+    shell,
+    meson.current_source_dir() / 'api-index.sh',
+    meson.current_source_dir(),
+    '@OUTPUT@',
+  ],
+  env: script_environment,
+  input: api_docs,
+  output: 'api-index.txt',
+)
+
+custom_target(
+  command: asciidoc_html_options,
+  input: api_index,
+  output: 'api-index.html',
+  depends: documentation_deps,
+  install: true,
+  install_dir: get_option('datadir') / 'doc/git-doc/technical',
+)
+
+foreach article : api_docs + articles
+  custom_target(
+    command: asciidoc_html_options,
+    input: article,
+    output: fs.stem(article) + '.html',
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc/technical',
+  )
+endforeach

-- 
2.47.1.668.gf74b3f243a.dirty

[PATCH 10/10] meson: install static files for HTML documentation

[Patrick Steinhardt]

Now that we generate man pages, articles and user manual with Meson the
only thing that is still missing in an installation of HTML documents is
a couple of static files. Wire these up to finalize Meson's support for
generating HTML documentation.

Diffing an installation that uses our Makefile with an installation that
uses Meson only surfaces a couple of discepancies now:

  - Meson doesn't install "everyday.html" and "git-remote-helpers.html".
    These files are marked as obsolete and don't contain any useful
    information anymore: they simply point to their modern equivalents.

  - Meson doesn't install "*.txt" files when asking for HTML docs. I'm
    not sure why our Makefiles do this in the first place, and it does
    seem like the resulting installation is fully functional even
    without those files.

Other than that, both layout and file contents are the exact same.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 1dd84af2d6bcf3214cfa1ec78c00359f64040fca..751d32c4847289da5dbbe63efbce309d36719f4f 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -382,6 +382,27 @@ foreach manpage, category : manpages
 endforeach
 
 if get_option('docs').contains('html')
+  configure_file(
+    input: 'docinfo-html.in',
+    output: 'docinfo.html',
+    copy: true,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+
+  configure_file(
+    input: 'docbook-xsl.css',
+    output: 'docbook-xsl.css',
+    copy: true,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+
+  install_symlink('index.html',
+    install_dir: get_option('datadir') / 'doc/git-doc',
+    pointing_to: 'git.html',
+  )
+
   xsltproc = find_program('xsltproc')
 
   user_manual_xml = custom_target(

-- 
2.47.1.668.gf74b3f243a.dirty

Re: [PATCH 00/10] meson: wire up missing HTML documentation

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> Hi,
>
> this patch series wires up missing HTML-based documentation with Meson.
> This includes a couple of missing manpages, the user manual as well as
> the random set of articles that we have. It also starts to generate the
> indices for API docs and howtos so that the result is a complete set of
> HTML docs, same as with our Makefile. It also fixes a couple of smaller
> issues I found while working on the series.
>
> Notably missing yet is an integration with CI as well as sanity checks
> for any kind of missing docs in Meson. I'll work on this in a separate
> patch series once the initial CI integration as well as this patch
> series here have landed.
>
> Further missing is the generation of both info pages and a user manual
> PDF. I couldn't find any users of these anywhere in downstream distros,
> so I decided to not care for now until somebody complains.
>
> The series is built on top of caacdb5dfd (The fifteenth batch,
> 2024-12-10) with ps/build at 904339edbd (Introduce support for the Meson
> build system, 2024-12-06) merged into it.

Hi Patrick,

I've been reading through the patches, and as far as I understand it
makes sense. But to be honest, I don't know how to use this. I have
almost no experience with Meson and I only know `meson setup` and `meson
compile`. But the `meson.build` from Documentation/ is marked as a
subdir() if option "docs" is given. But I don't understand how this
should be used. For `meson test` there are some instructions in the
root-level meson.build, but not for the docs. Should we add this as
well?

And a bit related to this, I saw you use `env: script_environment` in a
few places, how does this get injected from the root-level meson.build
file? Due to this, I assume it's intended to only use the root-level
meson.build directly, and not run `meson setup` in the Documentation/
folder?

-- 
Toon

Re: [PATCH 00/10] meson: wire up missing HTML documentation

[Patrick Steinhardt]

On Mon, Dec 23, 2024 at 12:51:54PM +0100, Toon Claes wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Hi,
> >
> > this patch series wires up missing HTML-based documentation with Meson.
> > This includes a couple of missing manpages, the user manual as well as
> > the random set of articles that we have. It also starts to generate the
> > indices for API docs and howtos so that the result is a complete set of
> > HTML docs, same as with our Makefile. It also fixes a couple of smaller
> > issues I found while working on the series.
> >
> > Notably missing yet is an integration with CI as well as sanity checks
> > for any kind of missing docs in Meson. I'll work on this in a separate
> > patch series once the initial CI integration as well as this patch
> > series here have landed.
> >
> > Further missing is the generation of both info pages and a user manual
> > PDF. I couldn't find any users of these anywhere in downstream distros,
> > so I decided to not care for now until somebody complains.
> >
> > The series is built on top of caacdb5dfd (The fifteenth batch,
> > 2024-12-10) with ps/build at 904339edbd (Introduce support for the Meson
> > build system, 2024-12-06) merged into it.
> 
> Hi Patrick,
> 
> I've been reading through the patches, and as far as I understand it
> makes sense. But to be honest, I don't know how to use this. I have
> almost no experience with Meson and I only know `meson setup` and `meson
> compile`. But the `meson.build` from Documentation/ is marked as a
> subdir() if option "docs" is given. But I don't understand how this
> should be used. For `meson test` there are some instructions in the
> root-level meson.build, but not for the docs. Should we add this as
> well?

I don't really think it makes sense to explicitly point out every option
that we have. We already document how to discover and set options, and
from hereon it follows that you can wire up docs by running for example
`meson setup -Ddocs=man ..`. It's just another option, and as such it
can be discovered by running `meson configure`.

The benefit of this is that it cannot grow stale like the build options
in our Makefile. These may or may not have documentation, and may or may
not be stale. With Meson, every build option is listed explicitly, has
documentation and is discoverable via `meson configure`.

> And a bit related to this, I saw you use `env: script_environment` in a
> few places, how does this get injected from the root-level meson.build
> file? Due to this, I assume it's intended to only use the root-level
> meson.build directly, and not run `meson setup` in the Documentation/
> folder?

Yup, you are always expected to set up the top-level source directory,
not any of the subdirectories. The build instructions are then processed
linearly in Meson, so variables declared before a call to `subdir()`
would be accessible in the subdirectory, as well.

Patrick

Re: How to use Meson (was: [PATCH 00/10] meson: wire up missing HTML documentation])

[Toon Claes]

Patrick Steinhardt <ps@pks.im> writes:

> I don't really think it makes sense to explicitly point out every
> option that we have. We already document how to discover and set
> options, and from hereon it follows that you can wire up docs by
> running for example `meson setup -Ddocs=man ..`. It's just another
> option, and as such it can be discovered by running `meson configure`.

This is something I wasn't aware of. Because I'm used to the Makefile
workflow and I'm not familiar with Meson, I didn't expect it to work
like that.

> The benefit of this is that it cannot grow stale like the build options
> in our Makefile. These may or may not have documentation, and may or may
> not be stale. With Meson, every build option is listed explicitly, has
> documentation and is discoverable via `meson configure`.

That's awesome, and I totally I agree we use the benefit of this
self-documenting feature of Meson. Again, I didn't know about that. It's
more of a me-problem than with your code.

> Yup, you are always expected to set up the top-level source directory,
> not any of the subdirectories. The build instructions are then processed
> linearly in Meson, so variables declared before a call to `subdir()`
> would be accessible in the subdirectory, as well.

With Makefiles I can build individual targets (like `make docs`), or run
`make` in the docs/ subdir, is something like that also possible with
Meson? Or are you always configuring what to build in `meson configure`
and building all that with `meson compile`?

-- 
Toon

Re: How to use Meson (was: [PATCH 00/10] meson: wire up missing HTML documentation])

[Patrick Steinhardt]

On Fri, Jan 03, 2025 at 08:58:00AM +0100, Toon Claes wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > Yup, you are always expected to set up the top-level source directory,
> > not any of the subdirectories. The build instructions are then processed
> > linearly in Meson, so variables declared before a call to `subdir()`
> > would be accessible in the subdirectory, as well.
> 
> With Makefiles I can build individual targets (like `make docs`), or run
> `make` in the docs/ subdir, is something like that also possible with
> Meson? Or are you always configuring what to build in `meson configure`
> and building all that with `meson compile`?

You can in theory. It's already possible to build individual parts of
Git, e.g.:

    # We need to discern these two `git` targets because the same name
    # is defined once as a static library and once as an executable.
    $ meson compile git:static_library
    $ meson compile git:executable
    $ meson compile Documentation/git-add.1

We can also have a target equivalent to `make docs` by adding
`alias_target()`s to Meson. I ain't got these wired up yet, but it could
look like the patch at the end of this mail. And then you can simply say
`meson compile docs`. It does require you to have docs configured
though, otherwise the 'Documentation/' subdirectory does not get pulled
included in the first place.

Patrick

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 2a26fa8a5f..4f8e2e7ebb 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -204,6 +204,8 @@ manpages = {
   'gitworkflows.txt' : 7,
 }
 
+docs_target = []
+
 docs_backend = get_option('docs_backend')
 if docs_backend == 'auto'
   if find_program('asciidoc', required: false).found()
@@ -364,10 +366,12 @@ foreach manpage, category : manpages
       install: true,
       install_dir: get_option('mandir') / 'man' + category.to_string(),
     )
+
+    docs_target += manpage_target
   endif
 
   if get_option('docs').contains('html')
-    custom_target(
+    docs_target += custom_target(
       command: asciidoc_common_options + [
         '--backend=' + asciidoc_html,
         '--doctype=manpage',
@@ -419,7 +423,7 @@ if get_option('docs').contains('html')
     depends: documentation_deps,
   )
 
-  custom_target(
+  docs_target += custom_target(
     command: [
       xsltproc,
       '--xinclude',
@@ -447,7 +451,7 @@ if get_option('docs').contains('html')
   ]
 
   foreach article : articles
-    custom_target(
+    docs_target += custom_target(
       command: asciidoc_common_options + [
         '--backend=' + asciidoc_html,
         '--out-file=@OUTPUT@',
@@ -502,3 +506,5 @@ if configured_manpages != actual_manpages
     error('Man page configured, but not found:\n\n - ' + '\n - '.join(superfluous_manpage))
   endif
 endif
+
+alias_target('docs', docs_target)

[PATCH v2 00/12] meson: wire up missing HTML documentation

[Patrick Steinhardt]

Hi,

this patch series wires up missing HTML-based documentation with Meson.
This includes a couple of missing manpages, the user manual as well as
the random set of articles that we have. It also starts to generate the
indices for API docs and howtos so that the result is a complete set of
HTML docs, same as with our Makefile. It also fixes a couple of smaller
issues I found while working on the series.

Notably missing yet is an integration with CI as well as sanity checks
for any kind of missing docs in Meson. I'll work on this in a separate
patch series once the initial CI integration as well as this patch
series here have landed.

Further missing is the generation of both info pages and a user manual
PDF. I couldn't find any users of these anywhere in downstream distros,
so I decided to not care for now until somebody complains.

Changes in v2:

  - Change the base to 76cf4f61c8 (Merge https://github.com/j6t/git-gui,
    2024-12-26). This is done to fix conflicts with in-flight topics and
    to pull in the CI setup.
  - Fix a typo.
  - Include another commit to auto-detect missing manpages in Meson both
    via Meson itself, but also via our Makefile.
  - Make the equivalent check in t/Makefile work with Dash.
  - Link to v1: https://lore.kernel.org/r/20241213-b4-pks-meson-docs-v1-0-0c7895952cd3@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (12):
      meson: wire up support for AsciiDoctor
      meson: properly wire up dependencies for our docs
      meson: fix generation of merge tools
      meson: generate HTML pages for all man page categories
      Documentation: inline user-manual.conf
      meson: generate user manual
      Documentation: refactor "api-index.sh" for out-of-tree builds
      Documentation: refactor "howto-index.sh" for out-of-tree builds
      meson: generate articles
      meson: install static files for HTML documentation
      t/Makefile: make "check-meson" work with Dash
      Documentation: wire up sanity checks for Meson

 Documentation/.gitignore                 |   1 +
 Documentation/Makefile                   |  24 ++-
 Documentation/asciidoc.conf.in           |  10 ++
 Documentation/{ => howto}/howto-index.sh |   2 +-
 Documentation/howto/meson.build          |  62 ++++++++
 Documentation/meson.build                | 255 ++++++++++++++++++++++++++-----
 Documentation/technical/api-index.sh     |  19 ++-
 Documentation/technical/meson.build      |  66 ++++++++
 Documentation/user-manual.conf           |  11 --
 meson_options.txt                        |   2 +
 t/.gitignore                             |   1 +
 t/Makefile                               |  12 +-
 12 files changed, 402 insertions(+), 63 deletions(-)

Range-diff versus v1:

 1:  e564c753c9 !  1:  376ed916ce meson: wire up support for AsciiDoctor
    @@ Documentation/meson.build: manpages = {
     -  input: meson.current_source_dir() / 'asciidoc.conf.in',
     -  output: 'asciidoc.conf',
     -  depends: [git_version_file],
    +-  env: version_gen_environment,
     -)
     +if docs_backend == 'asciidoc'
     +  asciidoc = find_program('asciidoc', required: true)
    @@ Documentation/meson.build: manpages = {
     +    input: meson.current_source_dir() / 'asciidoc.conf.in',
     +    output: 'asciidoc.conf',
     +    depends: [git_version_file],
    ++    env: version_gen_environment,
     +  )
     +
     +  asciidoc_common_options = [
    @@ Documentation/meson.build: manpages = {
     +    input: meson.current_source_dir() / 'asciidoctor-extensions.rb.in',
     +    output: 'asciidoctor-extensions.rb',
     +    depends: [git_version_file],
    ++    env: version_gen_environment,
     +  )
     +
     +  asciidoc_common_options = [
 2:  ce9bfd53f7 !  2:  6c6e593fad meson: properly wire up dependencies for our docs
    @@ Documentation/meson.build: if docs_backend == 'asciidoc'
     +    input: 'asciidoc.conf.in',
          output: 'asciidoc.conf',
          depends: [git_version_file],
    -   )
    +     env: version_gen_environment,
     @@ Documentation/meson.build: elif docs_backend == 'asciidoctor'
            '@INPUT@',
            '@OUTPUT@',
    @@ Documentation/meson.build: elif docs_backend == 'asciidoctor'
     +    input: 'asciidoctor-extensions.rb.in',
          output: 'asciidoctor-extensions.rb',
          depends: [git_version_file],
    -   )
    +     env: version_gen_environment,
     @@ Documentation/meson.build: cmd_lists = [
      documentation_deps += custom_target(
        command: [
 3:  905f220caa =  3:  b962455582 meson: fix generation of merge tools
 4:  ff35b7433a !  4:  32578c5cd2 meson: generate HTML pages for all man page categories
    @@ Commit message
         meson: generate HTML pages for all man page categories
     
         When generating HTML pages for our man pages we only generate them for
    -    category 1 in MEson, which are the pages corresponding to our built-in
    +    category 1 in Meson, which are the pages corresponding to our built-in
         commands. I cannot tell why I added this filter though: our Makefile
         installs all man pages, so a Meson-based build misses out on many of
         them.
 5:  bf8c278db5 !  5:  b704daf80c Documentation: inline user-manual.conf
    @@ Documentation/Makefile: manpage-cmd = $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $
      technical/api-index.txt: technical/api-index-skel.txt \
     
      ## Documentation/asciidoc.conf.in ##
    -@@ Documentation/asciidoc.conf.in: manmanual='Git Manual'
    - mansource='Git @GIT_VERSION@'
    - revdate='@GIT_DATE@'
    +@@ Documentation/asciidoc.conf.in: manmanual=Git Manual
    + mansource=Git @GIT_VERSION@
    + revdate=@GIT_DATE@
      
     +ifdef::doctype-book[]
     +[titles]
 6:  aaebbf0e94 =  6:  7eaf4f4267 meson: generate user manual
 7:  1cc7d42a55 =  7:  52b9e4c34b Documentation: refactor "api-index.sh" for out-of-tree builds
 8:  29fbda50a5 =  8:  b9c8e5fe4d Documentation: refactor "howto-index.sh" for out-of-tree builds
 9:  cd7f5ee207 =  9:  1f724c113a meson: generate articles
10:  d52f3db2bc = 10:  acb6c5f370 meson: install static files for HTML documentation
 -:  ---------- > 11:  2b893f7c0e t/Makefile: make "check-meson" work with Dash
 -:  ---------- > 12:  adf4835053 Documentation: wire up sanity checks for Meson

---
base-commit: 76cf4f61c87855ebf0784b88aaf737d6b09f504b
change-id: 20241212-b4-pks-meson-docs-2634bf3e7764

[PATCH v2 01/12] meson: wire up support for AsciiDoctor

[Patrick Steinhardt]

While our Makefile supports both Asciidoc and AsciiDoctor, our Meson
build instructions only support the former. Wire up support for the
latter, as well.

Our Makefile always favors Asciidoc, but Meson will automatically figure
out which of both to use based on whether they are installed or not. To
keep compatibility with our Makefile it favors Asciidoc over Asciidoctor
in case both are available.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 110 ++++++++++++++++++++++++++++++++++------------
 meson_options.txt         |   2 +
 2 files changed, 84 insertions(+), 28 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index fca3eab1f1360a5fdeda89c1766ab8cdb3267b89..acd6d86ec779e63230c88b7bff937aff330d2d4f 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -204,29 +204,87 @@ manpages = {
   'gitworkflows.txt' : 7,
 }
 
-asciidoc = find_program('asciidoc')
-git = find_program('git', required: false)
-xmlto = find_program('xmlto')
+docs_backend = get_option('docs_backend')
+if docs_backend == 'auto'
+  if find_program('asciidoc', required: false).found()
+    docs_backend = 'asciidoc'
+  elif find_program('asciidoctor', required: false).found()
+    docs_backend = 'asciidoctor'
+  else
+    error('Neither asciidoc nor asciidoctor were found.')
+  endif
+endif
 
-asciidoc_conf = custom_target(
-  command: [
-    shell,
-    meson.project_source_root() / 'GIT-VERSION-GEN',
-    meson.project_source_root(),
-    '@INPUT@',
-    '@OUTPUT@',
-  ],
-  input: meson.current_source_dir() / 'asciidoc.conf.in',
-  output: 'asciidoc.conf',
-  depends: [git_version_file],
-  env: version_gen_environment,
-)
+if docs_backend == 'asciidoc'
+  asciidoc = find_program('asciidoc', required: true)
+  asciidoc_html = 'xhtml11'
+  asciidoc_docbook = 'docbook'
+  xmlto_extra = [ ]
 
-asciidoc_common_options = [
-  asciidoc,
-  '--conf-file=' + asciidoc_conf.full_path(),
-  '--attribute=build_dir=' + meson.current_build_dir(),
-]
+  asciidoc_conf = custom_target(
+    command: [
+      shell,
+      meson.project_source_root() / 'GIT-VERSION-GEN',
+      meson.project_source_root(),
+      '@INPUT@',
+      '@OUTPUT@',
+    ],
+    input: meson.current_source_dir() / 'asciidoc.conf.in',
+    output: 'asciidoc.conf',
+    depends: [git_version_file],
+    env: version_gen_environment,
+  )
+
+  asciidoc_common_options = [
+    asciidoc,
+    '--conf-file=' + asciidoc_conf.full_path(),
+    '--attribute=build_dir=' + meson.current_build_dir(),
+  ]
+
+  documentation_deps = [
+    asciidoc_conf,
+  ]
+elif docs_backend == 'asciidoctor'
+  asciidoctor = find_program('asciidoctor', required: true)
+  asciidoc_html = 'xhtml5'
+  asciidoc_docbook = 'docbook5'
+  xmlto_extra = [
+    '--skip-validation',
+    '-x', meson.current_source_dir() / 'manpage.xsl',
+  ]
+
+  asciidoctor_extensions = custom_target(
+    command: [
+      shell,
+      meson.project_source_root() / 'GIT-VERSION-GEN',
+      meson.project_source_root(),
+      '@INPUT@',
+      '@OUTPUT@',
+    ],
+    input: meson.current_source_dir() / 'asciidoctor-extensions.rb.in',
+    output: 'asciidoctor-extensions.rb',
+    depends: [git_version_file],
+    env: version_gen_environment,
+  )
+
+  asciidoc_common_options = [
+    asciidoctor,
+    '--attribute', 'compat-mode',
+    '--attribute', 'tabsize=8',
+    '--attribute', 'litdd=&#x2d;&#x2d;',
+    '--attribute', 'docinfo=shared',
+    '--attribute', 'build_dir=' + meson.current_build_dir(),
+    '--load-path', meson.current_build_dir(),
+    '--require', 'asciidoctor-extensions',
+  ]
+
+  documentation_deps = [
+    asciidoctor_extensions,
+  ]
+endif
+
+git = find_program('git', required: false)
+xmlto = find_program('xmlto')
 
 cmd_lists = [
   'cmds-ancillaryinterrogators.txt',
@@ -243,10 +301,6 @@ cmd_lists = [
   'cmds-foreignscminterface.txt',
 ]
 
-documentation_deps = [
-  asciidoc_conf,
-]
-
 documentation_deps += custom_target(
   command: [
     perl,
@@ -278,7 +332,7 @@ foreach manpage, category : manpages
   if get_option('docs').contains('man')
     manpage_xml_target = custom_target(
       command: asciidoc_common_options + [
-        '--backend=docbook',
+        '--backend=' + asciidoc_docbook,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
         meson.current_source_dir() / manpage,
@@ -301,7 +355,7 @@ foreach manpage, category : manpages
         manpage_xml_target,
         '-o',
         meson.current_build_dir(),
-      ],
+      ] + xmlto_extra,
       output: manpage_path,
       install: true,
       install_dir: get_option('mandir') / 'man' + category.to_string(),
@@ -311,7 +365,7 @@ foreach manpage, category : manpages
   if get_option('docs').contains('html') and category == 1
     custom_target(
       command: asciidoc_common_options + [
-        '--backend=xhtml11',
+        '--backend=' + asciidoc_html,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
         meson.current_source_dir() / manpage,
diff --git a/meson_options.txt b/meson_options.txt
index 4be7eab39939178ae2ffde1ff9e78f83a1b482b2..f50bb40cdf6046529a0cea0a03a8cb696c3a6b18 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -85,6 +85,8 @@ option('docs', type: 'array', choices: ['man', 'html'], value: [],
   description: 'Which documenattion formats to build and install.')
 option('default_help_format', type: 'combo', choices: ['man', 'html'], value: 'man',
   description: 'Default format used when executing git-help(1).')
+option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto',
+  description: 'Which backend to use to generate documentation.')
 
 # Testing.
 option('tests', type: 'boolean', value: true,

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 02/12] meson: properly wire up dependencies for our docs

[Patrick Steinhardt]

A couple of Meson documentation targets use `meson.current_source_dir()`
to resolve inputs. This has the downside that it does not automagically
make Meson track these inputs as a dependency. After all, string
arguments really can be anything, even if they happen to match an actual
filesystem path.

Adapt these build targets to instead use inputs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index acd6d86ec779e63230c88b7bff937aff330d2d4f..b3c8b6c56339e10099f8c37a4d8198f402192520 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -229,7 +229,7 @@ if docs_backend == 'asciidoc'
       '@INPUT@',
       '@OUTPUT@',
     ],
-    input: meson.current_source_dir() / 'asciidoc.conf.in',
+    input: 'asciidoc.conf.in',
     output: 'asciidoc.conf',
     depends: [git_version_file],
     env: version_gen_environment,
@@ -261,7 +261,7 @@ elif docs_backend == 'asciidoctor'
       '@INPUT@',
       '@OUTPUT@',
     ],
-    input: meson.current_source_dir() / 'asciidoctor-extensions.rb.in',
+    input: 'asciidoctor-extensions.rb.in',
     output: 'asciidoctor-extensions.rb',
     depends: [git_version_file],
     env: version_gen_environment,
@@ -304,10 +304,11 @@ cmd_lists = [
 documentation_deps += custom_target(
   command: [
     perl,
-    meson.current_source_dir() / 'cmd-list.perl',
+    '@INPUT@',
     meson.project_source_root(),
     meson.current_build_dir(),
   ] + cmd_lists,
+  input: 'cmd-list.perl',
   output: cmd_lists
 )
 
@@ -315,7 +316,7 @@ foreach mode : [ 'diff', 'merge' ]
   documentation_deps += custom_target(
     command: [
       shell,
-      meson.current_source_dir() / 'generate-mergetool-list.sh',
+      '@INPUT@',
       '..',
       'diff',
       '@OUTPUT@'
@@ -324,6 +325,7 @@ foreach mode : [ 'diff', 'merge' ]
       'MERGE_TOOLS_DIR=' + meson.project_source_root() / 'mergetools',
       'TOOL_MODE=' + mode,
     ],
+    input: 'generate-mergetool-list.sh',
     output: 'mergetools-' + mode + '.txt',
   )
 endforeach
@@ -335,9 +337,10 @@ foreach manpage, category : manpages
         '--backend=' + asciidoc_docbook,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
-        meson.current_source_dir() / manpage,
+        '@INPUT@',
       ],
       depends: documentation_deps,
+      input: manpage,
       output: fs.stem(manpage) + '.xml',
     )
 
@@ -345,10 +348,8 @@ foreach manpage, category : manpages
     manpage_target = custom_target(
       command: [
         xmlto,
-        '-m',
-        meson.current_source_dir() / 'manpage-normal.xsl',
-        '-m',
-        meson.current_source_dir() / 'manpage-bold-literal.xsl',
+        '-m', '@INPUT0@',
+        '-m', '@INPUT1@',
         '--stringparam',
         'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'),
         'man',
@@ -356,6 +357,10 @@ foreach manpage, category : manpages
         '-o',
         meson.current_build_dir(),
       ] + xmlto_extra,
+      input: [
+        'manpage-normal.xsl',
+        'manpage-bold-literal.xsl',
+      ],
       output: manpage_path,
       install: true,
       install_dir: get_option('mandir') / 'man' + category.to_string(),
@@ -368,9 +373,10 @@ foreach manpage, category : manpages
         '--backend=' + asciidoc_html,
         '--doctype=manpage',
         '--out-file=@OUTPUT@',
-        meson.current_source_dir() / manpage,
+        '@INPUT@',
       ],
       depends: documentation_deps,
+      input: manpage,
       output: fs.stem(manpage) + '.html',
       install: true,
       install_dir: get_option('datadir') / 'doc/git-doc',

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 03/12] meson: fix generation of merge tools

[Patrick Steinhardt]

Our buildsystems generate a list of diff and merge tools that ultimately
end up in our documentation. And while Meson does wire up the logic, it
tries to use the TOOL_MODE environment variable to set up the mode. This
is wrong though: the mode is set via an argument that we have fixed to
'diff' mode by accident.

Fix this such that merge tools are properly generated.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index b3c8b6c56339e10099f8c37a4d8198f402192520..c2512328ca9b76a5dd512453ddbb776faea7967f 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -318,12 +318,11 @@ foreach mode : [ 'diff', 'merge' ]
       shell,
       '@INPUT@',
       '..',
-      'diff',
+      mode,
       '@OUTPUT@'
     ],
     env: [
       'MERGE_TOOLS_DIR=' + meson.project_source_root() / 'mergetools',
-      'TOOL_MODE=' + mode,
     ],
     input: 'generate-mergetool-list.sh',
     output: 'mergetools-' + mode + '.txt',

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 04/12] meson: generate HTML pages for all man page categories

[Patrick Steinhardt]

When generating HTML pages for our man pages we only generate them for
category 1 in Meson, which are the pages corresponding to our built-in
commands. I cannot tell why I added this filter though: our Makefile
installs all man pages, so a Meson-based build misses out on many of
them.

Fix this by removing the filter.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index c2512328ca9b76a5dd512453ddbb776faea7967f..48583e9a7f4b037de218358f16f59ce08141cbe8 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -366,7 +366,7 @@ foreach manpage, category : manpages
     )
   endif
 
-  if get_option('docs').contains('html') and category == 1
+  if get_option('docs').contains('html')
     custom_target(
       command: asciidoc_common_options + [
         '--backend=' + asciidoc_html,

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 05/12] Documentation: inline user-manual.conf

[Patrick Steinhardt]

When generating our user manual we set up a bit of extra configuration
compared to our normal configuration. This is done by having an extra
"user-manual.conf" file that Asciidoc seems to pull in automatically due
to matching filenames with "user-manual.txt". This dependency is quite
hidden though and thus easy to miss. Furthermore, it seems that Asciidoc
does not know to pull it in for out-of-tree builds where we use relative
paths.

The setup in AsciiDoctor is somewhat different: instead of having two
sets of configuration, we condition the use of manual-specific configs
based on whether the document type is "book". And as we only build our
user manual with that type this is sufficient.

Use the same trick for our user manual by inlining the configuration
into "asciidoc.conf.in" and making it conditional on whether or not
"doctype-book" is defined.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile         |  2 +-
 Documentation/asciidoc.conf.in | 10 ++++++++++
 Documentation/user-manual.conf | 11 -----------
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index a89823e1d1ee5042367bdcca6ed426196d49ce89..4f152077dded75bedd59abd56db5f6f0693908de 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -362,7 +362,7 @@ manpage-cmd = $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
 %.xml : %.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_XML) -d manpage -o $@ $<
 
-user-manual.xml: user-manual.txt user-manual.conf $(ASCIIDOC_DEPS)
+user-manual.xml: user-manual.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_XML) -d book -o $@ $<
 
 technical/api-index.txt: technical/api-index-skel.txt \
diff --git a/Documentation/asciidoc.conf.in b/Documentation/asciidoc.conf.in
index b89bccf2309d782ba29ea716a132b888c1421669..f2aef6cb79f47cf132b97d88a7e74fb40da8ac8d 100644
--- a/Documentation/asciidoc.conf.in
+++ b/Documentation/asciidoc.conf.in
@@ -25,12 +25,22 @@ manmanual=Git Manual
 mansource=Git @GIT_VERSION@
 revdate=@GIT_DATE@
 
+ifdef::doctype-book[]
+[titles]
+	underlines="__","==","--","~~","^^"
+endif::doctype-book[]
+
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]
+ifndef::doctype-book[]
 {0%{target}}
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+endif::doctype-book[]
+ifdef::doctype-book[]
+<ulink url="{target}.html">{target}{0?({0})}</ulink>
+endif::doctype-book[]
 
 [literal-inlinemacro]
 {eval:re.sub(r'(&lt;[-a-zA-Z0-9.]+&gt;)', r'<emphasis>\1</emphasis>', re.sub(r'([\[\s|()>]|^|\]|&gt;)(\.?([-a-zA-Z0-9:+=~@,\/_^\$]+\.?)+)',r'\1<literal>\2</literal>', re.sub(r'(\.\.\.?)([^\]$.])', r'<literal>\1</literal>\2', macros.passthroughs[int(attrs['passtext'][1:-1])] if attrs['passtext'][1:-1].isnumeric() else attrs['passtext'][1:-1])))}
diff --git a/Documentation/user-manual.conf b/Documentation/user-manual.conf
deleted file mode 100644
index 0148f126dcdf6aca15a5560fb5b122b85b022461..0000000000000000000000000000000000000000
--- a/Documentation/user-manual.conf
+++ /dev/null
@@ -1,11 +0,0 @@
-[titles]
-	underlines="__","==","--","~~","^^"
-
-[attributes]
-caret=^
-startsb=&#91;
-endsb=&#93;
-tilde=&#126;
-
-[linkgit-inlinemacro]
-<ulink url="{target}.html">{target}{0?({0})}</ulink>

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 06/12] meson: generate user manual

[Patrick Steinhardt]

Our documentation contains a user manual that gives people a short
introduction to Git. Our Makefile knows to generate the manual into
three different formats: an HTML page, a PDF and an info page. The Meson
build instructions don't yet generate any of these.

While wiring up all these formats I hit a couple of road blocks with how
we generate our info pages. Even though I eventually resolved these, it
made me question whether anybody actually uses info pages in the first
place. Checking through a couple of downstream consumers I couldn't find
a single user of either the info pages nor of our PDF manual in Arch
Linux, Debian, Fedora, Ubuntu, FreeBSD or OpenBSDFedora. So it's rather
safe to assume that there aren't really any users out there, and thus
the added complexity does not seem worth it.

Wire up support for building the user manual in HTML format and
conciously skip over the other two formats. This is basically a form of
silent deprecation: if people out there use the other two formats they
will eventually complain about them missing in Meson, which means we can
wire them up at a later point. If they don't we can phase out these
formats eventually.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 48583e9a7f4b037de218358f16f59ce08141cbe8..404cb20d10a2fbbe4e014bd8a7df74c49dad40a7 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -382,3 +382,35 @@ foreach manpage, category : manpages
     )
   endif
 endforeach
+
+if get_option('docs').contains('html')
+  xsltproc = find_program('xsltproc')
+
+  user_manual_xml = custom_target(
+    command: asciidoc_common_options + [
+      '--backend=' + asciidoc_docbook,
+      '--doctype=book',
+      '--out-file=@OUTPUT@',
+      '@INPUT@',
+    ],
+    input: 'user-manual.txt',
+    output: 'user-manual.xml',
+    depends: documentation_deps,
+  )
+
+  custom_target(
+    command: [
+      xsltproc,
+      '--xinclude',
+      '--stringparam', 'html.stylesheet', 'docbook-xsl.css',
+      '--param', 'generate.consistent.ids', '1',
+      '--output', '@OUTPUT@',
+      '@INPUT@',
+      user_manual_xml,
+    ],
+    input: 'docbook.xsl',
+    output: 'user-manual.html',
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+endif

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 07/12] Documentation: refactor "api-index.sh" for out-of-tree builds

[Patrick Steinhardt]

The "api-index.sh" script generates an index of API-related
documentation. The script does not handle out-of-tree builds and thus
cannot be used easily by Meson.

Refactor it to be independent of locations by both accepting a source
directory where the API docs live as well as a path to an output file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile               |  2 +-
 Documentation/technical/api-index.sh | 19 +++++++++++++++----
 2 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 4f152077dded75bedd59abd56db5f6f0693908de..b2d146c44f4ded750b5e0766eb66b25cb5ec08e3 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -367,7 +367,7 @@ user-manual.xml: user-manual.txt $(ASCIIDOC_DEPS)
 
 technical/api-index.txt: technical/api-index-skel.txt \
 	technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
-	$(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh
+	$(QUIET_GEN)'$(SHELL_PATH_SQ)' technical/api-index.sh ./technical ./technical/api-index.txt
 
 technical/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
 $(patsubst %,%.html,$(API_DOCS) technical/api-index $(TECH_DOCS)): %.html : %.txt \
diff --git a/Documentation/technical/api-index.sh b/Documentation/technical/api-index.sh
index 9c3f4131b8586408acd81d1e60912b51688575ed..296488557434b7fff60ab25f4246a4dc270729c0 100755
--- a/Documentation/technical/api-index.sh
+++ b/Documentation/technical/api-index.sh
@@ -1,6 +1,17 @@
 #!/bin/sh
 
+if test $# -ne 2
+then
+	echo >&2 "USAGE: $0 <SOURCE_DIR> <OUTPUT>"
+	exit 1
+fi
+
+SOURCE_DIR="$1"
+OUTPUT="$2"
+
 (
+	cd "$SOURCE_DIR"
+
 	c=////////////////////////////////////////////////////////////////
 	skel=api-index-skel.txt
 	sed -e '/^\/\/ table of contents begin/q' "$skel"
@@ -18,11 +29,11 @@
 	done
 	echo "$c"
 	sed -n -e '/^\/\/ table of contents end/,$p' "$skel"
-) >api-index.txt+
+) >"$OUTPUT"+
 
-if test -f api-index.txt && cmp api-index.txt api-index.txt+ >/dev/null
+if test -f "$OUTPUT" && cmp "$OUTPUT" "$OUTPUT"+ >/dev/null
 then
-	rm -f api-index.txt+
+	rm -f "$OUTPUT"+
 else
-	mv api-index.txt+ api-index.txt
+	mv "$OUTPUT"+ "$OUTPUT"
 fi

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 08/12] Documentation: refactor "howto-index.sh" for out-of-tree builds

[Patrick Steinhardt]

The "howto-index.sh" is used to generate an index of our how-to docs. It
receives as input the paths to these documents, which would typically be
relative to the "Documentation/" directory in Makefile-based builds. In
an out-of-tree build though it will get relative that may be rooted
somewhere else entirely.

The file paths do end up in the generated index, and the expectation is
that they should always start with "howto/". But for out-of-tree builds
we would populate it with the paths relative to the build directory,
which is wrong.

Fix the issue by using `$(basename "$file")` to generate the path. While
at it, move the script into "howto/" to align it with the location of
the comparable "api-index.sh" script.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/Makefile                   | 4 ++--
 Documentation/{ => howto}/howto-index.sh | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index b2d146c44f4ded750b5e0766eb66b25cb5ec08e3..e284ec8b98d6187ecb73011f6e490610dd3e7370 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -411,8 +411,8 @@ gitman.info: gitman.texi
 $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml
 	$(QUIET_DB2TEXI)$(DOCBOOK2X_TEXI) --to-stdout $*.xml >$@
 
-howto-index.txt: howto-index.sh $(HOWTO_TXT)
-	$(QUIET_GEN)'$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(HOWTO_TXT)) >$@
+howto-index.txt: howto/howto-index.sh $(HOWTO_TXT)
+	$(QUIET_GEN)'$(SHELL_PATH_SQ)' ./howto/howto-index.sh $(sort $(HOWTO_TXT)) >$@
 
 $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt $(ASCIIDOC_DEPS)
 	$(QUIET_ASCIIDOC)$(TXT_TO_HTML) $*.txt
diff --git a/Documentation/howto-index.sh b/Documentation/howto/howto-index.sh
similarity index 92%
rename from Documentation/howto-index.sh
rename to Documentation/howto/howto-index.sh
index 167b363668b8b53d752d5971798d3ca26c8f7f1f..eecd123a93607998e8b4eb8511f4165973f9d93e 100755
--- a/Documentation/howto-index.sh
+++ b/Documentation/howto/howto-index.sh
@@ -48,7 +48,7 @@ do
 		file="$txt"
 	fi
 
-	echo "* link:$file[$title] $from
+	echo "* link:howto/$(basename "$file")[$title] $from
 $abstract
 
 "

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 09/12] meson: generate articles

[Patrick Steinhardt]

While the Meson build system already knows to generate man pages and our
user manual, it does not yet generate the random assortment of articles
that we have. Plug this gap.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/howto/meson.build     | 62 ++++++++++++++++++++++++++++++++++
 Documentation/meson.build           | 36 ++++++++++++++++++++
 Documentation/technical/meson.build | 66 +++++++++++++++++++++++++++++++++++++
 3 files changed, 164 insertions(+)

diff --git a/Documentation/howto/meson.build b/Documentation/howto/meson.build
new file mode 100644
index 0000000000000000000000000000000000000000..c023c104161e61ca0399b5390d59e20343746621
--- /dev/null
+++ b/Documentation/howto/meson.build
@@ -0,0 +1,62 @@
+howto_sources = [
+  'coordinate-embargoed-releases.txt',
+  'keep-canonical-history-correct.txt',
+  'maintain-git.txt',
+  'new-command.txt',
+  'rebase-from-internal-branch.txt',
+  'rebuild-from-update-hook.txt',
+  'recover-corrupted-blob-object.txt',
+  'recover-corrupted-object-harder.txt',
+  'revert-a-faulty-merge.txt',
+  'revert-branch-rebase.txt',
+  'separating-topic-branches.txt',
+  'setup-git-server-over-http.txt',
+  'update-hook-example.txt',
+  'use-git-daemon.txt',
+  'using-merge-subtree.txt',
+  'using-signed-tag-in-pull-request.txt',
+]
+
+howto_index = custom_target(
+  command: [
+    shell,
+    meson.current_source_dir() / 'howto-index.sh',
+    '@INPUT@',
+  ],
+  env: script_environment,
+  capture: true,
+  input: howto_sources,
+  output: 'howto-index.txt',
+)
+
+custom_target(
+  command: asciidoc_html_options,
+  input: howto_index,
+  output: 'howto-index.html',
+  depends: documentation_deps,
+  install: true,
+  install_dir: get_option('datadir') / 'doc/git-doc',
+)
+
+foreach howto : howto_sources
+  howto_stripped = custom_target(
+    command: [
+      find_program('sed'),
+      '-e',
+      '1,/^$/d',
+      '@INPUT@',
+    ],
+    input: howto,
+    output: fs.stem(howto) + '.stripped',
+    capture: true,
+  )
+
+  custom_target(
+    command: asciidoc_html_options,
+    input: howto_stripped,
+    output: fs.stem(howto_stripped.full_path()) + '.html',
+    depends: documentation_deps,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc/howto',
+  )
+endforeach
diff --git a/Documentation/meson.build b/Documentation/meson.build
index 404cb20d10a2fbbe4e014bd8a7df74c49dad40a7..8c6ff0bce1206d988cc0d3b7997fa0f338d01194 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -413,4 +413,40 @@ if get_option('docs').contains('html')
     install: true,
     install_dir: get_option('datadir') / 'doc/git-doc',
   )
+
+  articles = [
+    'DecisionMaking.txt',
+    'MyFirstContribution.txt',
+    'MyFirstObjectWalk.txt',
+    'ReviewingGuidelines.txt',
+    'SubmittingPatches',
+    'ToolsForGit.txt',
+    'git-bisect-lk2009.txt',
+    'git-tools.txt',
+  ]
+
+  foreach article : articles
+    custom_target(
+      command: asciidoc_common_options + [
+        '--backend=' + asciidoc_html,
+        '--out-file=@OUTPUT@',
+        '@INPUT@',
+      ],
+      input: article,
+      output: fs.stem(article) + '.html',
+      depends: documentation_deps,
+      install: true,
+      install_dir: get_option('datadir') / 'doc/git-doc',
+    )
+  endforeach
+
+  asciidoc_html_options = asciidoc_common_options + [
+    '--backend=' + asciidoc_html,
+    '--out-file=@OUTPUT@',
+    '--attribute', 'git-relative-html-prefix=../',
+    '@INPUT@',
+  ]
+
+  subdir('howto')
+  subdir('technical')
 endif
diff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build
new file mode 100644
index 0000000000000000000000000000000000000000..21dfb8b5c9d93490f16c24bab1d49631e0395571
--- /dev/null
+++ b/Documentation/technical/meson.build
@@ -0,0 +1,66 @@
+api_docs = [
+  'api-error-handling.txt',
+  'api-merge.txt',
+  'api-parse-options.txt',
+  'api-simple-ipc.txt',
+  'api-trace2.txt',
+]
+
+articles = [
+  'bitmap-format.txt',
+  'build-systems.txt',
+  'bundle-uri.txt',
+  'commit-graph.txt',
+  'directory-rename-detection.txt',
+  'hash-function-transition.txt',
+  'long-running-process-protocol.txt',
+  'multi-pack-index.txt',
+  'packfile-uri.txt',
+  'pack-heuristics.txt',
+  'parallel-checkout.txt',
+  'partial-clone.txt',
+  'platform-support.txt',
+  'racy-git.txt',
+  'reftable.txt',
+  'remembering-renames.txt',
+  'repository-version.txt',
+  'rerere.txt',
+  'scalar.txt',
+  'send-pack-pipeline.txt',
+  'shallow.txt',
+  'sparse-checkout.txt',
+  'sparse-index.txt',
+  'trivial-merge.txt',
+  'unit-tests.txt',
+]
+
+api_index = custom_target(
+  command: [
+    shell,
+    meson.current_source_dir() / 'api-index.sh',
+    meson.current_source_dir(),
+    '@OUTPUT@',
+  ],
+  env: script_environment,
+  input: api_docs,
+  output: 'api-index.txt',
+)
+
+custom_target(
+  command: asciidoc_html_options,
+  input: api_index,
+  output: 'api-index.html',
+  depends: documentation_deps,
+  install: true,
+  install_dir: get_option('datadir') / 'doc/git-doc/technical',
+)
+
+foreach article : api_docs + articles
+  custom_target(
+    command: asciidoc_html_options,
+    input: article,
+    output: fs.stem(article) + '.html',
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc/technical',
+  )
+endforeach

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 10/12] meson: install static files for HTML documentation

[Patrick Steinhardt]

Now that we generate man pages, articles and user manual with Meson the
only thing that is still missing in an installation of HTML documents is
a couple of static files. Wire these up to finalize Meson's support for
generating HTML documentation.

Diffing an installation that uses our Makefile with an installation that
uses Meson only surfaces a couple of discepancies now:

  - Meson doesn't install "everyday.html" and "git-remote-helpers.html".
    These files are marked as obsolete and don't contain any useful
    information anymore: they simply point to their modern equivalents.

  - Meson doesn't install "*.txt" files when asking for HTML docs. I'm
    not sure why our Makefiles do this in the first place, and it does
    seem like the resulting installation is fully functional even
    without those files.

Other than that, both layout and file contents are the exact same.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 8c6ff0bce1206d988cc0d3b7997fa0f338d01194..4d9511156502653292144fe6962bd3411558d96a 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -384,6 +384,27 @@ foreach manpage, category : manpages
 endforeach
 
 if get_option('docs').contains('html')
+  configure_file(
+    input: 'docinfo-html.in',
+    output: 'docinfo.html',
+    copy: true,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+
+  configure_file(
+    input: 'docbook-xsl.css',
+    output: 'docbook-xsl.css',
+    copy: true,
+    install: true,
+    install_dir: get_option('datadir') / 'doc/git-doc',
+  )
+
+  install_symlink('index.html',
+    install_dir: get_option('datadir') / 'doc/git-doc',
+    pointing_to: 'git.html',
+  )
+
   xsltproc = find_program('xsltproc')
 
   user_manual_xml = custom_target(

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

[PATCH v2 11/12] t/Makefile: make "check-meson" work with Dash

[Patrick Steinhardt]

The "check-meson" target uses process substitution to check whether
extracted contents from "meson.build" match expected contents. Process
substitution is unportable though and thus the target will fail when
using for example Dash.

Fix this by writing data into a temporary directory.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/.gitignore |  1 +
 t/Makefile   | 12 +++++++-----
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/t/.gitignore b/t/.gitignore
index 91cf5772fe5643dbe075da98ed5166e1899b9a54..3e6b0f2cc57ffed0394d1cd2efc1e374f1c2169b 100644
--- a/t/.gitignore
+++ b/t/.gitignore
@@ -2,4 +2,5 @@
 /test-results
 /.prove
 /chainlinttmp
+/mesontmp
 /out/
diff --git a/t/Makefile b/t/Makefile
index 290fb03ff011d39c31c5073c796aa6f4dc966283..daa5fcae86f3480079b8c9743dd28e3fd304c27b 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -103,6 +103,7 @@ clean-except-prove-cache: clean-chainlint
 
 clean: clean-except-prove-cache
 	$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
+	$(RM) -r mesontmp
 	$(RM) .prove
 
 clean-chainlint:
@@ -116,16 +117,17 @@ check-chainlint:
 
 check-meson:
 	@# awk acts up when trying to match single quotes, so we use \047 instead.
-	@printf "%s\n" \
+	@mkdir -p mesontmp && \
+	printf "%s\n" \
 		"integration_tests t[0-9][0-9][0-9][0-9]-*.sh" \
 		"unit_test_programs unit-tests/t-*.c" \
 		"clar_test_suites unit-tests/u-*.c" | \
 	while read -r variable pattern; do \
-		meson_tests=$$(awk "/^$$variable = \[\$$/ {flag=1 ; next } /^]$$/ { flag=0 } flag { gsub(/^  \047/, \"\"); gsub(/\047,\$$/, \"\"); print }" meson.build) && \
-		actual_tests=$$(ls $$pattern) && \
-		if test "$$meson_tests" != "$$actual_tests"; then \
+		awk "/^$$variable = \[\$$/ {flag=1 ; next } /^]$$/ { flag=0 } flag { gsub(/^  \047/, \"\"); gsub(/\047,\$$/, \"\"); print }" meson.build >mesontmp/meson.txt && \
+		ls $$pattern >mesontmp/actual.txt && \
+		if ! cmp mesontmp/meson.txt mesontmp/actual.txt; then \
 			echo "Meson tests differ from actual tests:"; \
-			diff -u <(echo "$$meson_tests") <(echo "$$actual_tests"); \
+			diff -u mesontmp/meson.txt mesontmp/actual.txt; \
 			exit 1; \
 		fi; \
 	done

-- 
2.48.0.rc0.311.gb6c66824c1.dirty

Re: [PATCH v2 11/12] t/Makefile: make "check-meson" work with Dash

[Jonathan Nieder]

Hi,

Patrick Steinhardt wrote:

> The "check-meson" target uses process substitution to check whether
> extracted contents from "meson.build" match expected contents. Process
> substitution is unportable though and thus the target will fail when
> using for example Dash.
>
> Fix this by writing data into a temporary directory.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/.gitignore |  1 +
>  t/Makefile   | 12 +++++++-----
>  2 files changed, 8 insertions(+), 5 deletions(-)

Without this, I get the error described in
https://lore.kernel.org/git/CAHWeT-boK3x6mup11boEinNDQiAxxf0vwvZkxsGRc_GRvXYA8g@mail.gmail.com/
('/bin/sh: 10: Syntax error: "(" unexpected'), and with this, the
build in the Debian buildd environment succeeds.

Tested-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks for fixing it.

Sincerely,
Jonathan

Re: [PATCH v2 11/12] t/Makefile: make "check-meson" work with Dash

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Without this, I get the error described in
> https://lore.kernel.org/git/CAHWeT-boK3x6mup11boEinNDQiAxxf0vwvZkxsGRc_GRvXYA8g@mail.gmail.com/
> ('/bin/sh: 10: Syntax error: "(" unexpected'), and with this, the
> build in the Debian buildd environment succeeds.
>
> Tested-by: Jonathan Nieder <jrnieder@gmail.com>
>
> Thanks for fixing it.

Thanks.

Re: [PATCH v2 11/12] t/Makefile: make "check-meson" work with Dash

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Without this, I get the error described in
>> https://lore.kernel.org/git/CAHWeT-boK3x6mup11boEinNDQiAxxf0vwvZkxsGRc_GRvXYA8g@mail.gmail.com/
>> ('/bin/sh: 10: Syntax error: "(" unexpected'), and with this, the
>> build in the Debian buildd environment succeeds.
>>
>> Tested-by: Jonathan Nieder <jrnieder@gmail.com>
>>
>> Thanks for fixing it.
>
> Thanks.

Now the fix is in 'master'.

Thanks, all.

[PATCH v2 12/12] Documentation: wire up sanity checks for Meson

[Patrick Steinhardt]

Wire up sanity checks for Meson to verify that no man pages are missing.
This check is similar to the same check we already have for our tests.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/.gitignore  |  1 +
 Documentation/Makefile    | 16 ++++++++++++++++
 Documentation/meson.build | 31 +++++++++++++++++++++++++++++++
 3 files changed, 48 insertions(+)

diff --git a/Documentation/.gitignore b/Documentation/.gitignore
index 649df89474d357ccc91109b5c35fe2d0910f968a..9f4bb3c4bf9e9e84f740b3210c570ec25e15e4fe 100644
--- a/Documentation/.gitignore
+++ b/Documentation/.gitignore
@@ -12,6 +12,7 @@ cmds-*.txt
 mergetools-*.txt
 SubmittingPatches.txt
 tmp-doc-diff/
+tmp-meson-diff/
 GIT-ASCIIDOCFLAGS
 /.build/
 /GIT-EXCLUDED-PROGRAMS
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e284ec8b98d6187ecb73011f6e490610dd3e7370..aedfe99d1d35889aa24ed6b9085e614dbc240096 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -339,6 +339,7 @@ clean:
 	$(RM) $(cmds_txt) $(mergetools_txt) *.made
 	$(RM) GIT-ASCIIDOCFLAGS
 	$(RM) asciidoc.conf asciidoctor-extensions.rb
+	$(RM) -rf tmp-meson-diff
 
 docinfo.html: docinfo-html.in
 	$(QUIET_GEN)$(RM) $@ && cat $< >$@
@@ -494,6 +495,20 @@ lint-docs-fsck-msgids: $(LINT_DOCS_FSCK_MSGIDS)
 lint-docs-manpages:
 	$(QUIET_GEN)./lint-manpages.sh
 
+.PHONY: lint-docs-meson
+lint-docs-meson:
+	@# awk acts up when trying to match single quotes, so we use \047 instead.
+	@mkdir -p tmp-meson-diff && \
+	awk "/^manpages = {$$/ {flag=1 ; next } /^}$$/ { flag=0 } flag { gsub(/^  \047/, \"\"); gsub(/\047 : [157],\$$/, \"\"); print }" meson.build | \
+		grep -v -e '#' -e '^$$' | \
+		sort >tmp-meson-diff/meson.txt && \
+	ls git*.txt scalar.txt | grep -v -e git-bisect-lk2009.txt -e git-tools.txt >tmp-meson-diff/actual.txt && \
+	if ! cmp tmp-meson-diff/meson.txt tmp-meson-diff/actual.txt; then \
+		echo "Meson man pages differ from actual man pages:"; \
+		diff -u tmp-meson-diff/meson.txt tmp-meson-diff/actual.txt; \
+		exit 1; \
+	fi
+
 ## Lint: list of targets above
 .PHONY: lint-docs
 lint-docs: lint-docs-fsck-msgids
@@ -501,6 +516,7 @@ lint-docs: lint-docs-gitlink
 lint-docs: lint-docs-man-end-blurb
 lint-docs: lint-docs-man-section-order
 lint-docs: lint-docs-manpages
+lint-docs: lint-docs-meson
 
 ifeq ($(wildcard po/Makefile),po/Makefile)
 doc-l10n install-l10n::
diff --git a/Documentation/meson.build b/Documentation/meson.build
index 4d9511156502653292144fe6962bd3411558d96a..2a26fa8a5fedc0f46a82fcc31bf1d6457f6d082c 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -471,3 +471,34 @@ if get_option('docs').contains('html')
   subdir('howto')
   subdir('technical')
 endif
+
+# Sanity check that we are not missing any tests present in 't/'. This check
+# only runs once at configure time and is thus best-effort, only. Furthermore,
+# it only verifies man pages for the sake of simplicity.
+configured_manpages = manpages.keys() + [ 'git-bisect-lk2009.txt', 'git-tools.txt' ]
+actual_manpages = run_command(shell, '-c', 'ls git*.txt scalar.txt',
+  check: true,
+  env: script_environment,
+).stdout().strip().split('\n')
+
+if configured_manpages != actual_manpages
+  missing_manpage = [ ]
+  foreach actual_manpage : actual_manpages
+    if actual_manpage not in configured_manpages
+      missing_manpage += actual_manpage
+    endif
+  endforeach
+  if missing_manpage.length() > 0
+    error('Man page found, but not configured:\n\n - ' + '\n - '.join(missing_manpage))
+  endif
+
+  superfluous_manpage = [ ]
+  foreach configured_manpage : configured_manpages
+    if configured_manpage not in actual_manpages
+      superfluous_manpage += configured_manpage
+    endif
+  endforeach
+  if superfluous_manpage.length() > 0
+    error('Man page configured, but not found:\n\n - ' + '\n - '.join(superfluous_manpage))
+  endif
+endif

-- 
2.48.0.rc0.311.gb6c66824c1.dirty