[PATCH v7 0/2] docs: Integrate rustdoc into Rust documentation

[PATCH v7 0/2] docs: Integrate rustdoc into Rust documentation

[Carlos Bilbao]

Include HTML output generated with rustdoc into the Linux kernel
documentation on Rust.

Carlos Bilbao:
 docs: Move rustdoc output, cross-reference it
 docs: Integrate rustdoc generation into htmldocs

---
V6 is https://lore.kernel.org/lkml/20230127165728.119507-1-carlos.bilbao@amd.com/
Changes since V6:
 - Skip Rust document generation entirely if .config doesn't exist.

Changes since V5:
 - Continue executing htmldocs even if rustdoc fails.

Changes since V4:
 - Limit rustdoc note only to html outputs.

Changes since V3:
 - Added Reviewed-Bys from Akira Yokosawa.
 - PATCH 1/2: Avoid error 404 adding tag `rustdoc` for Sphinx.
 - PATCH 1/2: Don't use "here" as link text, describe destination instead.
 - PATCH 2/2: Check CONFIG_RUST in a way that allows us to skip generation.
 - PATCH 2/2: Reoder Sphinx runs so they complete even if rustdoc fails.

Changes since V2:
 - Split v2 into two-patch series.
 - Add "only:: html" directive in Documentation/rust/index.rst reference

Changes since V1:
 - Work on top of v6.1-rc1.
 - Don't use rustdoc.rst, instead add link to Documentation/rust/index.rst.
 - In Documentation/Makefile, replace @make rustdoc for $(Q)$(MAKE) rustdoc.
 - Don't do LLVM=1 for all rustdoc generation within `make htmldocs`.
 - Add spaces on definition of RUSTDOC_OUTPUT, for consistency.

---
 Documentation/Makefile       | 20 ++++++++++++++++++++
 Documentation/rust/index.rst |  8 ++++++++
 rust/Makefile                | 15 +++++++++------
 3 files changed, 37 insertions(+), 6 deletions(-)

[PATCH v7 1/2] docs: Move rustdoc output, cross-reference it

[Carlos Bilbao]

Generate rustdoc documentation with the rest of subsystem's documentation
in Documentation/output. Add a cross reference to the generated rustdoc in
Documentation/rust/index.rst if Sphinx target rustdoc is set.

Reviewed-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Carlos Bilbao <carlos.bilbao@amd.com>
---
 Documentation/rust/index.rst |  8 ++++++++
 rust/Makefile                | 15 +++++++++------
 2 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/Documentation/rust/index.rst b/Documentation/rust/index.rst
index 4ae8c66b94fa..e599be2cec9b 100644
--- a/Documentation/rust/index.rst
+++ b/Documentation/rust/index.rst
@@ -6,6 +6,14 @@ Rust
 Documentation related to Rust within the kernel. To start using Rust
 in the kernel, please read the quick-start.rst guide.
 
+.. only:: rustdoc and html
+
+	You can also browse `rustdoc documentation <rustdoc/kernel/index.html>`_.
+
+.. only:: not rustdoc and html
+
+	This documentation does not include rustdoc generated information.
+
 .. toctree::
     :maxdepth: 1
 
diff --git a/rust/Makefile b/rust/Makefile
index 7c9d9f11aec5..3d502818e5d4 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -1,5 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 
+# Where to place rustdoc generated documentation
+RUSTDOC_OUTPUT = $(objtree)/Documentation/output/rust/rustdoc
+
 obj-$(CONFIG_RUST) += core.o compiler_builtins.o
 always-$(CONFIG_RUST) += exports_core_generated.h
 
@@ -65,7 +68,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
 	OBJTREE=$(abspath $(objtree)) \
 	$(RUSTDOC) $(if $(rustdoc_host),$(rust_common_flags),$(rust_flags)) \
 		$(rustc_target_flags) -L$(objtree)/$(obj) \
-		--output $(objtree)/$(obj)/doc \
+		--output $(RUSTDOC_OUTPUT) \
 		--crate-name $(subst rustdoc-,,$@) \
 		@$(objtree)/include/generated/rustc_cfg $<
 
@@ -82,15 +85,15 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
 # and then retouch the generated files.
 rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \
     rustdoc-alloc rustdoc-kernel
-	$(Q)cp $(srctree)/Documentation/images/logo.svg $(objtree)/$(obj)/doc
-	$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(objtree)/$(obj)/doc
-	$(Q)find $(objtree)/$(obj)/doc -name '*.html' -type f -print0 | xargs -0 sed -Ei \
+	$(Q)cp $(srctree)/Documentation/images/logo.svg $(RUSTDOC_OUTPUT)
+	$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(RUSTDOC_OUTPUT)
+	$(Q)find $(RUSTDOC_OUTPUT) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
 		-e 's:rust-logo\.svg:logo.svg:g' \
 		-e 's:rust-logo\.png:logo.svg:g' \
 		-e 's:favicon\.svg:logo.svg:g' \
 		-e 's:<link rel="alternate icon" type="image/png" href="[./]*favicon-(16x16|32x32)\.png">::g'
 	$(Q)echo '.logo-container > img { object-fit: contain; }' \
-		>> $(objtree)/$(obj)/doc/rustdoc.css
+		>> $(RUSTDOC_OUTPUT)/rustdoc.css
 
 rustdoc-macros: private rustdoc_host = yes
 rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
@@ -154,7 +157,7 @@ quiet_cmd_rustdoc_test = RUSTDOC T $<
 		@$(objtree)/include/generated/rustc_cfg \
 		$(rustc_target_flags) $(rustdoc_test_target_flags) \
 		--sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \
-		-L$(objtree)/$(obj)/test --output $(objtree)/$(obj)/doc \
+		-L$(objtree)/$(obj)/test --output $(RUSTDOC_OUTPUT) \
 		--crate-name $(subst rusttest-,,$@) $<
 
 # We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
-- 
2.34.1

[PATCH v7 2/2] docs: Integrate rustdoc generation into htmldocs

[Carlos Bilbao]

Change target `make htmldocs` to combine RST Sphinx and the generation of
Rust documentation, when support is available and .config exists.

Reviewed-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Carlos Bilbao <carlos.bilbao@amd.com>
---
 Documentation/Makefile | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 023fa658a0a8..3c375c34fd81 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -59,6 +59,12 @@ PAPEROPT_letter = -D latex_paper_size=letter
 KERNELDOC       = $(srctree)/scripts/kernel-doc
 KERNELDOC_CONF  = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
 ALLSPHINXOPTS   =  $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) $(SPHINXOPTS)
+ifneq ($(wildcard $(srctree)/.config),)
+ifeq ($(CONFIG_RUST),y)
+	# Let Sphinx know we will include rustdoc
+	ALLSPHINXOPTS   +=  -t rustdoc
+endif
+endif
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 
@@ -95,6 +101,20 @@ htmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
 
+# If Rust support is available and .config exists, add rustdoc generated contents.
+# If there are any, the errors from this make rustdoc will be displayed but
+# won't stop the execution of htmldocs
+
+ifneq ($(wildcard $(srctree)/.config),)
+ifeq ($(CONFIG_RUST),y)
+	$(Q)$(MAKE) rustdoc || true
+else
+	@echo "   Skipping Rust documentation since CONFIG_RUST is not y."
+endif
+else
+	@echo "   Skipping Rust documentation since .config was not found."
+endif
+
 texinfodocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
-- 
2.34.1

Re: [PATCH v7 0/2] docs: Integrate rustdoc into Rust documentation

[Carlos Bilbao]

On 7/17/23 10:16, Carlos Bilbao wrote:
> Include HTML output generated with rustdoc into the Linux kernel
> documentation on Rust.

Please, note that you can directly test the case in which .config is
not present by running `make htmldocs`. However, to test the case in which
CONFIG_RUST !=y, it is not enough to include a .config; you need to let the
kernel compile for a few seconds first. In other words, until the
compilation process is initiated (using `make`), the build system does not
parse the configuration file or use the defined options.

> 
> Carlos Bilbao:
>   docs: Move rustdoc output, cross-reference it
>   docs: Integrate rustdoc generation into htmldocs
> 
> ---
> V6 is https://lore.kernel.org/lkml/20230127165728.119507-1-carlos.bilbao@amd.com/
> Changes since V6:
>   - Skip Rust document generation entirely if .config doesn't exist.
> 
> Changes since V5:
>   - Continue executing htmldocs even if rustdoc fails.
> 
> Changes since V4:
>   - Limit rustdoc note only to html outputs.
> 
> Changes since V3:
>   - Added Reviewed-Bys from Akira Yokosawa.
>   - PATCH 1/2: Avoid error 404 adding tag `rustdoc` for Sphinx.
>   - PATCH 1/2: Don't use "here" as link text, describe destination instead.
>   - PATCH 2/2: Check CONFIG_RUST in a way that allows us to skip generation.
>   - PATCH 2/2: Reoder Sphinx runs so they complete even if rustdoc fails.
> 
> Changes since V2:
>   - Split v2 into two-patch series.
>   - Add "only:: html" directive in Documentation/rust/index.rst reference
> 
> Changes since V1:
>   - Work on top of v6.1-rc1.
>   - Don't use rustdoc.rst, instead add link to Documentation/rust/index.rst.
>   - In Documentation/Makefile, replace @make rustdoc for $(Q)$(MAKE) rustdoc.
>   - Don't do LLVM=1 for all rustdoc generation within `make htmldocs`.
>   - Add spaces on definition of RUSTDOC_OUTPUT, for consistency.
> 
> ---
>   Documentation/Makefile       | 20 ++++++++++++++++++++
>   Documentation/rust/index.rst |  8 ++++++++
>   rust/Makefile                | 15 +++++++++------
>   3 files changed, 37 insertions(+), 6 deletions(-)
> 

Thanks,
Carlos

Re: [PATCH v7 1/2] docs: Move rustdoc output, cross-reference it

[Miguel Ojeda]

On Mon, Jul 17, 2023 at 5:16 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>
> +# Where to place rustdoc generated documentation
> +RUSTDOC_OUTPUT = $(objtree)/Documentation/output/rust/rustdoc

I think we may be able to get away with just `:=` instead of `=`.

Also, I assume this is intended to be overridable by the user, right?
i.e. that is why you wrote the identifier as uppercase.

In addition, I thought about basing it on `BUILDDIR` from the Doc's
`Makefile`, but that probably needs moving that one so that we can
access it here (in the case where we are not building as part of
`htmldocs`), and thus maybe it is not worth it.

(Cc'ing the rust-for-linux list, by the way)

Cheers,
Miguel

Re: [PATCH v7 2/2] docs: Integrate rustdoc generation into htmldocs

[Miguel Ojeda]

On Mon, Jul 17, 2023 at 5:17 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>
> +       # Let Sphinx know we will include rustdoc
> +       ALLSPHINXOPTS   +=  -t rustdoc

This is needed in the previous patch, no? I don't know if it fails or
just does not work, but either way, perhaps you can split that from
the first commit to put it last.

> +else
> +       @echo "   Skipping Rust documentation since CONFIG_RUST is not y."
> +endif

Is this printed for everybody? If so, then I think it is a good idea
for the future, but perhaps a bit annoying right for most users right
now.

Cheers,
Miguel

Re: [PATCH v7 1/2] docs: Move rustdoc output, cross-reference it

[Carlos Bilbao]

On 7/17/23 11:37, Miguel Ojeda wrote:
> On Mon, Jul 17, 2023 at 5:16 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>>
>> +# Where to place rustdoc generated documentation
>> +RUSTDOC_OUTPUT = $(objtree)/Documentation/output/rust/rustdoc
> 
> I think we may be able to get away with just `:=` instead of `=`.

Yes, for v8 we can simply use `:=`.

> 
> Also, I assume this is intended to be overridable by the user, right?
> i.e. that is why you wrote the identifier as uppercase.

That's true, I don't see any reason to make this uppercase.

> 
> In addition, I thought about basing it on `BUILDDIR` from the Doc's
> `Makefile`, but that probably needs moving that one so that we can
> access it here (in the case where we are not building as part of
> `htmldocs`), and thus maybe it is not worth it.
> 
> (Cc'ing the rust-for-linux list, by the way)

Thanks for CC'ing, I should have done it.

> 
> Cheers,
> Miguel

Thanks,
Carlos

Re: [PATCH v7 2/2] docs: Integrate rustdoc generation into htmldocs

[Carlos Bilbao]

On 7/17/23 11:39, Miguel Ojeda wrote:
> On Mon, Jul 17, 2023 at 5:17 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>>
>> +       # Let Sphinx know we will include rustdoc
>> +       ALLSPHINXOPTS   +=  -t rustdoc
> 
> This is needed in the previous patch, no? I don't know if it fails or
> just does not work, but either way, perhaps you can split that from
> the first commit to put it last.

The previous patch works without this, the generated HTML will show in the
Rust index:

  This documentation does not include rustdoc generated information.

> 
>> +else
>> +       @echo "   Skipping Rust documentation since CONFIG_RUST is not y."
>> +endif
> 
> Is this printed for everybody? If so, then I think it is a good idea
> for the future, but perhaps a bit annoying right for most users right
> now.

Fair point, will remove as part of v8.

> 
> Cheers,
> Miguel

(CC'ed rust-for-linux list)


Thanks,
Carlos

Re: [PATCH v7 1/2] docs: Move rustdoc output, cross-reference it

[Miguel Ojeda]

On Tue, Jul 18, 2023 at 3:50 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>
> On 7/17/23 11:37, Miguel Ojeda wrote:
> >
> > Also, I assume this is intended to be overridable by the user, right?
> > i.e. that is why you wrote the identifier as uppercase.
>
> That's true, I don't see any reason to make this uppercase.

I don't know -- perhaps users may want to override the output
location. `BUILDDIR` is intended to be overridable, so we should
consider what should be the behavior when one overrides one but not
the other. Or perhaps this one shouldn't be overridable, like you did
in v8, in which case we should still make sure things work if that one
(`BUILDDIR`) is overridden.

Cheers,
Miguel

Re: [PATCH v7 2/2] docs: Integrate rustdoc generation into htmldocs

[Miguel Ojeda]

On Tue, Jul 18, 2023 at 3:54 PM Carlos Bilbao <carlos.bilbao@amd.com> wrote:
>
> The previous patch works without this, the generated HTML will show in the
> Rust index:
>
>   This documentation does not include rustdoc generated information.

I see, thanks, then it does not matter much. I guess you put it in the
first commit because the docs were moved to the URL the docs talk
about, but it still means we are using a tag that does not yet exist,
so I would have put the "link/URL feature" as a separate commit
(together with the creation of the tag).

Cheers,
Miguel