[PATCH 0/2] docs: Add automatic cross-reference for C types

[PATCH 0/2] docs: Add automatic cross-reference for C types

[Nícolas F. R. A. Prado]

In order to cross-reference C types in the documentation, Sphinx
requires the syntax :c:type:`type_name`, or even :c:type:`struct
type_name <type_name>` in order to have the link text different from the
target text.
This patch series removes the need for that markup.

The first commit extends the automarkup script to enable automatic
cross-reference of C types by matching any "struct|union|enum|typedef type_name"
expression.
This makes the documentation's plain text cleaner and adds cross-reference to
types without any additional effort by the author.

The second commit updates the "Cross-referencing from
reStructuredText" section in Documentation/doc-guide/kernel-doc.rst
to reflect that no additional syntax is needed when cross-referencing both types
and functions anymore.

When testing this, I did find an edge-case from the output of
Documentation/output/scsi/scsi_mid_low_api.rst on the "typedef struct scsi_cmnd
Scsi_Cmnd;", where 'typedef struct' is being identified as a reference, but
there isn't any named 'struct', so it renders bold.
I thought of adding an ignore_names list just like there is one for functions,
with 'struct' in it, to workaround this edge case, but since it was the only
one I found, and also because it was unclear what the desired output was
(cross-reference 'struct scsi_cmnd' or leave the whole expression as plain text)
I wanted to get some feedback beforehand.

After getting this merged I intend to start removing the occurrences of :c:type.

Thanks,
Nícolas

Nícolas F. R. A. Prado (2):
  docs: Add automatic cross-reference for C types
  kernel-doc: Update "cross-referencing from rST" section to use
    automarkup

 Documentation/doc-guide/kernel-doc.rst | 33 ++++++++++++-----------
 Documentation/sphinx/automarkup.py     | 37 +++++++++++++++++---------
 2 files changed, 41 insertions(+), 29 deletions(-)

-- 
2.28.0

[PATCH 1/2] docs: Add automatic cross-reference for C types

[Nícolas F. R. A. Prado]

In order to cross-reference C types in the documentation, Sphinx
requires the syntax :c:type:`type_name`, or even :c:type:`struct
type_name <type_name>` in order to have the link text different from the
target text.

Extend automarkup to enable automatic cross-reference of C types by
matching any "struct|union|enum|typedef type_name" expression.
This makes the documentation's plain text cleaner and adds
cross-reference to types without any additional effort by the author.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
---
 Documentation/sphinx/automarkup.py | 37 +++++++++++++++++++-----------
 1 file changed, 24 insertions(+), 13 deletions(-)

diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index b18236370742..272eb2bdfab1 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -13,6 +13,7 @@ if sphinx.version_info[0] < 2 or \
 else:
     from sphinx.errors import NoUri
 import re
+from itertools import chain
 
 #
 # Regex nastiness.  Of course.
@@ -21,7 +22,8 @@ import re
 # :c:func: block (i.e. ":c:func:`mmap()`s" flakes out), so the last
 # bit tries to restrict matches to things that won't create trouble.
 #
-RE_function = re.compile(r'([\w_][\w\d_]+\(\))')
+RE_function = re.compile(r'(([\w_][\w\d_]+)\(\))')
+RE_type = re.compile(r'(struct|union|enum|typedef)\s+([\w_][\w\d_]+)')
 
 #
 # Many places in the docs refer to common system calls.  It is
@@ -35,31 +37,39 @@ Skipfuncs = [ 'open', 'close', 'read', 'write', 'fcntl', 'mmap',
               'socket' ]
 
 #
-# Find all occurrences of function() and try to replace them with
-# appropriate cross references.
+# Find all occurrences of C references (function() and struct/union/enum/typedef
+# type_name) and try to replace them with appropriate cross references.
 #
-def markup_funcs(docname, app, node):
+def markup_c_refs(docname, app, node):
+    class_str = {RE_function: 'c-func', RE_type: 'c-type'}
+    reftype_str = {RE_function: 'function', RE_type: 'type'}
+
     cdom = app.env.domains['c']
     t = node.astext()
     done = 0
     repl = [ ]
-    for m in RE_function.finditer(t):
+    #
+    # Sort all C references by the starting position in text
+    #
+    sorted_matches = sorted(chain(RE_type.finditer(t), RE_function.finditer(t)),
+                            key=lambda m: m.start())
+    for m in sorted_matches:
         #
-        # Include any text prior to function() as a normal text node.
+        # Include any text prior to match as a normal text node.
         #
         if m.start() > done:
             repl.append(nodes.Text(t[done:m.start()]))
         #
         # Go through the dance of getting an xref out of the C domain
         #
-        target = m.group(1)[:-2]
-        target_text = nodes.Text(target + '()')
+        target = m.group(2)
+        target_text = nodes.Text(m.group(0))
         xref = None
-        if target not in Skipfuncs:
-            lit_text = nodes.literal(classes=['xref', 'c', 'c-func'])
+        if not (m.re == RE_function and target in Skipfuncs):
+            lit_text = nodes.literal(classes=['xref', 'c', class_str[m.re]])
             lit_text += target_text
             pxref = addnodes.pending_xref('', refdomain = 'c',
-                                          reftype = 'function',
+                                          reftype = reftype_str[m.re],
                                           reftarget = target, modname = None,
                                           classname = None)
             #
@@ -68,7 +78,8 @@ def markup_funcs(docname, app, node):
             #
             try:
                 xref = cdom.resolve_xref(app.env, docname, app.builder,
-                                         'function', target, pxref, lit_text)
+                                         reftype_str[m.re], target, pxref,
+                                         lit_text)
             except NoUri:
                 xref = None
         #
@@ -97,7 +108,7 @@ def auto_markup(app, doctree, name):
     for para in doctree.traverse(nodes.paragraph):
         for node in para.traverse(nodes.Text):
             if not isinstance(node.parent, nodes.literal):
-                node.parent.replace(node, markup_funcs(name, app, node))
+                node.parent.replace(node, markup_c_refs(name, app, node))
 
 def setup(app):
     app.connect('doctree-resolved', auto_markup)
-- 
2.28.0

[PATCH 2/2] kernel-doc: Update "cross-referencing from rST" section to use automarkup

[Nícolas F. R. A. Prado]

Update text and examples in the "Cross-referencing from
reStructuredText" section to reflect that no additional syntax is needed
anymore.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
---
 Documentation/doc-guide/kernel-doc.rst | 33 +++++++++++++-------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index fff6604631ea..4fd86c21397b 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -387,22 +387,23 @@ Domain`_ references.
 Cross-referencing from reStructuredText
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To cross-reference the functions and types defined in the kernel-doc comments
-from reStructuredText documents, please use the `Sphinx C Domain`_
-references. For example::
-
-  See function :c:func:`foo` and struct/union/enum/typedef :c:type:`bar`.
-
-While the type reference works with just the type name, without the
-struct/union/enum/typedef part in front, you may want to use::
-
-  See :c:type:`struct foo <foo>`.
-  See :c:type:`union bar <bar>`.
-  See :c:type:`enum baz <baz>`.
-  See :c:type:`typedef meh <meh>`.
-
-This will produce prettier links, and is in line with how kernel-doc does the
-cross-references.
+No additional syntax is needed to cross-reference the functions and types
+defined in the kernel-doc comments from reStructuredText documents.
+Just end function names with ``()`` and write ``struct``, ``union``, ``enum``
+or ``typedef`` before types.
+For example::
+
+  See foo().
+  See struct foo.
+  See union bar.
+  See enum baz.
+  See typedef meh.
+
+However, if you want custom text in the cross-reference link, that can be done
+through the following syntax::
+
+  See :c:func:`my custom link text for function foo <foo>`.
+  See :c:type:`my custom link text for struct bar <bar>`.
 
 For further details, please refer to the `Sphinx C Domain`_ documentation.
 
-- 
2.28.0

Re: [PATCH 0/2] docs: Add automatic cross-reference for C types

[Randy Dunlap]

On 9/2/20 5:58 PM, Nícolas F. R. A. Prado wrote:
> In order to cross-reference C types in the documentation, Sphinx
> requires the syntax :c:type:`type_name`, or even :c:type:`struct
> type_name <type_name>` in order to have the link text different from the
> target text.
> This patch series removes the need for that markup.
> 
> The first commit extends the automarkup script to enable automatic
> cross-reference of C types by matching any "struct|union|enum|typedef type_name"
> expression.
> This makes the documentation's plain text cleaner and adds cross-reference to
> types without any additional effort by the author.
> 
> The second commit updates the "Cross-referencing from
> reStructuredText" section in Documentation/doc-guide/kernel-doc.rst
> to reflect that no additional syntax is needed when cross-referencing both types
> and functions anymore.
> 
> When testing this, I did find an edge-case from the output of
> Documentation/output/scsi/scsi_mid_low_api.rst on the "typedef struct scsi_cmnd
> Scsi_Cmnd;", where 'typedef struct' is being identified as a reference, but
> there isn't any named 'struct', so it renders bold.

There also isn't any file name scsi_typedefs.h any longer,
so maybe we can update scsi_mid_low_api.rst as well.

Thanks.

> I thought of adding an ignore_names list just like there is one for functions,
> with 'struct' in it, to workaround this edge case, but since it was the only
> one I found, and also because it was unclear what the desired output was
> (cross-reference 'struct scsi_cmnd' or leave the whole expression as plain text)
> I wanted to get some feedback beforehand.
> 
> After getting this merged I intend to start removing the occurrences of :c:type.
> 
> Thanks,
> Nícolas
> 
> Nícolas F. R. A. Prado (2):
>   docs: Add automatic cross-reference for C types
>   kernel-doc: Update "cross-referencing from rST" section to use
>     automarkup
> 
>  Documentation/doc-guide/kernel-doc.rst | 33 ++++++++++++-----------
>  Documentation/sphinx/automarkup.py     | 37 +++++++++++++++++---------
>  2 files changed, 41 insertions(+), 29 deletions(-)
> 


-- 
~Randy

Re: [PATCH 0/2] docs: Add automatic cross-reference for C types

[Jonathan Corbet]

On Thu, 03 Sep 2020 00:58:09 +0000
Nícolas F. R. A. Prado <nfraprado@protonmail.com> wrote:

> In order to cross-reference C types in the documentation, Sphinx
> requires the syntax :c:type:`type_name`, or even :c:type:`struct
> type_name <type_name>` in order to have the link text different from the
> target text.
> This patch series removes the need for that markup.
> 
> The first commit extends the automarkup script to enable automatic
> cross-reference of C types by matching any "struct|union|enum|typedef type_name"
> expression.
> This makes the documentation's plain text cleaner and adds cross-reference to
> types without any additional effort by the author.
> 
> The second commit updates the "Cross-referencing from
> reStructuredText" section in Documentation/doc-guide/kernel-doc.rst
> to reflect that no additional syntax is needed when cross-referencing both types
> and functions anymore.

So I've looked this over, applied it, looked at how the output changes,
and generally put an untoward amount of effort into finding something to
grumble about.

I failed.

This is a great change - thanks for stepping up and doing it!

> When testing this, I did find an edge-case from the output of
> Documentation/output/scsi/scsi_mid_low_api.rst on the "typedef struct scsi_cmnd
> Scsi_Cmnd;", where 'typedef struct' is being identified as a reference, but
> there isn't any named 'struct', so it renders bold.
> I thought of adding an ignore_names list just like there is one for functions,
> with 'struct' in it, to workaround this edge case, but since it was the only
> one I found, and also because it was unclear what the desired output was
> (cross-reference 'struct scsi_cmnd' or leave the whole expression as plain text)
> I wanted to get some feedback beforehand.

As Randy pointed out, the documentation in question is obsolete and
incorrect anyway; there's really not much that can be done in the docs
build system to fix *that*.  I'm not going to worry about this particular
glitch.  If you felt so inspired, a patch to the SCSI maintainers just
removing that paragraph might be well received.

I'm going to go ahead and push this out to linux-next; we'll see if
anything explodes from there.

Thanks,

jon