[PATCH 0/4] docs: add support for :nosymbols:

[PATCH 0/4] docs: add support for :nosymbols:

[Sean Christopherson]

Linux's kernel-doc directive for Sphinx allows .rst files to selectively
include "functions", i.e. symbols, but does not allow .rst files to
explicitly exclude symbols.  The actual kernel-doc script already
supports excluding symbols, i.e. most of the plumbing is in place, but
needs a bit of cleanup to get things working.

The intended user of this is SGX, which introduces a uapi header that
defines structs for ioctls and for a new vDSO function.  Ideally, the
documentation for the vDSO struct will be placed into a different
section.

I'm sending this as a standalone series as the SGX patch set is already
ridiculously big[1], but I can fold these patches into the SGX series
if that's preferred.

[1] https://lkml.kernel.org/r/20190320162119.4469-1-jarkko.sakkinen@linux.intel.com


Sean Christopherson (4):
  kernel-doc: Revert "scripts/kernel-doc: Processing -nofunc for
    functions only"
  kernel-doc: Rename -nofunction option to -nosymbol
  kernel-doc: Rename -function to -symbol
  docs: sphinx: Add support :nosymbols: to exclude specific symbols

 Documentation/sphinx/kerneldoc.py |  9 +++++++--
 scripts/kernel-doc                | 19 ++++++++++---------
 2 files changed, 17 insertions(+), 11 deletions(-)

-- 
2.21.0

[PATCH 1/4] kernel-doc: Revert "scripts/kernel-doc: Processing -nofunc for functions only"

[Sean Christopherson]

Now that Docbook has been deprecated in favor of Sphinx, the -nofunction
option in kernel-doc is defunct, e.g. Sphinx doesn't currently support
it.  Furthermore, "functions only" behavior was used by Docproc to avoid
duplicating exported symbols, which is now handled by -export and
-internal.

The end goal is to enable using :nofunction: in .rst files to split
documentation of structures into separate categories.

This reverts commit 23aebb3c05f3b3fb06a68bf6b1539a05a5f8aaab.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 scripts/kernel-doc | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 3350e498b4ce..1b40b10794da 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1026,9 +1026,10 @@ sub output_declaration {
 	(($output_selection == OUTPUT_INCLUDE ||
 	  $output_selection == OUTPUT_EXPORTED) &&
 	 defined($function_table{$name})) ||
-	(($output_selection == OUTPUT_EXCLUDE ||
-	  $output_selection == OUTPUT_INTERNAL) &&
-	 !($functype eq "function" && defined($function_table{$name}))))
+	($output_selection == OUTPUT_INTERNAL &&
+	 !($functype eq "function" && defined($function_table{$name}))) ||
+	($output_selection == OUTPUT_EXCLUDE &&
+	 !defined($function_table{$name})))
     {
 	&$func(@_);
 	$section_counter++;
-- 
2.21.0

Re: [PATCH 1/4] kernel-doc: Revert "scripts/kernel-doc: Processing -nofunc for functions only"

[Jonathan Corbet]

On Fri,  5 Apr 2019 14:44:50 -0700
Sean Christopherson <sean.j.christopherson@intel.com> wrote:

> Now that Docbook has been deprecated in favor of Sphinx, the -nofunction
> option in kernel-doc is defunct, e.g. Sphinx doesn't currently support
> it.  Furthermore, "functions only" behavior was used by Docproc to avoid
> duplicating exported symbols, which is now handled by -export and
> -internal.
> 
> The end goal is to enable using :nofunction: in .rst files to split
> documentation of structures into separate categories.
> 
> This reverts commit 23aebb3c05f3b3fb06a68bf6b1539a05a5f8aaab.
> 
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>

It seems strange to express this as a revert - it's far from a clean
revert of the original patch.  I think it's probably better to just merge
this series into a single patch that implements the functionality you're
after.

As for the single/plural question, let's go with singular to match
--function.

Thanks,

jon

[PATCH 2/4] kernel-doc: Rename -nofunction option to -nosymbol

[Sean Christopherson]

...now that it applies to all symbol types.  Do the rename before
Sphinx starts using the existing option.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 scripts/kernel-doc | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 1b40b10794da..9190110b9802 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -66,8 +66,8 @@ Output selection (mutually exclusive):
   -function NAME	Only output documentation for the given function(s)
 			or DOC: section title(s). All other functions and DOC:
 			sections are ignored. May be specified multiple times.
-  -nofunction NAME	Do NOT output documentation for the given function(s);
-			only output documentation for the other functions and
+  -nosymbol NAME	Do NOT output documentation for the given symbol(s);
+			only output documentation for the other symbols and
 			DOC: sections. May be specified multiple times.
 
 Output selection modifiers:
@@ -409,11 +409,11 @@ while ($ARGV[0] =~ m/^--?(.*)/) {
 	$output_mode = "none";
     } elsif ($cmd eq "module") { # not needed for XML, inherits from calling document
 	$modulename = shift @ARGV;
-    } elsif ($cmd eq "function") { # to only output specific functions
+    } elsif ($cmd eq "function") { # to only output specific function
 	$output_selection = OUTPUT_INCLUDE;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
-    } elsif ($cmd eq "nofunction") { # output all except specific functions
+    } elsif ($cmd eq "nosymbol") { # output all except specific symbols
 	$output_selection = OUTPUT_EXCLUDE;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
-- 
2.21.0

[PATCH 3/4] kernel-doc: Rename -function to -symbol

[Sean Christopherson]

The -function option applies to all symbol types, not just functions.
Rename it and update its help text and comments to reflect reality.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 Documentation/sphinx/kerneldoc.py | 4 ++--
 scripts/kernel-doc                | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index 9d0a7f08f93b..c52b6caac356 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -73,12 +73,12 @@ class KernelDocDirective(Directive):
             cmd += ['-internal']
             export_file_patterns = str(self.options.get('internal')).split()
         elif 'doc' in self.options:
-            cmd += ['-function', str(self.options.get('doc'))]
+            cmd += ['-symbol', str(self.options.get('doc'))]
         elif 'functions' in self.options:
             functions = self.options.get('functions').split()
             if functions:
                 for f in functions:
-                    cmd += ['-function', f]
+                    cmd += ['-symbol', f]
             else:
                 cmd += ['-no-doc-sections']
 
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9190110b9802..60ef90222a51 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -63,10 +63,10 @@ Output selection (mutually exclusive):
   -internal		Only output documentation for symbols that have NOT been
 			exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
                         in any input FILE or -export-file FILE.
-  -function NAME	Only output documentation for the given function(s)
-			or DOC: section title(s). All other functions and DOC:
+  -symbols	NAME	Only output documentation for the given symbol(s)
+			or DOC: section title(s). All other symbols and DOC:
 			sections are ignored. May be specified multiple times.
-  -nosymbol NAME	Do NOT output documentation for the given symbol(s);
+  -nosymbol	NAME	Do NOT output documentation for the given symbol(s);
 			only output documentation for the other symbols and
 			DOC: sections. May be specified multiple times.
 
@@ -409,7 +409,7 @@ while ($ARGV[0] =~ m/^--?(.*)/) {
 	$output_mode = "none";
     } elsif ($cmd eq "module") { # not needed for XML, inherits from calling document
 	$modulename = shift @ARGV;
-    } elsif ($cmd eq "function") { # to only output specific function
+    } elsif ($cmd eq "symbol") { # to only output specific symbol
 	$output_selection = OUTPUT_INCLUDE;
 	$function = shift @ARGV;
 	$function_table{$function} = 1;
-- 
2.21.0

Re: [PATCH 3/4] kernel-doc: Rename -function to -symbol

[Jani Nikula]

On Fri, 05 Apr 2019, Sean Christopherson <sean.j.christopherson@intel.com> wrote:
> The -function option applies to all symbol types, not just functions.
> Rename it and update its help text and comments to reflect reality.
>
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> ---
>  Documentation/sphinx/kerneldoc.py | 4 ++--
>  scripts/kernel-doc                | 8 ++++----
>  2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
> index 9d0a7f08f93b..c52b6caac356 100644
> --- a/Documentation/sphinx/kerneldoc.py
> +++ b/Documentation/sphinx/kerneldoc.py
> @@ -73,12 +73,12 @@ class KernelDocDirective(Directive):
>              cmd += ['-internal']
>              export_file_patterns = str(self.options.get('internal')).split()
>          elif 'doc' in self.options:
> -            cmd += ['-function', str(self.options.get('doc'))]
> +            cmd += ['-symbol', str(self.options.get('doc'))]
>          elif 'functions' in self.options:
>              functions = self.options.get('functions').split()
>              if functions:
>                  for f in functions:
> -                    cmd += ['-function', f]
> +                    cmd += ['-symbol', f]
>              else:
>                  cmd += ['-no-doc-sections']
>  
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 9190110b9802..60ef90222a51 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -63,10 +63,10 @@ Output selection (mutually exclusive):
>    -internal		Only output documentation for symbols that have NOT been
>  			exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
>                          in any input FILE or -export-file FILE.
> -  -function NAME	Only output documentation for the given function(s)
> -			or DOC: section title(s). All other functions and DOC:
> +  -symbols	NAME	Only output documentation for the given symbol(s)
> +			or DOC: section title(s). All other symbols and DOC:
>  			sections are ignored. May be specified multiple times.
> -  -nosymbol NAME	Do NOT output documentation for the given symbol(s);
> +  -nosymbol	NAME	Do NOT output documentation for the given symbol(s);
>  			only output documentation for the other symbols and
>  			DOC: sections. May be specified multiple times.

Please decide whether to use singular or prular, and stick to it
throughout.

BR,
Jani.


>  
> @@ -409,7 +409,7 @@ while ($ARGV[0] =~ m/^--?(.*)/) {
>  	$output_mode = "none";
>      } elsif ($cmd eq "module") { # not needed for XML, inherits from calling document
>  	$modulename = shift @ARGV;
> -    } elsif ($cmd eq "function") { # to only output specific function
> +    } elsif ($cmd eq "symbol") { # to only output specific symbol
>  	$output_selection = OUTPUT_INCLUDE;
>  	$function = shift @ARGV;
>  	$function_table{$function} = 1;

-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [PATCH 3/4] kernel-doc: Rename -function to -symbol

[Sean Christopherson]

On Mon, Apr 08, 2019 at 01:03:49PM +0300, Jani Nikula wrote:
> On Fri, 05 Apr 2019, Sean Christopherson <sean.j.christopherson@intel.com> wrote:
> > The -function option applies to all symbol types, not just functions.
> > Rename it and update its help text and comments to reflect reality.
> >
> > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> > ---
> >  Documentation/sphinx/kerneldoc.py | 4 ++--
> >  scripts/kernel-doc                | 8 ++++----
> >  2 files changed, 6 insertions(+), 6 deletions(-)
> >
> > diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
> > index 9d0a7f08f93b..c52b6caac356 100644
> > --- a/Documentation/sphinx/kerneldoc.py
> > +++ b/Documentation/sphinx/kerneldoc.py
> > @@ -73,12 +73,12 @@ class KernelDocDirective(Directive):
> >              cmd += ['-internal']
> >              export_file_patterns = str(self.options.get('internal')).split()
> >          elif 'doc' in self.options:
> > -            cmd += ['-function', str(self.options.get('doc'))]
> > +            cmd += ['-symbol', str(self.options.get('doc'))]
> >          elif 'functions' in self.options:
> >              functions = self.options.get('functions').split()
> >              if functions:
> >                  for f in functions:
> > -                    cmd += ['-function', f]
> > +                    cmd += ['-symbol', f]
> >              else:
> >                  cmd += ['-no-doc-sections']
> >  
> > diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> > index 9190110b9802..60ef90222a51 100755
> > --- a/scripts/kernel-doc
> > +++ b/scripts/kernel-doc
> > @@ -63,10 +63,10 @@ Output selection (mutually exclusive):
> >    -internal		Only output documentation for symbols that have NOT been
> >  			exported using EXPORT_SYMBOL() or EXPORT_SYMBOL_GPL()
> >                          in any input FILE or -export-file FILE.
> > -  -function NAME	Only output documentation for the given function(s)
> > -			or DOC: section title(s). All other functions and DOC:
> > +  -symbols	NAME	Only output documentation for the given symbol(s)
> > +			or DOC: section title(s). All other symbols and DOC:
> >  			sections are ignored. May be specified multiple times.
> > -  -nosymbol NAME	Do NOT output documentation for the given symbol(s);
> > +  -nosymbol	NAME	Do NOT output documentation for the given symbol(s);
> >  			only output documentation for the other symbols and
> >  			DOC: sections. May be specified multiple times.
> 
> Please decide whether to use singular or prular, and stick to it
> throughout.

"-symbols NAME" is a typo, should be "-symbol NAME".


Any preference on keeping or removing the "(s)" in the description, e.g.

  Only output documentation for the given function(s) or DOC: section title(s).

versus:

  Only output documentation for the given symbol or DOC: section title.

I think it makes sense to drop the "(s)" as each option can only specify a
single symbole, i.e. the "(s)" behavior is covered by "May be specified
multiple times."

> 
> BR,
> Jani.
> 
> 
> >  
> > @@ -409,7 +409,7 @@ while ($ARGV[0] =~ m/^--?(.*)/) {
> >  	$output_mode = "none";
> >      } elsif ($cmd eq "module") { # not needed for XML, inherits from calling document
> >  	$modulename = shift @ARGV;
> > -    } elsif ($cmd eq "function") { # to only output specific function
> > +    } elsif ($cmd eq "symbol") { # to only output specific symbol
> >  	$output_selection = OUTPUT_INCLUDE;
> >  	$function = shift @ARGV;
> >  	$function_table{$function} = 1;
> 
> -- 
> Jani Nikula, Intel Open Source Graphics Center

[PATCH 4/4] docs: sphinx: Add support :nosymbols: to exclude specific symbols

[Sean Christopherson]

Add :nosymbols: so that documentation of symbols can be organized in the
final output of makedocs.  For example, files that define a large number
of symbols may want to categorize their symbols.  Another example is
SGX, which introduces a uapi header that defines structs for ioctls and
for a new vDSO function[1].  Ideally, the documentation for the vDSO
struct will be placed into a different section.

[1] https://lkml.kernel.org/r/20190320162119.4469-25-jarkko.sakkinen@linux.intel.com

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 Documentation/sphinx/kerneldoc.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index c52b6caac356..d1eb1907858b 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -48,6 +48,7 @@ class KernelDocDirective(Directive):
     option_spec = {
         'doc': directives.unchanged_required,
         'functions': directives.unchanged,
+        'nosymbols': directives.unchanged,
         'export': directives.unchanged,
         'internal': directives.unchanged,
     }
@@ -81,6 +82,10 @@ class KernelDocDirective(Directive):
                     cmd += ['-symbol', f]
             else:
                 cmd += ['-no-doc-sections']
+        elif 'nosymbols' in self.options:
+            symbols = self.options.get('nosymbols').split()
+            for s in symbols:
+                cmd += ['-nosymbol', s]
 
         for pattern in export_file_patterns:
             for f in glob.glob(env.config.kerneldoc_srctree + '/' + pattern):
-- 
2.21.0