[PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Randy Dunlap]

Memory profiling introduces macros as hooks for function-level
allocation profiling[1]. Memory allocation functions that are profiled
are named like xyz_alloc() for API access to the function. xyz_alloc()
then calls xyz_alloc_noprof() to do the allocation work.

The kernel-doc comments for the memory allocation functions are
introduced with the xyz_alloc() function names but the function
implementations are the xyz_alloc_noprof() names.
This causes kernel-doc warnings for mismatched documentation and
function prototype names.
By dropping the "_noprof" part of the function name, the kernel-doc
function name matches the function prototype name, so the warnings
are resolved.

[1] https://lore.kernel.org/all/20240321163705.3067592-1-surenb@google.com/

Fixes: c64e38ed88d1 ("mm/slab: enable slab allocation tagging for kmalloc and friends")
Fixes: ea7b8933f21b ("mempool: hook up to memory allocation profiling")
Fixes: 576477564ede ("mm: vmalloc: enable memory allocation profiling")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Closes: https://lore.kernel.org/all/20240325123603.1bdd6588@canb.auug.org.au/
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
---
a. Suren, Kent, feel free to correct my comments on memory profiling.
I haven't read the documentation in a few weeks.

b. There are some instances of the kernel-doc function name including
"_noprof". Suren is correcting those cases so that all _noprof warnings
will be resolved.

 scripts/kernel-doc |    1 +
 1 file changed, 1 insertion(+)

diff -- a/scripts/kernel-doc b/scripts/kernel-doc
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1723,6 +1723,7 @@ sub dump_function($$) {
     $prototype =~ s/__must_check +//;
     $prototype =~ s/__weak +//;
     $prototype =~ s/__sched +//;
+    $prototype =~ s/_noprof//;
     $prototype =~ s/__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +//;
     $prototype =~ s/__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +//;
     $prototype =~ s/__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +//;

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Suren Baghdasaryan]

On Mon, Mar 25, 2024 at 10:41 PM Randy Dunlap <rdunlap@infradead.org> wrote:
>
> Memory profiling introduces macros as hooks for function-level
> allocation profiling[1]. Memory allocation functions that are profiled
> are named like xyz_alloc() for API access to the function. xyz_alloc()
> then calls xyz_alloc_noprof() to do the allocation work.
>
> The kernel-doc comments for the memory allocation functions are
> introduced with the xyz_alloc() function names but the function
> implementations are the xyz_alloc_noprof() names.
> This causes kernel-doc warnings for mismatched documentation and
> function prototype names.
> By dropping the "_noprof" part of the function name, the kernel-doc
> function name matches the function prototype name, so the warnings
> are resolved.
>
> [1] https://lore.kernel.org/all/20240321163705.3067592-1-surenb@google.com/
>
> Fixes: c64e38ed88d1 ("mm/slab: enable slab allocation tagging for kmalloc and friends")
> Fixes: ea7b8933f21b ("mempool: hook up to memory allocation profiling")
> Fixes: 576477564ede ("mm: vmalloc: enable memory allocation profiling")
> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Closes: https://lore.kernel.org/all/20240325123603.1bdd6588@canb.auug.org.au/
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Kent Overstreet <kent.overstreet@linux.dev>
> Cc: Suren Baghdasaryan <surenb@google.com>

Tried it out and didn't see the warnings anymore. Thanks Randy!

Tested-by: Suren Baghdasaryan <surenb@google.com>

> ---
> a. Suren, Kent, feel free to correct my comments on memory profiling.
> I haven't read the documentation in a few weeks.
>
> b. There are some instances of the kernel-doc function name including
> "_noprof". Suren is correcting those cases so that all _noprof warnings
> will be resolved.
>
>  scripts/kernel-doc |    1 +
>  1 file changed, 1 insertion(+)
>
> diff -- a/scripts/kernel-doc b/scripts/kernel-doc
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -1723,6 +1723,7 @@ sub dump_function($$) {
>      $prototype =~ s/__must_check +//;
>      $prototype =~ s/__weak +//;
>      $prototype =~ s/__sched +//;
> +    $prototype =~ s/_noprof//;
>      $prototype =~ s/__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +//;
>      $prototype =~ s/__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +//;
>      $prototype =~ s/__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +//;

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Matthew Wilcox]

On Mon, Mar 25, 2024 at 10:41:49PM -0700, Randy Dunlap wrote:
> Memory profiling introduces macros as hooks for function-level
> allocation profiling[1]. Memory allocation functions that are profiled
> are named like xyz_alloc() for API access to the function. xyz_alloc()
> then calls xyz_alloc_noprof() to do the allocation work.
> 
> The kernel-doc comments for the memory allocation functions are
> introduced with the xyz_alloc() function names but the function
> implementations are the xyz_alloc_noprof() names.
> This causes kernel-doc warnings for mismatched documentation and
> function prototype names.
> By dropping the "_noprof" part of the function name, the kernel-doc
> function name matches the function prototype name, so the warnings
> are resolved.

This turns out not to be enough.  For example, krealloc() is
currently undocumented.  This is because we match the function name
in EXPORT_SYMBOL() against the function name in the comment, and they
don't match.  This patch restores the documentation, although only
for the python version of kernel-doc, and I'm pretty sure there's a
better way to do it (eg building it into the export_symbol* regexes).
I can turn this into a proper patch if this is the way to go, but for
now it's just to illustrate the problem.

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 062453eefc7a..bdfa698d5570 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1176,11 +1176,15 @@ class KernelDoc:

         if export_symbol.search(line):
             symbol = export_symbol.group(2)
+            # See alloc_tags.h
+            symbol = symbol.removesuffix('_noprof')
             function_set.add(symbol)
             return

         if export_symbol_ns.search(line):
             symbol = export_symbol_ns.group(2)
+            # See alloc_tags.h
+            symbol = symbol.removesuffix('_noprof')
             function_set.add(symbol)

     def process_normal(self, ln, line):

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Jonathan Corbet]

Matthew Wilcox <willy@infradead.org> writes:

> On Mon, Mar 25, 2024 at 10:41:49PM -0700, Randy Dunlap wrote:
>> Memory profiling introduces macros as hooks for function-level
>> allocation profiling[1]. Memory allocation functions that are profiled
>> are named like xyz_alloc() for API access to the function. xyz_alloc()
>> then calls xyz_alloc_noprof() to do the allocation work.
>> 
>> The kernel-doc comments for the memory allocation functions are
>> introduced with the xyz_alloc() function names but the function
>> implementations are the xyz_alloc_noprof() names.
>> This causes kernel-doc warnings for mismatched documentation and
>> function prototype names.
>> By dropping the "_noprof" part of the function name, the kernel-doc
>> function name matches the function prototype name, so the warnings
>> are resolved.
>
> This turns out not to be enough.  For example, krealloc() is
> currently undocumented.  This is because we match the function name
> in EXPORT_SYMBOL() against the function name in the comment, and they
> don't match.  This patch restores the documentation, although only
> for the python version of kernel-doc, and I'm pretty sure there's a
> better way to do it (eg building it into the export_symbol* regexes).
> I can turn this into a proper patch if this is the way to go, but for
> now it's just to illustrate the problem.

FWIW, I have no problem with leaving the perl version behind, I expect
we'll drop it in 6.17.

We see other variants of this problem out there, where we want to
document foo(), but that's really just a macro calling _foo(), where the
real code is.

I wonder if we could add some sort of a marker to the kerneldoc comment
saying "we are documenting foo(), but do you checks against _foo()"
instead?  That would be more general than trying to keep a list of
suffixes to hack off.

I'll try to ponder on this...

(Meanwhile I don't object to your fix as a short-term workaround)

jon

> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 062453eefc7a..bdfa698d5570 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1176,11 +1176,15 @@ class KernelDoc:
>
>          if export_symbol.search(line):
>              symbol = export_symbol.group(2)
> +            # See alloc_tags.h
> +            symbol = symbol.removesuffix('_noprof')
>              function_set.add(symbol)
>              return
>
>          if export_symbol_ns.search(line):
>              symbol = export_symbol_ns.group(2)
> +            # See alloc_tags.h
> +            symbol = symbol.removesuffix('_noprof')
>              function_set.add(symbol)
>
>      def process_normal(self, ln, line):

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Matthew Wilcox]

On Thu, Jun 05, 2025 at 01:18:50PM -0600, Jonathan Corbet wrote:
> Matthew Wilcox <willy@infradead.org> writes:
> > This turns out not to be enough.  For example, krealloc() is
> > currently undocumented.  This is because we match the function name
> > in EXPORT_SYMBOL() against the function name in the comment, and they
> > don't match.  This patch restores the documentation, although only
> > for the python version of kernel-doc, and I'm pretty sure there's a
> > better way to do it (eg building it into the export_symbol* regexes).
> > I can turn this into a proper patch if this is the way to go, but for
> > now it's just to illustrate the problem.
> 
> FWIW, I have no problem with leaving the perl version behind, I expect
> we'll drop it in 6.17.
> 
> (Meanwhile I don't object to your fix as a short-term workaround)

OK, will give Mauro 24 hours to comment, then resend as a patch if
there are no objections.

> We see other variants of this problem out there, where we want to
> document foo(), but that's really just a macro calling _foo(), where the
> real code is.
> 
> I wonder if we could add some sort of a marker to the kerneldoc comment
> saying "we are documenting foo(), but do you checks against _foo()"
> instead?  That would be more general than trying to keep a list of
> suffixes to hack off.

kernel-doc is our own format, so sure, we can add whatever marker
we want to it.  I think it's not quite general enough because we have
situations like:

static inline void foo(int x)
{
	numa_foo(x, NUMA_NO_NODE);
};

/**
 * foo - Frobnicate
 * @x: How many
 * @nid: Which node
 */
void numa_foo(int x, int node)
{ .. }

and now we're documenting a parameter that doesn't exist.  The only
solution is to move the kdoc to the header file, which is annoying for
everyone.  Or replicate the declaration in the C file and kdoc it there.

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Jonathan Corbet]

Matthew Wilcox <willy@infradead.org> writes:

> kernel-doc is our own format, so sure, we can add whatever marker
> we want to it.  I think it's not quite general enough because we have
> situations like:
>
> static inline void foo(int x)
> {
> 	numa_foo(x, NUMA_NO_NODE);
> };
>
> /**
>  * foo - Frobnicate
>  * @x: How many
>  * @nid: Which node
>  */
> void numa_foo(int x, int node)
> { .. }
>
> and now we're documenting a parameter that doesn't exist.  The only
> solution is to move the kdoc to the header file, which is annoying for
> everyone.  Or replicate the declaration in the C file and kdoc it there.

In this case, though, is there a reason to not document foo() as a
separate function?

jon

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Mauro Carvalho Chehab]

Em Thu, 5 Jun 2025 20:31:02 +0100
Matthew Wilcox <willy@infradead.org> escreveu:

> On Thu, Jun 05, 2025 at 01:18:50PM -0600, Jonathan Corbet wrote:
> > Matthew Wilcox <willy@infradead.org> writes:  
> > > This turns out not to be enough.  For example, krealloc() is
> > > currently undocumented.  This is because we match the function name
> > > in EXPORT_SYMBOL() against the function name in the comment, and they
> > > don't match.  This patch restores the documentation, although only
> > > for the python version of kernel-doc, and I'm pretty sure there's a
> > > better way to do it (eg building it into the export_symbol* regexes).
> > > I can turn this into a proper patch if this is the way to go, but for
> > > now it's just to illustrate the problem.  
> > 
> > FWIW, I have no problem with leaving the perl version behind, I expect
> > we'll drop it in 6.17.

I agree with Jon: it is time to retire the perl version. 

> > 
> > (Meanwhile I don't object to your fix as a short-term workaround)  
> 
> OK, will give Mauro 24 hours to comment, then resend as a patch if
> there are no objections.
> 
> > We see other variants of this problem out there, where we want to
> > document foo(), but that's really just a macro calling _foo(), where the
> > real code is.

The problem is that one may want to document both _foo() and foo(),
if they have different arguments. I'm pretty sure we have cases like
that.

> > 
> > I wonder if we could add some sort of a marker to the kerneldoc comment
> > saying "we are documenting foo(), but do you checks against _foo()"
> > instead?  That would be more general than trying to keep a list of
> > suffixes to hack off.  
> 
> kernel-doc is our own format, so sure, we can add whatever marker
> we want to it.  I think it's not quite general enough because we have
> situations like:
> 
> static inline void foo(int x)
> {
> 	numa_foo(x, NUMA_NO_NODE);
> };
> 
> /**
>  * foo - Frobnicate
>  * @x: How many
>  * @nid: Which node
>  */
> void numa_foo(int x, int node)
> { .. }

If I"m not mistaken, if you do things like that, kernel-doc.py will
complain that "foo" is not "numa_foo". It will also complain that
"nid" doesn't exist and "node" is not documented.

Basically, there is a strict check there (if it got properly 
backported from the Perl version) which checks if kernel-doc
is documenting the next function prototype name and argument
names.

The rationale is that we caught several cases where a function was
removed, renamed and/or have their parameters renamed without the
corresponding kernel-doc change. So, the verification is now
stricter (*). When we enabled such check, we fixed several bad
kernel-doc markups.

(*) Also, kernel-doc handles files in one pass at read time,
    sequentially. It could be possible to change it, but kernel-doc 
    is already complex enough, and placing the markup just before
    the function is a good practice anyway.

> and now we're documenting a parameter that doesn't exist.  The only
> solution is to move the kdoc to the header file, which is annoying for
> everyone.  Or replicate the declaration in the C file and kdoc it there.

Heh, for my taste, having kernel-docs at header files look better, as
this is where the kAPIs should be defined anyway. 

Ok, one may change the behavior of a function without touching the
arguments and forget to update kernel-doc at the header files to tell
about such change, but this is problematic anyway, as, if someone is
relying on a certain behavior of a kAPI function, changing its behavior
may result on unexpected results at the current callers - and even
future callers.

Thanks,
Mauro

Re: [PATCH] scripts/kernel-doc: drop "_noprof" on function prototypes

[Mauro Carvalho Chehab]

Em Thu, 05 Jun 2025 13:18:50 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Matthew Wilcox <willy@infradead.org> writes:
> 
> > On Mon, Mar 25, 2024 at 10:41:49PM -0700, Randy Dunlap wrote:  
> >> Memory profiling introduces macros as hooks for function-level
> >> allocation profiling[1]. Memory allocation functions that are profiled
> >> are named like xyz_alloc() for API access to the function. xyz_alloc()
> >> then calls xyz_alloc_noprof() to do the allocation work.
> >> 
> >> The kernel-doc comments for the memory allocation functions are
> >> introduced with the xyz_alloc() function names but the function
> >> implementations are the xyz_alloc_noprof() names.
> >> This causes kernel-doc warnings for mismatched documentation and
> >> function prototype names.
> >> By dropping the "_noprof" part of the function name, the kernel-doc
> >> function name matches the function prototype name, so the warnings
> >> are resolved.  
> >
> > This turns out not to be enough.  For example, krealloc() is
> > currently undocumented.  This is because we match the function name
> > in EXPORT_SYMBOL() against the function name in the comment, and they
> > don't match.  This patch restores the documentation, although only
> > for the python version of kernel-doc, and I'm pretty sure there's a
> > better way to do it (eg building it into the export_symbol* regexes).
> > I can turn this into a proper patch if this is the way to go, but for
> > now it's just to illustrate the problem.  
> 
> FWIW, I have no problem with leaving the perl version behind, I expect
> we'll drop it in 6.17.
> 
> We see other variants of this problem out there, where we want to
> document foo(), but that's really just a macro calling _foo(), where the
> real code is.
> 
> I wonder if we could add some sort of a marker to the kerneldoc comment
> saying "we are documenting foo(), but do you checks against _foo()"
> instead?  That would be more general than trying to keep a list of
> suffixes to hack off.
> 
> I'll try to ponder on this...
> 
> (Meanwhile I don't object to your fix as a short-term workaround)

If we willing to place hacks like that, better to bold it:

	# FIXME: this is not what we should do in long term

> > diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> > index 062453eefc7a..bdfa698d5570 100644
> > --- a/scripts/lib/kdoc/kdoc_parser.py
> > +++ b/scripts/lib/kdoc/kdoc_parser.py
> > @@ -1176,11 +1176,15 @@ class KernelDoc:
> >
> >          if export_symbol.search(line):
> >              symbol = export_symbol.group(2)
> > +            # See alloc_tags.h
> > +            symbol = symbol.removesuffix('_noprof')

If we're willing to do that, I would prefer to place "_noprof" into an array, 
as we may have other similar cases. Also, please comment why we need it and
where we have those "_noprof". We tent to forget why rules are added. As the
code churns, we may end dropping things without updating kernel-doc.

---

for a more long term solution, maybe one option for cases like that would
be to have something like:

/**
  * foo(), foo_noprof() - common function description (is it possible to have 
  *      a single description for both - as they're semantically different?)
  * @_size: size description
  * @_flags: flags description
  *
  * some description, including an explanation what are the differences
  * between both
  */
#define foo(_size, _flags)                    foo_node(_size, _flags, NUMA_NO_NODE)
#define foo_noprof(_size, _flags)             foo_node_noprof(_size, _flags, NUMA_NO_NODE)

Still, another kernel-doc markup will be needed for foo_node variants, as 
the parameters are different anyway.

Please notice that this is easier said than done as the above may break the
kernel-doc's sequential state machine at the parser if not done with care,
specially since one might eventually modify the arguments on just one of
the variants, like:

#define foo(_size, _flags, _bar)              foo_node(_size, _flags, bar, NUMA_NO_NODE)
#define foo_noprof(_size, _flags)             foo_node_noprof(_size, _flags, NUMA_NO_NODE)
	
Btw, we do have things like that: there are several register 
functions/macros that have THIS_MODULE on one of their variants,
like this:

	#define acpi_bus_register_driver(drv) \
	        __acpi_bus_register_driver(drv, THIS_MODULE)

I didn't find yet a good way to have a single kernel-doc markup
that would fill both cases and won't add too much complexity on
both kernel-doc syntax and at the kernel-doc code.

At the above, we probably don't want to document the __foo
variant, as all kAPI calls should use the variant that doesn't
have THIS_MODULE, but there are other similar cases where the
__foo variant, for instance, don't have some mutex or semaphore,
and we may still want both documented.


Thanks,
Mauro