[PATCH RFCv2 0/5]Implement kernel-doc in Python

[PATCH RFCv2 0/5]Implement kernel-doc in Python

[Mauro Carvalho Chehab]

Hi Jon,

That's the second version of the Python kernel-doc tool.

As the previous version, I tried to stay as close as possible of the original
Perl implementation, as it helps to double check if each function was 
properly translated to Python.  This have been helpful debugging troubles
that happened during the conversion.

On this version, the ReST output for the files included under Documentation
is identical to the perl version, except for:

 - blank lines;
 - white spaces;
 - this version suppresses return (and description) sections on a few places
   where this is left in blank.

This version also replaces the old script by the new one for doc generation,
to make easier to test its results.

It should be noticed that  there are some sutile nuances on the way Perl 
and Python handle  regular expressions. In particular, Perl compiled 
expressions with qr{}  behaves as non-capturing groups, sometimes optional
ones. I documented it on one place, but I had to use non-capturing groups
on other parts of the logic to achieve the desired results.

This version is on a minimal integration scenario: it just replaces the
exec file to use the new one and gets rid of an obsolete parameter
as on this version I didn't implement support for Sphinx < 3.1.

On other words, it assumes that the patch requiring Sphinx >= 3.4
will be merged.

I also didn't remove on this series the old kernel-doc script, as this makes
easier to test both variations, e. g. using:

	make cleandocs && make KERNELDOC=scripts/kernel-doc htmldocs

will make it use the original Perl version.

Comments?

Mauro Carvalho Chehab (5):
  include/asm-generic/io.h: fix kerneldoc markup
  scripts/kernel-doc: remove an obscure logic from kernel-doc
  scripts/kernel-doc: don't add not needed new lines
  scripts/kernel-doc.py: add a Python parser
  docs: use kernel-doc.py script for kerneldoc output

 Documentation/Makefile            |    2 +-
 Documentation/conf.py             |    2 +-
 Documentation/sphinx/kerneldoc.py |    5 -
 include/asm-generic/io.h          |    6 +-
 scripts/kernel-doc                |   23 +-
 scripts/kernel-doc.py             | 2752 +++++++++++++++++++++++++++++
 6 files changed, 2762 insertions(+), 28 deletions(-)
 create mode 100755 scripts/kernel-doc.py

-- 
2.48.1

[PATCH RFCv2 1/5] include/asm-generic/io.h: fix kerneldoc markup

[Mauro Carvalho Chehab]

Kerneldoc requires a "-" after the name of a function for it
to be recognized as a function.

Add it.

Fix those kernel-doc warnings:

include/asm-generic/io.h:1215: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
 * memset_io    Set a range of I/O memory to a constant value
include/asm-generic/io.h:1227: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
 * memcpy_fromio        Copy a block of data from I/O memory
include/asm-generic/io.h:1239: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
 * memcpy_toio          Copy a block of data into I/O memory

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 include/asm-generic/io.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
index a5cbbf3e26ec..3c61c29ff6ab 100644
--- a/include/asm-generic/io.h
+++ b/include/asm-generic/io.h
@@ -1212,7 +1212,7 @@ static inline void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 
 #ifndef memset_io
 /**
- * memset_io	Set a range of I/O memory to a constant value
+ * memset_io -	Set a range of I/O memory to a constant value
  * @addr:	The beginning of the I/O-memory range to set
  * @val:	The value to set the memory to
  * @count:	The number of bytes to set
@@ -1224,7 +1224,7 @@ void memset_io(volatile void __iomem *addr, int val, size_t count);
 
 #ifndef memcpy_fromio
 /**
- * memcpy_fromio	Copy a block of data from I/O memory
+ * memcpy_fromio -	Copy a block of data from I/O memory
  * @dst:		The (RAM) destination for the copy
  * @src:		The (I/O memory) source for the data
  * @count:		The number of bytes to copy
@@ -1236,7 +1236,7 @@ void memcpy_fromio(void *dst, const volatile void __iomem *src, size_t count);
 
 #ifndef memcpy_toio
 /**
- * memcpy_toio		Copy a block of data into I/O memory
+ * memcpy_toio -	Copy a block of data into I/O memory
  * @dst:		The (I/O memory) destination for the copy
  * @src:		The (RAM) source for the data
  * @count:		The number of bytes to copy
-- 
2.48.1

Re: [PATCH RFCv2 1/5] include/asm-generic/io.h: fix kerneldoc markup

[Arnd Bergmann]

On Thu, Feb 13, 2025, at 13:06, Mauro Carvalho Chehab wrote:
> Kerneldoc requires a "-" after the name of a function for it
> to be recognized as a function.
>
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Acked-by: Arnd Bergmann <arnd@arndb.de>

I assume this will be merged through the documentation tree,
let me know if you prefer me to add it to the asm-generic
tree instead.

Re: [PATCH RFCv2 0/5]Implement kernel-doc in Python

[Randy Dunlap]

Hi Mauro,


On 2/13/25 4:06 AM, Mauro Carvalho Chehab wrote:
> Hi Jon,
> 
> That's the second version of the Python kernel-doc tool.
> 
> As the previous version, I tried to stay as close as possible of the original
> Perl implementation, as it helps to double check if each function was 
> properly translated to Python.  This have been helpful debugging troubles
> that happened during the conversion.

Since this new version is supposed to be bug-for-bug compatible, I will wait
until later to test the current known bugs that I know about in (Perl) kernel-doc.

For a preview of most of them, you can read:
https://lore.kernel.org/linux-doc/3a6a7dd0-72f1-44c6-b0bc-b1ce76fca76a@infradead.org/

and its follow-up email (today).

There are quite a few problems with parsing function parameters that use
typedefs.


-- 
~Randy

Re: [PATCH RFCv2 0/5]Implement kernel-doc in Python

[Mauro Carvalho Chehab]

Em Thu, 13 Feb 2025 19:15:28 -0800
Randy Dunlap <rdunlap@infradead.org> escreveu:

> Hi Mauro,
> 
> 
> On 2/13/25 4:06 AM, Mauro Carvalho Chehab wrote:
> > Hi Jon,
> > 
> > That's the second version of the Python kernel-doc tool.
> > 
> > As the previous version, I tried to stay as close as possible of the original
> > Perl implementation, as it helps to double check if each function was 
> > properly translated to Python.  This have been helpful debugging troubles
> > that happened during the conversion.  
> 
> Since this new version is supposed to be bug-for-bug compatible,

Yes, that's the goal: I'm checking all discrepancies by hand to ensure that
the output net result will be identical at the final version - maybe
except for blank lines/whitespace (and eventually empty Return sections
that the current script produces). Getting blank lines and whitespaces identical
have been hard.

So, yeah, if something is not handled well by the Perl version, the
Python version shall produce an identical result. I'm refraining to
try fixing any already existing issues there.

> I will wait
> until later to test the current known bugs that I know about in (Perl) kernel-doc.
> 
> For a preview of most of them, you can read:
> https://lore.kernel.org/linux-doc/3a6a7dd0-72f1-44c6-b0bc-b1ce76fca76a@infradead.org/
> 
> and its follow-up email (today).
> 
> There are quite a few problems with parsing function parameters that use
> typedefs.

Thanks for that!

Let's address when we finish the transition.

Thanks,
Mauro