Re: [RFC v2 18/38] docs: sphinx/kernel_abi: use AbiParser directly

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Tue, 04 Feb 2025 10:12:22 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > Hi Jon,  
> 
> > As I'll be preparing such patches for merge along this week, I'd
> > like to know what do you prefer in terms of directories:
> >
> > 1. Keep it as-is;
> > 2. have a separate library directory for Python modules
> >    (scripts/lib?);
> > 3. place python modules inside scripts/;
> > 4. place python modules inside Documentation/sphinx (IMO a bad
> >    idea);
> > 5. something else  
> 
> Honestly, I'm not sure.  I do feel that importing out of scripts/ is
> inelegant at best; having a dedicated directory for modules meant to be
> imported would be better.  So maybe scripts/lib?  Or lib/python, though
> that might raise eyebrows elsewhere, dunno.  Pick something you like,
> and we'll give that a try.

I would go for scripts/lib then.

> 
> > Btw, I'm considering to also submit later a patchset similar to
> > this one converting kernel-doc to Python. I already started writing
> > something like that (written from the scratch, following as much
> > as possible what we have today on Perl to avoid regressions).
> >
> > I would probably split the code into separate classes to make the code 
> > more readable/maintainable (a base class, a class with rest output,
> > another one with man output, and a few other helper classes).  
> 
> I definitely approve of the idea - I've pondered doing such a thing, but
> have never come close to finding the time.  It's probably worth looking
> at the rewrite Markus did years ago as a starting point?

I took a look on Markus work: it was licensed under GPL 3.0 and it was
written in 2016. There were several changes on kerneldoc since them,
including the addition of a regex that it is not compatible with
Python re[1]:

        $members =~ s/\bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*;/$2/gos;

This one use:

	- recursive patterns: ?1
	- atomic grouping (?>...)

Also, it is hard to map what he does with the existing script. I'm
opting to write a new script from scratch. I have already a version
that does ReST conversion (still missing man pages and filtering).
I'll send a RFC patch once I get it to the point it works properly.

[1] In the specific case of such regex, added on this commit:

	 50d7bd38c3aa ("stddef: Introduce struct_group() helper macro")

    I'm not sure why a very complex regex was used there. I'm currently 
    stuck on handling it properly in Python. An option would be do to:

	try:
		import regex as re
		has_regex_advanced_features = True
	except:
		import re
		has_regex_advanced_features = False

    And, if has_regex_advanced_features is false, use a simpler regex 
    that would work in Python or would just ignore struct_group
    expressions.

    Another option would be to re-implement such regexes without using
    such advanced patterns.

Thanks,
Mauro