Re: [PATCH RFC 0/2] Add Kconfig pages and cross-references to Documentation

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Fri, 4 Apr 2025 12:24:27 -0400
Nícolas F. R. A. Prado <nfraprado@collabora.com> escreveu:

> On Fri, Apr 04, 2025 at 08:31:35AM -0600, Jonathan Corbet wrote:
> > Nícolas F. R. A. Prado <nfraprado@collabora.com> writes:
> >   
> > > This series adds Kconfig pages (patch 1) to the Documentation, and
> > > automarkups CONFIG_* text as cross-references to those pages (patch 2).
> > >
> > > There is a huge change in build time with this series, so we'd either
> > > have to so some optimization and/or put this behind a flag in make so it
> > > is only generated when desired (for instance for the online
> > > documentation):
> > >
> > >   (On an XPS 13 9300)
> > >   
> > >   Before:
> > >   
> > >   real	6m43.576s
> > >   user	23m32.611s
> > >   sys	1m48.220s
> > >   
> > >   After:
> > >   
> > >   real	11m56.845s
> > >   user	47m40.528s
> > >   sys	2m27.382s
> > >
> > > There are also some issues that were solved in ad-hoc ways (eg the
> > > sphinx warnings due to repeated Kconfigs, by embedding the list of
> > > repeated configs in the script). Hence the RFC.  
> > 
> > I'm still digging out from LSFMM, so have only glanced at this ... I can
> > see the appeal of doing this, but nearly doubling the docs build time
> > really isn't going to fly.  Have you looked to see what is taking all of
> > that time?  The idea that it takes as long to process KConfig entries as
> > it does to build the entire rest of the docs seems ... a bit wrong.  
> 
> I have not yet. Thought I'd get some feedback before looking into the
> performance. But I agree with the sentiment.

My feeling is that the issue is using :glob" and a lot of wildcards
inside Sphinx. Instead, you should use something similar to what
I've done to get *.[ch] for the new kernel-doc.py implementation.

Placing it as an extension on a similar way to what i did with
get_abi.py would likely help as well.

> > I wonder what it would take to create a Sphinx extension that would
> > simply walk the source tree and slurp up the KConfig entries directly?
> > That would be nicer than adding a separate script in any case.  
> 
> That is what is currently done for the ABI, AFAIK, so definitely seems doable.

Yes, doing that via an extension is doable. If done right, it can also be
fast.

> The key difference between the ABI approach and this here, is that my goal was
> to reflect the Kconfig file hierarchy in the Documentation. So each Kconfig
> file gets its own documentation page, while the ABI approach collects the
> contents of all ABI files into just a few documentation pages (stable, testing,
> etc). (So there's a non-constant number of .rst files, which means they have to
> be generated and can't be a sphinx plugin in this approach).

Actually, get-api.py (the new version, merged for 6.15) generates a dict
just once. Then, Sphinx rst files filters part of the doc, but I see your
point: for every entry, we would need a .rst file if we follow the same
approach.

That's said, it may have a way to tell Sphinx to threat Kconfig files
on a similar way it handles ".txt" and ".rst" files. Something like the
extension to handle markdown works:

	https://www.sphinx-doc.org/en/master/usage/markdown.html

Another alternative would be to use:

	https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-include_patterns

but this would require Sphinx 5.1, which is above our current minimal
version. That's said, nothing prevents to only enable generating such
documentatation if the Sphinx version supports it.


> I went for this approach because the filesystem hierarchy seemed the most
> logical way to group the Kconfig symbols. Also Kconfig files have directives like
> 'menu' that should be present in the documentation in the same order they appear
> in the file to fully describe dependencies of the symbols, and having all of
> that in the same page seems like it would be confusing. But given the potential
> benefits it's worth a try for sure.
> 
> Now that I think about it, seems quite likely that a lot of the time spent comes
> from creating a subshell and running the script for every Kconfig file. So
> making a single script or sphinx extension that itself handles iterating over
> all the files would likely greatly reduce the run time. I'll test that.
> 
> Thanks,
> Nícolas
> 
> > 
> > I'll try to look closer, but I'll remain a bit distracted for a little
> > while yet.
> > 
> > Thanks,
> > 
> > jon