Re: [PATCH v6 00/15] Don't generate netlink .rst files inside $(srctree)

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Thu, 19 Jun 2025 00:46:15 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Mauro,
> 
> On 2025/06/18 20:46, Mauro Carvalho Chehab wrote:
> > As discussed at:
> >    https://lore.kernel.org/all/20250610101331.62ba466f@foz.lan/
> > 
> > changeset f061c9f7d058 ("Documentation: Document each netlink family")
> > added a logic which generates *.rst files inside $(srctree). This is bad
> > when O=<BUILDDIR> is used.
> > 
> > A recent change renamed the yaml files used by Netlink, revealing a bad
> > side effect: as "make cleandocs" don't clean the produced files and symbols
> > appear duplicated for people that don't build the kernel from scratch.
> > 
> > This series adds an yaml parser extension and uses an index file with glob for
> > *. We opted to write such extension in a way that no actual yaml conversion
> > code is inside it. This makes it flexible enough to handle other types of yaml
> > files in the future. The actual yaml conversion logic were placed at 
> > netlink_yml_parser.py. 
> > 
> > As requested by YNL maintainers, this version has netlink_yml_parser.py
> > inside tools/net/ynl/pyynl/ directory. I don't like mixing libraries with
> > binaries, nor to have Python libraries spread all over the Kernel. IMO,
> > the best is to put all of them on a common place (scripts/lib, python/lib,
> > lib/python, ...) but, as this can be solved later, for now let's keep it this
> > way.
> > 
> > ---
> > 
> > v6:
> > - YNL doc parser is now at tools/net/ynl/pyynl/lib/doc_generator.py;
> > - two patches got merged;
> > - added instructions to test docs with Sphinx 3.4.3 (minimal supported
> >   version);
> > - minor fixes.  
> 
> Quick tests against Sphinx 3.4.3 using container images based on
> debian:bullseye and almalinux:9, both of which have 3.4.3 as their distro
> packages, emits a *bunch* of warnings like the following:
> 
> /<srcdir>/Documentation/netlink/specs/conntrack.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
> /<srcdir>/Documentation/netlink/specs/devlink.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
> /<srcdir>/Documentation/netlink/specs/dpll.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
> /<srcdir>/Documentation/netlink/specs/ethtool.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
> /<srcdir>/Documentation/netlink/specs/fou.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
> [...]
> 
> I suspect there should be a minimal required minimal version of PyYAML.

Likely yes. From my side, I didn't change anything related to PyYAML, 
except by adding a loader at the latest patch to add line numbers.

The above warnings don't seem related. So, probably this was already
an issue.

Funny enough, I did, on my venv:

	$ pip install PyYAML==5.1
	$ tools/net/ynl/pyynl/ynl_gen_rst.py -i Documentation/netlink/specs/dpll.yaml -o Documentation/output/netlink/specs/dpll.rst -v
	...
	$ make clean; make SPHINXDIRS="netlink/specs" htmldocs
	...

but didn't get any issue (I have a later version installed outside
venv - not sure it it will do the right thing).

That's what I have at venv:

----------------------------- ---------
Package                       Version
----------------------------- ---------
alabaster                     0.7.13
babel                         2.17.0
certifi                       2025.6.15
charset-normalizer            3.4.2
docutils                      0.17.1
idna                          3.10
imagesize                     1.4.1
Jinja2                        2.8.1
MarkupSafe                    1.1.1
packaging                     25.0
pip                           25.1.1
Pygments                      2.19.1
PyYAML                        5.1
requests                      2.32.4
setuptools                    80.1.0
snowballstemmer               3.0.1
Sphinx                        3.4.3
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
urllib3                       2.4.0
----------------------------- ---------

> "pip freeze" based on almalinux:9 says:
> 
>     PyYAML==5.4.1
> 
> "pip freeze" based on debian:bullseye says:
> 
>     PyYAML==5.3.1
> 
> What is the minimal required version here?

Breno, what's the minimal version? Please update requirements.txt
to ensure that, and modify ./scripts/sphinx-pre-install to
check for it.

> 
> And if users of those old distros need to manually upgrade PyYAML,
> why don't you suggest them to upgrade Sphinx as well?

The criteria we used to define minimal version for python/sphinx
was having them released at the end of 2020/beginning 2021. So,
up to ~4 years old. We also double-checked latest LTS versions
from major distros.

With that, PyYAML 5.4.1 met the ~4 years old, and so 5.3.1 and
5.1.

funny enough:

	$ git grep tab_width Documentation/netlink/

doesn't return anything. Yet, tab_width is used by sphinx
extensions. The in-kernel ones do it the right way using
get:

	tab_width = self.options.get('tab-width',
				     self.state.document.settings.tab_width)

But perhaps some other extension you might have installed on your
environment has issues, or maybe Documentation/sphinx/parser_yaml.py
need to expand tabs with certain versions of docutils.

Please compare the versions that you're using on your test
environment with the ones I used here.

Regards,
Mauro