Re: [PATCH 1/1] configure: prefer python's sphinx module

[John Snow <jsnow@redhat.com>]

On 6/18/20 5:56 AM, Peter Maydell wrote:
> On Tue, 16 Jun 2020 at 20:09, John Snow <jsnow@redhat.com> wrote:
>> Using an explicit entry path script for sphinx can lead to confusing
>> results: If the python binary belongs to a virtual environment, our
>> configure script may still select a sphinx script that belongs to the
>> system distribution packages.
>>
>> It is likely best to use python itself (whichever one the user provides)
>> to resolve the sphinx script.
> 
> I'm not convinced. How do I find out which sphinx-build this
> is actually going to use ? ("python3 -m sphinx" doesn't list a
> path to anything.)
> 

> python3 -c 'import sphinx; print(sphinx.__file__)'
/usr/lib/python3.7/site-packages/sphinx/__init__.py

> How do I use the system python but a venv sphinx-build? At the

> python3 -m venv myvenv
> cd myvenv/bin
> ls -l python*

lrwxrwxrwx. 1 jsnow jsnow  7 Jun 19 13:23 python -> python3*
lrwxrwxrwx. 1 jsnow jsnow 16 Jun 19 13:23 python3 -> /usr/bin/python3*

The venv uses symlinks, so it will continue to use your system version,
but you can install sphinx here.

I'm proposing you do either one of:

A) ./configure --python=/home/petmay01/python-env/bin/python3

B) source ~/python-env/bin/activate
   ./configure

> moment I can easily do that with
>   --sphinx-build=/home/petmay01/python-env/bin/sphinx-build
> because scripts inside a venv have #! lines that make them
> work without having to manually activate the venv. I don't
> want to have to use some random non-system Python just
> because I have a newer Sphinx.
> 

I was under the impression that it would be best if sphinx was executed
using the user's specified python binary instead of allowing
scripts/qapi to run under the user's python but sphinx to run under a
different python.

One of the reasons I came to this belief was to ensure that when
operating inside of a venv that QEMU was always using that venv's python
and sphinx instead of "leaking" out to the system's installation. It
felt more explicit.

A problem with looking for 'sphinx-build-3' and 'sphinx-build' entry
scripts is that the /usr/bin/xxx namespace is shared between python2 and
python3 packages and we may wind up selecting a sphinx for the wrong
python version entirely -- and from what I could tell, there wasn't a
way to interrogate sphinx to get it to tell us what python it was using,
or any other way to force this kind of scripted entrypoint to use *my*
python.

Fedora gets into trouble here because we want 'sphinx-build-3', but this
ignores our venv version because the script entrypoint in a venv is
'sphinx-build' -- which might be the system's python2 version.

Using `python -m sphinx` seemed nice because:

1. Works the same inside and outside of venv
2. Don't need to interrogate python version, we already know it
3. We are confident it uses the venv.

Or, put another way, I think it's quite odd to:

1. Activate a venv
2. Specify an explicit python version to ./configure
3. Have sphinx use a python/sphinx that is not necessarily correlated
with either #1 or #2.

> Put another way, I don't think the fact that sphinx-build
> happens to be implemented in Python means that we should
> let the user's decision about which Python they want us to
> use control which version of sphinx-build we should use.
> 

I see your point: Why not treat sphinx-build as a black box executable
instead of treating it like a python plugin?

Mostly,

(1) The aforementioned problems locating and interrogating the correct
script entrypoint to use based on the user's environment and configure
options

(2) By treating it as a Python dependency instead, I can ensure that we
have a correctly modern sphinx in a venv as a repeatable build step for
platforms that do not ship a modern-enough sphinx.

--js