[PATCH] changes.rst: explain the usage of virtual environment

[PATCH] changes.rst: explain the usage of virtual environment

[Mauro Carvalho Chehab]

As the Sphinx build seems very fragile, specially for
PDF output, add a notice about how to use it on a virtual
environment.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---

 Documentation/process/changes.rst | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 3aed751e0cb5..8a51f37bce4a 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -322,6 +322,24 @@ PDF outputs, it is recommended to use version 1.4.6.
   functionalities required for ``XeLaTex`` to work. For PDF output you'll also
   need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
 
+Most distributions are shipped with Sphinx, but sometimes you may need
+to use a version different than the one shipped. For those cases, you
+can use Sphinx on a virtual environment. For example, if you want to
+install the Python 3 Sphinx version 1.4.9, do::
+
+       $ virtualenv-3 sphinx_1.4
+       $ . sphinx_1.4/bin/activate
+       $ pip install 'Sphinx==1.4.9'
+
+.. note::
+
+   #) It is recommended to use the RTD theme for html output. Depending
+      on the version, it should be installed  in separate,
+      with ``pip install sphinx_rtd_theme``.
+
+   #) The command ``. sphinx_1.4/bin/activate`` sets the environment to
+      use the virtual environment you just install. It should be called
+      when you start a new shell, before running ``make htmldocs``.
 
 Getting updated software
 ========================
-- 
2.9.4

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Christoph Hellwig]

On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> As the Sphinx build seems very fragile, specially for
> PDF output, add a notice about how to use it on a virtual
> environment.

Please don't.  python venv are good at one thing only, and that is
making a mess of your python module path.  Don't ever use them on a
system that has proper package management.

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Jonathan Corbet]

On Mon, 19 Jun 2017 06:08:37 -0700
Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> > As the Sphinx build seems very fragile, specially for
> > PDF output, add a notice about how to use it on a virtual
> > environment.  
> 
> Please don't.  python venv are good at one thing only, and that is
> making a mess of your python module path.  Don't ever use them on a
> system that has proper package management.

Well, they are also good for running specific versions of a given package
without disrupting the setup of your system as a whole, and when even a
system with proper package management may not offer the version you need.

I guess my question with the patch is whether this text really belongs in
changes.rst, or whether it should be part of the doc-guide.

jon

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Markus Heiser]

> Am 19.06.2017 um 15:46 schrieb Jonathan Corbet <corbet@lwn.net>:
> 
> On Mon, 19 Jun 2017 06:08:37 -0700
> Christoph Hellwig <hch@infradead.org> wrote:
> 
>> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
>>> As the Sphinx build seems very fragile, specially for
>>> PDF output, add a notice about how to use it on a virtual
>>> environment.  
>> 
>> Please don't.  python venv are good at one thing only, and that is
>> making a mess of your python module path.  Don't ever use them on a
>> system that has proper package management.
> 
> Well, they are also good for running specific versions of a given package
> without disrupting the setup of your system as a whole, and when even a
> system with proper package management may not offer the version you need.
> 
> I guess my question with the patch is whether this text really belongs in
> changes.rst, or whether it should be part of the doc-guide.

Hi, here are my 5cents:

Typically I have a PY_ENV target in my projects, building a virtualenv
in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:


PY ?=3
PYTHON ?= python$(PY)
..
VIRTUALENV = virtualenv --python=$(PYTHON)
VTENV_OPTS = "--no-site-packages"
PY_ENV = ./local/py$(PY)
..
quiet_cmd_virtualenv = PYENV $@
cmd_virtualenv = \
	if [ ! -d "./$(PY_ENV)" ];then \
	$(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
	else \
	echo "using virtualenv from $2"; \
	fi
...
# to build *local* environment, python and virtualenv from the OS is needed!
$(PY_ENV): virtualenv-exe python-exe
	$(call cmd,virtualenv,$(PY_ENV))
	@$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt
..

And the sphinxbuild coammand is used from there::

SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build

By this I can stick versions packages. E.g. to select last version of
RTD theme and Sphinx version 1.5 or upper, I add the following lines to
my reqierements.txt::

Sphinx>=1.5
sphinx_rtd_theme

If you are interested, I can prepare a patch, to add such functionality 
(as option) to Documents/Makefile (which will be documented in the doc-guide).

[1] https://github.com/return42/linuxdoc/blob/master/utils/makefile.python

-- Markus --






> 
> jon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Mauro Carvalho Chehab]

HI Markus,

Em Mon, 19 Jun 2017 16:11:56 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 19.06.2017 um 15:46 schrieb Jonathan Corbet <corbet@lwn.net>:
> > 
> > On Mon, 19 Jun 2017 06:08:37 -0700
> > Christoph Hellwig <hch@infradead.org> wrote:
> >   
> >> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:  
> >>> As the Sphinx build seems very fragile, specially for
> >>> PDF output, add a notice about how to use it on a virtual
> >>> environment.    
> >> 
> >> Please don't.  python venv are good at one thing only, and that is
> >> making a mess of your python module path.  Don't ever use them on a
> >> system that has proper package management.  
> > 
> > Well, they are also good for running specific versions of a given package
> > without disrupting the setup of your system as a whole, and when even a
> > system with proper package management may not offer the version you need.
> > 
> > I guess my question with the patch is whether this text really belongs in
> > changes.rst, or whether it should be part of the doc-guide.  
> 
> Hi, here are my 5cents:
> 
> Typically I have a PY_ENV target in my projects, building a virtualenv
> in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:
> 
> 
> PY ?=3
> PYTHON ?= python$(PY)
> ..
> VIRTUALENV = virtualenv --python=$(PYTHON)
> VTENV_OPTS = "--no-site-packages"
> PY_ENV = ./local/py$(PY)

I would split the PATH name on a separate var. This way, if one would
like to have multiple Sphinx versions, all it would need would be to
change the directory.
I would prefer to call it as "./sphinx" (or something similar).

So, I would do:

  PY_DIR = ./sphinx
  PY_ENV = $(PY_DIR)/py$(PY)

Don't forget to add $PY_DIR directory to .gitignore on your patch.

> ..
> quiet_cmd_virtualenv = PYENV $@
> cmd_virtualenv = \
> 	if [ ! -d "./$(PY_ENV)" ];then \
> 	$(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
> 	else \
> 	echo "using virtualenv from $2"; \
> 	fi
> ...
> # to build *local* environment, python and virtualenv from the OS is needed!
> $(PY_ENV): virtualenv-exe python-exe
> 	$(call cmd,virtualenv,$(PY_ENV))
> 	@$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt

Shouldn't it be using "pip$(PY)" instead?

Also, better to seek for requirements on a file under Documentation/sphinx/
directory.

> ..
> 
> And the sphinxbuild coammand is used from there::
> 
> SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build
> 
> By this I can stick versions packages. E.g. to select last version of
> RTD theme and Sphinx version 1.5 or upper, I add the following lines to
> my reqierements.txt::
> 
> Sphinx>=1.5  
> sphinx_rtd_theme
> 
> If you are interested, I can prepare a patch, to add such functionality 
> (as option) to Documents/Makefile (which will be documented in the doc-guide).

Yeah, IMHO, it makes sense to have something like that at the main build,
as an optional feature, e. g. perhaps adding a new make target, like:

	$ make sphinx_virtenv

That would create and populate PY_DIR. If $PY_DIR/bin/sphinx-build exists
and it is executable file, run it. Otherwise, use the system's sphinx, 
if available.

> 
> [1] https://github.com/return42/linuxdoc/blob/master/utils/makefile.python
> 
> -- Markus --
> 
> 
> 
> 
> 
> 
> > 
> > jon
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html  
> 



Thanks,
Mauro

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Markus Heiser]

> Am 19.06.2017 um 16:38 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> HI Markus,
> 

Hi Mauro :)

[...]

>> Typically I have a PY_ENV target in my projects, building a virtualenv
>> in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:
>> 
>> PY ?=3
>> PYTHON ?= python$(PY)
>> ..
>> VIRTUALENV = virtualenv --python=$(PYTHON)
>> VTENV_OPTS = "--no-site-packages"
>> PY_ENV = ./local/py$(PY)
> 
> I would split the PATH name on a separate var. This way, if one would
> like to have multiple Sphinx versions, all it would need would be to
> change the directory.
> I would prefer to call it as "./sphinx" (or something similar).

I will take this in mind.

> So, I would do:
> 
>  PY_DIR = ./sphinx
>  PY_ENV = $(PY_DIR)/py$(PY)
> 
> Don't forget to add $PY_DIR directory to .gitignore on your patch.

good point ;)

>> ..
>> quiet_cmd_virtualenv = PYENV $@
>> cmd_virtualenv = \
>> 	if [ ! -d "./$(PY_ENV)" ];then \
>> 	$(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
>> 	else \
>> 	echo "using virtualenv from $2"; \
>> 	fi
>> ...
>> # to build *local* environment, python and virtualenv from the OS is needed!
>> $(PY_ENV): virtualenv-exe python-exe
>> 	$(call cmd,virtualenv,$(PY_ENV))
>> 	@$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt
> 
> Shouldn't it be using "pip$(PY)" instead?

No, with @$(PY_ENV_BIN)/pip you always use the pip from the environment
and this is always named pip. E.g::

 ./local/py3/bin/pip
 ./local/py2/bin/pip

Or: same as "sphinx-build" command .. there is no sohinx-build3 ;)

> Also, better to seek for requirements on a file under Documentation/sphinx/
> directory.

right, the above was only a sloppily C&P from what I have in my project.

>> ..
>> 
>> And the sphinxbuild coammand is used from there::
>> 
>> SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build
>> 
>> By this I can stick versions packages. E.g. to select last version of
>> RTD theme and Sphinx version 1.5 or upper, I add the following lines to
>> my reqierements.txt::
>> 
>> Sphinx>=1.5  
>> sphinx_rtd_theme
>> 
>> If you are interested, I can prepare a patch, to add such functionality 
>> (as option) to Documents/Makefile (which will be documented in the doc-guide).
> 
> Yeah, IMHO, it makes sense to have something like that at the main build,
> as an optional feature, e. g. perhaps adding a new make target, like:
> 
> 	$ make sphinx_virtenv
> 
> That would create and populate PY_DIR. If $PY_DIR/bin/sphinx-build exists
> and it is executable file, run it. Otherwise, use the system's sphinx, 
> if available.

OK, I will prepare a RFC patch ... I will have time for this only on the
weekend, so have some patience with me.

-- Markus --

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Markus Heiser]

[...]

> Am 19.06.2017 um 17:13 schrieb Markus Heiser <markus.heiser@darmarIT.de>:
> 
>>> Typically I have a PY_ENV target in my projects, building a virtualenv
>>> in a folder named ./local.

[...]

>> Yeah, IMHO, it makes sense to have something like that at the main build,
>> as an optional feature,

> OK, I will prepare a RFC patch ...

For those who are interested in / lets discuss further in context
of my patch:

  https://www.mail-archive.com/linux-doc@vger.kernel.org/msg12844.html

Thanks!

-- Markus --

Re: [PATCH] changes.rst: explain the usage of virtual environment

[Mauro Carvalho Chehab]

Em Mon, 19 Jun 2017 07:46:53 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Mon, 19 Jun 2017 06:08:37 -0700
> Christoph Hellwig <hch@infradead.org> wrote:
> 
> > On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:  
> > > As the Sphinx build seems very fragile, specially for
> > > PDF output, add a notice about how to use it on a virtual
> > > environment.    
> > 
> > Please don't.  python venv are good at one thing only, and that is
> > making a mess of your python module path.  Don't ever use them on a
> > system that has proper package management.  
> 
> Well, they are also good for running specific versions of a given package
> without disrupting the setup of your system as a whole, and when even a
> system with proper package management may not offer the version you need.

Yes. Based on my experience with Sphinx, upgrading it to a new
version is painful, as it can cause regressions on docs build.

Also, when a new version is released, the previously stable version
becomes abandoned. Not even trivial and/or compatibility fixes are
accepted against old versions.

So, at least for me, it works best if an specific version is installed 
inside a virtual environment, as it won't cause problems when I update
other packages on my distro.

The one I'm using here is version 1.4.9 (the latest 1.4.x version).

Right now, vesion 1.5.x require a series of patches from Jon to
fix PDF build, and more stuff will be needed to fix PDF builds with
1.6.x.

> I guess my question with the patch is whether this text really belongs in
> changes.rst, or whether it should be part of the doc-guide.

Jon,

If you prefer, I can move such description to doc-guide, but I
suspect that we may need to specify, at changes.rst, what Sphinx
versions are known to work fine.

Thanks,
Mauro