commands to do "pip install" in docs README file need enhancing

commands to do "pip install" in docs README file need enhancing

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]


  I just went through this with another Sphinx-based documentation
repo -- installation instructions that advise the reader to install
Python modules with "pip install" fail on my Debian 13 system with:


$ pip install sphinx-lint
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.13/README.venv for more information.

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at
the risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.


  Yes, there are solutions, such as using "pipx" instead of "pip", and
what have you, but it seems that the README file should be augmented
with an explanation as to how to deal with the above given that the
"pip install" commands are, in some cases, guaranteed to fail.

rday

Re: commands to do "pip install" in docs README file need enhancing

[tgamblin]

On Thu Aug 20, 2026 at 7:20 AM EDT, Robert P. J. Day wrote:
>
>   I just went through this with another Sphinx-based documentation
> repo -- installation instructions that advise the reader to install
> Python modules with "pip install" fail on my Debian 13 system with:
>
>
> $ pip install sphinx-lint
> error: externally-managed-environment
>
> × This environment is externally managed
> ╰─> To install Python packages system-wide, try apt install
>     python3-xyz, where xyz is the package you are trying to
>     install.
>
>     If you wish to install a non-Debian-packaged Python package,
>     create a virtual environment using python3 -m venv path/to/venv.
>     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
>     sure you have python3-full installed.
>
>     If you wish to install a non-Debian packaged Python application,
>     it may be easiest to use pipx install xyz, which will manage a
>     virtual environment for you. Make sure you have pipx installed.
>
>     See /usr/share/doc/python3.13/README.venv for more information.
>
> note: If you believe this is a mistake, please contact your Python
> installation or OS distribution provider. You can override this, at
> the risk of breaking your Python installation or OS, by passing
> --break-system-packages.
> hint: See PEP 668 for the detailed specification.
>
>
>   Yes, there are solutions, such as using "pipx" instead of "pip", and
> what have you, but it seems that the README file should be augmented
> with an explanation as to how to deal with the above given that the
> "pip install" commands are, in some cases, guaranteed to fail.

Creating a venv like the error suggests (or via uv) is simple enough and IMO
best practice for anything like this. You could submit a change to provide short
examples, even replacing the references to pipenv earlier in the file.
>
> rday

Re: [docs] commands to do "pip install" in docs README file need enhancing

[Quentin Schulz]

On 8/20/26 3:23 PM, Trevor Gamblin via lists.yoctoproject.org wrote:
> On Thu Aug 20, 2026 at 7:20 AM EDT, Robert P. J. Day wrote:
>>
>>    I just went through this with another Sphinx-based documentation
>> repo -- installation instructions that advise the reader to install
>> Python modules with "pip install" fail on my Debian 13 system with:
>>
>>
>> $ pip install sphinx-lint
>> error: externally-managed-environment
>>
>> × This environment is externally managed
>> ╰─> To install Python packages system-wide, try apt install
>>      python3-xyz, where xyz is the package you are trying to
>>      install.
>>
>>      If you wish to install a non-Debian-packaged Python package,
>>      create a virtual environment using python3 -m venv path/to/venv.
>>      Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
>>      sure you have python3-full installed.
>>
>>      If you wish to install a non-Debian packaged Python application,
>>      it may be easiest to use pipx install xyz, which will manage a
>>      virtual environment for you. Make sure you have pipx installed.
>>
>>      See /usr/share/doc/python3.13/README.venv for more information.
>>
>> note: If you believe this is a mistake, please contact your Python
>> installation or OS distribution provider. You can override this, at
>> the risk of breaking your Python installation or OS, by passing
>> --break-system-packages.
>> hint: See PEP 668 for the detailed specification.
>>
>>
>>    Yes, there are solutions, such as using "pipx" instead of "pip", and
>> what have you, but it seems that the README file should be augmented
>> with an explanation as to how to deal with the above given that the
>> "pip install" commands are, in some cases, guaranteed to fail.
> 
> Creating a venv like the error suggests (or via uv) is simple enough and IMO
> best practice for anything like this. You could submit a change to provide short
> examples, even replacing the references to pipenv earlier in the file.
>>

It's not that simple (though not that difficult). A venv by default is 
completely isolated and doesn't use anything from the host. The issue is 
that by doing so, you won't be able to build the docs, only run 
sphinx-lint, which isn't ideal. You should probably reuse the same venv 
as documented in documentation/tools/host_packages_scripts/pip3_docs.sh 
instead.

You probably want to update the instructions for Vale as well. We still 
support pipenv though, via documentation/Pipfile as far as I remember, 
so that's another option. Maybe add something in dev-packages for 
example. No clue, I don't use pipenv.

Cheers,
Quentin

Re: [docs] commands to do "pip install" in docs README file need enhancing

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 3563 bytes --]

On Thu, 20 Aug 2026, Quentin Schulz wrote:

> On 8/20/26 3:23 PM, Trevor Gamblin via lists.yoctoproject.org wrote:
> > On Thu Aug 20, 2026 at 7:20 AM EDT, Robert P. J. Day wrote:
> > >
> > >    I just went through this with another Sphinx-based documentation
> > > repo -- installation instructions that advise the reader to install
> > > Python modules with "pip install" fail on my Debian 13 system with:
> > >
> > >
> > > $ pip install sphinx-lint
> > > error: externally-managed-environment
> > >
> > > × This environment is externally managed
> > > ╰─> To install Python packages system-wide, try apt install
> > >      python3-xyz, where xyz is the package you are trying to
> > >      install.
> > >
> > >      If you wish to install a non-Debian-packaged Python package,
> > >      create a virtual environment using python3 -m venv path/to/venv.
> > >      Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
> > >      sure you have python3-full installed.
> > >
> > >      If you wish to install a non-Debian packaged Python application,
> > >      it may be easiest to use pipx install xyz, which will manage a
> > >      virtual environment for you. Make sure you have pipx installed.
> > >
> > >      See /usr/share/doc/python3.13/README.venv for more information.
> > >
> > > note: If you believe this is a mistake, please contact your Python
> > > installation or OS distribution provider. You can override this, at
> > > the risk of breaking your Python installation or OS, by passing
> > > --break-system-packages.
> > > hint: See PEP 668 for the detailed specification.
> > >
> > >
> > >    Yes, there are solutions, such as using "pipx" instead of "pip", and
> > > what have you, but it seems that the README file should be augmented
> > > with an explanation as to how to deal with the above given that the
> > > "pip install" commands are, in some cases, guaranteed to fail.
> >
> > Creating a venv like the error suggests (or via uv) is simple enough and IMO
> > best practice for anything like this. You could submit a change to provide
> > short
> > examples, even replacing the references to pipenv earlier in the file.
>
> It's not that simple (though not that difficult). A venv by default
> is completely isolated and doesn't use anything from the host. The
> issue is that by doing so, you won't be able to build the docs, only
> run sphinx-lint, which isn't ideal. You should probably reuse the
> same venv as documented in
> documentation/tools/host_packages_scripts/pip3_docs.sh instead.
>
> You probably want to update the instructions for Vale as well. We
> still support pipenv though, via documentation/Pipfile as far as I
> remember, so that's another option. Maybe add something in
> dev-packages for example. No clue, I don't use pipenv.

  i just *knew* i would regret asking that question but since this
absolutely *needs* to be resolved so that the instructions in the
README work, here's one solution.

  forget about venvs (at least for now). the only two packages that
need installation to run the executable commands sphinx-lint and vale
are the packages with the same names, and they are special cases since
their purpose is to supply those commands, so the simple solution is
to install using "pipx", which is designed precisely for that purpose.

  one can install pipx with pip, then follow that with:

  $ pipx install sphinx-lint
  $ pipx install vale

and it all now works just fine on debian 13. maybe down the road get
fancier with "uv" or whatever but the above will just work, no?

rday

Re: [docs] commands to do "pip install" in docs README file need enhancing

[Quentin Schulz]

On 8/20/26 4:58 PM, Robert P. J. Day wrote:
> On Thu, 20 Aug 2026, Quentin Schulz wrote:
> 
>> On 8/20/26 3:23 PM, Trevor Gamblin via lists.yoctoproject.org wrote:
>>> On Thu Aug 20, 2026 at 7:20 AM EDT, Robert P. J. Day wrote:
>>>>
>>>>     I just went through this with another Sphinx-based documentation
>>>> repo -- installation instructions that advise the reader to install
>>>> Python modules with "pip install" fail on my Debian 13 system with:
>>>>
>>>>
>>>> $ pip install sphinx-lint
>>>> error: externally-managed-environment
>>>>
>>>> × This environment is externally managed
>>>> ╰─> To install Python packages system-wide, try apt install
>>>>       python3-xyz, where xyz is the package you are trying to
>>>>       install.
>>>>
>>>>       If you wish to install a non-Debian-packaged Python package,
>>>>       create a virtual environment using python3 -m venv path/to/venv.
>>>>       Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
>>>>       sure you have python3-full installed.
>>>>
>>>>       If you wish to install a non-Debian packaged Python application,
>>>>       it may be easiest to use pipx install xyz, which will manage a
>>>>       virtual environment for you. Make sure you have pipx installed.
>>>>
>>>>       See /usr/share/doc/python3.13/README.venv for more information.
>>>>
>>>> note: If you believe this is a mistake, please contact your Python
>>>> installation or OS distribution provider. You can override this, at
>>>> the risk of breaking your Python installation or OS, by passing
>>>> --break-system-packages.
>>>> hint: See PEP 668 for the detailed specification.
>>>>
>>>>
>>>>     Yes, there are solutions, such as using "pipx" instead of "pip", and
>>>> what have you, but it seems that the README file should be augmented
>>>> with an explanation as to how to deal with the above given that the
>>>> "pip install" commands are, in some cases, guaranteed to fail.
>>>
>>> Creating a venv like the error suggests (or via uv) is simple enough and IMO
>>> best practice for anything like this. You could submit a change to provide
>>> short
>>> examples, even replacing the references to pipenv earlier in the file.
>>
>> It's not that simple (though not that difficult). A venv by default
>> is completely isolated and doesn't use anything from the host. The
>> issue is that by doing so, you won't be able to build the docs, only
>> run sphinx-lint, which isn't ideal. You should probably reuse the
>> same venv as documented in
>> documentation/tools/host_packages_scripts/pip3_docs.sh instead.
>>
>> You probably want to update the instructions for Vale as well. We
>> still support pipenv though, via documentation/Pipfile as far as I
>> remember, so that's another option. Maybe add something in
>> dev-packages for example. No clue, I don't use pipenv.
> 
>    i just *knew* i would regret asking that question but since this
> absolutely *needs* to be resolved so that the instructions in the
> README work, here's one solution.
> 
>    forget about venvs (at least for now). the only two packages that

It's what we tell the user to do to build the docs.

> need installation to run the executable commands sphinx-lint and vale
> are the packages with the same names, and they are special cases since
> their purpose is to supply those commands, so the simple solution is
> to install using "pipx", which is designed precisely for that purpose.
> 
>    one can install pipx with pip, then follow that with:
> 
>    $ pipx install sphinx-lint
>    $ pipx install vale
> 
> and it all now works just fine on debian 13. maybe down the road get

Does it? I tried from within a container. First it tells me the path 
pipx installed to isn't in PATH so it won't be available. Then it tells 
me to run pipx ensurepath, which I do, and it then asks me to logoff or 
source ~/.bashrc. Only then do i have sphinx-lint. What a pain. Then I 
follow our instructions and install the sphinx packages in a venv:

sh ./documentation/tools/host_packages_scripts/pip3_docs.sh

I then source the venv:

. yocto-docs-venv/bin/activate

and then I run

make sphinx-lint

and you don't have sphinx-lint available. So you either build the docs, 
or run the linter, but not both without doing some gymnastics...

> fancier with "uv" or whatever but the above will just work, no?
> 

The tool needs to be available in the package feed of all supported 
distros for us to consider using it I think. I wouldn't want to document 
installing uv via pip and going into venv inceptions.

How about:

"""
diff --git a/documentation/Pipfile b/documentation/Pipfile
index 67fce078d..737f8bcea 100644
--- a/documentation/Pipfile
+++ b/documentation/Pipfile
@@ -4,6 +4,8 @@ url = "https://pypi.org/simple"
  verify_ssl = true

  [dev-packages]
+vale = "*"
+sphinx-lint = "*"

  [packages]
  sphinx = "*"
diff --git a/documentation/README b/documentation/README
index 4701357c3..5a90ad0ca 100644
--- a/documentation/README
+++ b/documentation/README
@@ -127,19 +127,19 @@ to validate the text style.

  To install Vale:

- $ pip install vale
+ $ pipenv install --dev

  To run Vale:

- $ make stylecheck
+ $ pipenv run make stylecheck

  Style checking the whole documentation might take some time and generate a
  lot of warnings/errors, thus one can run Vale on a subset of files or
  directories:

-  $ make stylecheck VALEDOCS=<file>
-  $ make stylecheck VALEDOCS="<file1> <file2>"
-  $ make stylecheck VALEDOCS=<dir>
+  $ pipenv run make stylecheck VALEDOCS=<file>
+  $ pipenv run make stylecheck VALEDOCS="<file1> <file2>"
+  $ pipenv run make stylecheck VALEDOCS=<dir>

  Lint checking the Yocto Project documentation
  =============================================
@@ -149,19 +149,19 @@ the project uses sphinx-lint 
(https://github.com/sphinx-contrib/sphinx-lint).

  To install sphinx-lint:

- $ pip install sphinx-lint
+ $ pipenv install --dev

  To run sphinx-lint:

- $ make sphinx-lint
+ $ pipenv run make sphinx-lint

  Lint checking the whole documentation might take some time and generate a
  lot of warnings/errors, thus one can run sphinx-lint on a subset of files
  or directories:

-  $ make sphinx-lint SPHINXLINTDOCS=<file>
-  $ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
-  $ make sphinx-lint SPHINXLINTDOCS=<dir>
+  $ pipenv run make sphinx-lint SPHINXLINTDOCS=<file>
+  $ pipenv run make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
+  $ pipenv run make sphinx-lint SPHINXLINTDOCS=<dir>

  Checking for broken links in the Yocto Project documentation
  ============================================================

"""

Cheers,
Quentin

Re: [docs] commands to do "pip install" in docs README file need enhancing

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 7220 bytes --]

On Thu, 20 Aug 2026, Quentin Schulz wrote:

>
>
> On 8/20/26 4:58 PM, Robert P. J. Day wrote:
> > On Thu, 20 Aug 2026, Quentin Schulz wrote:
> >
> > > On 8/20/26 3:23 PM, Trevor Gamblin via lists.yoctoproject.org wrote:
> > > > On Thu Aug 20, 2026 at 7:20 AM EDT, Robert P. J. Day wrote:
> > > > >
> > > > >     I just went through this with another Sphinx-based documentation
> > > > > repo -- installation instructions that advise the reader to install
> > > > > Python modules with "pip install" fail on my Debian 13 system with:
> > > > >
> > > > >
> > > > > $ pip install sphinx-lint
> > > > > error: externally-managed-environment
> > > > >
> > > > > × This environment is externally managed
> > > > > ╰─> To install Python packages system-wide, try apt install
> > > > >       python3-xyz, where xyz is the package you are trying to
> > > > >       install.
> > > > >
> > > > >       If you wish to install a non-Debian-packaged Python package,
> > > > >       create a virtual environment using python3 -m venv path/to/venv.
> > > > >       Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
> > > > >       sure you have python3-full installed.
> > > > >
> > > > >       If you wish to install a non-Debian packaged Python application,
> > > > >       it may be easiest to use pipx install xyz, which will manage a
> > > > >       virtual environment for you. Make sure you have pipx installed.
> > > > >
> > > > >       See /usr/share/doc/python3.13/README.venv for more information.
> > > > >
> > > > > note: If you believe this is a mistake, please contact your Python
> > > > > installation or OS distribution provider. You can override this, at
> > > > > the risk of breaking your Python installation or OS, by passing
> > > > > --break-system-packages.
> > > > > hint: See PEP 668 for the detailed specification.
> > > > >
> > > > >
> > > > >     Yes, there are solutions, such as using "pipx" instead of "pip",
> > > > > and
> > > > > what have you, but it seems that the README file should be augmented
> > > > > with an explanation as to how to deal with the above given that the
> > > > > "pip install" commands are, in some cases, guaranteed to fail.
> > > >
> > > > Creating a venv like the error suggests (or via uv) is simple enough and
> > > > IMO
> > > > best practice for anything like this. You could submit a change to
> > > > provide
> > > > short
> > > > examples, even replacing the references to pipenv earlier in the file.
> > >
> > > It's not that simple (though not that difficult). A venv by default
> > > is completely isolated and doesn't use anything from the host. The
> > > issue is that by doing so, you won't be able to build the docs, only
> > > run sphinx-lint, which isn't ideal. You should probably reuse the
> > > same venv as documented in
> > > documentation/tools/host_packages_scripts/pip3_docs.sh instead.
> > >
> > > You probably want to update the instructions for Vale as well. We
> > > still support pipenv though, via documentation/Pipfile as far as I
> > > remember, so that's another option. Maybe add something in
> > > dev-packages for example. No clue, I don't use pipenv.
> >
> >    i just *knew* i would regret asking that question but since this
> > absolutely *needs* to be resolved so that the instructions in the
> > README work, here's one solution.
> >
> >    forget about venvs (at least for now). the only two packages that
>
> It's what we tell the user to do to build the docs.
>
> > need installation to run the executable commands sphinx-lint and vale
> > are the packages with the same names, and they are special cases since
> > their purpose is to supply those commands, so the simple solution is
> > to install using "pipx", which is designed precisely for that purpose.
> >
> >    one can install pipx with pip, then follow that with:
> >
> >    $ pipx install sphinx-lint
> >    $ pipx install vale
> >
> > and it all now works just fine on debian 13. maybe down the road get
>
> Does it? I tried from within a container. First it tells me the path pipx
> installed to isn't in PATH so it won't be available. Then it tells me to run
> pipx ensurepath, which I do, and it then asks me to logoff or source
> ~/.bashrc. Only then do i have sphinx-lint. What a pain. Then I follow our
> instructions and install the sphinx packages in a venv:
>
> sh ./documentation/tools/host_packages_scripts/pip3_docs.sh
>
> I then source the venv:
>
> . yocto-docs-venv/bin/activate
>
> and then I run
>
> make sphinx-lint
>
> and you don't have sphinx-lint available. So you either build the docs, or run
> the linter, but not both without doing some gymnastics...
>
> > fancier with "uv" or whatever but the above will just work, no?
> >
>
> The tool needs to be available in the package feed of all supported distros
> for us to consider using it I think. I wouldn't want to document installing uv
> via pip and going into venv inceptions.
>
> How about:
>
> """
> diff --git a/documentation/Pipfile b/documentation/Pipfile
> index 67fce078d..737f8bcea 100644
> --- a/documentation/Pipfile
> +++ b/documentation/Pipfile
> @@ -4,6 +4,8 @@ url = "https://pypi.org/simple"
>  verify_ssl = true
>
>  [dev-packages]
> +vale = "*"
> +sphinx-lint = "*"
>
>  [packages]
>  sphinx = "*"
> diff --git a/documentation/README b/documentation/README
> index 4701357c3..5a90ad0ca 100644
> --- a/documentation/README
> +++ b/documentation/README
> @@ -127,19 +127,19 @@ to validate the text style.
>
>  To install Vale:
>
> - $ pip install vale
> + $ pipenv install --dev
>
>  To run Vale:
>
> - $ make stylecheck
> + $ pipenv run make stylecheck
>
>  Style checking the whole documentation might take some time and generate a
>  lot of warnings/errors, thus one can run Vale on a subset of files or
>  directories:
>
> -  $ make stylecheck VALEDOCS=<file>
> -  $ make stylecheck VALEDOCS="<file1> <file2>"
> -  $ make stylecheck VALEDOCS=<dir>
> +  $ pipenv run make stylecheck VALEDOCS=<file>
> +  $ pipenv run make stylecheck VALEDOCS="<file1> <file2>"
> +  $ pipenv run make stylecheck VALEDOCS=<dir>
>
>  Lint checking the Yocto Project documentation
>  =============================================
> @@ -149,19 +149,19 @@ the project uses sphinx-lint
> (https://github.com/sphinx-contrib/sphinx-lint).
>
>  To install sphinx-lint:
>
> - $ pip install sphinx-lint
> + $ pipenv install --dev
>
>  To run sphinx-lint:
>
> - $ make sphinx-lint
> + $ pipenv run make sphinx-lint
>
>  Lint checking the whole documentation might take some time and generate a
>  lot of warnings/errors, thus one can run sphinx-lint on a subset of files
>  or directories:
>
> -  $ make sphinx-lint SPHINXLINTDOCS=<file>
> -  $ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
> -  $ make sphinx-lint SPHINXLINTDOCS=<dir>
> +  $ pipenv run make sphinx-lint SPHINXLINTDOCS=<file>
> +  $ pipenv run make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
> +  $ pipenv run make sphinx-lint SPHINXLINTDOCS=<dir>
>
>  Checking for broken links in the Yocto Project documentation
>  ============================================================
>
> """
>
> Cheers,
> Quentin

  I swear, there are days when I am simply scared to ask what I think
are innocuous questions.

rday

Re: [docs] commands to do "pip install" in docs README file need enhancing

[Robert P. J. Day]

On Thu, 20 Aug 2026, Quentin Schulz wrote:

>
>
> On 8/20/26 4:58 PM, Robert P. J. Day wrote:

... snip ...
> > need installation to run the executable commands sphinx-lint and vale
> > are the packages with the same names, and they are special cases since
> > their purpose is to supply those commands, so the simple solution is
> > to install using "pipx", which is designed precisely for that purpose.
> >
> >    one can install pipx with pip, then follow that with:
> >
> >    $ pipx install sphinx-lint
> >    $ pipx install vale
> >
> > and it all now works just fine on debian 13. maybe down the road get
>
> Does it? I tried from within a container. First it tells me the path
> pipx installed to isn't in PATH so it won't be available. Then it
> tells me to run pipx ensurepath, which I do, and it then asks me to
> logoff or source ~/.bashrc. Only then do i have sphinx-lint. What a
> pain.

  FWIW, here's why this just worked for me. on debian 13,
/etc/skel/.profile contains:

  # set PATH so it includes user's private bin if it exists
  if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
  fi

well, it just so happens that, for whatever reason, on my debian 13
system, i have that directory so that it's automatically added to my
search PATH upon login, which is exactly where pipx installs commands
like "vale" and "sphinx-lint", so stuff like that just worked for me
the instant i installed those two packages with pipx.

rday