[LTP] [PATCH v5 0/3] Update doc related Makefile

[LTP] [PATCH v5 0/3] Update doc related Makefile

[Petr Vorel]

Changes from v4:
* Use sphinx version used on readthedocs.org (to really test in devel
  and github CI what will be deployed).
* Don't install sphinx in github CI (python3-sphinx package)
* Drop last 2 patches ("Makefile: Update 'doc' target, add 'doc-clean'"
  and "doc: Note 'make doc' in the building doc" => doc/Makefile should
  be rewritten to use generic_leaf_target.mk, then integration to the
  top level Makefile will not be a hack).

Tested:
https://app.readthedocs.org/projects/linux-test-project/builds/27705957/

Link to v4
https://patchwork.ozlabs.org/project/ltp/list/?series=450372&state=*
https://lore.kernel.org/ltp/20250328095747.169011-1-pvorel@suse.cz/#r

Link to v3:
https://patchwork.ozlabs.org/project/ltp/list/?series=449828&state=*
https://lore.kernel.org/ltp/20250324234016.367228-1-pvorel@suse.cz/T/#t

Link to v2:
https://patchwork.ozlabs.org/project/ltp/list/?series=443894&state=*
https://lore.kernel.org/ltp/20250211233552.1990618-1-pvorel@suse.cz/#r

Link to v1:
https://patchwork.ozlabs.org/project/ltp/patch/20250206143421.1571918-4-pvorel@suse.cz/
https://lore.kernel.org/ltp/20250206143421.1571918-4-pvorel@suse.cz/



Petr Vorel (3):
  doc/Makefile: Remove also metadata/ltp.json
  doc: Add sphinx to requirements.txt
  doc/Makefile: Allow to create and use .venv

 .github/workflows/ci-sphinx-doc.yml |  2 +-
 doc/Makefile                        | 25 ++++++++++++++++++++++---
 doc/developers/documentation.rst    |  2 --
 doc/requirements.txt                |  4 ++++
 4 files changed, 27 insertions(+), 6 deletions(-)

-- 
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v5 1/3] doc/Makefile: Remove also metadata/ltp.json

[Petr Vorel]

Because there is no detection whether tests changed it's better to
remove JSON file on metadata cleanup.

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
* The same as v4

 doc/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/doc/Makefile b/doc/Makefile
index a07df04d5c..3c5682ad00 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -15,4 +15,5 @@ spelling:
 	sphinx-build -b spelling -d build/doctree . build/spelling
 
 clean:
-	rm -rf html/ build/ _static/syscalls.rst _static/tests.rst syscalls.tbl
+	rm -rf html/ build/ _static/syscalls.rst _static/tests.rst syscalls.tbl \
+		${abs_top_builddir}/metadata/ltp.json
-- 
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v5 2/3] doc: Add sphinx to requirements.txt

[Petr Vorel]

Although that slightly prolongs creating virtualenv (and CI job), but
it's probably better to handle it as the other requirements.

Use the same sphinx and sphinx-rtd-theme versions as on readthedocs.org.
That way we get doc generation tested by CI and by local build (to avoid
incompatibility changes).

Also don't install python3-sphinx via apt in CI job to keep usage the
same as in readthedocs.org.

Suggested-by: Andrea Cervesato <andrea.cervesato@suse.com>
Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
Changes from v4:
* Use sphinx version used on readthedocs.org (to really test in devel
  and github CI what will be deployed).
* Don't install sphinx in github CI (python3-sphinx package)

 .github/workflows/ci-sphinx-doc.yml | 2 +-
 doc/developers/documentation.rst    | 2 --
 doc/requirements.txt                | 4 ++++
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci-sphinx-doc.yml b/.github/workflows/ci-sphinx-doc.yml
index a5f9d74d1f..e77c766871 100644
--- a/.github/workflows/ci-sphinx-doc.yml
+++ b/.github/workflows/ci-sphinx-doc.yml
@@ -23,7 +23,7 @@ jobs:
       - name: Install sphinx and autotools
         run: |
           sudo apt update
-          sudo apt install autoconf make python3-sphinx python3-virtualenv
+          sudo apt install autoconf make python3-virtualenv
 
       - name: Run configure
         run: |
diff --git a/doc/developers/documentation.rst b/doc/developers/documentation.rst
index 27c847e125..a303253693 100644
--- a/doc/developers/documentation.rst
+++ b/doc/developers/documentation.rst
@@ -23,8 +23,6 @@ Before building, make sure you have python3 ``virtualenv`` module installed.
     # prepare virtual environment
     python3 -m virtualenv .venv
     . .venv/bin/activate
-
-    pip install sphinx # usually packaged in distros
     pip install -r requirements.txt
 
     # build documentation
diff --git a/doc/requirements.txt b/doc/requirements.txt
index 742fb8b4bb..6302ecd9f2 100644
--- a/doc/requirements.txt
+++ b/doc/requirements.txt
@@ -1,3 +1,7 @@
+# Use the same sphinx as on readthedocs.org. When updated, make sure
+# sphinx-rtd-theme is compatible with sphinx.
+sphinx==5.3.0
 sphinx-rtd-theme==2.0.0
+
 linuxdoc==20231020
 sphinxcontrib-spelling==7.7.0
-- 
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 2/3] doc: Add sphinx to requirements.txt

[Andrea Cervesato via ltp]

Hi!

On 4/7/25 17:01, Petr Vorel wrote:
> Although that slightly prolongs creating virtualenv (and CI job), but
> it's probably better to handle it as the other requirements.
>
> Use the same sphinx and sphinx-rtd-theme versions as on readthedocs.org.
> That way we get doc generation tested by CI and by local build (to avoid
> incompatibility changes).
>
> Also don't install python3-sphinx via apt in CI job to keep usage the
> same as in readthedocs.org.
>
> Suggested-by: Andrea Cervesato <andrea.cervesato@suse.com>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> Changes from v4:
> * Use sphinx version used on readthedocs.org (to really test in devel
>    and github CI what will be deployed).
> * Don't install sphinx in github CI (python3-sphinx package)
>
>   .github/workflows/ci-sphinx-doc.yml | 2 +-
>   doc/developers/documentation.rst    | 2 --
>   doc/requirements.txt                | 4 ++++
>   3 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/.github/workflows/ci-sphinx-doc.yml b/.github/workflows/ci-sphinx-doc.yml
> index a5f9d74d1f..e77c766871 100644
> --- a/.github/workflows/ci-sphinx-doc.yml
> +++ b/.github/workflows/ci-sphinx-doc.yml
> @@ -23,7 +23,7 @@ jobs:
>         - name: Install sphinx and autotools
>           run: |
>             sudo apt update
> -          sudo apt install autoconf make python3-sphinx python3-virtualenv
> +          sudo apt install autoconf make python3-virtualenv
>   
>         - name: Run configure
>           run: |
> diff --git a/doc/developers/documentation.rst b/doc/developers/documentation.rst
> index 27c847e125..a303253693 100644
> --- a/doc/developers/documentation.rst
> +++ b/doc/developers/documentation.rst
> @@ -23,8 +23,6 @@ Before building, make sure you have python3 ``virtualenv`` module installed.
>       # prepare virtual environment
>       python3 -m virtualenv .venv
>       . .venv/bin/activate
> -
> -    pip install sphinx # usually packaged in distros
>       pip install -r requirements.txt
>   
>       # build documentation
> diff --git a/doc/requirements.txt b/doc/requirements.txt
> index 742fb8b4bb..6302ecd9f2 100644
> --- a/doc/requirements.txt
> +++ b/doc/requirements.txt
> @@ -1,3 +1,7 @@
> +# Use the same sphinx as on readthedocs.org. When updated, make sure
> +# sphinx-rtd-theme is compatible with sphinx.
> +sphinx==5.3.0
I'm still not 100% sure about this, since readthedocs might change the 
version and "pip install sphinx==5.3.0" would override it. But we can 
try and see how it goes in the future.
>   sphinx-rtd-theme==2.0.0
> +
>   linuxdoc==20231020
>   sphinxcontrib-spelling==7.7.0

Acked-by: Andrea Cervesato <andrea.cervesato@suse.com>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Petr Vorel]

Add 'setup' target (alias to '.venv') to create virtualenv directory.
This is an optional target (not run by default).
If .venv exists, it's used in other targets, activation supports only
fish and bash/zsh (known shells used by LTP developers, csh/tcsh is
ignored atm).

This helps to use virtualenv for development, but avoid using it by
default (readthedoc uses container with virtualenv, creating it would be
waste of time).

Add 'distclean' target which removes also .venv/ directory.

Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>
Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
* The same as v4

NOTE: doc/Makefile should be rewritten to use generic_leaf_target.mk,
then integration to the top level Makefile will not be a hack).

 doc/Makefile | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/doc/Makefile b/doc/Makefile
index 3c5682ad00..2062d6e935 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -5,15 +5,33 @@ top_srcdir		?= ..
 
 include $(top_srcdir)/include/mk/env_pre.mk
 
+PYTHON := python3
+VENV_DIR := .venv
+
+# only fish and bash/zsh supported
+VENV_CMD := if [ "x${FISH_VERSION}" != "x" ]; then . $(VENV_DIR)/bin/activate.fish; else . $(VENV_DIR)/bin/activate; fi
+
+RUN_VENV := if [ -d $(VENV_DIR) ]; then $(VENV_CMD); fi
+
+$(VENV_DIR):
+	$(PYTHON) -m virtualenv $(VENV_DIR)
+	$(VENV_CMD) && pip install -r requirements.txt
+
+.PHONY: setup
+setup: $(VENV_DIR)
+
 ${abs_top_builddir}/metadata/ltp.json:
 	$(MAKE) -C ${abs_top_builddir}/metadata
 
 all: ${abs_top_builddir}/metadata/ltp.json
-	sphinx-build -b html . html
+	$(RUN_VENV); sphinx-build -b html . html
 
 spelling:
-	sphinx-build -b spelling -d build/doctree . build/spelling
+	$(RUN_VENV); sphinx-build -b spelling -d build/doctree . build/spelling
 
 clean:
 	rm -rf html/ build/ _static/syscalls.rst _static/tests.rst syscalls.tbl \
 		${abs_top_builddir}/metadata/ltp.json
+
+distclean: clean
+	rm -rf $(VENV_DIR)
-- 
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Andrea Cervesato via ltp]

Hi Petr,

some comments below.

On 4/7/25 17:01, Petr Vorel wrote:
> Add 'setup' target (alias to '.venv') to create virtualenv directory.
> This is an optional target (not run by default).
> If .venv exists, it's used in other targets, activation supports only
> fish and bash/zsh (known shells used by LTP developers, csh/tcsh is
> ignored atm).
>
> This helps to use virtualenv for development, but avoid using it by
> default (readthedoc uses container with virtualenv, creating it would be
> waste of time).
>
> Add 'distclean' target which removes also .venv/ directory.
>
> Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> * The same as v4
>
> NOTE: doc/Makefile should be rewritten to use generic_leaf_target.mk,
> then integration to the top level Makefile will not be a hack).
>
>   doc/Makefile | 22 ++++++++++++++++++++--
>   1 file changed, 20 insertions(+), 2 deletions(-)
>
> diff --git a/doc/Makefile b/doc/Makefile
> index 3c5682ad00..2062d6e935 100644
> --- a/doc/Makefile
> +++ b/doc/Makefile
> @@ -5,15 +5,33 @@ top_srcdir		?= ..
>   
>   include $(top_srcdir)/include/mk/env_pre.mk
>   
> +PYTHON := python3
> +VENV_DIR := .venv
> +
> +# only fish and bash/zsh supported
> +VENV_CMD := if [ "x${FISH_VERSION}" != "x" ]; then . $(VENV_DIR)/bin/activate.fish; else . $(VENV_DIR)/bin/activate; fi
I had to think carefully about this, but I think you are right. It's 
better not to over-complicate this and to support other shells but bash.
make command can override environment variables, so it's better to use 
that feature instead of complicating Makefile that can be really messy 
when adding new features.
> +
> +RUN_VENV := if [ -d $(VENV_DIR) ]; then $(VENV_CMD); fi
> +
> +$(VENV_DIR):
> +	$(PYTHON) -m virtualenv $(VENV_DIR)
> +	$(VENV_CMD) && pip install -r requirements.txt
> +
> +.PHONY: setup
> +setup: $(VENV_DIR)
> +
>   ${abs_top_builddir}/metadata/ltp.json:
>   	$(MAKE) -C ${abs_top_builddir}/metadata
>   
>   all: ${abs_top_builddir}/metadata/ltp.json
> -	sphinx-build -b html . html
> +	$(RUN_VENV); sphinx-build -b html . html
>   
>   spelling:
> -	sphinx-build -b spelling -d build/doctree . build/spelling
> +	$(RUN_VENV); sphinx-build -b spelling -d build/doctree . build/spelling
>   
>   clean:
>   	rm -rf html/ build/ _static/syscalls.rst _static/tests.rst syscalls.tbl \
>   		${abs_top_builddir}/metadata/ltp.json
> +
> +distclean: clean
> +	rm -rf $(VENV_DIR)

The rest looks fine.

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Petr Vorel]

> Hi Petr,

> some comments below.

> On 4/7/25 17:01, Petr Vorel wrote:
> > Add 'setup' target (alias to '.venv') to create virtualenv directory.
> > This is an optional target (not run by default).
> > If .venv exists, it's used in other targets, activation supports only
> > fish and bash/zsh (known shells used by LTP developers, csh/tcsh is
> > ignored atm).

> > This helps to use virtualenv for development, but avoid using it by
> > default (readthedoc uses container with virtualenv, creating it would be
> > waste of time).

> > Add 'distclean' target which removes also .venv/ directory.

> > Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>

I hope you still agree with this ^. Or shell I wait for your ack?

> > Signed-off-by: Petr Vorel <pvorel@suse.cz>
> > ---
> > * The same as v4

> > NOTE: doc/Makefile should be rewritten to use generic_leaf_target.mk,
> > then integration to the top level Makefile will not be a hack).

> >   doc/Makefile | 22 ++++++++++++++++++++--
> >   1 file changed, 20 insertions(+), 2 deletions(-)

> > diff --git a/doc/Makefile b/doc/Makefile
> > index 3c5682ad00..2062d6e935 100644
> > --- a/doc/Makefile
> > +++ b/doc/Makefile
> > @@ -5,15 +5,33 @@ top_srcdir		?= ..
> >   include $(top_srcdir)/include/mk/env_pre.mk
> > +PYTHON := python3
> > +VENV_DIR := .venv
> > +
> > +# only fish and bash/zsh supported
> > +VENV_CMD := if [ "x${FISH_VERSION}" != "x" ]; then . $(VENV_DIR)/bin/activate.fish; else . $(VENV_DIR)/bin/activate; fi
> I had to think carefully about this, but I think you are right. It's better
> not to over-complicate this and to support other shells but bash.
> make command can override environment variables, so it's better to use that
> feature instead of complicating Makefile that can be really messy when
> adding new features.

This supports fish, bash and zsh. If anybody asks for t{,csh} or anything else
in .venv/bin/activate* it can be added.

> > +
> > +RUN_VENV := if [ -d $(VENV_DIR) ]; then $(VENV_CMD); fi
> > +
> > +$(VENV_DIR):
> > +	$(PYTHON) -m virtualenv $(VENV_DIR)
> > +	$(VENV_CMD) && pip install -r requirements.txt
> > +
> > +.PHONY: setup
> > +setup: $(VENV_DIR)
> > +
> >   ${abs_top_builddir}/metadata/ltp.json:
> >   	$(MAKE) -C ${abs_top_builddir}/metadata
> >   all: ${abs_top_builddir}/metadata/ltp.json
> > -	sphinx-build -b html . html
> > +	$(RUN_VENV); sphinx-build -b html . html
> >   spelling:
> > -	sphinx-build -b spelling -d build/doctree . build/spelling
> > +	$(RUN_VENV); sphinx-build -b spelling -d build/doctree . build/spelling
> >   clean:
> >   	rm -rf html/ build/ _static/syscalls.rst _static/tests.rst syscalls.tbl \
> >   		${abs_top_builddir}/metadata/ltp.json
> > +
> > +distclean: clean
> > +	rm -rf $(VENV_DIR)

> The rest looks fine.

Thanks!

Kind regards,
Petr

> Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Andrea Cervesato via ltp]

Hi,

On 4/8/25 12:14, Petr Vorel wrote:
>>> Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>
> I hope you still agree with this ^. Or shell I wait for your ack?
Yes, I just meant that you can feel free to stick with the initial idea 
of supporting bash only, so we get rid of the more complex scenario.
>
>
Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Petr Vorel]

> Hi,

> On 4/8/25 12:14, Petr Vorel wrote:
> > > > Reviewed-by: Andrea Cervesato <andrea.cervesato@suse.com>
> > I hope you still agree with this ^. Or shell I wait for your ack?
> Yes, I just meant that you can feel free to stick with the initial idea of
> supporting bash only, so we get rid of the more complex scenario.

Thank you. You reported you're using fish, let's keep that support as well.

Kind regards,
Petr

> Andrea

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v5 3/3] doc/Makefile: Allow to create and use .venv

[Petr Vorel]

Hi Andrea, Ricardo,

FYI patchset merged.
Thanks for your review!

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp