[LTP] [PATCH v2] doc: Add basic shell test description

[LTP] [PATCH v2] doc: Add basic shell test description

[Cyril Hrubis]

Signed-off-by: Cyril Hrubis <chrubis@suse.cz>
Reviewed-by: Avinesh Kumar <akumar@suse.de>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
---
 doc/conf.py                        |  1 +
 doc/developers/api_shell_tests.rst | 91 ++++++++++++++++++++++++++++++
 2 files changed, 92 insertions(+)

Changes in v2:

- fixed typos
- added links as requested by Petr

diff --git a/doc/conf.py b/doc/conf.py
index 86f6fe2b7..e8488bff7 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -34,6 +34,7 @@ exclude_patterns = ["html*", '_static*', '.venv*']
 extlinks = {
     'repo': (f'{ltp_repo}/%s', '%s'),
     'master': (f'{ltp_repo}/blob/master/%s', '%s'),
+    'shell_lib': (f'{ltp_repo}/blob/master/testcases/lib/%s', '%s'),
     'git_man': ('https://git-scm.com/docs/git-%s', 'git %s'),
     'man2': ('https://man7.org/linux/man-pages/man2/%s.2.html', '%s(2)'),
     'man3': ('https://man7.org/linux/man-pages/man3/%s.3.html', '%s(3)'),
diff --git a/doc/developers/api_shell_tests.rst b/doc/developers/api_shell_tests.rst
index b6e8560d9..ccfc0e071 100644
--- a/doc/developers/api_shell_tests.rst
+++ b/doc/developers/api_shell_tests.rst
@@ -2,3 +2,94 @@
 
 LTP shell API
 =============
+
+Shell API overview
+------------------
+
+First lines of the shell test should be a shebang, a license, and copyrights.
+
+.. code-block:: shell
+
+   #!/bin/sh
+   # SPDX-License-Identifier: GPL-2.0-or-later
+   # Copyright 2099 Foo Bar <foo.bar@example.org>
+
+A documentation comment block formatted in ReStructuredText should follow right
+after these lines. This comment is parsed and exported into the LTP
+documentation at https://linux-test-project.readthedocs.io/en/latest/users/test_catalog.html
+
+.. code-block:: shell
+
+   # ---
+   # doc
+   # Test for a foo bar.
+   #
+   # This test is testing foo by checking that bar is doing xyz.
+   # ---
+
+The shell loader test library uses the :doc:`../developers/api_c_tests`
+internally by parsing a special JSON formatted comment and
+initializing it accordingly. The JSON format is nearly 1:1 serialization of the
+:ref:`struct tst_test` into a JSON. The environment must be always preset even
+when it's empty.
+
+.. code-block:: shell
+
+   # ---
+   # env
+   # {
+   #  "needs_root": true,
+   #  "needs_tmpdir": true,
+   #  "needs_kconfigs": ["CONFIG_NUMA=y"],
+   #  "tags": {
+   #   ["linux-git", "432fd03240fa"]
+   #  }
+   # }
+
+After the documentation and environment has been laid out we finally import the
+:shell_lib:`tst_loader.sh`. This will, among other things, start the
+:shell_lib:`tst_run_shell.c` binary, that will parse the shell test environment
+comment and initialize the C test library accordingly.
+
+.. code-block:: shell
+
+   . tst_loader.sh
+
+At this point everything has been set up and we can finally write the test
+function. The test results are reported by the usual functions :ref:`tst_res` and
+:ref:`tst_brk`. As in the C API these functions store results into a piece of shared
+memory as soon as they return so there is no need to propagate results event
+from child processes.
+
+.. code-block:: shell
+
+   tst_test()
+   {
+        tst_res TPASS "Bar enabled Foo"
+   }
+
+In order for the test to be actually executed the very last line of the script
+must source the :shell_lib:`tst_run.sh` script.
+
+.. code-block:: shell
+
+   . tst_run.sh
+
+In order to run a test from a LTP tree a few directories has to be added to the
+`$PATH`. Note that the number of `../` may depend on the depth of the current
+directory relative to the LTP root.
+
+.. code-block:: shell
+
+   $ PATH=$PATH:$PWD:$PWD/../../lib/ ./foo01.sh
+
+Test setup and cleanup
+----------------------
+
+The test setup and cleanup functions are optional and passed via variables.
+Similarily to the C API the setup is executed exactly once at the start of the
+test and the test cleanup is executed at the test end or when test was
+interrupted by ref:`tst_brk`.
+
+   .. literalinclude:: ../../testcases/lib/tests/shell_loader_setup_cleanup.sh
+   :language: shell
-- 
2.51.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2] doc: Add basic shell test description

[Petr Vorel]

Hi Cyril,

TL;DR You already add my RBT (thanks), please amend your patch with the diff below.

...
> +    'shell_lib': (f'{ltp_repo}/blob/master/testcases/lib/%s', '%s'),
+1. I was thinking about it previously for C helpers, then I preferred to show
full path. I guess I'll later use 'shell_lib' prefix for these. I suppose we can
keep lib/*.c path, but sure, we could also (later) add 'lib' and 'include'.

I wish extlinks module allow to pass more params than just one.

...
> +The test setup and cleanup functions are optional and passed via variables.
> +Similarily to the C API the setup is executed exactly once at the start of the

Similarily => Similarly

> +test and the test cleanup is executed at the test end or when test was
> +interrupted by ref:`tst_brk`.
=> missing leading ':' causes text "by ref:tst_brk."

interrupted by :ref:`tst_brk`.

> +
> +   .. literalinclude:: ../../testcases/lib/tests/shell_loader_setup_cleanup.sh
> +   :language: shell

+1, nice feature.  "language : shell" require another level of indentation,
otherwise it just shows as a test below code.

Kind regards,
Petr

+++ doc/developers/api_shell_tests.rst
@@ -87,9 +87,9 @@ Test setup and cleanup
 ----------------------
 
 The test setup and cleanup functions are optional and passed via variables.
-Similarily to the C API the setup is executed exactly once at the start of the
+Similarly to the C API the setup is executed exactly once at the start of the
 test and the test cleanup is executed at the test end or when test was
-interrupted by ref:`tst_brk`.
+interrupted by :ref:`tst_brk`.
 
    .. literalinclude:: ../../testcases/lib/tests/shell_loader_setup_cleanup.sh
-   :language: shell
+      :language: shell

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2] doc: Add basic shell test description

[Cyril Hrubis]

Hi!
> > +    'shell_lib': (f'{ltp_repo}/blob/master/testcases/lib/%s', '%s'),
> +1. I was thinking about it previously for C helpers, then I preferred to show
> full path. I guess I'll later use 'shell_lib' prefix for these. I suppose we can
> keep lib/*.c path, but sure, we could also (later) add 'lib' and 'include'.

Well this is going to be easier to maintain if we ever decided to move
the content of the testcases/lib into a top level directory, where it
probably belongs.

> I wish extlinks module allow to pass more params than just one.

Yes.

> +++ doc/developers/api_shell_tests.rst
> @@ -87,9 +87,9 @@ Test setup and cleanup
>  ----------------------
>  
>  The test setup and cleanup functions are optional and passed via variables.
> -Similarily to the C API the setup is executed exactly once at the start of the
> +Similarly to the C API the setup is executed exactly once at the start of the
>  test and the test cleanup is executed at the test end or when test was
> -interrupted by ref:`tst_brk`.
> +interrupted by :ref:`tst_brk`.
>  
>     .. literalinclude:: ../../testcases/lib/tests/shell_loader_setup_cleanup.sh
> -   :language: shell
> +      :language: shell

Fixed and pushed, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp