* [LTP] [PATCH 01/10] Add the new basics documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 02/10] Add new mailing list setup documentation Andrea Cervesato
` (9 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
This is the first patch introducing the new LTP documentation. The
new documentation is now using Sphinx as the main framework,
providing a solid substitute to Github wiki pages.
This patch is meant to be the starting point, where we start rewriting
the old asciidoc documentation format.
A .readthedocs.yaml configuration has been added to provide an initial
documentation publish via readthedocs service.
---
.readthedocs.yml | 15 +++
doc_new/.gitignore | 1 +
doc_new/Makefile | 5 +
doc_new/_static/custom.css | 4 +
doc_new/conf.py | 27 ++++++
doc_new/developers/api_c_tests.rst | 4 +
doc_new/developers/api_kvm_tests.rst | 4 +
doc_new/developers/api_network_tests.rst | 4 +
doc_new/developers/api_shell_tests.rst | 4 +
doc_new/developers/build_system.rst | 4 +
doc_new/developers/debugging.rst | 10 ++
doc_new/developers/ltp_library.rst | 4 +
doc_new/developers/setup_mailinglist.rst | 4 +
doc_new/developers/test_case_tutorial.rst | 4 +
doc_new/developers/writing_tests.rst | 20 ++++
doc_new/index.rst | 124 +++++++++++++++++++++++++
doc_new/maintainers/ltp_release_procedure.rst | 4 +
doc_new/maintainers/patch_review.rst | 4 +
doc_new/requirements.txt | 1 +
doc_new/users/quick_start.rst | 126 ++++++++++++++++++++++++++
doc_new/users/setup_tests.rst | 104 +++++++++++++++++++++
doc_new/users/supported_systems.rst | 110 ++++++++++++++++++++++
22 files changed, 587 insertions(+)
diff --git a/.readthedocs.yml b/.readthedocs.yml
new file mode 100644
index 000000000..636e29c97
--- /dev/null
+++ b/.readthedocs.yml
@@ -0,0 +1,15 @@
+version: 2
+
+build:
+ os: "ubuntu-22.04"
+ tools:
+ python: "3.6"
+
+# Build from the docs/ directory with Sphinx
+sphinx:
+ configuration: doc_new/conf.py
+
+# Explicitly set the version of Python and its requirements
+python:
+ install:
+ - requirements: doc_new/requirements.txt
diff --git a/doc_new/.gitignore b/doc_new/.gitignore
new file mode 100644
index 000000000..5ccff1a6b
--- /dev/null
+++ b/doc_new/.gitignore
@@ -0,0 +1 @@
+html/
diff --git a/doc_new/Makefile b/doc_new/Makefile
new file mode 100644
index 000000000..aea89f641
--- /dev/null
+++ b/doc_new/Makefile
@@ -0,0 +1,5 @@
+all:
+ sphinx-build -b html . html
+
+clean:
+ rm -rf html/
diff --git a/doc_new/_static/custom.css b/doc_new/_static/custom.css
new file mode 100644
index 000000000..cb4d3821d
--- /dev/null
+++ b/doc_new/_static/custom.css
@@ -0,0 +1,4 @@
+/* set multiline tables cells */
+.wy-table-responsive table td {
+ white-space: normal;
+}
diff --git a/doc_new/conf.py b/doc_new/conf.py
new file mode 100644
index 000000000..b333ebf2a
--- /dev/null
+++ b/doc_new/conf.py
@@ -0,0 +1,27 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'Linux Test Project'
+copyright = '2024, Linux Test Project'
+author = 'Linux Test Project'
+release = '1.0'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = []
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+html_static_path = ['_static']
+
+def setup(app):
+ app.add_css_file('custom.css')
diff --git a/doc_new/developers/api_c_tests.rst b/doc_new/developers/api_c_tests.rst
new file mode 100644
index 000000000..c03f9caf1
--- /dev/null
+++ b/doc_new/developers/api_c_tests.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using C API
+======================
diff --git a/doc_new/developers/api_kvm_tests.rst b/doc_new/developers/api_kvm_tests.rst
new file mode 100644
index 000000000..1ef7ec452
--- /dev/null
+++ b/doc_new/developers/api_kvm_tests.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using KVM API
+========================
diff --git a/doc_new/developers/api_network_tests.rst b/doc_new/developers/api_network_tests.rst
new file mode 100644
index 000000000..4fe45f5f6
--- /dev/null
+++ b/doc_new/developers/api_network_tests.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using network API
+============================
diff --git a/doc_new/developers/api_shell_tests.rst b/doc_new/developers/api_shell_tests.rst
new file mode 100644
index 000000000..2fb2a5068
--- /dev/null
+++ b/doc_new/developers/api_shell_tests.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using shell API
+==========================
diff --git a/doc_new/developers/build_system.rst b/doc_new/developers/build_system.rst
new file mode 100644
index 000000000..cfcaf2d04
--- /dev/null
+++ b/doc_new/developers/build_system.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Build system
+============
diff --git a/doc_new/developers/debugging.rst b/doc_new/developers/debugging.rst
new file mode 100644
index 000000000..18a4a67e9
--- /dev/null
+++ b/doc_new/developers/debugging.rst
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Debugging
+=========
+
+The new test library runs the actual test, i.e. the `test()` function in a
+forked process. To get stack trace of a crashing test in gdb it's needed to
+`set follow-fork-mode child <https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_25.html>`_.
+To trace the test, please use `strace -f` to enable tracing also for the
+forked processes.
diff --git a/doc_new/developers/ltp_library.rst b/doc_new/developers/ltp_library.rst
new file mode 100644
index 000000000..b780b6870
--- /dev/null
+++ b/doc_new/developers/ltp_library.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+LTP Library development
+=======================
diff --git a/doc_new/developers/setup_mailinglist.rst b/doc_new/developers/setup_mailinglist.rst
new file mode 100644
index 000000000..b7320b9a2
--- /dev/null
+++ b/doc_new/developers/setup_mailinglist.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Setting up the Mailing list
+===========================
diff --git a/doc_new/developers/test_case_tutorial.rst b/doc_new/developers/test_case_tutorial.rst
new file mode 100644
index 000000000..43563c234
--- /dev/null
+++ b/doc_new/developers/test_case_tutorial.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Test case tutorial
+==================
diff --git a/doc_new/developers/writing_tests.rst b/doc_new/developers/writing_tests.rst
new file mode 100644
index 000000000..ba0188aee
--- /dev/null
+++ b/doc_new/developers/writing_tests.rst
@@ -0,0 +1,20 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Writing tests
+=============
+
+Testing builds with GitHub Actions
+----------------------------------
+
+Master branch is tested in `GitHub Actions <https://github.com/linux-test-project/ltp/actions>`_
+to ensure LTP builds in various distributions, including old, current and
+bleeding edge. ``gcc`` and ``clang`` toolchains are also tested for various
+architectures using cross-compilation. For a full list of tested distros, please
+check ``.github/workflows/ci.yml``.
+
+.. note::
+
+ Passing the GitHub Actions CI means that LTP compiles in a variety of
+ different distributions on their **newest releases**.
+ The CI also checks for code linting, running ``make check`` in the whole
+ LTP project.
diff --git a/doc_new/index.rst b/doc_new/index.rst
new file mode 100644
index 000000000..a8778b3d8
--- /dev/null
+++ b/doc_new/index.rst
@@ -0,0 +1,124 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Linux Test Project
+==================
+
+Linux Test Project is a joint project started by SGI, OSDL and Bull developed
+and maintained by SUSE, Red Hat, Fujitsu, IBM, Cisco, Oracle and others. The
+project goal is to deliver tests to the open source community that validate
+reliability, robustness, and stability of the Linux Kernel.
+
+The testing suites contain a collection of tools for testing the Linux kernel
+and related features. Our goal is to improve the Linux kernel and system
+libraries by bringing test automation.
+
+.. warning::
+
+ LTP tests shouldn't run in production systems. In particular,
+ growfiles, doio, and iogen, stress the I/O capabilities of the systems and
+ they are intended to find (or cause) problems.
+
+Some references:
+
+* `Source code <https://github.com/linux-test-project/ltp>`_
+* `Releases <https://github.com/linux-test-project/ltp/releases>`_
+* `Mailing List <http://lists.linux.it/listinfo/ltp>`_
+* `Working patches (patchwork) <https://patchwork.ozlabs.org/project/ltp/list/>`_
+* `Working patches (lore.kernel.org) <https://lore.kernel.org/ltp>`_
+* `#ltp @ libera chat <https://libera.chat/>`_
+
+.. toctree::
+ :maxdepth: 3
+ :hidden:
+ :caption: For users
+
+ users/quick_start
+ users/setup_tests
+ users/supported_systems
+
+.. toctree::
+ :maxdepth: 3
+ :hidden:
+ :caption: For developers
+
+ developers/setup_mailinglist
+ developers/writing_tests
+ developers/test_case_tutorial
+ developers/api_c_tests
+ developers/api_shell_tests
+ developers/api_network_tests
+ developers/api_kvm_tests
+ developers/ltp_library
+ developers/build_system
+ developers/debugging
+
+.. toctree::
+ :maxdepth: 3
+ :hidden:
+ :caption: For maintainers
+
+ maintainers/patch_review
+ maintainers/ltp_release_procedure
+
+For users
+---------
+
+.. descriptions here are active
+
+:doc:`users/quick_start`
+ How to build and use LTP framework in few steps
+
+:doc:`users/setup_tests`
+ How to setup tests execution
+
+:doc:`users/supported_systems`
+ A list of supported technologies by the LTP framework
+
+For developers
+--------------
+
+.. descriptions here are active
+
+:doc:`developers/setup_mailinglist`
+ How to configure git and to start sending patches via ``git send-email``
+
+:doc:`developers/writing_tests`
+ Starting guide on writing tests
+
+:doc:`developers/test_case_tutorial`
+ A tutorial showing how to write a test from scratch using C API
+
+:doc:`developers/api_c_tests`
+ Walk through the C API features
+
+:doc:`developers/api_shell_tests`
+ Walk through the Shell API features
+
+:doc:`developers/api_network_tests`
+ Walk through the network API features
+
+:doc:`developers/api_kvm_tests`
+ Walk through the KVM API features
+
+:doc:`developers/ltp_library`
+ Developing new features in the LTP library
+
+:doc:`developers/build_system`
+ Short introduction to the LTP build system
+
+:doc:`developers/debugging`
+ How to debug LTP tests
+
+For maintainers
+---------------
+
+:doc:`maintainers/patch_review`
+ Steps to follow when reviewing patches
+
+:doc:`maintainers/ltp_release_procedure`
+ Steps to follow for a new LTP release
+
+
+Getting help
+------------
+To report a problem or suggest any feature, please write at ltp@lists.linux.it
diff --git a/doc_new/maintainers/ltp_release_procedure.rst b/doc_new/maintainers/ltp_release_procedure.rst
new file mode 100644
index 000000000..808ff2211
--- /dev/null
+++ b/doc_new/maintainers/ltp_release_procedure.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Release process
+===============
diff --git a/doc_new/maintainers/patch_review.rst b/doc_new/maintainers/patch_review.rst
new file mode 100644
index 000000000..1fb76b8bc
--- /dev/null
+++ b/doc_new/maintainers/patch_review.rst
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Patch reviewing
+===============
diff --git a/doc_new/requirements.txt b/doc_new/requirements.txt
new file mode 100644
index 000000000..59709ec72
--- /dev/null
+++ b/doc_new/requirements.txt
@@ -0,0 +1 @@
+sphinx-rtd-theme==2.0.0
diff --git a/doc_new/users/quick_start.rst b/doc_new/users/quick_start.rst
new file mode 100644
index 000000000..40d8dd9c6
--- /dev/null
+++ b/doc_new/users/quick_start.rst
@@ -0,0 +1,126 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Installation and tests execution
+================================
+
+Basics requirements to build LTP are the following:
+
+* git
+* autoconf
+* automake
+* make
+* gcc
+* m4
+* pkgconf / pkg-config
+* libc headers
+* linux headers
+
+.. code-block:: console
+
+ $ git clone https://github.com/linux-test-project/ltp.git
+ $ cd ltp
+ $ make autotools
+ $ ./configure
+
+.. note::
+
+ For optional library dependencies, take a look at the scripts inside ``ci/``
+ directory.
+
+Running single tests
+--------------------
+
+LTP provides the possibility to build and run single tests:
+
+.. code-block:: console
+
+ $ cd testcases/kernel/syscalls/foo
+ $ make
+ $ PATH=$PATH:$PWD ./foo01
+
+Shell testcases are a bit more complicated, since they need to setup ``PATH``
+as well as to compiled binary helpers:
+
+.. code-block:: console
+
+ $ cd testcases/lib
+ $ make
+ $ cd ../commands/foo
+ $ PATH=$PATH:$PWD:$PWD/../../lib/ ./foo01.sh
+
+Open Posix Testsuite has it's own build system which needs Makefiles to be
+generated first:
+
+.. code-block:: console
+
+ $ cd testcases/open_posix_testsuite/
+ $ make generate-makefiles
+ $ cd conformance/interfaces/foo
+ $ make
+ $ ./foo_1-1.run-test
+
+Compiling and installing all testcases
+--------------------------------------
+
+To compile all tests is really simple:
+
+.. code-block:: console
+
+ $ make
+
+ $ # install LTP inside /opt/ltp by default
+ $ make install
+
+.. note::
+
+ Some tests will be disabled if ``configure`` script won't find the build
+ dependencies.
+
+Running tests
+-------------
+
+To run all the test suites
+
+.. code-block:: console
+
+ $ cd /opt/ltp
+
+ $ # run syscalls testing suite
+ $ ./kirk -f ltp -r syscalls
+
+.. note::
+
+ Many test cases have to be executed as root.
+
+Test suites (e.g. syscalls) are defined in the ``runtest`` directory. Each file
+contains a list of test cases in a simple format.
+
+Each test case has its own executable or script that can directly executed:
+
+.. code-block:: console
+
+ $ testcases/bin/abort01
+
+ $ # some tests have arguments
+ $ testcases/bin/mesgq_nstest -m none
+
+ $ # vast majority of tests have a help
+ $ testcases/bin/ioctl01 -h
+
+ $ # Many require certain environment variables to be set
+ $ LTPROOT=/opt/ltp PATH="$PATH:$LTPROOT/testcases/bin" testcases/bin/wc01.sh
+
+Most commonly, the ``PATH`` variable needs to be set and also ``LTPROOT``, but
+there are a number of other variables which usually ``kirk`` sets for you.
+
+.. note::
+
+ All shell scripts need the ``PATH`` to be set. However, this is not limited
+ to shell scripts and some C based tests need environment variables as well.
+ They usually raise a configuration error when this is needed.
+
+Network tests
+-------------
+
+Network tests usually require a certain setup that is described in
+``testcases/network/README.md``.
diff --git a/doc_new/users/setup_tests.rst b/doc_new/users/setup_tests.rst
new file mode 100644
index 000000000..910cc2315
--- /dev/null
+++ b/doc_new/users/setup_tests.rst
@@ -0,0 +1,104 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Tests setup
+===========
+
+The internal LTP library provides a set of features that permits to customize
+tests behaviour by setting environment variables and using specific tests
+arguments.
+
+Library environment variables
+-----------------------------
+
+Following environment variables are expected to be set by LTP users. Therefore,
+with some exceptions, they have ``LTP_`` prefix. Environment variables with
+``TST_`` prefix are used inside LTP shell API and should **not** be set by
+users.
+
+.. list-table::
+ :header-rows: 1
+
+ * - Variable
+ - Note
+
+ * - KCONFIG_PATH
+ - The path to the kernel config file, (if not set, it tries the usual paths
+ ``/boot/config-RELEASE`` or ``/proc/config.gz``)
+
+ * - KCONFIG_SKIP_CHECK
+ - Skip kernel config check if variable set (not set by default)
+
+ * - LTPROOT
+ - Prefix for installed LTP. **Should be always set**, since some tests
+ need it to use data files (``LTP_DATAROOT``). LTP is by default installed
+ into ``/opt/ltp``
+
+ * - LTP_COLORIZE_OUTPUT
+ - By default LTP colorizes it's output unless it's redirected to a pipe or
+ file. Force colorized output behaviour: ``y`` or ``1``: always colorize,
+ ``n`` or ``0``: never colorize.
+
+ * - LTP_DEV
+ - Path to the block device to be used. C Language: ``.needs_device = 1``.
+ Shell language: ``TST_NEEDS_DEVICE=1``.
+
+ * - LTP_SINGLE_FS_TYPE
+ - Testing only - specifies filesystem instead all supported
+ (for tests with ``.all_filesystems``).
+
+ * - LTP_DEV_FS_TYPE
+ - Filesystem used for testing (default: ``ext2``).
+
+ * - LTP_TIMEOUT_MUL
+ - Multiplies timeout, must be number >= 0.1 (> 1 is useful for slow
+ machines to avoid unexpected timeout).
+
+ * - LTP_RUNTIME_MUL
+ - Multiplies maximal test iteration runtime. Tests that run for more than a
+ second or two are capped on runtime. You can scale the default runtime
+ both up and down with this multiplier. This is not yet implemented in the
+ shell API.
+
+ * - LTP_VIRT_OVERRIDE
+ - Overrides virtual machine detection in the test library. Setting it to
+ empty string, tella the library that system is not a virtual machine.
+ Other possible values are ``kvm``, ``xen``, ``zvm`` and ``microsoft``
+ that describe different types supervisors.
+
+ * - PATH
+ - It's required to addjust path: ``PATH="$PATH:$LTPROOT/testcases/bin"``
+
+ * - TMPDIR
+ - Base directory for template directory (C language: ``.needs_tmpdir = 1``
+ and shell: ``TST_NEEDS_TMPDIR=1``). Must be an absolute path (default:
+ '/tmp').
+
+ * - LTP_NO_CLEANUP
+ - Disable running test cleanup (defined in ``TST_CLEANUP``).
+ Shell API only.
+
+ * - LTP_ENABLE_DEBUG
+ - Enable debug info (value ``1`` or ``y``).
+
+Test execution time and timeout
+-------------------------------
+
+The limit on how long a test can run does compose of two parts: ``max_runtime``
+and ``timeout``. The limit does apply to a single test variant. That means, for
+example, that tests which run for all available filesystems will apply this
+limit for a single filesystem only.
+
+The ``max_runtime`` is a cap on how long the ``run()`` function can take and for
+most testcases this part is set to zero. For tests that do run for more than a
+second or two the ``max_runtime`` has to be defined and the ``run()`` function
+has to check actively how much runtime is left.
+
+Test runtime can be scaled up and down with ``LTP_RUNTIME_MUL`` environment
+variable or set on a commandline by the ``-I`` parameter. Hoewever,
+setting the runtime too low will cause long running tests to exit prematurely,
+possibly before having a chance to actually test anything.
+
+The timeout is a limit for test setup and cleanup and it's also a safety
+margin for the runtime accounting. It's currently set to 30 seconds but it may
+change later. If your target machine is too slow, it can be scaled up with the
+``LTP_TIMEOUT_MUL`` environment variable.
diff --git a/doc_new/users/supported_systems.rst b/doc_new/users/supported_systems.rst
new file mode 100644
index 000000000..6220a96e0
--- /dev/null
+++ b/doc_new/users/supported_systems.rst
@@ -0,0 +1,110 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Supported systems
+=================
+
+Kernel version
+--------------
+
+Minimal supported kernel version is **3.10**.
+
+Oldest tested distributions
+---------------------------
+
+.. list-table::
+ :header-rows: 1
+
+ * - Distro
+ - Kernel
+ - glibc
+ - gcc
+ - clang
+
+ * - openSUSE Leap 42.2
+ - 4.4
+ - 2.22
+ - 4.8.5
+ - \-
+
+ * - Ubuntu 18.04 LTS bionic
+ - 4.15
+ - 2.27
+ - 7.3.0
+ - \-
+
+ * - Debian 11 (bullseye)
+ - 5.10
+ - 2.31
+ - 10.2.1
+ - 11.0.1
+
+ * - RHEL-7.9 Maipo
+ - 3.10
+ - 4.8.5
+ - 2.17
+ - \-
+
+Older distributions are not officially supported, which means that it
+may or may not work. It all depends on your luck. It should be possible
+to compile latest LTP even on slightly older distributions than we
+support with a few manual tweaks, e.g. disabling manually tests for
+newly added syscalls, etc. **Trivial fixes/workarounds may be accepted,
+but users are encouraged to move to a newer distro.**
+
+If latest LTP cannot be compiled even with some amount of workarounds,
+you may result to older LTP releases, however these are **not** supported
+in any way. Also if you are trying to run LTP on more than 10 years old
+distribution you may as well reconsider you life choices.
+
+Tested architectures
+--------------------
+
+.. list-table::
+ :header-rows: 1
+
+ * - Architecture
+ - Build
+
+ * - x86_64
+ - native
+
+ * - x86 emulation
+ - native
+
+ * - aarch64
+ - cross compilation
+
+ * - ppc64le
+ - cross compilation
+
+ * - s390x
+ - cross compilation
+
+Supported C libraries
+---------------------
+
+.. list-table::
+ :header-rows: 1
+
+ * - C library
+ - Note
+
+ * - `glibc <https://www.gnu.org/software/libc/>`_
+ - Targeted libc, tested both compilation and actual test results.
+
+ * - `uClibc-ng <https://uclibc-ng.org/>`_
+ - Although not being tested, it should work as it attempt to maintain a glibc compatible interface.
+
+ * - `uClibc <https://www.uclibc.org/>`_
+ - Older uClibc might have problems.
+
+ * - `musl <https://musl.libc.org/>`_
+ - Not yet fully supported. Check ``ci/alpine.sh`` script.
+
+ * - Android
+ - Please use `AOSP fork <https://android.googlesource.com/platform/external/ltp>`_
+
+C version
+---------
+
+LTP is compiled with ``-std=gnu99``.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 02/10] Add new mailing list setup documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 01/10] Add the new basics documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 03/10] Rewrite writing tests guidelines documentation Andrea Cervesato
` (8 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Mailing list setup documentation explains how to setup the email in
order to send patches to the LTP mailing list.
---
doc_new/developers/setup_mailinglist.rst | 46 ++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/doc_new/developers/setup_mailinglist.rst b/doc_new/developers/setup_mailinglist.rst
index b7320b9a2..1eee0ae9f 100644
--- a/doc_new/developers/setup_mailinglist.rst
+++ b/doc_new/developers/setup_mailinglist.rst
@@ -2,3 +2,49 @@
Setting up the Mailing list
===========================
+
+Before using ``git send-email``, you need to set up your email client to send
+emails from the command line. This typically involves configuring an SMTP server
+and authentication details.
+
+Open a terminal and configure Git with your email settings using the following
+commands:
+
+.. code-block:: bash
+
+ git config --global sendemail.from "Your Name <your_email@example.com>"
+ git config --global sendemail.smtpserver "smtp.example.com"
+ git config --global sendemail.smtpuser "your_email@example.com"
+ git config --global sendemail.smtpserverport 587
+ git config --global sendemail.smtpencryption tls
+
+Replace ``smtp.example.com`` with the SMTP server address provided by your email
+provider. Replace ``your_email@example.com`` with your email address. Adjust the
+SMTP port and encryption settings according to your email provider's
+requirements.
+
+To test the configuration you can use ``--dry-run`` parameter.
+
+.. code-block:: bash
+
+ git send-email --dry-run --to "ltp@lists.linux.it" --subject "Test Email" --body "This is a test email." HEAD^
+
+Depending on your SMTP server's configuration, you may need to authenticate
+before sending emails. If required, configure authentication settings using:
+
+.. code-block:: bash
+
+ git config --global sendemail.smtpuser "your_email@example.com"
+ git config --global sendemail.smtppass "your_password"
+
+Replace ``your_email@example.com`` with your email address and ``your_password``
+with your email account password.
+
+For any corner case, please take a look at the
+`email + git <https://git-send-email.io/>`_ documentation.
+
+.. note::
+
+ This method still works in most of the cases, but nowadays we often
+ require to setup a two factor authentication. If this is the case, please
+ consider setting up Git accordingly.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 03/10] Rewrite writing tests guidelines documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 01/10] Add the new basics documentation Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 02/10] Add new mailing list setup documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 04/10] Refactor patch review " Andrea Cervesato
` (7 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The writing tests guidelines documentation has been adapted into
the RST format, as well as fixing typos and syntax errors.
make check errors messages are documented here, after being
previously inside user guidelines.
---
doc_new/developers/writing_tests.rst | 517 +++++++++++++++++++++++++++++++++++
1 file changed, 517 insertions(+)
diff --git a/doc_new/developers/writing_tests.rst b/doc_new/developers/writing_tests.rst
index ba0188aee..7b4b70bd0 100644
--- a/doc_new/developers/writing_tests.rst
+++ b/doc_new/developers/writing_tests.rst
@@ -3,6 +3,343 @@
Writing tests
=============
+This document describes LTP guidelines and it's intended for anybody who wants
+to write or to modify a LTP testcase. It's not a definitive guide and it's not,
+by any means, a substitute for common sense.
+
+Guide to clean and understandable code
+--------------------------------------
+
+Testcases require that the source code is easy to follow. When a test starts to
+fail, the failure has to be analyzed and clean test codebase makes this task
+much easier and quicker.
+
+Keep things simple
+~~~~~~~~~~~~~~~~~~
+
+It's worth to keep testcases simple or, better, as simple as possible.
+
+The kernel and libc are tricky beasts and the complexity imposed by their
+interfaces is quite high. Concentrate on the interface you want to test and
+follow the UNIX philosophy.
+
+It's a good idea to make the test as self-contained as possible too, ideally
+tests should not depend on tools or libraries that are not widely available.
+
+Do not reinvent the wheel!
+
+* Use LTP standard interface
+* Do not add custom PASS/FAIL reporting functions
+* Do not write Makefiles from scratch, use LTP build system instead
+* Etc.
+
+Keep functions and variable names short
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Choosing a good name for an API functions or even variables is a difficult
+task do not underestimate it.
+
+There are a couple of customary names for different things that help people to
+understand code, for example:
+
+* For loop variables are usually named with a single letter ``i``, ``j``, ...
+* File descriptors ``fd`` or ``fd_foo``.
+* Number of bytes stored in file are usually named as ``size`` or ``len``
+* Etc.
+
+Do not overcomment
+~~~~~~~~~~~~~~~~~~
+
+Comments can sometimes save your day, but they can easily do more harm than
+good. There has been several cases where comments and actual implementation
+drifted slowly apart which yielded into API misuses and hard to find bugs.
+Remember there is only one thing worse than no documentation: wrong
+documentation.
+
+Ideally, everybody should write code that is obvious, which unfortunately isn't
+always possible. If there is a code that requires to be commented, keep it
+short and to the point. These comments should explain *why* and not *how*
+things are done.
+
+Never ever comment the obvious.
+
+In case of LTP testcases, it's customary to add an asciidoc formatted comment
+paragraph with highlevel test description at the beginning of the file right
+under the GPL SPDX header. This helps other people to understand the overall
+goal of the test before they dive into the technical details. It's also
+exported into generated documentation hence it should mostly explain what is
+tested.
+
+DRY (Code duplication)
+~~~~~~~~~~~~~~~~~~~~~~
+
+Copy & paste is a good servant but very poor master. If you are about to copy a
+large part of the code from one testcase to another, think what would happen if
+you find bug in the code that has been copied all around the tree. What about
+moving it to a library instead?
+
+The same goes for short but complicated parts, whenever you are about to copy &
+paste a syscall wrapper that packs arguments accordingly to machine
+architecture or similarly complicated code, put it into a header instead.
+
+C coding style
+--------------
+
+LTP adopted `Linux kernel coding style <https://www.kernel.org/doc/html/latest/process/coding-style.html>`_.
+Run ``make check`` in the test's directory and/or use ``make check-$TCID``, it
+uses (among other checks) our vendored version of
+`checkpatch.pl <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl>`_
+script from kernel git tree.
+
+.. note::
+ If ``make check`` does not report any problems, the code still may be wrong
+ as all tools used for checking only look for common mistakes.
+
+The following linting code can be found when we run ``make check``:
+
+.. list-table::
+ :header-rows: 1
+
+ * - Linting code
+ - Message
+ - Explanation
+
+ * - LTP-001
+ - Library source files have ``tst_`` prefix
+ - API source code is inside headers in ``include/{empty}*.h``,
+ ``include/lapi/{empty}*.h`` (backward compatibility for old kernel and
+ libc) and C sources in ``lib/{empty}*.c``. Files must have ``tst_``
+ prefix.
+
+ * - LTP-002
+ - ``TST_RET`` and ``TST_ERR`` are never modified by test library functions
+ - The test author is guaranteed that the test API will not modify these
+ variables. This prevents silent errors where the return value and
+ errno are overwritten before the test has chance to check them.
+
+ The macros which are clearly intended to update these variables. That
+ is ``TEST`` and those in ``tst_test_macros.h``. Are of course allowed to
+ update these variables.
+
+ * - LTP-003
+ - Externally visible library symbols have the ``tst_`` prefix
+ - Functions, types and variables in the public test API should have the
+ ``tst_`` prefix. With some exceptions for symbols already prefixed with
+ ``safe_`` or ``ltp_``.
+
+ Static (private) symbols should not have the prefix.
+
+ * - LTP-004
+ - Test executable symbols are marked ``static``
+ - Test executables should not export symbols unnecessarily. This means
+ that all top-level variables and functions should be marked with the
+ ``static`` keyword. The only visible symbols should be those included
+ from shared object files.
+
+ * - LTP-005
+ - Array must terminate with a sentinel value (i.e. ``NULL`` or ``{}``)
+ - When defining arrays in the ``struct tst_test`` structure, we need to
+ end the array items with a sentinel ``NULL`` value.
+
+Shell coding style
+------------------
+
+When writing testcases in shell, write in *portable shell* only, it's a good
+idea to try to run the test using alternative shell (alternative to bash, for
+example dash) too.
+
+*Portable shell* means Shell Command Language as defined by POSIX with an
+exception of few widely used extensions, namely **local** keyword used inside of
+functions and ``-o`` and ``-a`` test parameters (that are marked as obsolete in
+POSIX).
+
+You can either try to run the testcases in Debian which has ``/bin/sh`` pointing
+to ``dash`` by default, or to install ``dash`` on your favorite distribution,
+then use it to run tests. If your distribution lacks ``dash`` package, you can
+always compile it from `sources <http://gondor.apana.org.au/~herbert/dash/files/>`_.
+
+Run ``make check`` in the test's directory and/or use ``make check-$TCID.sh``.
+It uses (among other checks) our vendored version of
+`checkbashism.pl <https://salsa.debian.org/debian/devscripts/raw/master/scripts/checkbashisms.pl>`_
+from Debian that is used to check for non-portable shell code.
+
+.. note::
+
+ If ``make check`` does not report any problems the code still may be wrong,
+ as ``checkbashisms.pl`` is used for checking only common mistakes.
+
+Here there are some common sense style rules for shell
+
+* Keep lines under 80 chars
+* Use tabs for indentation
+* Keep things simple, avoid unnecessary subshells
+* Don't do confusing things (i.e. don't name your functions like common shell
+ commands, etc.)
+* Quote variables
+* Be consistent
+
+3 Backwards compatibility
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+LTP test should be as backward compatible as possible. Think of an enterprise
+distributions with long term support (more than five years since the initial
+release) or of an embedded platform that needs to use several years old
+toolchain supplied by the manufacturer.
+
+Therefore LTP test for more current features should be able to cope with older
+systems. It should at least compile fine and if it's not appropriate for the
+configuration it should return ``TCONF``.
+
+There are several types of checks we use:
+
+* The *configure script* is usually used to detect availability of a function
+ declarations in system headers. It's used to disable tests at compile time or
+ to enable fallback definitions.
+
+* Checking the ``errno`` value is another type of runtime check. Most of the
+ syscalls returns either ``EINVAL`` or ``ENOSYS`` when syscall was not
+ implemented or was disabled upon kernel compilation.
+
+* LTP has kernel version detection that can be used to disable tests at runtime.
+ Unfortunately, the kernel version does not always corresponds to a well
+ defined feature set, as distributions tend to backport hundreds of patches
+ while the kernel version stays the same. Use with caution.
+
+* Lately, we added a kernel ``.config`` parser. A test can define a boolean
+ expression of kernel config variables that has to be satisfied in order to run
+ a test. At the moment, this is mostly used for kernel namespaces.
+
+* Sometimes it makes sense to define a few macros instead of creating a
+ configure test. One example is Linux specific POSIX clock ids in
+ ``include/lapi/posix_clocks.h``
+
+Dealing with messed up legacy code
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+LTP still contains a lot of old and messy code and we are cleaning it up as
+fast as we can but, despite the decade of efforts, there is still a lot of it.
+If you start modifying old or a messy testcase and your changes are more
+complicated than simple typo fixes, you should convert the test into a new
+library first.
+
+It's also much easier to review the changes if you split them into a smaller
+logical groups. The same goes for moving files: if you need to rename or to move
+files, do it in a separate patch.
+
+License
+~~~~~~~
+
+Code contributed to LTP should be licensed under GPLv2+ (GNU GPL version 2 or
+any later version).
+
+Use ``SPDX-License-Identifier: GPL-2.0-or-later``
+
+LTP Structure
+-------------
+
+The structure of LTP is quite simple. Each test is a binary written either in
+portable shell or C. The test gets a configuration via environment variables
+and/or command line parameters, it prints additional information into the
+stdout and reports overall success/failure via the exit value.
+
+Tests are generally placed under the ``testcases/`` directory. Everything that
+is a syscall or (slightly confusingly) libc syscall wrapper, goes under
+``testcases/kernel/syscalls/``.
+
+There is also ``testcases/open_posix_testsuite/`` which is a well maintained
+fork of the Open POSIX testsuite project, that has been dead since 2005.
+
+We also have a number of directories with tests for more specific features, such
+as containers, etc.
+
+Runtest Files
+~~~~~~~~~~~~~
+
+The list of tests to be executed is stored in runtest files under the
+``runtest/`` directory. The default set of runtest files to be executed is
+stored in ``scenario_groups/default``. When you add a test, you should add
+corresponding entries into some runtest file(s) as well.
+
+Each line of runtest file contains one test. The first item is the test name.
+All other items, separated by space will be executed as a command.
+
+.. code-block:: bash
+
+ shell_test01 echo "SUCCESS" | shell_pipe01.sh
+ splice02 splice02 -s 20
+
+Blank lines and lines starting with a ``#`` (comments) are ignored.
+
+Syscalls tests, placed under ``testcases/kernel/syscalls/``, use
+``runtest/syscalls`` file. For kernel related tests for memory management we
+have ``runtest/mm``, etc.
+
+.. note::
+
+ runtest files should have one entry per a test. Creating a
+ wrapper that runs all your tests and adding it as a single test
+ into runtest file is strongly discouraged.
+
+Datafiles
+---------
+
+If your test needs data files, these should be put into a subdirectory
+named ``datafiles`` and installed into the ``testcases/data/$TCID`` directory.
+This will require to add ``INSTALL_DIR := testcases/data/TCID`` into
+``datafiles/Makefile``.
+
+You can obtain path to datafiles via ``$TST_DATAROOT`` provided by ``test.sh``
+or via C function ``tst_dataroot()`` provided by libltp:
+
+.. code-block:: c
+
+ const char *dataroot = tst_dataroot();
+
+Datafiles can also be accessed as ``$LTPROOT/testcases/data/$TCID/...``,
+but ``$TST_DATAROOT`` and ``tst_dataroot()`` are preferred, as these can be used
+when running testcases directly in git tree as well as from install location.
+
+Subexecutables
+~~~~~~~~~~~~~~
+
+If your test needs to execute a binary, place it in the same directory of the
+testcase and name the binary with ``$TESTNAME_`` prefix, where ``$TESTNAME`` is
+the name of the test binary. Once the test is executed by the framework, the
+path to the directory with all LTP binaries is added to the ``$PATH`` and you
+can execute it via its name.
+
+.. note::
+
+ If you need to execute a test from the LTP tree, you can add ``PATH`` to
+ the current directory with ``PATH="$PATH:$PWD" ./foo01``.
+
+Test Contribution Checklist
+---------------------------
+
+#. Test compiles and it runs fine (check with ``-i 10`` and ``-i 0`` too)
+#. ``make check`` should not emit any warnings for the test you are working on
+ (hint: run it in the test's directory and/or use ``make check-$TCID``)
+#. The runtest entries are in place
+#. New test binaries are added into the corresponding ``.gitignore`` files
+#. Patches apply over the latest git
+
+About .gitignore files
+~~~~~~~~~~~~~~~~~~~~~~
+
+There are numerous ``.gitignore`` files in the LTP tree. Usually, there is a
+``.gitignore`` file for a group of tests. The reason of this setup is simple:
+it's easier to maintain a ``.gitignore`` file per tests' directory, rather
+than having a single file in the project root directory. In this way, we don't
+have to update all the gitignore files when moving directories, and they get
+deleted automatically when a directory with tests is removed.
+
+Testing pre-release kernel features
+-----------------------------------
+
+Tests for features not yet in the mainline kernel release are accepted. However,
+they must be added only to the **staging** runtest file. Once a feature is part
+of the stable kernel ABI, the associated test must be moved out of staging.
+
Testing builds with GitHub Actions
----------------------------------
@@ -18,3 +355,183 @@ check ``.github/workflows/ci.yml``.
different distributions on their **newest releases**.
The CI also checks for code linting, running ``make check`` in the whole
LTP project.
+
+LTP C And Shell Test API Comparison
+-----------------------------------
+
+.. list-table::
+ :header-rows: 1
+
+ * - C API ``struct tst_test`` members
+ - Shell API ``$TST_*`` variables
+
+ * - .all_filesystems
+ - TST_ALL_FILESYSTEMS
+
+ * - .bufs
+ - \-
+
+ * - .caps
+ - \-
+
+ * - .child_needs_reinit
+ - not applicable
+
+ * - .cleanup
+ - TST_CLEANUP
+
+ * - .dev_extra_opts
+ - TST_DEV_EXTRA_OPTS
+
+ * - .dev_fs_opts
+ - TST_DEV_FS_OPTS
+
+ * - .dev_fs_type
+ - TST_FS_TYPE
+
+ * - .dev_min_size
+ - not applicable
+
+ * - .format_device
+ - TST_FORMAT_DEVICE
+
+ * - .max_runtime
+ - \-
+
+ * - .min_cpus
+ - not applicable
+
+ * - .min_kver
+ - TST_MIN_KVER
+
+ * - .min_mem_avail
+ - not applicable
+
+ * - .mnt_flags
+ - TST_MNT_PARAMS
+
+ * - .min_swap_avail
+ - not applicable
+
+ * - .mntpoint | .mnt_data
+ - TST_MNTPOINT
+
+ * - .mount_device
+ - TST_MOUNT_DEVICE
+
+ * - .needs_cgroup_ctrls
+ - \-
+
+ * - .needs_checkpoints
+ - TST_NEEDS_CHECKPOINTS
+
+ * - .needs_cmds
+ - TST_NEEDS_CMDS
+
+ * - .needs_devfs
+ - \-
+
+ * - .needs_device
+ - TST_NEEDS_DEVICE
+
+ * - .needs_drivers
+ - TST_NEEDS_DRIVERS
+
+ * - .needs_kconfigs
+ - TST_NEEDS_KCONFIGS
+
+ * - .needs_overlay
+ - \-
+
+ * - .needs_rofs
+ - \-
+
+ * - .needs_root
+ - TST_NEEDS_ROOT
+
+ * - .needs_tmpdir
+ - TST_NEEDS_TMPDIR
+
+ * - .options
+ - TST_PARSE_ARGS | TST_OPTS
+
+ * - .resource_files
+ - \-
+
+ * - .restore_wallclock
+ - not applicable
+
+ * - .sample
+ - \-
+
+ * - .save_restore
+ - \-
+
+ * - .scall
+ - not applicable
+
+ * - .setup
+ - TST_SETUP
+
+ * - .skip_filesystems
+ - TST_SKIP_FILESYSTEMS
+
+ * - .skip_in_compat
+ - \-
+
+ * - .skip_in_lockdown
+ - TST_SKIP_IN_LOCKDOWN
+
+ * - .skip_in_secureboot
+ - TST_SKIP_IN_SECUREBOOT
+
+ * - .supported_archs
+ - not applicable
+
+ * - .tags
+ - \-
+
+ * - .taint_check
+ - \-
+
+ * - .tcnt
+ - TST_CNT
+
+ * - .tconf_msg
+ - not applicable
+
+ * - .test | .test_all
+ - TST_TESTFUNC
+
+ * - .test_variants
+ - \-
+
+ * - .timeout
+ - TST_TIMEOUT
+
+ * - .tst_hugepage
+ - not applicable
+
+ * - .ulimit
+ - not applicable
+
+ * - not applicable
+ - TST_NEEDS_KCONFIGS_IFS
+
+ * - not applicable
+ - TST_NEEDS_MODULE
+
+ * - not applicable
+ - TST_POS_ARGS
+
+ * - not applicable
+ - TST_USAGE
+
+.. list-table::
+ :header-rows: 1
+
+ * - C API other structs
+ - Shell API ``$TST_*`` variables
+
+ * - struct tst_device
+ - TST_DEVICE
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 04/10] Refactor patch review guidelines documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (2 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 03/10] Rewrite writing tests guidelines documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 05/10] Refactor LTP release procedure documentation Andrea Cervesato
` (6 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
Patch review guidelines documentation has been refactored adding RST
syntax and fixing a couple of typos.
---
doc_new/maintainers/patch_review.rst | 171 ++++++++++++++++++++++++++++++++++-
1 file changed, 169 insertions(+), 2 deletions(-)
diff --git a/doc_new/maintainers/patch_review.rst b/doc_new/maintainers/patch_review.rst
index 1fb76b8bc..81d198532 100644
--- a/doc_new/maintainers/patch_review.rst
+++ b/doc_new/maintainers/patch_review.rst
@@ -1,4 +1,171 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
-Patch reviewing
-===============
+Patch review
+============
+
+Anyone can and should review patches. It's the only way to get good at patch
+review and for the project to scale. For this reason, we have a short guide on
+what to do during the review process.
+
+Goals of patch review
+---------------------
+
+#. Prevent false positive test results
+#. Prevent false negative test results
+#. Keep the code as simple as possible, but no simpler
+
+How to find clear errors
+------------------------
+
+A clear error is one where there is unlikely to be any argument if you
+provide evidence of it. Evidence being an error trace or logical proof
+that an error will occur in a common situation.
+
+The following are examples and may not be appropriate for all tests.
+
+* Merge the patch locally. It should apply cleanly to master.
+* Compile the patch with default and non-default configurations.
+
+ * Use sanitizers e.g. undefined behaviour, address.
+ * Compile on non-x86
+ * Compile on x86 with ``-m32``
+ * Compile testing patches with GitHub Actions in LTP repo fork can cover
+ various distros/architectures
+
+* Use ``make check``
+* Run effected tests in a VM
+
+ * Use single vCPU
+ * Use many vCPUs and enable NUMA
+ * Restrict RAM to < 1GB.
+
+* Run effected tests on an embedded device
+* Run effected tests on non-x86 machine in general
+* Run reproducers on a kernel where the bug is present
+* Run tests with ``-i0`` and ``-i2``
+* Compare usage of system calls with man page descriptions
+* Compare usage of system calls with kernel code
+* Double check commit message
+* Search the LTP library for existing helper functions
+* Check doc formatting, i.e. ``make doc && chromium docparse/metadata.html``
+
+How to find subtle errors
+-------------------------
+
+A subtle error is one where you can expect some argument because you
+do not have clear evidence of an error. It is best to state these as
+questions and not make assertions if possible.
+
+Although if it is a matter of style or "taste" then senior maintainers
+can assert what is correct to avoid bike shedding.
+
+* Ask what happens if there is an error, could it be debugged just
+ with the test output?
+* Are we testing undefined behavior?
+
+ * Could future kernel behaviour change without "breaking userland"?
+ * Does the kernel behave differently depending on hardware?
+ * Does it behave differently depending on kernel configuration?
+ * Does it behave differently depending on the compiler?
+ * Would it behave differently if the order of checks on syscall parameters
+ changed in the kernel?
+
+* Will it scale to tiny and huge systems?
+
+ * What happens if there are 100+ CPUs?
+ * What happens if each CPU core is very slow?
+ * What happens if there are 2TB of RAM?
+
+* Are we repeating a pattern that can be turned into a library function?
+* Is a single test trying to do too much?
+* Could multiple similar tests be merged?
+* Race conditions
+
+ * What happens if a process gets preempted?
+ * Could checkpoints or fuzzsync by used instead?
+ * Note, usually you can insert a sleep to prove a race condition
+ exists however finding them is hard
+
+* Is there a simpler way to achieve the same kernel coverage?
+
+How to get patches merged
+-------------------------
+
+Once you think a patch is good enough you should add your ``Reviewed-by``
+and/or ``Tested-by`` tags. This means you will get some credit for getting
+the patch merged. Also some blame if there are problems.
+
+If you ran the test you can add the ``Tested-by`` tag. If you read the
+code or used static analysis tools on it, you can add the Reviewed-by
+tag.
+
+In addition you can expect others to review your patches and add their
+tags. This will speed up the process of getting your patches merged.
+
+Maintainers Checklist
+---------------------
+
+Patchset should be tested locally and ideally also in maintainer's fork in
+GitHub Actions on GitHub.
+
+.. note::
+
+ GitHub Actions do only build testing, passing the CI means only that
+ the test compiles fine on variety of different distributions and releases.
+
+The test should be executed at least once locally and should PASS as well.
+
+Commit messages should have
+
+* Author's ``Signed-off-by`` tag
+* Committer's ``Reviewed-by`` or ``Signed-off-by`` tag
+* Check also mailing lists for other reviewers / testers tags, notes and failure
+ reports
+* ``Fixes: hash`` if it fixes particular LTP commit
+* ``Fixes: #N`` if it fixes github issue number N, so it's automatically closed
+* LTP documentation should be kept up to date.
+
+After patch is accepted or rejected, set correct state and archive in the
+`LTP patchwork instance <https://patchwork.ozlabs.org/project/ltp/list/>`_.
+
+New tests
+---------
+
+New test should
+
+* Have a record in runtest file
+* Test should work fine with more than one iteration (e.g. run with ``-i 100``)
+* Run with ``-i 0`` to check that setup and cleanup are coded properly
+ (no test is being run)
+* Have a brief description
+* License: the default license for new tests is GPL v2 or later, use
+ ``GPL-2.0-or-later``; the licence for test (e.g. GPL-2.0) should not change
+ unless test is completely rewritten
+* Old copyrights should be kept unless test is completely rewritten
+
+C tests
+~~~~~~~
+
+* Use the new C API, implementing ``struct tst_test``
+* Test binaries are added into corresponding ``.gitignore`` files
+* Check coding style with ``make check``
+* Docparse documentation
+* If a test is a regression test it should include ``.tags`` in the
+ ``struct tst_test`` definition
+* When rewriting old tests, `uClinux <https://en.wikipedia.org/wiki/%CE%9CClinux>`_
+ support should be removed (project has been discontinued).
+ E.g. remove ``#ifdef UCLINUX``, replace ``FORK_OR_VFORK()`` with simple
+ ``fork()`` or ``SAFE_FORK()``.
+
+Shell tests
+~~~~~~~~~~~
+
+* Use new shell API
+* Check coding style with ``make check``
+* If a test is a regression test it should include related kernel or glibc
+ commits as a comment
+
+LTP library
+~~~~~~~~~~~
+
+For patchset touching the LTP library, follow the LTP library guidelines.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 05/10] Refactor LTP release procedure documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (3 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 04/10] Refactor patch review " Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 06/10] Refactor debugging documentation Andrea Cervesato
` (5 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The LTP release procedure documentation has been rewritten using the RST
format and some typo errors have been fixed.
---
doc_new/maintainers/ltp_release_procedure.rst | 169 ++++++++++++++++++++++++++
1 file changed, 169 insertions(+)
diff --git a/doc_new/maintainers/ltp_release_procedure.rst b/doc_new/maintainers/ltp_release_procedure.rst
index 808ff2211..f0bc01cee 100644
--- a/doc_new/maintainers/ltp_release_procedure.rst
+++ b/doc_new/maintainers/ltp_release_procedure.rst
@@ -2,3 +2,172 @@
Release process
===============
+
+Preparations
+------------
+
+The release procedure generally takes a few weeks. In the first week or two,
+patches that should go into the release are reviewed and possibly merged. These
+patches are either fixes or patches pointed out by the community.
+
+Patch review, when finished, is followed by a git freeze, which is a period
+where only fixes are pushed to the git. During that period community is
+expected to run a LTP pre-release tests, reports problems, and/or send fixes to
+the mailing list. In this period we are especially making sure that there are
+no regressions in the test results on a wide range of distributions and
+architectures.
+
+Once the stabilization period has ended the time has finally come to proceed
+with the release.
+
+Prepare the release notes
+-------------------------
+
+Part of the preparation is also to write the release notes, which are then
+added to the GitHub release and also sent as announcement to various mailing
+lists (see below).
+
+Have a look at `this release letter <https://lore.kernel.org/ltp/ZGNiQ1sMGvPU_ETp@yuki/>`_
+to get the idea how it should look.
+
+Tag the git and push changes to github
+--------------------------------------
+
+.. code-block:: bash
+
+ cd ltp
+ echo YYYYMMDD > VERSION
+ git commit -S -s -m 'LTP YYYYMMDD' VERSION
+ git tag -s -a YYYYMMDD -m 'LTP YYYYMMDD'
+ git push origin master:master
+ git push origin YYYYMMDD
+
+The string ``YYYYMMDD`` should be substituted to the current date.
+
+You can use `tools/tag-release.sh <https://github.com/linux-test-project/ltp/blob/master/tools/tag-release.sh>`_
+script to have the above automated process.
+It allows you to verify the tag before pushing it and does other checks.
+
+.. code-block:: bash
+
+ $ ./tools/tag-release.sh
+ ===== git push =====
+ new tag: 'YYYYMMDD', previous tag: '20230127'
+ tag YYYYMMDD
+ Tagger: Person-who-released LTP <foo@example.com>
+ Date: ...
+
+ LTP YYYYMMDD
+ -----BEGIN PGP SIGNATURE-----
+ ...
+ -----END PGP SIGNATURE-----
+
+ commit 3ebc2dfa85c2445bb68d8c0d66e33c4da1e1b3a7
+ gpg: using RSA key ...
+ ...
+ Primary key fingerprint: ...
+ Author: Person-who-released LTP <foo@example.com>
+ Date: ...
+
+ LTP YYYYMMDD
+
+ Signed-off-by: Person-who-released LTP <foo@example.com>
+
+ diff --git a/VERSION b/VERSION
+ index af4c41fec..ae488c0e7 100644
+ --- a/VERSION
+ +++ b/VERSION
+ @@ -1 +1 @@
+ -20230127
+ +YYYYMMDD
+
+ Please check tag and signature. Proceed? [N/y]: y
+ Pushing changes to upstream git. Proceed? [N/y]: y
+ ...
+ To github.com:linux-test-project/ltp.git
+ * [new tag] YYYYMMDD -> YYYYMMDD
+
+Prepare tarballs and metadata documentation
+-------------------------------------------
+
+The following procedure will show how to create the release archives and the
+metadata documentation:
+
+.. code-block:: bash
+
+ # clone already clonned git repository to new folder
+ cd ..
+ git clone ltp ltp-full-YYYYMMDD
+ cd ltp-full-YYYYMMDD
+
+ # update all submodules
+ git submodule update --init
+
+ # Generate configure script
+ make autotools
+
+ # Generate tarballs
+ cd ..
+ tar -cjf ltp-full-YYYYMMDD.tar.bz2 ltp-full-YYYYMMDD --exclude .git
+ tar -cJf ltp-full-YYYYMMDD.tar.xz ltp-full-YYYYMMDD --exclude .git
+
+ # Generate checksums
+ md5 ltp-full-YYYYMMDD.tar.xz > ltp-full-YYYYMMDD.tar.xz.md5
+ sha1 ltp-full-YYYYMMDD.tar.xz > ltp-full-YYYYMMDD.tar.xz.sha1
+ sha256sum ltp-full-YYYYMMDD.tar.xz > ltp-full-YYYYMMDD.tar.xz.sha256
+
+ # Generate metadata documentation
+ ./configure --with-metadata-generator=asciidoctor
+ make -C metadata
+ cp -v docparse/metadata.html ../metadata.YYYYMMDD.html
+
+You can use `tools/create-tarballs-metadata.sh <https://github.com/linux-test-project/ltp/blob/master/tools/create-tarballs-metadata.sh>`_
+script to have the above procedure automated. All generated files are placed
+in the ``ltp-release-YYYYMMDD`` directory.
+
+.. code-block:: bash
+
+ $ ./tools/create-tarballs-metadata.sh
+ ===== git clone =====
+ Cloning into 'ltp-full-YYYYMMDD'...
+ done.
+ ===== Update submodules =====
+ Submodule 'tools/kirk' (https://github.com/linux-test-project/kirk.git) registered for path 'tools/kirk'
+ ...
+ ===== Generate configure script =====
+ sed -n '1{s:LTP-:m4_define([LTP_VERSION],[:;s:$:]):;p;q}' VERSION > m4/ltp-version.m4
+ aclocal -I m4
+ ...
+ ===== Generate tarballs =====
+ ===== Generate checksums =====
+ ===== Generate metadata documentation =====
+ checking for a BSD-compatible install... /usr/bin/install -c
+ ...
+ 'docparse/metadata.html' -> '/home/foo/ltp-release-YYYYMMDD/metadata.YYYYMMDD.html'
+ Generated files are in '/home/foo/ltp-release-YYYYMMDD', upload them to github
+
+Upload the generated files to GitHub
+------------------------------------
+
+Click on `Releases <https://github.com/linux-test-project/ltp/releases>`_, then
+switch to `Tags <https://github.com/linux-test-project/ltp/tags>`_. Click on
+'Add release notes'. There should be 'Attach binaries ...' link at the
+bottom of the page.
+
+Don't forget to upload checksums for the tarballs and metadata documentation
+as well.
+
+Send release announcement
+-------------------------
+
+The announcement is sent to:
+
+* ltp at lists.linux.it
+* linux-kernel at vger.kernel.org
+* libc-alpha at sourceware.org
+
+CCed to:
+
+* lwn at lwn.net
+* akpm at linux-foundation.org
+* torvalds at linux-foundation.org
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 06/10] Refactor debugging documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (4 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 05/10] Refactor LTP release procedure documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 07/10] Refactor LTP library guidelines documentation Andrea Cervesato
` (4 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The debugging documentation section explains how to debug test binaries
and to enable TDEBUG messages on tests run.
---
doc_new/developers/debugging.rst | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/doc_new/developers/debugging.rst b/doc_new/developers/debugging.rst
index 18a4a67e9..7a4e6b2ad 100644
--- a/doc_new/developers/debugging.rst
+++ b/doc_new/developers/debugging.rst
@@ -3,8 +3,19 @@
Debugging
=========
-The new test library runs the actual test, i.e. the `test()` function in a
+This section explains some tricks which can be used to debug test binaries.
+
+Debug messages
+--------------
+
+The LTP framework currently supports ``TDEBUG`` flag test debug messages. These
+messages can be enabled using the ``-D`` test's argument.
+
+Tracing and debugging syscalls
+------------------------------
+
+The new test library runs the actual test (i.e. the ``test()`` function) in a
forked process. To get stack trace of a crashing test in gdb it's needed to
`set follow-fork-mode child <https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_25.html>`_.
-To trace the test, please use `strace -f` to enable tracing also for the
+To trace the test, please use ``strace -f`` to enable tracing also for the
forked processes.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 07/10] Refactor LTP library guidelines documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (5 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 06/10] Refactor debugging documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 08/10] Refactor build system documentation Andrea Cervesato
` (3 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The LTP library guidelines documentation has been refactored,
simplifying the overall structure, cleaning up the make check error
messages which were moved into the test writing guidelines.
---
doc_new/developers/ltp_library.rst | 42 ++++++++++++++++++++++++++++++++++++--
1 file changed, 40 insertions(+), 2 deletions(-)
diff --git a/doc_new/developers/ltp_library.rst b/doc_new/developers/ltp_library.rst
index b780b6870..85b69c11c 100644
--- a/doc_new/developers/ltp_library.rst
+++ b/doc_new/developers/ltp_library.rst
@@ -1,4 +1,42 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
-LTP Library development
-=======================
+LTP Library guidelines
+======================
+
+General Rules
+-------------
+
+When we extend library API, we need to apply the same general rules that we use
+when writing tests, plus:
+
+#. LTP library tests must go inside ``lib/newlib_tests`` directory
+#. LTP documentation has to be updated according to API changes
+
+Shell API
+---------
+
+API source code is in ``tst_test.sh``, ``tst_security.sh`` and ``tst_net.sh``
+(all in ``testcases/lib`` directory).
+
+Changes in the shell API should not introduce uncommon dependencies
+(use basic commands installed everywhere by default).
+
+Shell libraries
+~~~~~~~~~~~~~~~
+
+Aside from shell API libraries in ``testcases/lib``, it's worth putting
+common code for a group of tests into a shell library. The filename
+should end with ``_lib.sh`` and the library should load ``tst_test.sh`` or
+``tst_net.sh``.
+
+Shell libraries should have conditional expansion for ``TST_SETUP`` or
+``TST_CLEANUP``, to avoid surprises when test specific setup/cleanup function is
+redefined by shell library.
+
+.. code-block:: bash
+
+ # ipsec_lib.sh
+ # SPDX-License-Identifier: GPL-2.0-or-later
+ TST_SETUP="${TST_SETUP:-ipsec_lib_setup}"
+ ...
+ . tst_test.sh
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 08/10] Refactor build system documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (6 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 07/10] Refactor LTP library guidelines documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 09/10] Refactor test case tutorial documentation Andrea Cervesato
` (2 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The build system documentation has been refactored using RST format and
fixing typos or syntax errors. The last section referring to INSTALL and
TODO has been removed, since those are files which we are not
maintaining anymore.
---
doc_new/developers/build_system.rst | 209 ++++++++++++++++++++++++++++++++++++
1 file changed, 209 insertions(+)
diff --git a/doc_new/developers/build_system.rst b/doc_new/developers/build_system.rst
index cfcaf2d04..1a6ad4c56 100644
--- a/doc_new/developers/build_system.rst
+++ b/doc_new/developers/build_system.rst
@@ -2,3 +2,212 @@
Build system
============
+
+The following document briefly describes the steps and methodologies used for
+the new and improved Makefile system.
+
+The Problem
+-----------
+
+The problem with the old Makefile system is that it was very difficult to
+maintain and it lacked any sense of formal structure, thus developing for LTP
+and including new targets was more difficult than it should have been
+(maintenance). Furthermore, proper option-based cross-compilation was
+impossible due to the fact that the Makefiles didn't support a prefixing
+system, and the appropriate implicit / static rules hadn't been configured to
+compile into multiple object directories for out-of-tree build support (ease of
+use / functionality). Finally, there wasn't a means to setup dependencies
+between components, such that if a component required ``libltp.a`` in order to
+compile, it would go off and compile ``libltp.a`` first (ease of use).
+
+These items needed to be fixed to reduce maintenance nightmares for the
+development community contributing to LTP, and the project maintainers.
+
+Design
+------
+
+The system was designed such that including a single GNU Makefile compatible
+set in each new directory component is all that's essentially required to
+build the system.
+
+Say you had a directory like the following (with ``.c`` files in them which
+directly tie into applications, e.g. baz.c -> baz):
+
+.. code-block::
+
+ .../foo/
+ |--> Makefile
+ |
+ --> bar/
+ |
+ --> Makefile
+ |
+ --> baz.c
+
+.. code-block:: make
+ :caption: .../foo/Makefile
+
+ #
+ # Copyright disclaimer goes here -- please use GPLv2.
+ #
+
+ top_srcdir ?= ..
+
+ include $(top_srcdir)/include/mk/env_pre.mk
+ include $(top_srcdir)/include/mk/generic_trunk_target.mk
+
+.. code-block:: make
+ :caption: .../foo/bar/Makefile
+
+ #
+ # Copyright disclaimer goes here -- please use GPLv2.
+ #
+
+ top_srcdir ?= ../..
+
+ include $(top_srcdir)/include/mk/env_pre.mk
+ include $(top_srcdir)/include/mk/generic_leaf_target.mk
+
+Kernel Modules
+--------------
+
+Some of the tests need to build kernel modules, happily LTP has
+infrastructure for this.
+
+.. code-block:: make
+
+ ifneq ($(KERNELRELEASE),)
+
+ obj-m := module01.o
+
+ else
+
+ top_srcdir ?= ../../../..
+ include $(top_srcdir)/include/mk/testcases.mk
+
+ REQ_VERSION_MAJOR := 2
+ REQ_VERSION_PATCH := 6
+ MAKE_TARGETS := test01 test02 module01.ko
+
+ include $(top_srcdir)/include/mk/module.mk
+ include $(top_srcdir)/include/mk/generic_leaf_target.mk
+
+ endif
+
+This is a Makefile example that allows you to build kernel modules inside LTP.
+The prerequisites for the build are detected by the ``configure`` script.
+
+The ``REQ_VERSION_MAJOR`` and ``REQ_VERSION_PATCH`` describe minimal kernel
+version for which the build system tries to build the module.
+
+The build system is also forward compatible with changes in Linux kernel
+internal API so that, if module fails to build, the failure is ignored both on
+build and installation. If the userspace counterpart of the test fails to load
+the module because the file does not exists, the test is skipped.
+
+Note the ``ifneq($(KERNELRELEASE),)``. The reason it exists, it is that the
+Makefile is executed twice: once by LTP build system and once by kernel
+kbuild, see ``Documentation/kbuild/modules.rst`` in the Linux kernel tree for
+details on external module build.
+
+Make Rules and Make Variables
+-----------------------------
+
+When using make rules, avoid writing ad hoc rules like:
+
+.. code-block:: make
+
+ [prog]: [dependencies]
+ cc -I../../include $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(LDLIBS) \
+ -o [prog] [dependencies]
+
+This makes cross-compilation and determinism difficult, if not impossible.
+Besides, implicit rules are your friends and as long as you use ``MAKEOPTS=;``
+in the top-level caller (or do ``$(subst r,$(MAKEOPTS)``) to remove ``-r``),
+the compile will complete successfully, assuming all other prerequisites have
+been fulfilled (libraries, headers, etc).
+
+.. list-table::
+ :header-rows: 1
+
+ * - Variable
+ - Explaination
+
+ * - $(AR)
+ - The library archiver
+
+ * - $(CC)
+ - The system C compiler
+
+ * - $(CCP)
+ - The system C preprocessor
+
+ * - $(CFLAGS)
+ - C compiler flags
+
+ * - $(CPPFLAGS)
+ - Preprocessor flags, e.g. ``-I`` arguments
+
+ * - $(DEBUG_CFLAGS)
+ - Debug flags to pass to ``$(CC)``, ``-g``, etc
+
+ * - $(KVM_LD)
+ - Special linker for wrapping KVM payload binaries into linkable object
+ files. Defaults to ``$(LD)``. Change this variable if the KVM Makefile
+ fails to build files named ``*-payload.o``
+
+ * - $(LD)
+ - The system linker (typically ``$(CC)``, but not necessarily)
+
+ * - $(LDFLAGS)
+ - What to pass in to the linker, including ``-L`` arguments and other ld
+ arguments, apart from ``-l`` library includes (see ``$(LDLIBS)``).
+ This should be done in the ``$(CC)`` args passing style when
+ ``LD := $(CC)``, e.g. ``-Wl,-foo``, as opposed to ``-foo``
+
+ * - $(LDLIBS)
+ - Libraries to pass to the linker (e.g. ``-lltp``, etc)
+
+ * - $(LTPLDLIBS)
+ - LTP internal libraries i.e. these in libs/ directory
+
+ * - $(OPT_CFLAGS)
+ - Optimization flags to pass into the C compiler, ``-O2``, etc. If you
+ specify ``-O2`` or higher, you should also specify
+ ``-fno-strict-aliasing``, because of gcc fstrict-aliasing optimization
+ bugs in the tree optimizer. Search for **fstrict-aliasing optimization
+ bug** with your favorite search engine.
+
+ Examples of more recent bugs: tree-optimization/17510
+ and tree-optimization/39100.
+
+ Various bugs have occurred in the past due to buggy logic in the
+ tree-optimization portion of the gcc compiler, from 3.3.x to 4.4.
+
+ * - $(RANLIB)
+ - What to run after archiving a library
+
+ * - $(WCFLAGS)
+ - Warning flags to pass to ``$(CC)``, e.g. ``-Werror``, ``-Wall``, etc.
+
+Make System Variables
+---------------------
+
+A series of variables are used within the make system that direct what actions
+need to be taken. Rather than listing the variables here, please refer to the
+comments contained in ``.../include/mk/env_pre.mk``.
+
+Guidelines and Recommendations
+------------------------------
+
+Of course, GNU Make manual is the key to understand the Make system, but
+following manuals are probably the most important:
+
+* `Implicit Rules <http://www.gnu.org/software/make/manual/make.html#Implicit-Rules>`_
+* `Variables and Expansion <http://www.gnu.org/software/make/manual/make.html#Using-Variables>`_
+* `Origin Use <http://www.gnu.org/software/make/manual/make.html#Origin-Function>`_
+* `VPath Use <http://www.gnu.org/software/make/manual/make.html#Directory-Search>`_
+
+.. warning::
+
+ Rebuild from scratch before committing anything in the build system.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 09/10] Refactor test case tutorial documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (7 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 08/10] Refactor build system documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-20 16:20 ` [LTP] [PATCH 10/10] Introduce kernel-doc API support Andrea Cervesato
2024-03-21 8:25 ` [LTP] [PATCH 00/10] New LTP documentation Petr Vorel
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
The test case tutorial documentation has been refactored using the RST
format. Most hte text was already pretty good, so no major changes have
been made.
---
doc_new/developers/test_case_tutorial.rst | 1034 +++++++++++++++++++++++++++++
1 file changed, 1034 insertions(+)
diff --git a/doc_new/developers/test_case_tutorial.rst b/doc_new/developers/test_case_tutorial.rst
index 43563c234..ae7d59959 100644
--- a/doc_new/developers/test_case_tutorial.rst
+++ b/doc_new/developers/test_case_tutorial.rst
@@ -2,3 +2,1037 @@
Test case tutorial
==================
+
+This is a step-by-step tutorial on writing a simple C LTP test, where topics
+of the LTP and Linux kernel testing will be introduced gradually using a
+concrete example. Most sections will include exercises, some trivial and
+others not so much. If you find an exercise is leading you off at too much of
+a tangent, just leave it for later and move on.
+
+LTP tests can be written in C or Shell script. This tutorial is **only for tests
+written in C** using the new LTP test API. Note that while we go into some
+detail on using Git, this is not intended as a canonical or complete guide
+for Git.
+
+Assumptions & Feedback
+----------------------
+
+We assume the reader is familiar with C, Git and common Unix/Linux/GNU tools
+and has some general knowledge of Operating Systems. Experienced Linux
+developers may find it too verbose while people new to system level Linux
+development may find it overwhelming.
+
+Comments and feedback are welcome, please direct them to the Mailing list.
+
+Getting Started
+---------------
+
+First of all, make sure you have a copy of LTP in the current folder
+and we recommended cloning the Linux kernel repository for reference, this
+guide will refer to files and directories.
+
+.. code-block:: bash
+
+ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+
+There are a number of other repositories which are useful for reference as
+well, including the GNU C library ``glibc`` and the alternative C library
+``musl``. Some system calls are partially or even entirely implemented in user
+land as part of the standard C library. So in these cases, the C library is an
+important reference. ``glibc`` is the most common C library for Linux, however
+``musl`` is generally easier to understand.
+
+How system calls are implemented varies from one architecture to another and
+across kernel and C library versions. To find out whether a system call is
+actually accessing the kernel (whether it is actually a system call) on any
+given machine you can use the ``strace`` utility. This intercepts system calls
+made by an executable and prints them. We will use this later in the tutorial.
+
+Choose a System Call to test
+----------------------------
+
+We will use the ``statx()`` system call, to provide a concrete example of a
+test. At the time of writing there is no test for this call which was
+introduced in Linux kernel version 4.11.
+
+Linux system call specific tests are primarily contained in
+``testcases/kernel/syscalls``, but you should also ``git grep`` the entire LTP
+repository to check for any existing usages of a system call.
+
+One way to find a system call which is not currently tested by the LTP is to
+look at ``include/linux/syscalls.h`` in the kernel tree.
+
+Something the LTP excels to ensure bug-fixes are back ported to
+maintenance releases, so targeting a specific regression is another
+option.
+
+Find an untested System call
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Try to find an untested system call which has a manual page (i.e. ``man
+syscall`` produces a result). It is a good idea to Git-clone the latest kernel
+man-pages repository.
+
+.. code-block:: bash
+
+ git clone git://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
+
+At the time of writing, the difference between the latest man-pages release and
+the ``HEAD`` of the repository (usually the latest commit) is well over 100
+commits. This represents about 9 weeks of changes. If you are using a stable
+Linux distribution, your man-pages package may well be years old. So as with
+the kernel, it is best to have the Git repository as a reference.
+
+You could also find a system call with untested parameters or use whatever it
+is you are planning to use the LTP for.
+
+Create the test skeleton
+------------------------
+
+I shall call my test ``statx01.c``, by the time you read this that file name
+will probably be taken, so increment the number in the file name as
+appropriate or replace ``statx`` with the system call in the chosen exercise.
+
+.. code-block:: bash
+
+ mkdir testcases/kernel/syscalls/statx
+ cd testcases/kernel/syscalls/statx
+ echo statx >> .gitignore
+
+Next open ``statx01.c`` and add the following boilerplate. Make sure to change
+the copyright notice to your name/company, correct the test name and minimum
+kernel version if necessary. I will explain what the code does below.
+
+.. code-block:: c
+
+ // SPDX-License-Identifier: GPL-2.0-or-later
+ /*
+ * Copyright (c) 2017 Instruction Ignorer <"can't"@be.bothered.com>
+ */
+
+ /*\
+ * [Description]
+ *
+ * All tests should start with a description of _what_ we are testing.
+ * Non-trivial explanations of _how_ the code works should also go here.
+ * Include relevant links, Git commit hashes and CVE numbers.
+ * Inline comments should be avoided.
+ */
+
+ #include "tst_test.h"
+
+ static void run(void)
+ {
+ tst_res(TPASS, "Doing hardly anything is easy");
+ }
+
+ static struct tst_test test = {
+ .test_all = run,
+ .min_kver = "4.11",
+ };
+
+Starting with the ``#include`` statement we copy in the main LTP test library
+headers. This includes the most common test API functions and the test harness
+initialisation code. It is important to note that this is a completely
+ordinary, independent C program, however ``main()`` is missing because it is
+implemented in ``tst_test.h``.
+
+We specify what code we want to run as part of the test using the ``tst_test
+test`` structure. Various callbacks can be set by the test writer, including
+``test.test_all``, which we have set to ``run()``. The test harness will execute
+this callback in a separate process (using ``fork()``), forcibly terminating it
+if it does not return after ``test.timeout`` seconds.
+
+We have also set ``test.min_kver`` to the kernel version where ``statx`` was
+introduced. The test library will determine the kernel version at runtime. If
+the version is less than 4.11 then the test harness will return ``TCONF``,
+indicating that this test is not suitable for the current system
+configuration.
+
+Occasionally features are back ported to older kernel versions, so ``statx`` may
+exist on kernels with a lower version. However we don't need to worry about
+that unless there is evidence of it happening.
+
+As mentioned in the code itself, you should specify what you are testing and
+the expected outcome, even if it is relatively simple. If your program flow is
+necessarily complex and difficult to understand (which is often the case when
+trying to manipulate the kernel into doing something bad), then a detailed
+explanation of how the code works is welcome.
+
+What you should not do, is use inline comments or include the same level of
+explanation which is written here. As a general rule, if something is easy to
+document, then the code should also be easy to read. So don't document the easy
+stuff (except for the basic test specification).
+
+Before continuing we should compile this and check that the basics work. In
+order to compile the test we need a ``Makefile`` in the same subdirectory. If
+one already exists, then nothing needs to be done, otherwise add one with the
+following contents.
+
+.. code-block:: make
+
+ # SPDX-License-Identifier: GPL-2.0-or-later
+ # Copyright (c) 2019 Linux Test Project
+
+ top_srcdir ?= ../../../..
+
+ include $(top_srcdir)/include/mk/testcases.mk
+
+ include $(top_srcdir)/include/mk/generic_leaf_target.mk
+
+This will automatically add ``statx01.c`` as a build target producing a
+``statx01`` executable. Unless you have heavily deviated from the tutorial, and
+probably need to change ``top_srcdir``, nothing else needs to be done.
+
+Normally, if you were starting a Makefile from scratch, then you would need to
+add ``statx01`` as a build target. Specifying that you would like to run some
+program (e.g. ``gcc`` or ``clang``) to transform ``statx01.c`` into ``statx01``.
+Here we don't need to do that, but sometimes it is still necessary. For example,
+if we needed to link to the POSIX threading library, then we could add the
+following line after ``testcases.mk``.
+
+.. code-block:: make
+
+ statx01: CFLAGS += -pthread
+
+Assuming you are in the test's subdirectory ``testcases/kernel/syscalls/statx``,
+please do:
+
+.. code-block:: bash
+
+ make
+ ./statx01
+
+This should build the test and then run it. However, even though the test is
+in the ``syscalls`` directory it won't be automatically ran as part of the
+_syscalls_ test group (remember ``./runltp -f syscalls`` from the
+``README.md``?). For this we need to add it to the ``runtest`` file. So open
+``runtest/syscalls`` and add the lines starting with a ``+``.
+
+.. code-block::
+
+ statvfs01 statvfs01
+ statvfs02 statvfs02
+
+ +statx01 statx01
+ +
+ stime01 stime01
+ stime02 stime02
+
+The ``runtest`` files are in a two column format. The first column is the test
+name, which is mainly used by test runners for reporting and filtering. It is
+just a single string of text with no spaces. The second column, which can
+contain spaces, is passed to the shell in order to execute the test. Often it
+is just the executable name, but some tests also take arguments (the LTP has a
+library for argument parsing, by the way).
+
+If you haven't done so already, we should add all these new files to Git. It
+is vitally important that you do not make changes to the master branch. If you
+do then pulling changes from upstream becomes a major issue. So first of all
+create a new branch.
+
+.. code-block:: bash
+
+ git checkout -b statx01 master
+
+Now we want to add the files we have created or modified, but before doing a
+commit make sure you have configured Git correctly. You need to at least set
+your Name and e-mail address in ``~/.gitconfig``, but there are some other
+settings which come in handy too. My relatively simple configuration is similar
+to the below:
+
+.. code-block:: ini
+
+ [user]
+ name = Sarah Jane
+ email = sjane@e-mail.address
+ [core]
+ editor = emacs
+ [sendemail]
+ smtpServer = smtp.server.address
+
+Obviously you need to at least change your name and e-mail. The SMTP server is
+useful for ``git send-email``, which we will discuss later. The editor value is
+used for things like writing commits (without the ``-m`` option).
+
+.. code-block:: bash
+
+ git add -v :/testcases/kernel/syscalls/statx :/runtest/syscalls
+ git commit -m "statx01: Add new test for statx syscall"
+
+This should add all the new files in the ``statx`` directory and the ``runtest``
+file. It is good practice to commit early and often. Later on we will do a
+Git-rebase, which allows us to clean up the commit history. So don't worry
+about how presentable your commit log is for now. Also don't hesitate to
+create a new branch when doing the exercises or experimenting. This will allow
+you to diverge from the tutorial and then easily come back again.
+
+I can't emphasize enough that Git makes things easy through branching and that
+things quickly get complicated if you don't do it. However if you do get into
+a mess, Git-reflog and Git-reset, will usually get you out of it. If you also
+mess that up then it may be possible to cherry pick 'dangling' commits out of
+the database into a branch.
+
+Report TCONF instead of TPASS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Maybe the test should report ``TCONF: Not implemented`` instead or perhaps
+``TBROK``. Try changing it do so.
+
+Check Git ignores the executable
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Is your ``.gitignore`` correct?
+
+Run make check
+~~~~~~~~~~~~~~~~~~
+
+Check coding style with ``make check``.
+
+Install the LTP and run the test with runtest
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Run ``statx01`` on its own, also using ``-I0`` amd ``-I10``.
+
+Call the system call
+--------------------
+
+At the time of writing ``statx`` has no ``glibc`` wrapper. It is also fairly common
+for a distribution's C library version to be older than its kernel or it may use a
+cut down C library in comparison to the GNU one. So we must call ``statx()``
+using the general ``syscall()`` interface.
+
+The LTP contains a library for dealing with the ``syscall`` interface, which is
+located in ``include/lapi``. System call numbers are listed against the relevant
+call in the ``*.in`` files (e.g. ``x86_64.in``) which are used to generate
+``syscalls.h``, which is the header you should include. On rare occasions you
+may find the system call number is missing from the ``*.in`` files and will need
+to add it (see ``include/lapi/syscalls/strip_syscall.awk``).
+
+System call numbers vary between architectures, hence there are multiple
+``*.in`` files for each architecture. You can find the various values for the
+``statx`` system call across a number of ``unistd.h`` files in the Linux kernel.
+
+Note that we don't use the system-call-identifier value available in
+``/usr/include/linux/uinstd.h`` because the kernel might be much newer than the
+user land development packages.
+
+For ``statx`` we had to add ``statx 332`` to ``include/lapi/syscalls/x86_64.in``,
+``statx 383`` to ``include/lapi/syscalls/powerpc.in``, etc. Now lets look at
+the code, which I will explain in more detail further down.
+
+.. code-block:: c
+
+ /*
+ * Test statx
+ *
+ * Check if statx exists and what error code it returns when we give it dodgy
+ * data.
+ */
+
+ #include <stdint.h>
+ #include "tst_test.h"
+ #include "lapi/syscalls.h"
+
+ struct statx_timestamp {
+ int64_t tv_sec;
+ uint32_t tv_nsec;
+ int32_t __reserved;
+ };
+
+ struct statx {
+ uint32_t stx_mask;
+ uint32_t stx_blksize;
+ uint64_t stx_attributes;
+ uint32_t stx_nlink;
+ uint32_t stx_uid;
+ uint32_t stx_gid;
+ uint16_t stx_mode;
+ uint16_t __spare0[1];
+ uint64_t stx_ino;
+ uint64_t stx_size;
+ uint64_t stx_blocks;
+ uint64_t stx_attributes_mask;
+ struct statx_timestamp stx_atime;
+ struct statx_timestamp stx_btime;
+ struct statx_timestamp stx_ctime;
+ struct statx_timestamp stx_mtime;
+ uint32_t stx_rdev_major;
+ uint32_t stx_rdev_minor;
+ uint32_t stx_dev_major;
+ uint32_t stx_dev_minor;
+ uint64_t __spare2[14];
+ };
+
+ static int sys_statx(int dirfd, const char *pathname, int flags,
+ unsigned int mask, struct statx *statxbuf)
+ {
+ return tst_syscall(__NR_statx, dirfd, pathname, flags, mask, statxbuf);
+ }
+
+ ...
+
+So the top part of the code is now boiler plate for calling ``statx``. It is
+common for the kernel to be newer than the user land libraries and headers. So
+for new system calls like ``statx``, we copy, with a few modifications, the
+relevant definitions into the LTP. This is somewhat like 'vendoring', although
+we are usually just copying headers required for interacting with the Kernel's
+ABI (Application Binary Interface), rather than internalising actual
+functionality.
+
+So from the top we include the ``stdint.h`` library which gives us the standard
+``(u)int*_t`` type definitions. We use these in place of the Kernel type
+definitions such as ``__u64`` in ``linux/types.h``. We then have a couple of
+structure definitions which form part of the ``statx`` API. These were copied
+from ``include/uapi/linux/stat.h`` in the Kernel tree.
+
+After that, there is a wrapper function, which saves us from writing
+``tst_syscall(__NR_statx, ...``, every time we want to make a call to
+``statx``. This also provides a stub for when ``statx`` is eventually integrated
+into the LTP library and also implemented by the C library. At that point we
+can switch to using the C library implementation if available or fallback to
+our own.
+
+The advantage of using the C library implementation is that it will often be
+better supported across multiple architectures. It will also mean we are using
+the system call in the same way most real programs would. Sometimes there are
+advantages to bypassing the C library, but in general it should not be our
+first choice.
+
+The final test should do a check during configuration (i.e. when we run
+``./configure`` before building) which checks if the ``statx`` system call and
+associated structures exists. This requires writing an ``m4`` file for use with
+``configure.ac`` which is processed during ``make autotools`` and produces the
+configure script.
+
+For the time being though we shall just ignore this. All you need to know for
+now is that this is a problem which eventually needs to be dealt with and that
+there is a system in place to handle it.
+
+.. code-block:: c
+
+ ...
+
+ static void run(void)
+ {
+ struct statx statxbuf = { 0 };
+
+ TEST(sys_statx(0, NULL, 0, 0, &statxbuf));
+
+ if (TST_RET == 0)
+ tst_res(TFAIL, "statx thinks it can stat NULL");
+ else if (TST_ERR == EFAULT)
+ tst_res(TPASS, "statx set errno to EFAULT as expected");
+ else
+ tst_res(TFAIL | TERRNO, "statx set errno to some unexpected value");
+ }
+
+ static struct tst_test test = {
+ .test_all = run,
+ .min_kver = "4.11",
+ };
+
+The ``TEST`` macro sets ``TST_RET`` to the return value of ``tst_statx()`` and
+``TST_ERR`` to the value of ``errno`` immediately after the functions
+return. This is mainly just for convenience, although it potentially could
+have other uses.
+
+We check whether the return value indicates success and if it doesn't also
+check the value of ``errno``. The last call to ``tst_res`` includes ``TERRNO``,
+which will print the current error number and associated description in
+addition to the message we have provided. Note that it uses the current value
+of ``errno`` not ``TST_ERR``.
+
+What we should have done in the example above is use ``TTERRNO`` which takes the
+value of ``TST_ERR``.
+
+If we try to run the test on a kernel where ``statx`` does not exist, then
+``tst_syscall`` will cause it to fail gracefully with ``TCONF``. Where ``TCONF``
+indicates the test is not applicable to our configuration.
+
+The function ``tst_syscall`` calls ``tst_brk(TCONF,...)`` on failure. ``tst_brk``
+causes the test to exit immediately, which prevents any further test code from
+being run.
+
+What are the differences between ``tst_brk`` and ``tst_res``?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See ``include/tst_test.h``. Also what do they have in common?
+
+What happens if you call ``tst_res(TINFO, ...)`` after ``sys_statx``?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Does the test still function correctly?
+
+Extend the test to handle other basic error conditions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For example, see if you can trigger ``ENOENT`` instead. You shouldn't
+have to create any files, which is discussed in the next section.
+
+Setup, Cleanup and files
+------------------------
+
+Some tests require resources to be allocated, or system settings to be
+changed, before the test begins. This ``setup`` only has to be done once at the
+beginning and at the end of the test needs to be removed or reverted. The
+``cleanup`` also has to be done regardless of whether the test breaks.
+
+Fortunately, like most test libraries, we have setup and cleanup (teardown)
+callbacks. ``setup`` is called once before ``run`` and ``cleanup`` is called once
+afterwards. Note that ``run`` itself can be called multiple times by the test
+harness, but that ``setup`` and ``cleanup`` are only called once.
+
+If either your code, a ``SAFE_*`` macro or a library function such as
+``tst_syscall`` call ``tst_brk``, then ``run`` will exit immediately and the
+``cleanup`` function is then called. Once ``cleanup`` is completed, the test
+executable will then exit altogether abandoning any remaining iterations of
+``run``.
+
+For ``statx`` we would like to create some files or file like objects which we
+have control over. Deciding where to create the files is easy, we just create
+it in the current working directory and let the LTP test harness handle where
+that should be by setting ``.needs_tmpdir = 1``.
+
+.. code-block:: c
+
+ /*
+ * Test statx
+ *
+ * Check if statx exists and what error code it returns when we give it dodgy
+ * data. Then stat a file and check it returns success.
+ */
+
+ #include <stdint.h>
+ #include "tst_test.h"
+ #include "lapi/syscalls.h"
+ #include "lapi/fcntl.h"
+
+ #define FNAME "file_to_stat"
+ #define STATX_BASIC_STATS 0x000007ffU
+
+ /*************** statx structure and wrapper goes here ! ***************/
+ ...
+
+We have added an extra include ``lapi/fcntl.h`` which wraps the system header by
+the same name (``#include <fcntl.h>``). This header ensures we have definitions
+for recently added macros such as ``AT_FDCWD`` by providing fall backs if the
+system header does not have them. The ``lapi`` directory contains a number of
+headers like this.
+
+At some point we may wish to add ``lapi/stat.h`` to provide a fall back for
+macros such as ``STATX_BASIC_STATS``. However for the time being we have just
+defined it in the test.
+
+
+.. code-block:: c
+
+ ...
+
+ static void setup(void)
+ {
+ SAFE_TOUCH(FNAME, 0777, NULL);
+ }
+
+ static void run(void)
+ {
+ struct statx statxbuf = { 0 };
+
+ TEST(sys_statx(0, NULL, 0, 0, &statxbuf));
+ if (TST_RET == 0)
+ tst_res(TFAIL, "statx thinks it can stat NULL");
+ else if (TST_ERR == EFAULT)
+ tst_res(TPASS, "statx set errno to EFAULT as expected");
+ else
+ tst_res(TFAIL | TERRNO, "statx set errno to some unexpected value");
+
+ TEST(sys_statx(AT_FDCWD, FNAME, 0, STATX_BASIC_STATS, &statxbuf));
+ if (TST_RET == 0)
+ tst_res(TPASS, "It returned zero so it must have worked!");
+ else
+ tst_res(TFAIL | TERRNO, "statx can not stat a basic file");
+ }
+
+ static struct tst_test test = {
+ .setup = setup,
+ .test_all = run,
+ .min_kver = "4.11",
+ .needs_tmpdir = 1
+ };
+
+The ``setup`` callback uses one of the LTP's ``SAFE`` functions to create an empty
+file ``file_to_stat``. Because we have set ``.needs_tmpdir``, we can just create
+this file in the present working directory. We don't need to create a
+``cleanup`` callback yet because the LTP test harness will recursively delete
+the temporary directory and its contents.
+
+The ``run`` function can be called multiple times by the test harness, however
+``setup`` and ``cleanup`` callbacks will only be ran once.
+
+.. warning::
+
+ By this point you may have begun to explore the LTP library headers or older
+ tests. In which case you will have come across functions from the old API such
+ as ``tst_brkm``. The old API is being phased out, so you should not use these
+ functions.
+
+So far we haven't had to do any clean up. So our example doesn't answer the
+question "what happens if part of the clean up fails?". To answer this we are
+going to modify the test to ask the (highly contrived) question "What happens
+if I create and open a file, then create a hard-link to it, then call open
+again on the hard-link, then ``stat`` the file".
+
+
+.. code-block:: c
+
+ #define LNAME "file_to_stat_link"
+
+ ...
+
+ static void setup(void)
+ {
+ fd = SAFE_OPEN(FNAME, O_CREAT, 0777);
+ SAFE_LINK(FNAME, LNAME);
+ lfd = SAFE_OPEN(LNAME, 0);
+ }
+
+ static void cleanup(void)
+ {
+ if (lfd != 0)
+ SAFE_CLOSE(lfd);
+
+ if (fd != 0)
+ SAFE_CLOSE(fd);
+ }
+
+ static void run(void)
+ {
+ ...
+
+ TEST(sys_statx(AT_FDCWD, LNAME, 0, STATX_BASIC_STATS, &statxbuf));
+ if (TST_RET == 0)
+ tst_res(TPASS, "It returned zero so it must have worked!");
+ else
+ tst_res(TFAIL | TERRNO, "statx can not stat a basic file");
+ }
+
+ static struct tst_test test = {
+ .setup = setup,
+ .cleanup = cleanup,
+ .test_all = run,
+ .tcnt = 2,
+ .min_kver = "4.11",
+ .needs_tmpdir = 1
+ };
+
+Because we are now opening a file, we need a ``cleanup`` function to close the
+file descriptors. We have to manually close the files to ensure the temporary
+directory is deleted by the test harness (see the
+https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines[test
+writing guidelines] for details).
+
+As a matter of good practice, the file descriptors are closed in reverse
+order. In some circumstances the order in which clean up is performed is
+significant. In that case resources created towards the end of 'setup' are
+dependent on ones near the beginning. So during 'cleanup' we remove the
+dependants before their dependencies.
+
+If, for some reason, the file descriptor ``lfd`` became invalid during the test,
+but ``fd`` was still open, we do not want ``SAFE_CLOSE(lfd)`` to cause the
+``cleanup`` function to exit prematurely. If it did, then ``fd`` would remain open
+which would cause problems on some file systems.
+
+Nor do we want to call ``cleanup`` recursively. So during 'cleanup' ``tst_brk``,
+and consequently the ``SAFE`` functions, do not cause the test to exit with
+``TBROK``. Instead they just print an error message with ``TWARN``.
+
+It is not entirely necessary to check if the file descriptors have a none zero
+value before attempting to close them. However it avoids a bunch of spurious
+warning messages if we fail to open ``file_to_stat``. Test case failures can be
+difficult to interpret at the best of times, so avoid filling the log with
+noise.
+
+Check ``statx`` returns the correct number of hard links
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The field ``statx.stx_nlink`` should be equal to 2, right?
+
+Git-branch
+~~~~~~~~~~
+
+We are about to make some organisational changes to the test, so now would be
+a good time to branch. Then we can switch between the old and new versions, to
+check the behavior has not been changed by accident.
+
+Split the test
+--------------
+
+In our current test, we have essentially rolled two different test cases into
+one. Firstly we check if an error is returned when bad arguments are provided
+and secondly we check what happens when we stat an actual file. Quite often it
+makes sense to call ``tst_res`` multiple times in a single test case because we
+are checking different properties of the same result, but here we are clearly
+testing two different scenarios.
+
+So we should split the test in two. One obvious way to do this is to create
+``statx02.c``, but that seems like overkill in order to separate two simple test
+cases. So, for now at least, we are going to do it a different way.
+
+.. code-block:: c
+
+ ...
+
+ static void run_stat_null(void)
+ {
+ struct statx statxbuf = { 0 };
+
+ TEST(sys_statx(0, NULL, 0, 0, &statxbuf));
+ if (TST_RET == 0)
+ tst_res(TFAIL, "statx thinks it can stat NULL");
+ else if (TST_ERR == EFAULT)
+ tst_res(TPASS, "statx set errno to EFAULT as expected");
+ else
+ tst_res(TFAIL | TERRNO, "statx set errno to some unexpected value");
+ }
+
+ static void run_stat_symlink(void)
+ {
+ struct statx statxbuf = { 0 };
+
+ TEST(sys_statx(AT_FDCWD, LNAME, 0, STATX_BASIC_STATS, &statxbuf));
+ if (TST_RET == 0)
+ tst_res(TPASS, "It returned zero so it must have worked!");
+ else
+ tst_res(TFAIL | TERRNO, "statx can not stat a basic file");
+ }
+
+ static void run(unsigned int i)
+ {
+ switch(i) {
+ case 0: run_stat_null();
+ case 1: run_stat_symlink();
+ }
+ }
+
+ static struct tst_test test = {
+ .setup = setup,
+ .cleanup = cleanup,
+ .test = run,
+ .tcnt = 2,
+ .min_kver = "4.11",
+ .needs_tmpdir = 1
+ };
+
+So we have used an alternative form of the ``test`` or ``run`` callback which
+accepts an index. Some tests use this index with an array of parameters and
+expected return values. Others do something similar to the above. The index
+can be used how you want so long as each iteration calls ``tst_res`` in a
+meaningful way.
+
+If an iteration fails to return a result (i.e. call ``tst_res`` with a value
+other than ``TINFO``) then the test harness will report ``TBROK`` and print the
+iteration which failed. This prevents a scenario in your test from silently
+failing due to some faulty logic.
+
+What is wrong with the switch statement?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Were you paying attention? Also see the output of ``make check``.
+
+Test a feature unique to statx
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+So far we have not tested anything which is unique to ``statx``. So, for
+example, you could check stx_btime is correct (possibly only to within a
+margin of error) and that it differs from ``stx_mtime`` after writing to the
+file.
+
+Alternatively you could check that ``stx_dev_major`` and ``stx_dev_minor`` are set
+correctly. Note that the LTP has helper functions for creating devices and
+file systems.
+
+This could be quite a challenging exercise. You may wish to tackle an
+altogether different test scenario instead. If you get stuck just move onto
+the next section and come back later.
+
+Submitting the test for review
+------------------------------
+
+Ignoring the fact we should probably create ``lapi/stat.h`` along with a bunch
+of fallback logic in the build system. We can now get our test ready for
+submission.
+
+The first thing you need to do before considering submitting your test is run
+``make check-statx01`` or ``make check`` in the test's directory. Again, we use
+the kernel style guidelines where possible. Next you should create a new
+branch, this will allow you to reshape your commit history without fear.
+
+After that we have the pleasure of doing an interactive ``rebase`` to clean up
+our commit history. In its current form the test only really needs a single
+commit, but if you have been using Git correctly then you should have
+many. The main reason we want to compress it to a single commit, is to make
+the LTP's Git-log readable. It also allows us to write a coherent description
+of the work as a whole in retrospective. Although, when adding a new test, the
+test description in the code will probably make the commit message redundant.
+
+Anyway, as an example, we shall look at my personal commit history from this
+tutorial and ``rebase`` it. You should try following along with your own
+repository. First lets look at the commit history since we branched from
+master.
+
+.. code-block:: bash
+
+ git log --oneline master..HEAD
+ 152d39fe7 (HEAD -> tutorial-rebase2, tutorial-rebase) tutorial: Start Submitting patch section
+ 70f7ce7ce statx01: Stop checkpatch from complaining
+ bb0332bd7 tutorial: Fix review problems
+ 6a87a084a statx01: Fix review problems
+ d784b1e85 test-writing-guidelines: Remove old API argument
+ c26e1be7a fixup! tutorial
+ 1e24a5fb5 (me/tutorial-rebase) fixup! tutorial
+ 568a3f7be fixup! tutorial
+ 09dd2c829 statx: stage 6
+ bfeef7902 statx: stage 5b
+ 76e03d714 statx: stage 5a
+ 98f5bc7ac statx: stage 4
+ 6f8c16438 statx: stage 3 (Add statx01)
+ 5d93b84d8 Add statx and other syscall numbers
+ 5ca627b78 tutorial: Add a step-by-step C test tutorial
+
+So we have told git to show all the commits which don't exist in ``master``, but
+are in ``HEAD``, where ``HEAD`` is the top of the current branch. The current
+branch is ``tutorial-rebase2`` which I just created. I have already done one
+``rebase`` and submitted a patch for review, so my original branch was just called
+``tutorial``.
+
+As usual my commit history is starting to look like a bit of mess! There is
+even a commit in there which should not be in the this branch (Remove old API
+argument), however it can be ignored for now and 'cherry picked' into a new branch
+later.
+
+For my patch I actually need at least two commits, one which contains the
+tutorial text and one which contains the test and associated files. So first
+of all I want to 'squash' (amalgamate) all the commits appended with
+``tutorial:`` into the bottom commit.
+
+.. code-block:: bash
+
+ $ git rebase -i 5ca627b78\^
+ ...
+
+This begins an interactive ``rebase`` where commit ``5ca6427b78`` is the earliest
+commit we want to edit. The ``^`` symbol after the commit hash, specifies the
+commit before this one. The interactive ``rebase`` command takes the last commit
+we want to keep unaltered as it's argument (in other words it takes a
+non-inclusive range).
+
+Upon entering a similar command you will be presented with a text file
+similar to the following. The file should be displayed in your text editor of
+choice, if it doesn't, then you may change the editor variable in
+``.gitconfig``.
+
+.. code-block:: bash
+
+ pick 5ca627b78 tutorial: Add a step-by-step C test tutorial
+ pick 5d93b84d8 Add statx and other syscall numbers
+ pick 6f8c16438 statx: stage 3 (Add statx01)
+ pick 98f5bc7ac statx: stage 4
+ pick 76e03d714 statx: stage 5a
+ pick bfeef7902 statx: stage 5b
+ pick 09dd2c829 statx: stage 6
+ pick 568a3f7be fixup! tutorial
+ pick 1e24a5fb5 fixup! tutorial
+ pick c26e1be7a fixup! tutorial
+ pick d784b1e85 test-writing-guidelines: Remove old API argument
+ pick 6a87a084a statx01: Fix review problems
+ pick bb0332bd7 tutorial: Fix review problems
+ pick 70f7ce7ce statx01: Stop checkpatch from complaining
+ pick 152d39fe7 tutorial: Start Submitting patch section
+
+The last commit from Git-log is shown at the top. The left hand column
+contains the commands we want to run on each commit. ``pick`` just means we
+re-apply the commit as-is. We can reorder the lines to apply the commits in a
+different order, but we need to be careful when reordering commits to the same
+file. If your ``rebase`` results in a merge conflict, then you have probably
+reordered some commits which contained changes to the same piece of code.
+
+Perhaps a better name for the interactive ``rebase`` command would be 'replay'. As
+we pick a point in the commit history, undo all those commits before that
+point, then reapply them one at a time. During the replay we can reorder the
+commits, drop, merge, split and edit them, creating a new history.
+
+The commands I am going to use are ``reword`` and ``fixup``. The ``reword`` command
+allows you to edit a single commit's message. The 'fixup' command 'squashes' a
+commit into the commit above/preceding it, merging the two commits into
+one. The commit which has ``fixup`` applied has its commit message deleted. If
+you think a commit might have something useful in its message then you can use
+``squash`` instead.
+
+.. code-block:: bash
+
+ reword 5ca627b78 tutorial: Add a step-by-step C test tutorial
+ fixup 568a3f7be fixup! tutorial
+ fixup 1e24a5fb5 fixup! tutorial
+ fixup c26e1be7a fixup! tutorial
+ fixup bb0332bd7 tutorial: Fix review problems
+ fixup 152d39fe7 tutorial: Start Submitting patch section
+ fixup 276edecab tutorial: Save changes before rebase
+ pick 5d93b84d8 Add statx and other syscall numbers
+ pick 6f8c16438 statx: stage 3 (Add statx01)
+ pick 98f5bc7ac statx: stage 4
+ pick 76e03d714 statx: stage 5a
+ pick bfeef7902 statx: stage 5b
+ pick 09dd2c829 statx: stage 6
+ pick d784b1e85 test-writing-guidelines: Remove old API argument
+ pick 6a87a084a statx01: Fix review problems
+
+So all the commits marked with ``fixup`` will be re-played by Git immediately
+after 5ca62 at the top. A new commit will then be created with the amalgamated
+changes of all the commits and 5ca62's log message. It turns out that I didn't
+need to reword anything, but there is no harm in checking. It is easy to
+forget the ``Signed-off-by:`` line.
+
+I could now do the same for the commits to the ``statx`` test, making the commit
+message prefixes consistent. However I am not actually going to submit the
+test (yet).
+
+I won't attempt to show you this, but if you need to do the opposite and split
+apart a commit. It is also possible using Git-rebase by marking a line with
+``edit``. This will pause Git just after replaying the marked commit. You can
+then use a 'soft' Git-reset to bring the selected commit's changes back into
+the 'index' where you are then able to un-stage some parts before
+re-committing.
+
+You can also use ``edit`` and ``git commit --amend`` together to change a commit
+deep in your history, but without resetting the 'index'. The 'index' contains
+changes which you have staged with ``git add``, but not yet committed.
+
+So now that the commit history has been cleaned up, we need to submit a patch
+to the mailing list or make a pull request on GitHub. The mailing list is the
+preferred place to make submissions and is more difficult for most people, so
+I will only cover that method.
+
+Just before we create the patch, we need to check that our changes will still
+apply to the master branch without problems. To do this we can use another
+type of ``rebase`` and then try rebuilding and running the test.
+
+.. code-block:: bash
+
+ git checkout master
+ git pull origin
+ git checkout tutorial-rebase2
+ git rebase master
+
+Above, I update the master branch and then replay our changes onto it using
+``git rebase master``. You may find that after the rebase there is a merge
+conflict. This will result in something which looks like the following (taken
+from a Makefile conflict which was caused by reordering commits in a ``rebase``).
+
+.. code-block:: diff
+
+ <<<<<<< HEAD
+ cve-2016-7117: LDFLAGS += -lpthread
+ =======
+ cve-2014-0196: LDFLAGS += -lpthread -lutil -lrt
+ cve-2016-7117: LDFLAGS += -lpthread -lrt
+ >>>>>>> 4dbfb8e79... Add -lrt
+
+The first line tells us this is the beginning of a conflict. The third line
+separates the two conflicting pieces of content and the last line is the end
+of the conflict. Usually, all you need to do is remove the lines you don't
+want, stage the changes and continue the ``rebase`` with ``git rebase
+--continue``.
+
+In order to create a patch e-mail we use
+`git format-patch <https://git-scm.com/docs/git-format-patch>`_,
+we can then send that e-mail using
+`git send-email <https://git-scm.com/docs/git-send-email>`_.
+It is also possible to import the patch (``mbox``) file into a number of e-mail
+programs.
+
+.. code-block:: bash
+
+ $ git format-patch -1 -v 2 -o output --to ltp@lists.linux.it fd3cc8596
+ output/v2-0001-tutorial-Add-a-step-by-step-C-test-tutorial.patch
+
+The first argument ``-1`` specifies we want one commit from fd3cc8596
+onwards. If we wanted this commit and the one after it we could specify ``-2``
+instead.
+
+This is my second patch submission so I have used ``-v 2``, which indicates this
+is the second version of a patch set. The ``-o`` option specifies the output
+directory (literally called ``output``). The ``--to`` option adds the ``To:`` e-mail
+header, which I have set to the LTP mailing list.
+
+We can then send this patch with the following command sans ``--dry-run``.
+
+.. code-block:: bash
+
+ git send-email --dry-run output/v2-0001-tutorial-Add-a-step-by-step-C-test-tutorial.patch
+
+Git will ask some questions (which you can ignore) and then tell you what it
+would do if this weren't a dry-run. In order for this to work you have to have
+a valid SMTP server set in ``.gitconfig`` and also be signed up to the LTP
+mailing list under the same e-mail address you have configured in Git. You can
+sign up at https://lists.linux.it/listinfo/ltp.
+
+Doing code review
+-----------------
+
+While waiting for your test to be reviewed, you are invited and encouraged to
+review other contributors' code. This may seem bizarre when you are completely
+new to the project, but there are two important ways in which you can
+contribute here:
+
+A. Point out logical errors in the code.
+B. Improve your own understanding
+
+It doesn't matter whether you know the canonical way of writing an LTP test in
+C. An error of logic, when properly explained, is usually indisputable. These
+are the most important errors to find as they always result in false test
+results. Once someone points out such an error it is usually obvious to
+everyone that it is a bug and needs to be fixed.
+
+Obviously testing the patch is one way of finding errors. You can apply
+patches using ``git am``. Then it is just a case of compiling and running the
+tests.
+
+Finally, reading and attempting to comment on other peoples patches, gives
+you a better understanding of the reviewers perspective. This is better for
+the project and for you.
+
+Style and organisational issues are best left to after you have found logical
+errors.
+
+Final notes
+-----------
+
+Hopefully you can now grasp the structure of an LTP test and have some idea of
+what is available in the LTP test library. There are a vast number of library
+functions available (mainly located in include and lib), some of which are
+documented in the test writing guidelines and many of which are not.
+
+We have only scratched the surface of the immense technical complexity of
+systems programming across multiple Kernel and C lib versions as well as
+different hardware architectures. The important thing to take away from this
+is that you have to be conscientious of what will happen on systems different
+from yours. The LTP has a huge and varied user base, so situations you may
+think are unlikely can and do happen to somebody.
+
+Of course you don't want to spend time allowing for situations which may never
+arise either, so you have to do your research and think about each situation
+critically. The more systems you can test on before submitting your changes,
+the better, although we understand not everyone has access to a lab.
+
+One important topic which has not been covered by this tutorial, is
+multi-process or multi-threaded testing. The LTP library functions work inside
+child processes and threads, but their semantics change slightly. There are
+also various helper functions for synchronising and forking processes.
+
+.. note::
+
+ When it comes time to submit a test, the preferred way to do it is on the
+ mailing list although you can also use GitHub. The LTP follows similar rules
+ to the kernel for formatting and submitting patches. Generally speaking the
+ review cycle is easier for small patches, so try to make small changes or
+ additions where possible.
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* [LTP] [PATCH 10/10] Introduce kernel-doc API support
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (8 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 09/10] Refactor test case tutorial documentation Andrea Cervesato
@ 2024-03-20 16:20 ` Andrea Cervesato
2024-03-21 8:25 ` [LTP] [PATCH 00/10] New LTP documentation Petr Vorel
10 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato @ 2024-03-20 16:20 UTC (permalink / raw)
To: ltp
From: Andrea Cervesato <andrea.cervesato@suse.com>
kernel-doc API support is introduced by linuxdoc package that includes
Linux kernel tools used to generate the documentation.
With this patch we fully support the kernel documentation format that
can be found here:
https://return42.github.io/linuxdoc/linuxdoc-howto/kernel-doc-syntax.html
---
doc_new/conf.py | 3 +--
doc_new/developers/api_c_tests.rst | 3 +++
doc_new/developers/api_kvm_tests.rst | 3 +++
doc_new/developers/api_network_tests.rst | 3 +++
doc_new/requirements.txt | 1 +
5 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/doc_new/conf.py b/doc_new/conf.py
index b333ebf2a..44122b761 100644
--- a/doc_new/conf.py
+++ b/doc_new/conf.py
@@ -14,8 +14,7 @@ release = '1.0'
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
-extensions = []
-
+extensions = ['linuxdoc.rstKernelDoc']
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
diff --git a/doc_new/developers/api_c_tests.rst b/doc_new/developers/api_c_tests.rst
index c03f9caf1..adee17f16 100644
--- a/doc_new/developers/api_c_tests.rst
+++ b/doc_new/developers/api_c_tests.rst
@@ -1,4 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
+.. Include headers in this file with:
+.. .. kernel-doc:: ../../include/tst_test.h
+
Developing using C API
======================
diff --git a/doc_new/developers/api_kvm_tests.rst b/doc_new/developers/api_kvm_tests.rst
index 1ef7ec452..5ec3fc3a4 100644
--- a/doc_new/developers/api_kvm_tests.rst
+++ b/doc_new/developers/api_kvm_tests.rst
@@ -1,4 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
+.. Include headers in this file with:
+.. .. kernel-doc:: ../../include/tst_test.h
+
Developing using KVM API
========================
diff --git a/doc_new/developers/api_network_tests.rst b/doc_new/developers/api_network_tests.rst
index 4fe45f5f6..3e487d7f2 100644
--- a/doc_new/developers/api_network_tests.rst
+++ b/doc_new/developers/api_network_tests.rst
@@ -1,4 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
+.. Include headers in this file with:
+.. .. kernel-doc:: ../../include/tst_test.h
+
Developing using network API
============================
diff --git a/doc_new/requirements.txt b/doc_new/requirements.txt
index 59709ec72..720e6870e 100644
--- a/doc_new/requirements.txt
+++ b/doc_new/requirements.txt
@@ -1 +1,2 @@
sphinx-rtd-theme==2.0.0
+linuxdoc==20231020
--
2.35.3
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [LTP] [PATCH 00/10] New LTP documentation
2024-03-20 16:20 [LTP] [PATCH 00/10] New LTP documentation Andrea Cervesato
` (9 preceding siblings ...)
2024-03-20 16:20 ` [LTP] [PATCH 10/10] Introduce kernel-doc API support Andrea Cervesato
@ 2024-03-21 8:25 ` Petr Vorel
2024-03-21 8:46 ` Andrea Cervesato via ltp
10 siblings, 1 reply; 16+ messages in thread
From: Petr Vorel @ 2024-03-21 8:25 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
> The new LTP documentation is meant to use Sphinx as the main
> documentation framework, with ReconStructedText as the main language.
> All the conversion has been done step-by-step, each chapter at time,
> updating english syntax and fixing typos. There are minor improvements,
> but overall the structure is the same.
> C / Networking / KVM API are not documented yet, because they will be
> probably integrated in the LTP library. C API documentation is already
> on going, but the others not.
> For a demo, please take a look at:
> https://ltp-acerv.readthedocs.io/en/latest/index.html
Out of curiosity, we need to register LTP on readthedocs.io.
Then we need GitHub Action hook (including some secret for auth) for publishing.
Correct?
https://docs.readthedocs.io/en/stable/guides/setup/git-repo-manual.html
I guess you now deploy to your fork manually, otherwise you would add GitHub
Action config in the patchset.
Also, what I like (for the future) offline formats (PDF, ePub, HTML) are
supported:
https://docs.readthedocs.io/en/stable/downloadable-documentation.html
Also, I would prefer when this merged it would actually replace the old github
docs (otherwise we maintainers endup maintaining both - getting fixes for both).
E.g. in the final version first deleting the old docs + github update hook.
Kind regards,
Petr
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [LTP] [PATCH 00/10] New LTP documentation
2024-03-21 8:25 ` [LTP] [PATCH 00/10] New LTP documentation Petr Vorel
@ 2024-03-21 8:46 ` Andrea Cervesato via ltp
2024-03-21 8:56 ` Petr Vorel
0 siblings, 1 reply; 16+ messages in thread
From: Andrea Cervesato via ltp @ 2024-03-21 8:46 UTC (permalink / raw)
To: Petr Vorel, Andrea Cervesato; +Cc: ltp
Hi Petr!
On 3/21/24 09:25, Petr Vorel wrote:
> Hi Andrea,
>
>> The new LTP documentation is meant to use Sphinx as the main
>> documentation framework, with ReconStructedText as the main language.
>> All the conversion has been done step-by-step, each chapter at time,
>> updating english syntax and fixing typos. There are minor improvements,
>> but overall the structure is the same.
>> C / Networking / KVM API are not documented yet, because they will be
>> probably integrated in the LTP library. C API documentation is already
>> on going, but the others not.
>> For a demo, please take a look at:
>> https://ltp-acerv.readthedocs.io/en/latest/index.html
> Out of curiosity, we need to register LTP on readthedocs.io.
> Then we need GitHub Action hook (including some secret for auth) for publishing.
> Correct?
>
> https://docs.readthedocs.io/en/stable/guides/setup/git-repo-manual.html
readthedocs works with any kind of git server. The only thing we need,
it's a .readthedocs.yaml file inside a specific branch (i.e. master).
So we need to create an account for the LTP team in readthedocs.com,
then we provide the git repo to readthedocs and site will be published
automatically.
ltp.readthedocs.io is available, so maybe we should create an account
already.
>
> I guess you now deploy to your fork manually, otherwise you would add GitHub
> Action config in the patchset.
It's automatic done by readthedocs and we don't need Github actions.
>
> Also, what I like (for the future) offline formats (PDF, ePub, HTML) are
> supported:
>
> https://docs.readthedocs.io/en/stable/downloadable-documentation.html
Sure we can try to generate it and see what happens. I never tried to do
it in sphinx.
>
> Also, I would prefer when this merged it would actually replace the old github
> docs (otherwise we maintainers endup maintaining both - getting fixes for both).
> E.g. in the final version first deleting the old docs + github update hook.
Sure, so let's finish the basics. I'm waiting for Cyril LTP library
documentation, since I don't have all the expertise which are requested.
The idea is that we will get rid of the current C-Test-API.asciidoc file
and we will only use documented headers.
Then we can proceed for sure.
>
> Kind regards,
> Petr
We also have networking/KVM/shell API documentation that should be
placed inside the headers. It would be nice to have a help on that.
Regards,
Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH 00/10] New LTP documentation
2024-03-21 8:46 ` Andrea Cervesato via ltp
@ 2024-03-21 8:56 ` Petr Vorel
2024-03-22 10:49 ` Wojtek Porczyk
[not found] ` <Zf1htmFkRl8BIFOX@invisiblethingslab.com>
0 siblings, 2 replies; 16+ messages in thread
From: Petr Vorel @ 2024-03-21 8:56 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
Hi Andrea,
> Hi Petr!
> On 3/21/24 09:25, Petr Vorel wrote:
> > Hi Andrea,
> > > The new LTP documentation is meant to use Sphinx as the main
> > > documentation framework, with ReconStructedText as the main language.
> > > All the conversion has been done step-by-step, each chapter at time,
> > > updating english syntax and fixing typos. There are minor improvements,
> > > but overall the structure is the same.
> > > C / Networking / KVM API are not documented yet, because they will be
> > > probably integrated in the LTP library. C API documentation is already
> > > on going, but the others not.
> > > For a demo, please take a look at:
> > > https://ltp-acerv.readthedocs.io/en/latest/index.html
> > Out of curiosity, we need to register LTP on readthedocs.io.
> > Then we need GitHub Action hook (including some secret for auth) for publishing.
> > Correct?
> > https://docs.readthedocs.io/en/stable/guides/setup/git-repo-manual.html
> readthedocs works with any kind of git server. The only thing we need, it's
> a .readthedocs.yaml file inside a specific branch (i.e. master).
> So we need to create an account for the LTP team in readthedocs.com, then we
> provide the git repo to readthedocs and site will be published
> automatically.
> ltp.readthedocs.io is available, so maybe we should create an account
> already.
ack.
> > I guess you now deploy to your fork manually, otherwise you would add GitHub
> > Action config in the patchset.
> It's automatic done by readthedocs and we don't need Github actions.
> > Also, what I like (for the future) offline formats (PDF, ePub, HTML) are
> > supported:
> > https://docs.readthedocs.io/en/stable/downloadable-documentation.html
> Sure we can try to generate it and see what happens. I never tried to do it
> in sphinx.
> > Also, I would prefer when this merged it would actually replace the old github
> > docs (otherwise we maintainers endup maintaining both - getting fixes for both).
> > E.g. in the final version first deleting the old docs + github update hook.
> Sure, so let's finish the basics. I'm waiting for Cyril LTP library
> documentation, since I don't have all the expertise which are requested. The
> idea is that we will get rid of the current C-Test-API.asciidoc file and we
> will only use documented headers.
> Then we can proceed for sure.
Cool (I can imagine it might take some time and energy).
> > Kind regards,
> > Petr
> We also have networking/KVM/shell API documentation that should be placed
> inside the headers. It would be nice to have a help on that.
Yeah. It might be another challenge to get Sphinx working on shell API.
Kind regards,
Petr
> Regards,
> Andrea
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [LTP] [PATCH 00/10] New LTP documentation
2024-03-21 8:56 ` Petr Vorel
@ 2024-03-22 10:49 ` Wojtek Porczyk
[not found] ` <Zf1htmFkRl8BIFOX@invisiblethingslab.com>
1 sibling, 0 replies; 16+ messages in thread
From: Wojtek Porczyk @ 2024-03-22 10:49 UTC (permalink / raw)
To: Andrea Cervesato; +Cc: ltp
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
[ sorry for repost, the list does not seem to like PGP/MIME ]
On Thu, Mar 21, 2024 at 09:56:56AM +0100, Petr Vorel wrote:
> > On 3/21/24 09:25, Petr Vorel wrote:
> > > > https://ltp-acerv.readthedocs.io/en/latest/index.html
> > > Out of curiosity, we need to register LTP on readthedocs.io.
> > > Then we need GitHub Action hook (including some secret for auth) for publishing.
> > > Correct?
>
> > > https://docs.readthedocs.io/en/stable/guides/setup/git-repo-manual.html
> > readthedocs works with any kind of git server. The only thing we need, it's
> > a .readthedocs.yaml file inside a specific branch (i.e. master).
> > So we need to create an account for the LTP team in readthedocs.com, then we
> > provide the git repo to readthedocs and site will be published
> > automatically.
> > ltp.readthedocs.io is available, so maybe we should create an account
> > already.
>
> ack.
>
> > > I guess you now deploy to your fork manually, otherwise you would add GitHub
> > > Action config in the patchset.
> > It's automatic done by readthedocs and we don't need Github actions.
If you're on GitHub on GitLab, may I recommend pull request previews:
https://docs.readthedocs.io/en/stable/pull-requests.html
https://docs.readthedocs.io/en/stable/guides/pull-requests.html
RTD will build partial (HTML-only) docs for each push to a PR. It's very nice
for online review, you won't need to push to your own ltp-something.rtfd.io
variant.
- --
pozdrawiam / best regards
Wojtek Porczyk
Gramine / Invisible Things Lab
I do not fear computers,
I fear lack of them.
-- Isaac Asimov
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEaO0VFfpr0tEF6hYkv2vZMhA6I1EFAmX9Yi4ACgkQv2vZMhA6
I1FLLhAAhmWu1bFmcQB8uKNb+3d9OuPA7lwsnuK7Lo7HXQNaWG4qGdQBQLIwi4by
SEBEIw0olU7BF2JVvRun+KGHB9j5PCSHJmXFdmbQy2WLetVdc6Aogwr9O/w2gIvf
/5+iQNBdxe5dfB6naK5MnrR05S83bqjrsRtsDE+Bvng9yWuVdD4xvQTnZNOmfsOK
nzpZs0gNJszzuZ6hHu8dKulvBmqVlRS/+K5Agnd1W1Vc1ZOrJcPuFcn3GipvROgq
s8boPhBaGRoXWfU2Ye/ROw/WcgO/H2ISOC9N9RpQ9y5oQnDmMd6NnrlMEVEjUFkL
5xgqJUsOoisiodemoAFr5fPqvwpnCQQzy7OV3B2VMkrV/arXj9TQKkvKz7LA2HA5
YI5MGajflEZNoL4t5BjmBDMhF3UdlXxYW/EK946qkIpWm8ylX52CEHNXb7FSpNqY
IKDvORWb3TH+mGyLJO2Zyywd3IPnd8Z5v17+qyLMZWg5zfwUJbTPu2c5IXYCA8vf
umoDLdIow0ccnBQE1JE8UTMKvLFuC0w4XJgrvc0b7d+uzV1tbaSoL87XQib3EWYA
tM2Y3o0LTebiaF2UH17D71nvBQbpTEdoKgQa9TUpqdLyzLRuByUcK0odbWC9WDkG
iMwUHyaf0+GGSbrpGK5oNJDsA89Q2ElcqeLBhVY/0W4z1jxHM20=
=acLP
-----END PGP SIGNATURE-----
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 16+ messages in thread[parent not found: <Zf1htmFkRl8BIFOX@invisiblethingslab.com>]
* Re: [LTP] [PATCH 00/10] New LTP documentation
[not found] ` <Zf1htmFkRl8BIFOX@invisiblethingslab.com>
@ 2024-03-22 10:56 ` Andrea Cervesato via ltp
0 siblings, 0 replies; 16+ messages in thread
From: Andrea Cervesato via ltp @ 2024-03-22 10:56 UTC (permalink / raw)
To: Petr Vorel, ltp
Hi Wojtek,
thanks for your suggestions. I will take a look at it.
Kind regards,
Andrea Cervesato
On 3/22/24 11:47, Wojtek Porczyk wrote:
> On Thu, Mar 21, 2024 at 09:56:56AM +0100, Petr Vorel wrote:
>>> On 3/21/24 09:25, Petr Vorel wrote:
>>>>> https://ltp-acerv.readthedocs.io/en/latest/index.html
>>>> Out of curiosity, we need to register LTP on readthedocs.io.
>>>> Then we need GitHub Action hook (including some secret for auth) for publishing.
>>>> Correct?
>>>> https://docs.readthedocs.io/en/stable/guides/setup/git-repo-manual.html
>>> readthedocs works with any kind of git server. The only thing we need, it's
>>> a .readthedocs.yaml file inside a specific branch (i.e. master).
>>> So we need to create an account for the LTP team in readthedocs.com, then we
>>> provide the git repo to readthedocs and site will be published
>>> automatically.
>>> ltp.readthedocs.io is available, so maybe we should create an account
>>> already.
>> ack.
>>
>>>> I guess you now deploy to your fork manually, otherwise you would add GitHub
>>>> Action config in the patchset.
>>> It's automatic done by readthedocs and we don't need Github actions.
> If you're on GitHub on GitLab, may I recommend pull request previews:
> https://docs.readthedocs.io/en/stable/pull-requests.html
> https://docs.readthedocs.io/en/stable/guides/pull-requests.html
>
> RTD will build partial (HTML-only) docs for each push to a PR. It's very nice
> for online review, you won't need to push to your own ltp-something.rtfd.io
> variant.
>
>
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 16+ messages in thread