[LTP] [PATCH v2] Remove TODO

[LTP] [PATCH v2] Remove TODO

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

TODO file is not updated and it talks about goals we already reached via
new LTP documentation. In general, it contains generic and random topics
which none is updating anymore, so it's better to remove it in order to
create less confusion for new comers.

Instead, we create a todo list in the new documentation, providing a
more clear intention of what are the current LTP goals.

Acked-by: Petr Vorel <pvorel@suse.cz>
Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
Changes in v2:
- add doc/users/todo.rst section
- Link to v1: https://lore.kernel.org/r/20251006-remove_todo-v1-1-5bab5f6f77f5@suse.com
---
 TODO               | 39 ---------------------------------------
 doc/index.rst      |  4 ++++
 doc/users/todo.rst | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 54 insertions(+), 39 deletions(-)

diff --git a/TODO b/TODO
deleted file mode 100644
index fe0a3ab9b..000000000
--- a/TODO
+++ /dev/null
@@ -1,39 +0,0 @@
-LTP TODO
---------
-
-
-Write more syscall tests
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-Syscalls and new syscall flags are added to Linux kernel each development cycle
-and LTP still falls behind. Unfortunately there is no single place that would
-store comprehensive list of syscalls, but there are a few places to look at:
-
-One of the options would be looking at changes in man-pages git[1] in man2/
-directory to find out newly documented functionality.
-
-Another good source of information are kernel pages in LWN[2] weekly
-editions.
-
-Then there is linux-api mailing list[3] where changes in kernel userspace API
-should be discussed.
-
-[1] http://git.kernel.org/cgit/docs/man-pages/man-pages.git
-[2] http://lwn.net
-[3] http://dir.gmane.org/gmane.linux.kernel.api
-
-
-Rewrite old and add new controller testcases
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-We already started working on this with rewriting cgroup_fj testcases and
-newly added pids testcases. Have a look at testcases/kernel/controllers/.
-
-
-Shell tests cleanups
-~~~~~~~~~~~~~~~~~~~~
-
-There are numerous testcases written in shell that does not follow single style
-and use the broken-by-design binaries from tools/apicmds. These should be
-cleaned up and fixed to use the test.sh test library. These include most of
-tests in testcases/commands/ for example.
diff --git a/doc/index.rst b/doc/index.rst
index acd16cdbf..6cfe21e8d 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -13,6 +13,7 @@
    users/supported_systems
    users/stats
    users/test_catalog
+   users/todo
 
 .. toctree::
    :maxdepth: 3
@@ -59,6 +60,9 @@ For users
 :doc:`users/test_catalog`
    The LTP test catalog
 
+:doc:`users/todo`
+   List of tasks maintainers are working on
+
 For developers
 --------------
 
diff --git a/doc/users/todo.rst b/doc/users/todo.rst
new file mode 100644
index 000000000..a8d8f9d37
--- /dev/null
+++ b/doc/users/todo.rst
@@ -0,0 +1,50 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+List of ongoing tasks
+=====================
+
+This is a comprehensive list of tasks where LTP maintainers are currently
+working on. Priorities might change over time, but these are the most important
+points which are currently being achieved.
+
+Test new syscalls
+-----------------
+
+Syscalls and new syscalls flags are added to Linux kernel each development
+cycle and LTP still falls behind. Unfortunately there is no single place that
+would store comprehensive list of syscalls, but there are a few places to look
+at:
+
+- `man-pages repo <http://git.kernel.org/cgit/docs/man-pages/man-pages.git>`_,
+  or the ``man2`` directory, where it's possible to find newly documented
+  functionalities.
+- `LWN <http://lwn.net>`_ weekly editions.
+- `linux-api mailing list <http://dir.gmane.org/gmane.linux.kernel.api>`_, where
+  changes in kernel userspace API are discussed.
+
+Rewrite old API tests
+---------------------
+
+LTP has a long story and, at certain point of its development, new API were
+introduced to make kernel testing easier and more efficient. This happened when
+lots of tests were still using old, messy API.
+
+Some of these tests have been converted to the new API, but this process is
+still ongoing for many others. To have an overview of the tests using old API,
+you can run the following command inside the LTP root folder:
+
+.. code-block:: bash
+
+        $ wc -l $(grep -R '"test\.h"' --include="*.c" testcases/kernel/ | cut -d: -f1) | sort -g
+
+Fade out shell scripts
+----------------------
+
+LTP was initially thought as a generic framework for running tests with both
+shell and plain-C languages. Even if writing tests in shell script might seem
+easy, the reality is that debugging and maintaining certain test cases is
+difficult and slow down the whole validation process.
+
+LTP maintainers are working on converting shell scripts to plain-C tests, in
+order to reduce the impact that shell scripts might have on the overall kernel
+testing.

---
base-commit: 79b7f212b3e56f1a4c33af3f61f66502b574cf1e
change-id: 20251006-remove_todo-c29a585e8b01

Best regards,
-- 
Andrea Cervesato <andrea.cervesato@suse.com>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2] Remove TODO

[Petr Vorel]

Hi Andrea,

Thanks!

LGTM, a few notes below.

Most important is switch gmane to lore. With that:
Reviewed-by: Petr Vorel <pvorel@suse.cz>

> -Rewrite old and add new controller testcases
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -We already started working on this with rewriting cgroup_fj testcases and
> -newly added pids testcases. Have a look at testcases/kernel/controllers/.
> -
> -
> -Shell tests cleanups
> -~~~~~~~~~~~~~~~~~~~~
> -
> -There are numerous testcases written in shell that does not follow single style
> -and use the broken-by-design binaries from tools/apicmds. These should be
> -cleaned up and fixed to use the test.sh test library. These include most of
> -tests in testcases/commands/ for example.
> diff --git a/doc/index.rst b/doc/index.rst
> index acd16cdbf..6cfe21e8d 100644
> --- a/doc/index.rst
> +++ b/doc/index.rst
> @@ -13,6 +13,7 @@
>     users/supported_systems
>     users/stats
>     users/test_catalog
> +   users/todo

>  .. toctree::
>     :maxdepth: 3
> @@ -59,6 +60,9 @@ For users
>  :doc:`users/test_catalog`
>     The LTP test catalog

> +:doc:`users/todo`
> +   List of tasks maintainers are working on
> +
>  For developers
>  --------------

> diff --git a/doc/users/todo.rst b/doc/users/todo.rst
> new file mode 100644
> index 000000000..a8d8f9d37
> --- /dev/null
> +++ b/doc/users/todo.rst
> @@ -0,0 +1,50 @@
> +.. SPDX-License-Identifier: GPL-2.0-or-later
> +
> +List of ongoing tasks
> +=====================
> +
> +This is a comprehensive list of tasks where LTP maintainers are currently
> +working on. Priorities might change over time, but these are the most important
> +points which are currently being achieved.

Maybe mention here https://github.com/linux-test-project/ltp/issues ?
(i.e. Fixing bugs in the tests?)

I'm not sure if easyhacks are still relevant (atm 8 tasks):
https://github.com/linux-test-project/ltp/issues?q=state%3Aopen%20label%3Aeasyhack

> +
> +Test new syscalls
> +-----------------
> +
> +Syscalls and new syscalls flags are added to Linux kernel each development
> +cycle and LTP still falls behind. Unfortunately there is no single place that
> +would store comprehensive list of syscalls, but there are a few places to look
> +at:
> +
> +- `man-pages repo <http://git.kernel.org/cgit/docs/man-pages/man-pages.git>`_,
nit: s/repo/repository/ (spoken vs. written English?)

> +  or the ``man2`` directory, where it's possible to find newly documented
> +  functionalities.
> +- `LWN <http://lwn.net>`_ weekly editions.
> +- `linux-api mailing list <http://dir.gmane.org/gmane.linux.kernel.api>`_, where

http://dir.gmane.org/gmane.linux.kernel.api even does not exist any more
(gmane.org is gone), please use lore:

https://lore.kernel.org/linux-api/

NOTE kernel community switched even lkml.org to lore (they prefer it as it's run
by Linux Foundation instead of 3rd party):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05a5f51ca566

> +  changes in kernel userspace API are discussed.
> +
> +Rewrite old API tests
> +---------------------
> +
> +LTP has a long story and, at certain point of its development, new API were
> +introduced to make kernel testing easier and more efficient. This happened when
> +lots of tests were still using old, messy API.
> +
> +Some of these tests have been converted to the new API, but this process is
> +still ongoing for many others. To have an overview of the tests using old API,
> +you can run the following command inside the LTP root folder:
> +
> +.. code-block:: bash
> +
> +        $ wc -l $(grep -R '"test\.h"' --include="*.c" testcases/kernel/ | cut -d: -f1) | sort -g

wc -l $(git grep 'include "test\.h"' testcases/ | cut -d: -f1) | sort -g

* there are other tests than in testcases/kernel/
* git grep is faster (we expect people working with git clone)

> +
> +Fade out shell scripts
> +----------------------
> +
> +LTP was initially thought as a generic framework for running tests with both
> +shell and plain-C languages. Even if writing tests in shell script might seem
> +easy, the reality is that debugging and maintaining certain test cases is
> +difficult and slow down the whole validation process.

+1 for pointing out shell is problematic and discourage users to write new
tests. Maybe mention also a shell loader?

I would say the problem is with unreliability/instability of shell
implementation (even we use C parts glued by shell it's not enough). Specially
cgroup related tests should be all rewritten in C as shell does not add enough
control + race condition.

> +
> +LTP maintainers are working on converting shell scripts to plain-C tests, in
> +order to reduce the impact that shell scripts might have on the overall kernel
> +testing.

I remember Wei converted recently test_6 to cpuset02.c and that's it. And we
~300 tests (both old and new API):

$ git grep -l -e '^\. .*_lib\.sh' -e '^\. .*test.sh' |wc -l
294

The reality is that we convert most of them to shell loader (way less work, also
for many tests e.g. in testcases/{commands,network}/ it could be sufficient
enough).

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2] Remove TODO

[Andrea Cervesato via ltp]

Hi,

I'm thinking that maybe we can even generate the list of tests which are
currently using the old API and show it. WDYT?

-- 
Andrea Cervesato
SUSE QE Automation Engineer Linux
andrea.cervesato@suse.com


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v2] Remove TODO

[Petr Vorel]

> Hi,

> I'm thinking that maybe we can even generate the list of tests which are
> currently using the old API and show it. WDYT?

Even it's trivial I would be ok with git grep example and invest the energy into
finishing the transition of the docs to REST (at least the C API part would be
nice to finish). But up to you of course.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp