Re: [LTP] [PATCH v2] Remove TODO

[Petr Vorel <pvorel@suse.cz>]

Hi Andrea,

Thanks!

LGTM, a few notes below.

Most important is switch gmane to lore. With that:
Reviewed-by: Petr Vorel <pvorel@suse.cz>

> -Rewrite old and add new controller testcases
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -We already started working on this with rewriting cgroup_fj testcases and
> -newly added pids testcases. Have a look at testcases/kernel/controllers/.
> -
> -
> -Shell tests cleanups
> -~~~~~~~~~~~~~~~~~~~~
> -
> -There are numerous testcases written in shell that does not follow single style
> -and use the broken-by-design binaries from tools/apicmds. These should be
> -cleaned up and fixed to use the test.sh test library. These include most of
> -tests in testcases/commands/ for example.
> diff --git a/doc/index.rst b/doc/index.rst
> index acd16cdbf..6cfe21e8d 100644
> --- a/doc/index.rst
> +++ b/doc/index.rst
> @@ -13,6 +13,7 @@
>     users/supported_systems
>     users/stats
>     users/test_catalog
> +   users/todo

>  .. toctree::
>     :maxdepth: 3
> @@ -59,6 +60,9 @@ For users
>  :doc:`users/test_catalog`
>     The LTP test catalog

> +:doc:`users/todo`
> +   List of tasks maintainers are working on
> +
>  For developers
>  --------------

> diff --git a/doc/users/todo.rst b/doc/users/todo.rst
> new file mode 100644
> index 000000000..a8d8f9d37
> --- /dev/null
> +++ b/doc/users/todo.rst
> @@ -0,0 +1,50 @@
> +.. SPDX-License-Identifier: GPL-2.0-or-later
> +
> +List of ongoing tasks
> +=====================
> +
> +This is a comprehensive list of tasks where LTP maintainers are currently
> +working on. Priorities might change over time, but these are the most important
> +points which are currently being achieved.

Maybe mention here https://github.com/linux-test-project/ltp/issues ?
(i.e. Fixing bugs in the tests?)

I'm not sure if easyhacks are still relevant (atm 8 tasks):
https://github.com/linux-test-project/ltp/issues?q=state%3Aopen%20label%3Aeasyhack

> +
> +Test new syscalls
> +-----------------
> +
> +Syscalls and new syscalls flags are added to Linux kernel each development
> +cycle and LTP still falls behind. Unfortunately there is no single place that
> +would store comprehensive list of syscalls, but there are a few places to look
> +at:
> +
> +- `man-pages repo <http://git.kernel.org/cgit/docs/man-pages/man-pages.git>`_,
nit: s/repo/repository/ (spoken vs. written English?)

> +  or the ``man2`` directory, where it's possible to find newly documented
> +  functionalities.
> +- `LWN <http://lwn.net>`_ weekly editions.
> +- `linux-api mailing list <http://dir.gmane.org/gmane.linux.kernel.api>`_, where

http://dir.gmane.org/gmane.linux.kernel.api even does not exist any more
(gmane.org is gone), please use lore:

https://lore.kernel.org/linux-api/

NOTE kernel community switched even lkml.org to lore (they prefer it as it's run
by Linux Foundation instead of 3rd party):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=05a5f51ca566

> +  changes in kernel userspace API are discussed.
> +
> +Rewrite old API tests
> +---------------------
> +
> +LTP has a long story and, at certain point of its development, new API were
> +introduced to make kernel testing easier and more efficient. This happened when
> +lots of tests were still using old, messy API.
> +
> +Some of these tests have been converted to the new API, but this process is
> +still ongoing for many others. To have an overview of the tests using old API,
> +you can run the following command inside the LTP root folder:
> +
> +.. code-block:: bash
> +
> +        $ wc -l $(grep -R '"test\.h"' --include="*.c" testcases/kernel/ | cut -d: -f1) | sort -g

wc -l $(git grep 'include "test\.h"' testcases/ | cut -d: -f1) | sort -g

* there are other tests than in testcases/kernel/
* git grep is faster (we expect people working with git clone)

> +
> +Fade out shell scripts
> +----------------------
> +
> +LTP was initially thought as a generic framework for running tests with both
> +shell and plain-C languages. Even if writing tests in shell script might seem
> +easy, the reality is that debugging and maintaining certain test cases is
> +difficult and slow down the whole validation process.

+1 for pointing out shell is problematic and discourage users to write new
tests. Maybe mention also a shell loader?

I would say the problem is with unreliability/instability of shell
implementation (even we use C parts glued by shell it's not enough). Specially
cgroup related tests should be all rewritten in C as shell does not add enough
control + race condition.

> +
> +LTP maintainers are working on converting shell scripts to plain-C tests, in
> +order to reduce the impact that shell scripts might have on the overall kernel
> +testing.

I remember Wei converted recently test_6 to cpuset02.c and that's it. And we
~300 tests (both old and new API):

$ git grep -l -e '^\. .*_lib\.sh' -e '^\. .*test.sh' |wc -l
294

The reality is that we convert most of them to shell loader (way less work, also
for many tests e.g. in testcases/{commands,network}/ it could be sufficient
enough).

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp