Re: [LTP] [PATCH v3] Remove TODO

[Andrea Cervesato via ltp <ltp@lists.linux.it>]

Hi Petr,

On Mon Oct 13, 2025 at 5:38 PM CEST, Petr Vorel wrote:
> Hi Andrea,
>
> once more, now Cc the list. I'm sorry for the noise.
>
> Generally LGTM, notes below.
>
> Reviewed-by: Petr Vorel <pvorel@suse.cz>
>
>> TODO file is not updated and it talks about goals we already reached via
>> new LTP documentation. In general, it contains generic and random topics
>> which none is updating anymore, so it's better to remove it in order to
>> create less confusion for new comers.
>
> nit: s/new comers/newcomers/
>
>> Instead, we create a todo list in the new documentation, providing a
>> more clear intention of what are the current LTP goals.
>
>> Acked-by: Petr Vorel <pvorel@suse.cz>
>> Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
>> ---
>> Changes in v3:
>> - fix typo
>> - update command to generate test with old API
>> - add kirk link in TODO
>> - shell loader reference
>> - Link to v2: https://lore.kernel.org/r/20251013-remove_todo-v2-1-d0a46ad14e34@suse.com
>
>> Changes in v2:
>> - add doc/users/todo.rst section
>> - Link to v1: https://lore.kernel.org/r/20251006-remove_todo-v1-1-5bab5f6f77f5@suse.com
>> ---
>>  TODO               | 39 ----------------------------
>>  doc/index.rst      |  4 +++
>>  doc/users/todo.rst | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 78 insertions(+), 39 deletions(-)
>
> ...
>> diff --git a/doc/users/todo.rst b/doc/users/todo.rst
>> new file mode 100644
>> index 000000000..2484eaf32
>> --- /dev/null
>> +++ b/doc/users/todo.rst
>
> IMHO this TODO page is for new LTP developers to see ideas what to work on,
> right?  Thinking about it twice, IMHO this should be in doc/developers/.
> At least normal LTP users (testers) don't bother what we want to develop.

Probably better yes, I will move it to developers.

>
>> @@ -0,0 +1,74 @@
>> +.. SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +List of ongoing tasks
>> +=====================
>> +
>> +This is a comprehensive list of tasks where LTP maintainers are currently
>> +working on. Priorities might change over time, but these are the most important
>> +points which are currently being achieved.
>> +
>> +Fade out old tests runner
>> +-------------------------
>> +
>> +``runltp`` script is old and unmaintaned. We are slowly shifting to
>> +`kirk <https://github.com/linux-test-project/kirk>`_, that will be our official
>> +tests runner in the next future.
>> +
>> +kirk provides support for remote testing via Qemu, SSH, LTX, parallel
>> +execution and much more.
>
> IMHO this TODO page is for new LTP developers to see ideas what to work on,
> right? Wouldn't be more useful to put it to the page which people using LTP will
> be reading? Maybe one of these:
>
> https://linux-test-project.readthedocs.io/en/latest/users/quick_start.html
> https://linux-test-project.readthedocs.io/en/latest/users/setup_tests.html

I don't know about this, it makes sense to have a separate place for
this list. TODO/quickstart/setup are different things.

>
>> +
>> +Test new syscalls
>> +-----------------
>> +
>> +Syscalls and new syscalls flags are added to Linux kernel each development
>> +cycle and LTP still falls behind. Unfortunately there is no single place that
>> +would store comprehensive list of syscalls, but there are a few places to look
>> +at:
>> +
>> +- `man-pages repository <http://git.kernel.org/cgit/docs/man-pages/man-pages.git>`_
>> +  or the ``man2`` directory, where it's possible to find newly documented
>> +  functionalities.
>> +- `LWN <http://lwn.net>`_ weekly editions.
>> +- `linux-api mailing list <https://lore.kernel.org/linux-api/>`_ where
>> +  changes in kernel userspace API are discussed.
>> +- `LTP Github issues <https://github.com/linux-test-project/ltp/issues>`_
>> +
>> +Rewrite old API tests
>> +---------------------
>> +
>> +LTP has a long story and, at certain point of its development, new API were
>> +introduced to make kernel testing easier and more efficient. This happened when
>> +lots of tests were still using old, messy API.
>> +
>> +Some of these tests have been converted to the new API, but this process is
>> +still ongoing for many others. To have an overview of the tests using old API,
>> +please run the following command inside the LTP root folder:
>> +
>> +.. code-block:: bash
>> +
>> +        git --no-pager grep 'include "test\.h"' testcases/ | cut -d: -f1
>
> I quite like your first version which showed number of lines. But if you want to
> just list of the old API tests, why not use git grep -l ?
>
> git --no-pager grep -l 'include "test\.h"' testcases/
>
>> +
>> +Fade out shell scripts
>> +----------------------
>> +
>> +LTP was initially thought as a generic framework for running tests with both
>> +shell and plain-C languages. Even if writing tests in shell script might seem
>> +easy, the reality is that debugging and maintaining certain test cases is
>> +difficult and slow down the whole validation process. This is particularly
>> +visible for cgroup tests, since shell doesn't add enough control over race
>> +conditions.
>
> That reminds me Cyril's saying "it's easy to write portable shell *interpreter*
> than portable shell *scripts*. :).
>
>> +
>> +LTP maintainers are working on converting shell scripts to plain-C tests, in
>> +order to reduce the impact that shell scripts might have on the overall kernel
>> +testing.
>> +
>> +For a complete list of shell tests, please run the following command inside the
>> +LTP root folder:
>> +
>> +.. code-block:: bash
>> +
>> +        git --no-pager grep -l -e '^\. .*_lib\.sh' -e '^\. .*test.sh'
>> +
>> +LTP also provides a shell loader implementation for plain-C tests using
>> +``tst_run_script`` features. Please take a look at the
>
> That's tst_run_shell. Could you please use link to the source?
>
> :master:`testcases/lib/tst_run_shell.c`
>
>> +:doc:`LTP API <../developers/api_c_tests>`.
> For shell loader are useful only struct tst_test tags in JSON comments.
> Maybe point directly to struct tst_test?
>
> Maybe the most descriptive are tests in testcases/lib/tests/.
>
> :master:`testcases/lib/tst_run_shell.c` features. Please take a look at the
> :ref:`struct tst_test` and examples in :master:`testcases/lib/tests/`.
>
> (NOTE: besides testcases/lib/tst_run_shell.c there is also
> testcases/lib/tst_run.sh and testcases/lib/tst_env.sh but we can ignore it in
> this short info.  If we really convert most of the shell tests to shell runner,
> we should write doc in a separate page. In that case we could leave
> doc/old/Shell-Test-API.asciidoc unconverted).

I will just skip the C API reference and point to the tst_run_shell
file.
>
> Kind regards,
> Petr


Kind regards,

-- 
Andrea Cervesato
SUSE QE Automation Engineer Linux
andrea.cervesato@suse.com


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp