[LTP] [PATCH 1/2] doc: Update blacklist and whitelist

[LTP] [PATCH 1/2] doc: Update blacklist and whitelist

[Petr Vorel]

From: Petr Vorel <pvorel@suse.cz>

getmsg, getpmsg, putmsg, putpmsg in man UNIMPLEMENTED(2),
move them from whitelist to blacklist.

seccomp is now detected, therefore remove from whitelist.

Fixes: 7248e5c5f7 ("doc: update syscalls statistics")
Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
 doc/conf.py | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index aed0cd50fd..e14c1387d0 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -67,8 +67,6 @@ def generate_syscalls_stats(_):
         'fanotify_init',
         'fanotify_mark',
         'getdents64',
-        'getmsg',
-        'getpmsg',
         'inotify_add_watch',
         'inotify_rm_watch',
         'io_uring_enter',
@@ -79,8 +77,6 @@ def generate_syscalls_stats(_):
         'landlock_restrict_self',
         'lsetxattr',
         'newfstatat',
-        'putmsg',
-        'putpmsg',
         'pkey_alloc',
         'pkey_free',
         'pkey_mprotect',
@@ -90,7 +86,6 @@ def generate_syscalls_stats(_):
         'pwrite64',
         'quotactl_fd',
         'rt_sigpending',
-        'seccomp',
         'semtimedop',
         'sethostname',
     ]
-- 
2.45.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 2/2] doc: Add URL to the sources in syscalls list

[Petr Vorel]

From: Petr Vorel <pvorel@suse.cz>

Provide links to the sources of all tested syscalls.
Users can inspect the test coverage :).

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
 doc/conf.py | 107 +++++++++++++++++++++++++++++++++-------------------
 1 file changed, 68 insertions(+), 39 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index e14c1387d0..7c3da9e112 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -28,9 +28,11 @@ extensions = [
 ]
 
 exclude_patterns = ["html*", '_static*']
+github = 'https://github.com/linux-test-project/ltp'
+master = f'{github}/blob/master'
 extlinks = {
-    'repo': ('https://github.com/linux-test-project/ltp/%s', '%s'),
-    'master': ('https://github.com/linux-test-project/ltp/blob/master/%s', '%s'),
+    'repo': (f'{github}/%s', '%s'),
+    'master': (f'{master}/%s', '%s'),
     'git_man': ('https://git-scm.com/docs/git-%s', 'git %s'),
     # TODO: allow 2nd parameter to show page description instead of plain URL
     'kernel_doc': ('https://docs.kernel.org/%s.html', 'https://docs.kernel.org/%s.html'),
@@ -56,39 +58,53 @@ def generate_syscalls_stats(_):
     generate a file that is included in the statistics documentation section.
     """
     output = '_static/syscalls.rst'
+    syscalls_dir = 'testcases/kernel/syscalls'
 
+    # format syscall_name : folder (for URL, optional)
     # sometimes checking testcases/kernel/syscalls file names are not enough,
     # because in some cases (i.e. io_ring) syscalls are tested, but they are
     # part of a more complex scenario. In the following list, we define syscalls
     # which we know they are 100% tested already.
-    white_list = [
-        'epoll_pwait2',
-        'fadvise64',
-        'fanotify_init',
-        'fanotify_mark',
-        'getdents64',
-        'inotify_add_watch',
-        'inotify_rm_watch',
-        'io_uring_enter',
-        'io_uring_register',
-        'io_uring_setup',
-        'landlock_add_rule',
-        'landlock_create_ruleset',
-        'landlock_restrict_self',
-        'lsetxattr',
-        'newfstatat',
-        'pkey_alloc',
-        'pkey_free',
-        'pkey_mprotect',
-        'prlimit64',
-        'pread64',
-        'pselect6',
-        'pwrite64',
-        'quotactl_fd',
-        'rt_sigpending',
-        'semtimedop',
-        'sethostname',
-    ]
+    # Or syscalls are here to get their folder.
+    white_list = {
+        'epoll_pwait2': 'epoll_pwait',
+        'fadvise64': 'fadvise',
+        'fanotify_init': 'fanotify',
+        'fanotify_mark': 'fanotify',
+        'futex_wait': 'futex',
+        'futex_waitv': 'futex',
+        'futex_wake': 'futex',
+        'getdents64': 'getdents',
+        'inotify_add_watch': 'inotify',
+        'inotify_rm_watch': 'inotify',
+        'inotify_init1': 'inotify',
+        'io_uring_enter': 'io_uring',
+        'io_uring_register': 'io_uring',
+        'io_uring_setup': 'io_uring',
+        'ioprio_get': 'ioprio',
+        'ioprio_set': 'ioprio',
+        'landlock_add_rule': 'landlock',
+        'landlock_create_ruleset': 'landlock',
+        'landlock_restrict_self': 'landlock',
+        'lsetxattr': 'lgetxattr', # llistxattr, lremovexattr
+        'newfstatat': 'fstatat',
+        'pkey_alloc': 'pkeys',
+        'pkey_free': 'pkeys',
+        'pkey_mprotect': 'pkeys',
+        'pread64': 'preadv',
+        'prlimit64': 'getrlimit',
+        'process_vm_readv': 'cma',
+        'process_vm_writev': 'cma',
+        'pselect6': 'select',
+        'pwrite64': 'pwrite',
+        'quotactl_fd': 'quotactl',
+        'rt_sigpending': 'sigpending',
+        'semtimedop': 'ipc/semop',
+        'sethostname': 'setdomainname',
+        'timerfd_gettime': 'timerfd',
+        'timerfd_settime': 'timerfd',
+        'timerfd_create': 'timerfd',
+    }
 
     # populate with not implemented, reserved, unmaintained syscalls defined
     # inside the syscalls file
@@ -144,28 +160,38 @@ def generate_syscalls_stats(_):
                 ker_syscalls.append(match.group('syscall'))
 
     # collect all LTP tested syscalls
-    ltp_syscalls = []
-    for root, _, files in os.walk('../testcases/kernel/syscalls'):
+    ltp_syscalls = {}
+    for root, _, files in os.walk('../' + syscalls_dir):
         for myfile in files:
             if myfile.endswith('.c'):
-                ltp_syscalls.append(myfile)
+                ltp_syscalls[myfile] = root[3:]
 
     # compare kernel syscalls with LTP tested syscalls
     syscalls = {}
+    url = {}
     for kersc in ker_syscalls:
         if kersc in black_list:
             continue
 
         if kersc not in syscalls:
-            if kersc in white_list:
+            if kersc in white_list.keys():
                 syscalls[kersc] = True
+                if white_list[kersc]:
+                    url[kersc] = syscalls_dir + '/' + white_list[kersc]
+
                 continue
 
             syscalls[kersc] = False
 
-        for ltpsc in ltp_syscalls:
+        for ltpsc in ltp_syscalls.keys():
             if ltpsc.startswith(kersc):
                 syscalls[kersc] = True
+                if kersc in url:
+                    continue
+                # Be conservative and use only directories which match exactly the syscall.
+                # Otherwise mkdir will be linked to mkdirat, openat to openat2, etc.
+                if os.path.basename(ltp_syscalls[ltpsc]) == kersc:
+                    url[kersc] = ltp_syscalls[ltpsc]
 
     # generate the statistics file
     tested_syscalls = [key for key, val in syscalls.items() if val]
@@ -192,18 +218,21 @@ def generate_syscalls_stats(_):
     ]
 
     for sysname, tested in syscalls.items():
+        name = f'{sysname}'
         if tested:
+            if sysname in url.keys():
+                name = f'`{sysname} <{github}/tree/master/{url[sysname]}>`_'
             if (index_tested % 3) == 0:
-                table_tested.append(f'    * - {sysname}\n')
+                table_tested.append(f'    * - {name}\n')
             else:
-                table_tested.append(f'      - {sysname}\n')
+                table_tested.append(f'      - {name}\n')
 
             index_tested += 1
         else:
             if (index_untest % 3) == 0:
-                table_untest.append(f'    * - {sysname}\n')
+                table_untest.append(f'    * - {name}\n')
             else:
-                table_untest.append(f'      - {sysname}\n')
+                table_untest.append(f'      - {name}\n')
 
             index_untest += 1
 
-- 
2.45.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/2] doc: Update blacklist and whitelist

[Petr Vorel]

> From: Petr Vorel <pvorel@suse.cz>

> getmsg, getpmsg, putmsg, putpmsg in man UNIMPLEMENTED(2),
> move them from whitelist to blacklist.

> seccomp is now detected, therefore remove from whitelist.

> Fixes: 7248e5c5f7 ("doc: update syscalls statistics")
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
>  doc/conf.py | 5 -----
>  1 file changed, 5 deletions(-)

> diff --git a/doc/conf.py b/doc/conf.py
> index aed0cd50fd..e14c1387d0 100644
> --- a/doc/conf.py
> +++ b/doc/conf.py
> @@ -67,8 +67,6 @@ def generate_syscalls_stats(_):
>          'fanotify_init',
>          'fanotify_mark',
>          'getdents64',
> -        'getmsg',
> -        'getpmsg',
Actually these 2 were supposed to be moved to blacklist.

>          'inotify_add_watch',
>          'inotify_rm_watch',
>          'io_uring_enter',
> @@ -79,8 +77,6 @@ def generate_syscalls_stats(_):
>          'landlock_restrict_self',
>          'lsetxattr',
>          'newfstatat',
> -        'putmsg',
> -        'putpmsg',
And these two as well.

>          'pkey_alloc',
>          'pkey_free',
>          'pkey_mprotect',
> @@ -90,7 +86,6 @@ def generate_syscalls_stats(_):
>          'pwrite64',
>          'quotactl_fd',
>          'rt_sigpending',
> -        'seccomp',
>          'semtimedop',
>          'sethostname',
>      ]

Diff to the correct code below. I'm sorry for the noise.
Corrected code is in my fork in case somebody wants to test it:
https://github.com/pevik/ltp/commits/refs/heads/doc/syscalls-url.fixes/

+++ doc/conf.py
@@ -96,11 +96,15 @@ def generate_syscalls_stats(_):
         '_newselect',
         '_sysctl',
         'afs_syscall',
+        'getmsg',
+        'getpmsg',
         'cachectl',
         'create_module',
         'get_kernel_syms',
         'mq_getsetattr',
         'nfsservctl',
+        'putmsg',
+        'putpmsg',
         'query_module',
         'reserved177',
         'reserved193',

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 2/2] doc: Add URL to the sources in syscalls list

[Andrea Cervesato via ltp]

Hi Petr,

On 11/15/24 04:39, Petr Vorel wrote:
> +    # Or syscalls are here to get their folder.
> +    white_list = {
> +        'epoll_pwait2': 'epoll_pwait',
> +        'fadvise64': 'fadvise',
> +        'fanotify_init': 'fanotify',
> +        'fanotify_mark': 'fanotify',
> +        'futex_wait': 'futex',
> +        'futex_waitv': 'futex',
> +        'futex_wake': 'futex',
> +        'getdents64': 'getdents',
> +        'inotify_add_watch': 'inotify',
> +        'inotify_rm_watch': 'inotify',
> +        'inotify_init1': 'inotify',
> +        'io_uring_enter': 'io_uring',
> +        'io_uring_register': 'io_uring',
> +        'io_uring_setup': 'io_uring',
> +        'ioprio_get': 'ioprio',
> +        'ioprio_set': 'ioprio',
> +        'landlock_add_rule': 'landlock',
> +        'landlock_create_ruleset': 'landlock',
> +        'landlock_restrict_self': 'landlock',
> +        'lsetxattr': 'lgetxattr', # llistxattr, lremovexattr
> +        'newfstatat': 'fstatat',
> +        'pkey_alloc': 'pkeys',
> +        'pkey_free': 'pkeys',
> +        'pkey_mprotect': 'pkeys',
> +        'pread64': 'preadv',
> +        'prlimit64': 'getrlimit',
> +        'process_vm_readv': 'cma',
> +        'process_vm_writev': 'cma',
> +        'pselect6': 'select',
> +        'pwrite64': 'pwrite',
> +        'quotactl_fd': 'quotactl',
> +        'rt_sigpending': 'sigpending',
> +        'semtimedop': 'ipc/semop',
> +        'sethostname': 'setdomainname',
> +        'timerfd_gettime': 'timerfd',
> +        'timerfd_settime': 'timerfd',
> +        'timerfd_create': 'timerfd',
> +    }

The problem with this approach and the reason why I didn't use it, is 
that if you change the testing folder, you need to update the docs 
generator script. But I also see the goal, so probably this is the only 
approach we should follow.

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 2/2] doc: Add URL to the sources in syscalls list

[Petr Vorel]

> Hi Petr,

> On 11/15/24 04:39, Petr Vorel wrote:
> > +    # Or syscalls are here to get their folder.
> > +    white_list = {
> > +        'epoll_pwait2': 'epoll_pwait',
> > +        'fadvise64': 'fadvise',
> > +        'fanotify_init': 'fanotify',
> > +        'fanotify_mark': 'fanotify',
> > +        'futex_wait': 'futex',
> > +        'futex_waitv': 'futex',
> > +        'futex_wake': 'futex',
> > +        'getdents64': 'getdents',
> > +        'inotify_add_watch': 'inotify',
> > +        'inotify_rm_watch': 'inotify',
> > +        'inotify_init1': 'inotify',
> > +        'io_uring_enter': 'io_uring',
> > +        'io_uring_register': 'io_uring',
> > +        'io_uring_setup': 'io_uring',
> > +        'ioprio_get': 'ioprio',
> > +        'ioprio_set': 'ioprio',
> > +        'landlock_add_rule': 'landlock',
> > +        'landlock_create_ruleset': 'landlock',
> > +        'landlock_restrict_self': 'landlock',
> > +        'lsetxattr': 'lgetxattr', # llistxattr, lremovexattr
> > +        'newfstatat': 'fstatat',
> > +        'pkey_alloc': 'pkeys',
> > +        'pkey_free': 'pkeys',
> > +        'pkey_mprotect': 'pkeys',
> > +        'pread64': 'preadv',
> > +        'prlimit64': 'getrlimit',
> > +        'process_vm_readv': 'cma',
> > +        'process_vm_writev': 'cma',
> > +        'pselect6': 'select',
> > +        'pwrite64': 'pwrite',
> > +        'quotactl_fd': 'quotactl',
> > +        'rt_sigpending': 'sigpending',
> > +        'semtimedop': 'ipc/semop',
> > +        'sethostname': 'setdomainname',
> > +        'timerfd_gettime': 'timerfd',
> > +        'timerfd_settime': 'timerfd',
> > +        'timerfd_create': 'timerfd',
> > +    }

> The problem with this approach and the reason why I didn't use it, is that
> if you change the testing folder, you need to update the docs generator
> script. But I also see the goal, so probably this is the only approach we
> should follow.

Other thing would be move around folders, but IMHO some syscalls are really
related, that's why it's good they are in "ipc" subfolder. OTOH bind(), send(),
recvmsg(), ... aren't in "net or "network" subfolder.

Whole this was inspired by Metan's suggestion to deriver info about syscalls
from the folder.

Kind regards,
Petr

> Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 2/2] doc: Add URL to the sources in syscalls list

[Petr Vorel]

Hi all,

> Hi Petr,

> On 11/15/24 04:39, Petr Vorel wrote:
> > +    # Or syscalls are here to get their folder.
> > +    white_list = {
> > +        'epoll_pwait2': 'epoll_pwait',
> > +        'fadvise64': 'fadvise',
> > +        'fanotify_init': 'fanotify',
> > +        'fanotify_mark': 'fanotify',
> > +        'futex_wait': 'futex',
> > +        'futex_waitv': 'futex',
> > +        'futex_wake': 'futex',
> > +        'getdents64': 'getdents',
> > +        'inotify_add_watch': 'inotify',
> > +        'inotify_rm_watch': 'inotify',
> > +        'inotify_init1': 'inotify',
> > +        'io_uring_enter': 'io_uring',
> > +        'io_uring_register': 'io_uring',
> > +        'io_uring_setup': 'io_uring',
> > +        'ioprio_get': 'ioprio',
> > +        'ioprio_set': 'ioprio',
> > +        'landlock_add_rule': 'landlock',
> > +        'landlock_create_ruleset': 'landlock',
> > +        'landlock_restrict_self': 'landlock',
> > +        'lsetxattr': 'lgetxattr', # llistxattr, lremovexattr
> > +        'newfstatat': 'fstatat',
> > +        'pkey_alloc': 'pkeys',
> > +        'pkey_free': 'pkeys',
> > +        'pkey_mprotect': 'pkeys',
> > +        'pread64': 'preadv',
> > +        'prlimit64': 'getrlimit',
> > +        'process_vm_readv': 'cma',
> > +        'process_vm_writev': 'cma',
> > +        'pselect6': 'select',
> > +        'pwrite64': 'pwrite',
> > +        'quotactl_fd': 'quotactl',
> > +        'rt_sigpending': 'sigpending',
> > +        'semtimedop': 'ipc/semop',
> > +        'sethostname': 'setdomainname',
> > +        'timerfd_gettime': 'timerfd',
> > +        'timerfd_settime': 'timerfd',
> > +        'timerfd_create': 'timerfd',
> > +    }

> The problem with this approach and the reason why I didn't use it, is that
> if you change the testing folder, you need to update the docs generator
> script. But I also see the goal, so probably this is the only approach we
> should follow.

Also, my first version was to match only syscalls, which are the folder with the
same name as the syscall. If we prefer this is better, I can post this version.

@Andrea @Cyril: Other think I would like to have our metadata doc somehow
generated for the master. But that would require to have installed asciidoctor
on container, that will not work for sphinx. Also our metadata syntax is somehow
LTP specific (at least /*\ starter) and having doc on 2 places
(readthedocs https://linux-test-project.readthedocs.io/ and static metadata doc
file uploaded to releases (e.g.
https://github.com/linux-test-project/ltp/releases/download/20240930/metadata.20240930.html)
is not optimal. I also like python scripting more than perl (used for metadata).

Other option would be to drop metadata syntax and transform docs to sphinx
format. But that would require a lot of scripting, we would not want to do it manually.

Also, doc added to releases vs. online doc:
* Should we add generated readthedocs to releases? One could have docs forever.
* Should we have also the latest release doc in online readthedocs? Or even for
* all releases? ATM we have just master.

Kind regards,
Petr

> Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 2/2] doc: Add URL to the sources in syscalls list

[Andrea Cervesato via ltp]

Hi Petr,

On 11/15/24 12:04, Petr Vorel wrote:
> @Andrea @Cyril: Other think I would like to have our metadata doc somehow
> generated for the master. But that would require to have installed asciidoctor
> on container, that will not work for sphinx. Also our metadata syntax is somehow
> LTP specific (at least /*\ starter) and having doc on 2 places
> (readthedocshttps://linux-test-project.readthedocs.io/ and static metadata doc
> file uploaded to releases (e.g.
> https://github.com/linux-test-project/ltp/releases/download/20240930/metadata.20240930.html)
> is not optimal. I also like python scripting more than perl (used for metadata).

I remember we talked with Cyril long time ago about this. Yes, it's 
possibly something which requires a move from asciidoc to sphinx syntax. 
It's not so difficult. The rules are pretty simple, since we really 
don't have much information inside the asciidoc format.

Most of the times, it's only a matter of replacing /*\ with /** and to 
change [Description/Algorithm] with ===== underline.

> Other option would be to drop metadata syntax and transform docs to sphinx
> format. But that would require a lot of scripting, we would not want to do it manually.
>
> Also, doc added to releases vs. online doc:
> * Should we add generated readthedocs to releases? One could have docs forever.
> * Should we have also the latest release doc in online readthedocs? Or even for
> * all releases? ATM we have just master.

I think it makes sense to have it in the release, but I don't know if 
it's possible to generate a single file in a easy way. Maybe take a look 
at singlehtml.

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp