[LTP] [PATCH v1] doc: Add new API conversion stats

[LTP] [PATCH v1] doc: Add new API conversion stats

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Show inside documentation what's the status of the current LTP
refactoring plan that aims to convert all tests into the new API.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 doc/.gitignore      |  1 +
 doc/conf.py         | 48 +++++++++++++++++++++++++++++++++++++++++++++
 doc/users/stats.rst |  1 +
 3 files changed, 50 insertions(+)

diff --git a/doc/.gitignore b/doc/.gitignore
index 173179852..ab979ad18 100644
--- a/doc/.gitignore
+++ b/doc/.gitignore
@@ -1,4 +1,5 @@
 html/
 build/
 _static/syscalls.rst
+_static/old_api.rst
 syscalls.tbl
diff --git a/doc/conf.py b/doc/conf.py
index fb3e83cf2..2cec90a64 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -39,6 +39,53 @@ html_theme = 'sphinx_rtd_theme'
 html_static_path = ['_static']
 
 
+def generate_old_api_stats(_):
+    """
+    Generate statistics for old API tests. We enter the tests folder and we
+    count all tests implementations using old API. The way we count them, is to
+    open every *.c file and to verify that "#include <test.h>" directive is
+    present.
+    """
+    output = '_static/old_api.rst'
+
+    old_regex = re.compile(r'#include.*[<"\']test\.h[>"\']')
+    new_regex = re.compile(r'#include.*[<"\']tst_test\.h[>"\']')
+
+    old_tests = 0
+    new_tests = 0
+
+    for root, _, files in os.walk('../testcases'):
+        for fname in files:
+            if not fname.endswith('.c'):
+                continue
+
+            path = os.path.join(root, fname)
+            with open(path, 'r', errors='ignore') as fdata:
+                for line in fdata:
+                    if old_regex.match(line):
+                        old_tests += 1
+                        break
+
+                    if new_regex.match(line):
+                        new_tests += 1
+                        break
+
+    if old_tests == 0 and new_tests == 0:
+        return
+
+    text = [
+        'New API conversion status\n',
+        '-------------------------\n\n',
+        'The current LTP refactoring plan aims to convert all LTP test\n',
+        f'cases using the new LTP API. There are currently **{old_tests}** tests\n',
+        f'which need to be converted and **{new_tests}** have been converted\n',
+        'already.\n',
+    ]
+
+    with open(output, 'w+') as stats:
+        stats.writelines(text)
+
+
 def generate_syscalls_stats(_):
     """
     Generate statistics for syscalls. We fetch the syscalls list from the kernel
@@ -198,4 +245,5 @@ def generate_syscalls_stats(_):
 
 def setup(app):
     app.add_css_file('custom.css')
+    app.connect('builder-inited', generate_old_api_stats)
     app.connect('builder-inited', generate_syscalls_stats)
diff --git a/doc/users/stats.rst b/doc/users/stats.rst
index 7073442aa..a8b322b5f 100644
--- a/doc/users/stats.rst
+++ b/doc/users/stats.rst
@@ -6,4 +6,5 @@ Statistics
 In this section we collect some statistics related to the current state of
 LTP tests.
 
+.. include:: ../_static/old_api.rst
 .. include:: ../_static/syscalls.rst
-- 
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v1] doc: Add new API conversion stats

[Cyril Hrubis]

> +def generate_old_api_stats(_):
> +    """
> +    Generate statistics for old API tests. We enter the tests folder and we
> +    count all tests implementations using old API. The way we count them, is to
> +    open every *.c file and to verify that "#include <test.h>" directive is
> +    present.
> +    """
> +    output = '_static/old_api.rst'
> +
> +    old_regex = re.compile(r'#include.*[<"\']test\.h[>"\']')
> +    new_regex = re.compile(r'#include.*[<"\']tst_test\.h[>"\']')
> +
> +    old_tests = 0
> +    new_tests = 0
> +
> +    for root, _, files in os.walk('../testcases'):
> +        for fname in files:
> +            if not fname.endswith('.c'):
> +                continue

There are at least two cases where the test.h header is included
indirectly from a test specific header. I guess that it would be cleaner
to actually fix these tests instead of working around it here.

> +            path = os.path.join(root, fname)
> +            with open(path, 'r', errors='ignore') as fdata:
> +                for line in fdata:
> +                    if old_regex.match(line):
> +                        old_tests += 1
> +                        break
> +
> +                    if new_regex.match(line):
> +                        new_tests += 1
> +                        break
> +
> +    if old_tests == 0 and new_tests == 0:
> +        return
> +
> +    text = [
> +        'New API conversion status\n',
> +        '-------------------------\n\n',
> +        'The current LTP refactoring plan aims to convert all LTP test\n',
> +        f'cases using the new LTP API. There are currently **{old_tests}** tests\n',
> +        f'which need to be converted and **{new_tests}** have been converted\n',
> +        'already.\n',
> +    ]
> +
> +    with open(output, 'w+') as stats:
> +        stats.writelines(text)

Maybe we should generate a table with the test filenames and links to
github code here? So that people interested in rewriting these tests
could check here and click on the filename to see the code?

>  def generate_syscalls_stats(_):
>      """
>      Generate statistics for syscalls. We fetch the syscalls list from the kernel
> @@ -198,4 +245,5 @@ def generate_syscalls_stats(_):
>  
>  def setup(app):
>      app.add_css_file('custom.css')
> +    app.connect('builder-inited', generate_old_api_stats)
>      app.connect('builder-inited', generate_syscalls_stats)
> diff --git a/doc/users/stats.rst b/doc/users/stats.rst
> index 7073442aa..a8b322b5f 100644
> --- a/doc/users/stats.rst
> +++ b/doc/users/stats.rst
> @@ -6,4 +6,5 @@ Statistics
>  In this section we collect some statistics related to the current state of
>  LTP tests.
>  
> +.. include:: ../_static/old_api.rst
>  .. include:: ../_static/syscalls.rst
> -- 
> 2.35.3
> 
> 
> -- 
> Mailing list info: https://lists.linux.it/listinfo/ltp

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v1] doc: Add new API conversion stats

[Andrea Cervesato via ltp]

Hi!

On 4/16/24 15:08, Cyril Hrubis wrote:
>> +def generate_old_api_stats(_):
>> +    """
>> +    Generate statistics for old API tests. We enter the tests folder and we
>> +    count all tests implementations using old API. The way we count them, is to
>> +    open every *.c file and to verify that "#include <test.h>" directive is
>> +    present.
>> +    """
>> +    output = '_static/old_api.rst'
>> +
>> +    old_regex = re.compile(r'#include.*[<"\']test\.h[>"\']')
>> +    new_regex = re.compile(r'#include.*[<"\']tst_test\.h[>"\']')
>> +
>> +    old_tests = 0
>> +    new_tests = 0
>> +
>> +    for root, _, files in os.walk('../testcases'):
>> +        for fname in files:
>> +            if not fname.endswith('.c'):
>> +                continue
> There are at least two cases where the test.h header is included
> indirectly from a test specific header. I guess that it would be cleaner
> to actually fix these tests instead of working around it here.
I can filter out .h and .c eventually, but the problem is that 
implementing inside headers is a bad practice by itself and it's not 
easy to recognize automatically.
>> +            path = os.path.join(root, fname)
>> +            with open(path, 'r', errors='ignore') as fdata:
>> +                for line in fdata:
>> +                    if old_regex.match(line):
>> +                        old_tests += 1
>> +                        break
>> +
>> +                    if new_regex.match(line):
>> +                        new_tests += 1
>> +                        break
>> +
>> +    if old_tests == 0 and new_tests == 0:
>> +        return
>> +
>> +    text = [
>> +        'New API conversion status\n',
>> +        '-------------------------\n\n',
>> +        'The current LTP refactoring plan aims to convert all LTP test\n',
>> +        f'cases using the new LTP API. There are currently **{old_tests}** tests\n',
>> +        f'which need to be converted and **{new_tests}** have been converted\n',
>> +        'already.\n',
>> +    ]
>> +
>> +    with open(output, 'w+') as stats:
>> +        stats.writelines(text)
> Maybe we should generate a table with the test filenames and links to
> github code here? So that people interested in rewriting these tests
> could check here and click on the filename to see the code?
I tried at first, but it's quite confusing to add this in the same 
section and creating a new page seems a bit too much.
>
>>   def generate_syscalls_stats(_):
>>       """
>>       Generate statistics for syscalls. We fetch the syscalls list from the kernel
>> @@ -198,4 +245,5 @@ def generate_syscalls_stats(_):
>>   
>>   def setup(app):
>>       app.add_css_file('custom.css')
>> +    app.connect('builder-inited', generate_old_api_stats)
>>       app.connect('builder-inited', generate_syscalls_stats)
>> diff --git a/doc/users/stats.rst b/doc/users/stats.rst
>> index 7073442aa..a8b322b5f 100644
>> --- a/doc/users/stats.rst
>> +++ b/doc/users/stats.rst
>> @@ -6,4 +6,5 @@ Statistics
>>   In this section we collect some statistics related to the current state of
>>   LTP tests.
>>   
>> +.. include:: ../_static/old_api.rst
>>   .. include:: ../_static/syscalls.rst
>> -- 
>> 2.35.3
>>
>>
>> -- 
>> Mailing list info: https://lists.linux.it/listinfo/ltp

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v1] doc: Add new API conversion stats

[Cyril Hrubis]

Hi!
> I can filter out .h and .c eventually, but the problem is that 
> implementing inside headers is a bad practice by itself and it's not 
> easy to recognize automatically.

The header that includes test.h tends to be also include from more than
one test, hence I suggested to fix the tests instead...

> I tried at first, but it's quite confusing to add this in the same 
> section and creating a new page seems a bit too much.

Fair enough.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp