[LTP] [PATCH v1 0/2] kirk: add --run-pattern option

[LTP] [PATCH v1 0/2] kirk: add --run-pattern option

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Add a new option --run-pattern to execute tests which are matching a
certain name pattern. For example, "sendfile" will execute all tests
which have "sendfile" inside their name.

This patch-set also removes some linting tests on multiple python
versions, since we always test the same linging configurations over
all python versions and that's not really needed.

Andrea Cervesato (2):
  ci: test linting only with python 3.9
  options: add new --run-pattern

 .github/workflows/linting.yml | 11 +------
 libkirk/main.py               |  8 +++++-
 libkirk/session.py            | 54 ++++++++++++++++++++++++++++++++---
 libkirk/tests/test_session.py | 13 +++++++++
 4 files changed, 71 insertions(+), 15 deletions(-)

-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v1 1/2] ci: test linting only with python 3.9

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Remove all python versions but the 3.9 from all liting commands.
The reason is that there's no need to run pylint on multiple python
versions if we check for the same issues.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 .github/workflows/linting.yml | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
index 66746ec..7df63b7 100644
--- a/.github/workflows/linting.yml
+++ b/.github/workflows/linting.yml
@@ -10,16 +10,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [
-          "3.6",
-          "3.7",
-          "3.8",
-          "3.9",
-          "3.10",
-          "3.11",
-          "3.12",
-          "3.13"
-        ]
+        python-version: ["3.9"]
 
     steps:
     - name: Show OS
-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH v1 2/2] options: add new --run-pattern

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

Add the --run-pattern|-S option to filter tests based on a specific
string pattern. This option accepts a list of strings, allowing users
to specify multiple patterns. A common usage example is as follows:

    kirk --run-pattern sendfile madvice

Each string is compared against the labels defined in the runtest file,
and only tests containing the specified patterns will be executed.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
 libkirk/main.py               |  8 +++++-
 libkirk/session.py            | 54 ++++++++++++++++++++++++++++++++---
 libkirk/tests/test_session.py | 13 +++++++++
 3 files changed, 70 insertions(+), 5 deletions(-)

diff --git a/libkirk/main.py b/libkirk/main.py
index 4c86c8b..6780e43 100644
--- a/libkirk/main.py
+++ b/libkirk/main.py
@@ -304,6 +304,7 @@ def _start_session(
             await session.run(
                 command=args.run_command,
                 suites=args.run_suite,
+                pattern=args.run_pattern,
                 report_path=args.json_report,
                 restore=restore_dir,
             )
@@ -414,6 +415,11 @@ def run(cmd_args: list = None) -> None:
         "-r",
         nargs="*",
         help="List of suites to run")
+    parser.add_argument(
+        "--run-pattern",
+        "-S",
+        nargs="*",
+        help="Filter tests using a pattern string")
     parser.add_argument(
         "--run-command",
         "-c",
@@ -465,7 +471,7 @@ def run(cmd_args: list = None) -> None:
     if args.json_report and os.path.exists(args.json_report):
         parser.error(f"JSON report file already exists: {args.json_report}")
 
-    if not args.run_suite and not args.run_command:
+    if not args.run_suite and not args.run_command and not args.run_pattern:
         parser.error("--run-suite/--run-command are required")
 
     if args.skip_file and not os.path.isfile(args.skip_file):
diff --git a/libkirk/session.py b/libkirk/session.py
index 53f07cf..2f27790 100644
--- a/libkirk/session.py
+++ b/libkirk/session.py
@@ -14,6 +14,7 @@ import libkirk.events
 from libkirk import KirkException
 from libkirk.sut import SUT
 from libkirk.sut import IOBuffer
+from libkirk.data import Suite
 from libkirk.results import TestResults
 from libkirk.export import JSONExporter
 from libkirk.scheduler import SuiteScheduler
@@ -186,16 +187,16 @@ class Session:
         await libkirk.events.fire("sut_stop", self._sut.name)
         await self._sut.stop(iobuffer=RedirectSUTStdout(self._sut, False))
 
-    async def _read_suites(self, suites: list, restore: str) -> list:
+    async def _get_suites_objects(self, names: list) -> list:
         """
-        Read suites and return a list of Suite objects.
+        Return suites objects by giving their names.
         """
         coros = []
-        for suite in suites:
+        for suite in names:
             coros.append(self._framework.find_suite(self._sut, suite))
 
         if not coros:
-            raise KirkException(f"Can't find suites: {suites}")
+            raise KirkException(f"Can't find suites: {names}")
 
         suites_obj = await asyncio.gather(*coros, return_exceptions=True)
         for suite in suites_obj:
@@ -205,6 +206,42 @@ class Session:
             if not suite:
                 raise KirkException("Can't find suite objects")
 
+        return suites_obj
+
+    async def _filter_tests(self, pattern: str) -> list:
+        """
+        Read all tests from all the testing suites which are matching the
+        pattern string.
+        """
+        if not pattern:
+            raise ValueError("pattern string is empty")
+
+        suites = await self._framework.get_suites(self._sut)
+        if not suites:
+            return []
+
+        suites_obj = await self._get_suites_objects(suites)
+
+        suites = []
+        for patt in pattern:
+            tests = []
+
+            for suite in suites_obj:
+                for test in suite.tests:
+                    if patt in test.name:
+                        tests.append(test)
+
+            suite = Suite(patt, tests)
+            suites.append(suite)
+
+        return suites
+
+    async def _read_suites(self, suites: list, restore: str) -> list:
+        """
+        Read suites and return a list of Suite objects.
+        """
+        suites_obj = await self._get_suites_objects(suites)
+
         restored = self._read_restored_session(restore)
         if restored:
             await libkirk.events.fire("session_restore", restore)
@@ -286,10 +323,13 @@ class Session:
             await libkirk.events.fire("session_stopped")
             self._stop = False
 
+    # consider changing the arguments handling if new ones must be added
+    # pylint: disable=too-many-positional-arguments
     async def run(
             self,
             command: str = None,
             suites: list = None,
+            pattern: str = None,
             report_path: str = None,
             restore: str = None) -> None:
         """
@@ -298,6 +338,8 @@ class Session:
         :type command: str
         :param suites: list of suites to execute
         :type suites: list
+        :param pattern: string pattern to match tests
+        :types pattern: str
         :param report_path: JSON report path
         :type report_path: str
         :param restore: temporary directory generated by a previous session
@@ -317,6 +359,10 @@ class Session:
                 if command:
                     await self._exec_command(command)
 
+                if pattern:
+                    suites_obj = await self._filter_tests(pattern)
+                    await self._scheduler.schedule(suites_obj)
+
                 if suites:
                     suites_obj = await self._read_suites(suites, restore)
                     await self._scheduler.schedule(suites_obj)
diff --git a/libkirk/tests/test_session.py b/libkirk/tests/test_session.py
index 983eb93..44ae2b4 100644
--- a/libkirk/tests/test_session.py
+++ b/libkirk/tests/test_session.py
@@ -44,6 +44,19 @@ class _TestSession:
         """
         await session.run(suites=["suite01", "suite02"])
 
+    async def test_run_pattern(self, tmpdir, session):
+        """
+        Test run method when executing tests filtered out with a pattern.
+        """
+        report = str(tmpdir / "report.json")
+        await session.run(
+            pattern=["test01", "test02"],
+            report_path=report)
+
+        with open(report, "r", encoding="utf-8") as report_file:
+            report_data = json.loads(report_file.read())
+            assert len(report_data["results"]) == 8
+
     async def test_run_report(self, tmpdir, session):
         """
         Test run method when executing suites, generating a report.
-- 
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v1 2/2] options: add new --run-pattern

[Cyril Hrubis]

Hi!
> Add the --run-pattern|-S option to filter tests based on a specific
> string pattern. This option accepts a list of strings, allowing users
> to specify multiple patterns. A common usage example is as follows:
> 
>     kirk --run-pattern sendfile madvice
> 
> Each string is compared against the labels defined in the runtest file,
> and only tests containing the specified patterns will be executed.

So this is a substring match, right? I wonder if we should use a regular
expression instead, so that we can do more complex matches.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH v1 2/2] options: add new --run-pattern

[Andrea Cervesato via ltp]

Hi!

On 2/14/25 11:16, Cyril Hrubis wrote:
> Hi!
>> Add the --run-pattern|-S option to filter tests based on a specific
>> string pattern. This option accepts a list of strings, allowing users
>> to specify multiple patterns. A common usage example is as follows:
>>
>>      kirk --run-pattern sendfile madvice
>>
>> Each string is compared against the labels defined in the runtest file,
>> and only tests containing the specified patterns will be executed.
> So this is a substring match, right? I wonder if we should use a regular
> expression instead, so that we can do more complex matches.
>
I thought about that, but I wanted to start with something easier and 
check for feedback. We are already doing it for skip-tests option.

Andrea


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp