qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	qemu-block@nongnu.org, Hanna Reitz <hreitz@redhat.com>,
	Cleber Rosa <crosa@redhat.com>, John Snow <jsnow@redhat.com>
Subject: [PATCH 09/13] iotests: split linters.py out from 297
Date: Mon,  4 Oct 2021 17:04:59 -0400	[thread overview]
Message-ID: <20211004210503.1455391-10-jsnow@redhat.com> (raw)
In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com>

Now, 297 is just the iotests-specific incantations and linters.py is as
minimal as I can think to make it. The only remaining element in here
that ought to be configuration and not code is the list of skip files,
but they're still numerous enough that repeating them for mypy and
pylint configurations both would be ... a hassle.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 tests/qemu-iotests/297        | 72 +++---------------------------
 tests/qemu-iotests/linters.py | 83 +++++++++++++++++++++++++++++++++++
 2 files changed, 88 insertions(+), 67 deletions(-)
 create mode 100644 tests/qemu-iotests/linters.py

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 4c54dd39b46..f79c80216bf 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -17,76 +17,14 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
-import re
 import shutil
-import subprocess
 import sys
-from typing import List, Mapping, Optional
 
 import iotests
+import linters
 
 
-# TODO: Empty this list!
-SKIP_FILES = (
-    '030', '040', '041', '044', '045', '055', '056', '057', '065', '093',
-    '096', '118', '124', '132', '136', '139', '147', '148', '149',
-    '151', '152', '155', '163', '165', '194', '196', '202',
-    '203', '205', '206', '207', '208', '210', '211', '212', '213', '216',
-    '218', '219', '224', '228', '234', '235', '236', '237', '238',
-    '240', '242', '245', '246', '248', '255', '256', '257', '258', '260',
-    '262', '264', '266', '274', '277', '280', '281', '295', '296', '298',
-    '299', '302', '303', '304', '307',
-    'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py'
-)
-
-
-def is_python_file(filename):
-    if not os.path.isfile(filename):
-        return False
-
-    if filename.endswith('.py'):
-        return True
-
-    with open(filename, encoding='utf-8') as f:
-        try:
-            first_line = f.readline()
-            return re.match('^#!.*python', first_line) is not None
-        except UnicodeDecodeError:  # Ignore binary files
-            return False
-
-
-def get_test_files() -> List[str]:
-    named_tests = [f'tests/{entry}' for entry in os.listdir('tests')]
-    check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES)
-    return list(filter(is_python_file, check_tests))
-
-
-def run_linter(
-        tool: str,
-        args: List[str],
-        env: Optional[Mapping[str, str]] = None,
-        suppress_output: bool = False,
-) -> int:
-    """
-    Run a python-based linting tool.
-
-    If suppress_output is True, capture stdout/stderr of the child
-    process and only print that information back to stdout if the child
-    process's return code was non-zero.
-    """
-    p = subprocess.run(
-        ('python3', '-m', tool, *args),
-        env=env,
-        check=False,
-        stdout=subprocess.PIPE if suppress_output else None,
-        stderr=subprocess.STDOUT if suppress_output else None,
-        universal_newlines=True,
-    )
-
-    if suppress_output and p.returncode != 0:
-        print(p.stdout)
-
-    return p.returncode
+# Looking for the list of files to exclude from linting? See linters.py.
 
 
 def main() -> None:
@@ -94,7 +32,7 @@ def main() -> None:
         if shutil.which(linter) is None:
             iotests.notrun(f'{linter} not found')
 
-    files = get_test_files()
+    files = linters.get_test_files()
 
     iotests.logger.debug('Files to be checked:')
     iotests.logger.debug(', '.join(sorted(files)))
@@ -104,11 +42,11 @@ def main() -> None:
 
     print('=== pylint ===')
     sys.stdout.flush()
-    run_linter('pylint', files, env=env)
+    linters.run_linter('pylint', files, env=env)
 
     print('=== mypy ===')
     sys.stdout.flush()
-    run_linter('mypy', files, env=env, suppress_output=True)
+    linters.run_linter('mypy', files, env=env, suppress_output=True)
 
 
 iotests.script_main(main)
diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py
new file mode 100644
index 00000000000..f6a2dc139fd
--- /dev/null
+++ b/tests/qemu-iotests/linters.py
@@ -0,0 +1,83 @@
+# Copyright (C) 2020 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import re
+import subprocess
+from typing import List, Mapping, Optional
+
+
+# TODO: Empty this list!
+SKIP_FILES = (
+    '030', '040', '041', '044', '045', '055', '056', '057', '065', '093',
+    '096', '118', '124', '132', '136', '139', '147', '148', '149',
+    '151', '152', '155', '163', '165', '194', '196', '202',
+    '203', '205', '206', '207', '208', '210', '211', '212', '213', '216',
+    '218', '219', '224', '228', '234', '235', '236', '237', '238',
+    '240', '242', '245', '246', '248', '255', '256', '257', '258', '260',
+    '262', '264', '266', '274', '277', '280', '281', '295', '296', '298',
+    '299', '302', '303', '304', '307',
+    'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py'
+)
+
+
+def is_python_file(filename):
+    if not os.path.isfile(filename):
+        return False
+
+    if filename.endswith('.py'):
+        return True
+
+    with open(filename, encoding='utf-8') as f:
+        try:
+            first_line = f.readline()
+            return re.match('^#!.*python', first_line) is not None
+        except UnicodeDecodeError:  # Ignore binary files
+            return False
+
+
+def get_test_files() -> List[str]:
+    named_tests = [f'tests/{entry}' for entry in os.listdir('tests')]
+    check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES)
+    return list(filter(is_python_file, check_tests))
+
+
+def run_linter(
+        tool: str,
+        args: List[str],
+        env: Optional[Mapping[str, str]] = None,
+        suppress_output: bool = False,
+) -> int:
+    """
+    Run a python-based linting tool.
+
+    If suppress_output is True, capture stdout/stderr of the child
+    process and only print that information back to stdout if the child
+    process's return code was non-zero.
+    """
+    p = subprocess.run(
+        ('python3', '-m', tool, *args),
+        env=env,
+        check=False,
+        stdout=subprocess.PIPE if suppress_output else None,
+        stderr=subprocess.STDOUT if suppress_output else None,
+        universal_newlines=True,
+    )
+
+    if suppress_output and p.returncode != 0:
+        print(p.stdout)
+
+    return p.returncode
+
-- 
2.31.1



  parent reply	other threads:[~2021-10-04 21:11 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-04 21:04 [PATCH 00/13] python/iotests: Run iotest linters during Python CI John Snow
2021-10-04 21:04 ` [PATCH 01/13] iotests/297: Move pylint config into pylintrc John Snow
2021-10-13 10:49   ` Hanna Reitz
2021-10-04 21:04 ` [PATCH 02/13] iotests/297: Split mypy configuration out into mypy.ini John Snow
2021-10-13 10:53   ` Hanna Reitz
2021-10-13 14:37     ` John Snow
2021-10-04 21:04 ` [PATCH 03/13] iotests/297: Add get_files() function John Snow
2021-10-13 10:58   ` Hanna Reitz
2021-10-04 21:04 ` [PATCH 04/13] iotests/297: Don't rely on distro-specific linter binaries John Snow
2021-10-04 21:04 ` [PATCH 05/13] iotests/297: Create main() function John Snow
2021-10-13 11:03   ` Hanna Reitz
2021-10-13 14:41     ` John Snow
2021-10-04 21:04 ` [PATCH 06/13] iotests/297: Separate environment setup from test execution John Snow
2021-10-13 11:07   ` Hanna Reitz
2021-10-04 21:04 ` [PATCH 07/13] iotests/297: Split run_linters apart into run_pylint and run_mypy John Snow
2021-10-13 11:18   ` Hanna Reitz
2021-10-04 21:04 ` [PATCH 08/13] iotests/297: refactor run_[mypy|pylint] as generic execution shim John Snow
2021-10-13 11:26   ` Hanna Reitz
2021-10-04 21:04 ` John Snow [this message]
2021-10-13 11:50   ` [PATCH 09/13] iotests: split linters.py out from 297 Hanna Reitz
2021-10-13 15:07     ` John Snow
2021-10-13 16:28       ` Hanna Reitz
2021-10-04 21:05 ` [PATCH 10/13] iotests/linters: Add entry point for linting via Python CI John Snow
2021-10-13 12:11   ` Hanna Reitz
2021-10-13 15:11     ` John Snow
2021-10-04 21:05 ` [PATCH 11/13] iotests/linters: Add workaround for mypy bug #9852 John Snow
2021-10-13 12:15   ` Hanna Reitz
2021-10-04 21:05 ` [PATCH 12/13] python: Add iotest linters to test suite John Snow
2021-10-13 12:21   ` Hanna Reitz
2021-10-04 21:05 ` [PATCH 13/13] iotests: [RFC] drop iotest 297 John Snow
2021-10-13 12:23   ` Hanna Reitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211004210503.1455391-10-jsnow@redhat.com \
    --to=jsnow@redhat.com \
    --cc=crosa@redhat.com \
    --cc=ehabkost@redhat.com \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).