[PATCH 0/2] Python: update CI tests for pylint 2.10 and 2.11

[PATCH 0/2] Python: update CI tests for pylint 2.10 and 2.11

[John Snow]

GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-pylint-211
CI: https://gitlab.com/jsnow/qemu/-/pipelines/372122981

While debating the fix for 2.10, 2.11 released and added new warnings to
suppress.

This version includes everything needed to get the Gitlab CI green again.

John Snow (2):
  python: Update for pylint 2.10
  python: pylint 2.11 support

 python/qemu/machine/machine.py | 7 ++++++-
 python/setup.cfg               | 3 ++-
 2 files changed, 8 insertions(+), 2 deletions(-)

-- 
2.31.1

[PATCH 1/2] python: Update for pylint 2.10

[John Snow]

A few new annoyances. Of note is the new warning for an unspecified
encoding when opening a text file, which actually does indicate a
potentially real problem; see
https://www.python.org/dev/peps/pep-0597/#motivation

Use LC_CTYPE to determine an encoding to use for interpreting QEMU's
terminal output. Note that Python states: "language code and encoding
may be None if their values cannot be determined" -- use a platform
default as a backup.

Notes: Passing encoding=None will generate a suppressed warning on
Python 3.10+ that 'None' should not be passed as the encoding
argument. This behavior may be deprecated in the future and the default
switched to be a ubiquitous UTF-8. Opting in to the locale default will
be done by passing the encoding 'locale', but that isn't available in
3.6 through 3.9. Presumably this warning will be unsuppressed some time
prior to the actual switch and we can re-investigate these issues at
that time if necessary.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/machine/machine.py | 7 ++++++-
 python/setup.cfg               | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index a7081b1845..34131884a5 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -19,6 +19,7 @@
 
 import errno
 from itertools import chain
+import locale
 import logging
 import os
 import shutil
@@ -290,8 +291,12 @@ def get_pid(self) -> Optional[int]:
         return self._subp.pid
 
     def _load_io_log(self) -> None:
+        # Assume that the output encoding of QEMU's terminal output is
+        # defined by our locale. If indeterminate, allow open() to fall
+        # back to the platform default.
+        _, encoding = locale.getlocale()
         if self._qemu_log_path is not None:
-            with open(self._qemu_log_path, "r") as iolog:
+            with open(self._qemu_log_path, "r", encoding=encoding) as iolog:
                 self._iolog = iolog.read()
 
     @property
diff --git a/python/setup.cfg b/python/setup.cfg
index 83909c1c97..0f0cab098f 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -104,6 +104,7 @@ good-names=i,
 [pylint.similarities]
 # Ignore imports when computing similarities.
 ignore-imports=yes
+ignore-signatures=yes
 
 # Minimum lines number of a similarity.
 # TODO: Remove after we opt in to Pylint 2.8.3. See commit msg.
-- 
2.31.1

Re: [PATCH 1/2] python: Update for pylint 2.10

[Eduardo Habkost]

On Thu, Sep 16, 2021 at 02:22:47PM -0400, John Snow wrote:
> A few new annoyances. Of note is the new warning for an unspecified
> encoding when opening a text file, which actually does indicate a
> potentially real problem; see
> https://www.python.org/dev/peps/pep-0597/#motivation
> 
> Use LC_CTYPE to determine an encoding to use for interpreting QEMU's
> terminal output. Note that Python states: "language code and encoding
> may be None if their values cannot be determined" -- use a platform
> default as a backup.
> 
> Notes: Passing encoding=None will generate a suppressed warning on
> Python 3.10+ that 'None' should not be passed as the encoding
> argument. This behavior may be deprecated in the future and the default
> switched to be a ubiquitous UTF-8. Opting in to the locale default will
> be done by passing the encoding 'locale', but that isn't available in
> 3.6 through 3.9. Presumably this warning will be unsuppressed some time
> prior to the actual switch and we can re-investigate these issues at
> that time if necessary.

So, in the very worst case this will trigger a warning that is
currently suppressed.  And that will happen only if we are in the
unlikely situation where we have absolutely no information about
the encoding being used by other parts of the system.

Sounds reasonable to me, so:

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>

> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/qemu/machine/machine.py | 7 ++++++-
>  python/setup.cfg               | 1 +
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
> index a7081b1845..34131884a5 100644
> --- a/python/qemu/machine/machine.py
> +++ b/python/qemu/machine/machine.py
> @@ -19,6 +19,7 @@
>  
>  import errno
>  from itertools import chain
> +import locale
>  import logging
>  import os
>  import shutil
> @@ -290,8 +291,12 @@ def get_pid(self) -> Optional[int]:
>          return self._subp.pid
>  
>      def _load_io_log(self) -> None:
> +        # Assume that the output encoding of QEMU's terminal output is
> +        # defined by our locale. If indeterminate, allow open() to fall
> +        # back to the platform default.
> +        _, encoding = locale.getlocale()
>          if self._qemu_log_path is not None:
> -            with open(self._qemu_log_path, "r") as iolog:
> +            with open(self._qemu_log_path, "r", encoding=encoding) as iolog:
>                  self._iolog = iolog.read()
>  
>      @property
> diff --git a/python/setup.cfg b/python/setup.cfg
> index 83909c1c97..0f0cab098f 100644
> --- a/python/setup.cfg
> +++ b/python/setup.cfg
> @@ -104,6 +104,7 @@ good-names=i,
>  [pylint.similarities]
>  # Ignore imports when computing similarities.
>  ignore-imports=yes
> +ignore-signatures=yes
>  
>  # Minimum lines number of a similarity.
>  # TODO: Remove after we opt in to Pylint 2.8.3. See commit msg.
> -- 
> 2.31.1
> 

-- 
Eduardo

Re: [PATCH 1/2] python: Update for pylint 2.10

[Willian Rampazzo]

On Thu, Sep 16, 2021 at 3:29 PM John Snow <jsnow@redhat.com> wrote:
>
> A few new annoyances. Of note is the new warning for an unspecified
> encoding when opening a text file, which actually does indicate a
> potentially real problem; see
> https://www.python.org/dev/peps/pep-0597/#motivation
>
> Use LC_CTYPE to determine an encoding to use for interpreting QEMU's
> terminal output. Note that Python states: "language code and encoding
> may be None if their values cannot be determined" -- use a platform
> default as a backup.
>
> Notes: Passing encoding=None will generate a suppressed warning on
> Python 3.10+ that 'None' should not be passed as the encoding
> argument. This behavior may be deprecated in the future and the default
> switched to be a ubiquitous UTF-8. Opting in to the locale default will
> be done by passing the encoding 'locale', but that isn't available in
> 3.6 through 3.9. Presumably this warning will be unsuppressed some time
> prior to the actual switch and we can re-investigate these issues at
> that time if necessary.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/qemu/machine/machine.py | 7 ++++++-
>  python/setup.cfg               | 1 +
>  2 files changed, 7 insertions(+), 1 deletion(-)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

[PATCH 2/2] python: pylint 2.11 support

[John Snow]

We're not ready to enforce f-strings everywhere, so just silence this
new warning.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/setup.cfg | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/python/setup.cfg b/python/setup.cfg
index 0f0cab098f..fdca265fec 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -87,7 +87,7 @@ ignore_missing_imports = True
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use "--disable=all --enable=classes
 # --disable=W".
-disable=
+disable=consider-using-f-string,
 
 [pylint.basic]
 # Good variable names which should always be accepted, separated by a comma.
-- 
2.31.1

Re: [PATCH 2/2] python: pylint 2.11 support

[Eduardo Habkost]

On Thu, Sep 16, 2021 at 02:22:48PM -0400, John Snow wrote:
> We're not ready to enforce f-strings everywhere, so just silence this
> new warning.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>

-- 
Eduardo

Re: [PATCH 2/2] python: pylint 2.11 support

[Willian Rampazzo]

On Thu, Sep 16, 2021 at 3:24 PM John Snow <jsnow@redhat.com> wrote:
>
> We're not ready to enforce f-strings everywhere, so just silence this
> new warning.

Oh, boy! We are good, thanks :)

>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/setup.cfg | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

Re: [PATCH 0/2] Python: update CI tests for pylint 2.10 and 2.11

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

On Thu, Sep 16, 2021 at 2:22 PM John Snow <jsnow@redhat.com> wrote:

> GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-pylint-211
> CI: https://gitlab.com/jsnow/qemu/-/pipelines/372122981
>
> While debating the fix for 2.10, 2.11 released and added new warnings to
> suppress.
>
> This version includes everything needed to get the Gitlab CI green again.
>
> John Snow (2):
>   python: Update for pylint 2.10
>   python: pylint 2.11 support
>
>  python/qemu/machine/machine.py | 7 ++++++-
>  python/setup.cfg               | 3 ++-
>  2 files changed, 8 insertions(+), 2 deletions(-)
>
>
Eduardo, Willian: Thank you very much!

I have staged this on my python branch:
https://gitlab.com/jsnow/qemu/-/commits/python

PR to follow as soon as the lights turn green.

--js

[-- Attachment #2: Type: text/html, Size: 1532 bytes --]