[PATCH v2 0/1] Update check-python-tox test for pylint 2.10

[PATCH v2 0/1] Update check-python-tox test for pylint 2.10

[John Snow]

V2: It's not safe to use sys.stderr.encoding to determine a "console
encoding", because that uses the "current" stderr and not a
hypothetically generic one -- and doing this causes the acceptance tests
to fail.

Use UTF-8 instead.

Question: What encoding do terminal programs use? Is there an inherent
encoding to fprintf et al, or does it just push whatever bytes you put
into it straight into the stdout/stderr pipe?

John Snow (1):
  python: Update for pylint 2.10

 python/qemu/machine/machine.py | 3 ++-
 python/setup.cfg               | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

-- 
2.31.1

[PATCH v2 1/1] python: Update for pylint 2.10

[John Snow]

A few new annoyances. Of note is the new warning for an unspecified
encoding when opening a text file, which actually does indicate a
potentially real problem; see
https://www.python.org/dev/peps/pep-0597/#motivation

It's not clear to me what the "right" encoding is; it depends on
whatever encoding QEMU is using when it prints to terminal. I'm going to
assume UTF-8 works.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/machine/machine.py | 3 ++-
 python/setup.cfg               | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index a7081b1845..a27a80497d 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -291,7 +291,8 @@ def get_pid(self) -> Optional[int]:
 
     def _load_io_log(self) -> None:
         if self._qemu_log_path is not None:
-            with open(self._qemu_log_path, "r") as iolog:
+            with open(self._qemu_log_path, "r",
+                      encoding='utf-8') as iolog:
                 self._iolog = iolog.read()
 
     @property
diff --git a/python/setup.cfg b/python/setup.cfg
index 83909c1c97..0f0cab098f 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -104,6 +104,7 @@ good-names=i,
 [pylint.similarities]
 # Ignore imports when computing similarities.
 ignore-imports=yes
+ignore-signatures=yes
 
 # Minimum lines number of a similarity.
 # TODO: Remove after we opt in to Pylint 2.8.3. See commit msg.
-- 
2.31.1

Re: [PATCH v2 0/1] Update check-python-tox test for pylint 2.10

[Daniel P. Berrangé]

On Wed, Sep 15, 2021 at 01:30:10AM -0400, John Snow wrote:
> V2: It's not safe to use sys.stderr.encoding to determine a "console
> encoding", because that uses the "current" stderr and not a
> hypothetically generic one -- and doing this causes the acceptance tests
> to fail.
> 
> Use UTF-8 instead.
> 
> Question: What encoding do terminal programs use? Is there an inherent
> encoding to fprintf et al, or does it just push whatever bytes you put
> into it straight into the stdout/stderr pipe?

Programs are expected to output data in the encoding that is set in
the various env variables LC_ALL/LC_CTYPE/LANG.

In traditional end user scenarios this almost always means UTF-8 charset.

There's plenty of cases which end up with the C locale though, which
would mean 7-bit ASCII on Linux, though apps are supposed to be 8-bit
clean allow data with the high bit to pass through without interpretation.
The latter is what python3 gets very wrong complaining if you output
8-bit high data in C locale.

There is increasing support for a C.UTF-8 locale to bring it closer to
other locales which are all UTF-8.

On macOS the C locale has been UTF-8 by default indefinitely.

Windows is a whole other world of fun and IIRC isn't UTF-8 by default,
but I don't recall details.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 0/1] Update check-python-tox test for pylint 2.10

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]

On Wed, Sep 15, 2021 at 5:10 AM Daniel P. Berrangé <berrange@redhat.com>
wrote:

> On Wed, Sep 15, 2021 at 01:30:10AM -0400, John Snow wrote:
> > V2: It's not safe to use sys.stderr.encoding to determine a "console
> > encoding", because that uses the "current" stderr and not a
> > hypothetically generic one -- and doing this causes the acceptance tests
> > to fail.
> >
> > Use UTF-8 instead.
> >
> > Question: What encoding do terminal programs use? Is there an inherent
> > encoding to fprintf et al, or does it just push whatever bytes you put
> > into it straight into the stdout/stderr pipe?
>
> Programs are expected to output data in the encoding that is set in
> the various env variables LC_ALL/LC_CTYPE/LANG.
>
> In traditional end user scenarios this almost always means UTF-8 charset.
>
> There's plenty of cases which end up with the C locale though, which
> would mean 7-bit ASCII on Linux, though apps are supposed to be 8-bit
> clean allow data with the high bit to pass through without interpretation.
> The latter is what python3 gets very wrong complaining if you output
> 8-bit high data in C locale.
>
> There is increasing support for a C.UTF-8 locale to bring it closer to
> other locales which are all UTF-8.
>
> On macOS the C locale has been UTF-8 by default indefinitely.
>
> Windows is a whole other world of fun and IIRC isn't UTF-8 by default,
> but I don't recall details.
>
>
> Regards,
> Daniel
> --
> |: https://berrange.com      -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-
> https://www.instagram.com/dberrange :|
>
>
Hm, I believe I can use `lang, encoding = locale.getlocale() ` in this case
-- I believe it follows LC_CTYPE. This ought to accurately match the
console output from QEMU.
I'll respin, actually. We don't test the Python packages on Windows, but I
see no reason to introduce a nasty timebomb.

Thanks!
--js

[-- Attachment #2: Type: text/html, Size: 3058 bytes --]