[PATCH 0/3] python/qemu: strictly typed mypy conversion, pt3

[PATCH 0/3] python/qemu: strictly typed mypy conversion, pt3

[John Snow]

This is actually quite short; it's already fully typed. Attached are two
fixes for settimeout and error handling. There are actually more fixes
that need to be made here, because use of readline() in non-blocking
mode is actually undefined behavior, so a more thorough re-work of the
error classes used by this library must be put on hold pending a more
aggressive re-write.

That's a problem for later, so for now, call the initial conversion to
the statically typed subset of python done so we can move on to adding
the regression tests that will maintain this baseline for us.

John Snow (3):
  python: add mypy config
  python/qemu/qmp.py: re-raise OSError when encountered
  python/qemu/qmp.py: Fix settimeout operation

 python/mypy.ini    |  4 ++++
 python/qemu/qmp.py | 30 +++++++++++++++++++++---------
 2 files changed, 25 insertions(+), 9 deletions(-)
 create mode 100644 python/mypy.ini

-- 
2.26.2

[PATCH 1/3] python: add mypy config

[John Snow]

Formalize the options used for checking the python library. You can run
mypy from the directory that mypy.ini is in by typing `mypy qemu/`.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/mypy.ini | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 python/mypy.ini

diff --git a/python/mypy.ini b/python/mypy.ini
new file mode 100644
index 00000000000..7a70eca47c6
--- /dev/null
+++ b/python/mypy.ini
@@ -0,0 +1,4 @@
+[mypy]
+strict = True
+python_version = 3.6
+warn_unused_configs = True
\ No newline at end of file
-- 
2.26.2

Re: [PATCH 1/3] python: add mypy config

[Bin Meng]

On Sat, Oct 10, 2020 at 1:54 AM John Snow <jsnow@redhat.com> wrote:
>
> Formalize the options used for checking the python library. You can run
> mypy from the directory that mypy.ini is in by typing `mypy qemu/`.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/mypy.ini | 4 ++++
>  1 file changed, 4 insertions(+)
>  create mode 100644 python/mypy.ini
>
> diff --git a/python/mypy.ini b/python/mypy.ini
> new file mode 100644
> index 00000000000..7a70eca47c6
> --- /dev/null
> +++ b/python/mypy.ini
> @@ -0,0 +1,4 @@
> +[mypy]
> +strict = True
> +python_version = 3.6
> +warn_unused_configs = True
> \ No newline at end of file

Missing "\n" at the end of file

Regards,
Bin

Re: [PATCH 1/3] python: add mypy config

[John Snow]

On 10/13/20 5:15 AM, Bin Meng wrote:
> On Sat, Oct 10, 2020 at 1:54 AM John Snow <jsnow@redhat.com> wrote:
>>
>> Formalize the options used for checking the python library. You can run
>> mypy from the directory that mypy.ini is in by typing `mypy qemu/`.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>>   python/mypy.ini | 4 ++++
>>   1 file changed, 4 insertions(+)
>>   create mode 100644 python/mypy.ini
>>
>> diff --git a/python/mypy.ini b/python/mypy.ini
>> new file mode 100644
>> index 00000000000..7a70eca47c6
>> --- /dev/null
>> +++ b/python/mypy.ini
>> @@ -0,0 +1,4 @@
>> +[mypy]
>> +strict = True
>> +python_version = 3.6
>> +warn_unused_configs = True
>> \ No newline at end of file
> 
> Missing "\n" at the end of file
> 
> Regards,
> Bin
> 

Easily fixed, thanks!

[PATCH 2/3] python/qemu/qmp.py: re-raise OSError when encountered

[John Snow]

Nested if conditions don't change when the exception block fires; we
need to explicitly re-raise the error if we didn't intend to capture and
suppress it.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/qmp.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index d911999da1f..4969e5741cb 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -165,14 +165,15 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
         """
 
         # Check for new events regardless and pull them into the cache:
-        self.__sock.setblocking(False)
         try:
+            self.__sock.setblocking(False)
             self.__json_read()
         except OSError as err:
-            if err.errno == errno.EAGAIN:
-                # No data available
-                pass
-        self.__sock.setblocking(True)
+            # EAGAIN: No data available; not critical
+            if err.errno != errno.EAGAIN:
+                raise
+        finally:
+            self.__sock.setblocking(True)
 
         # Wait for new events, if needed.
         # if wait is 0.0, this means "no wait" and is also implicitly false.
-- 
2.26.2

Re: [PATCH 2/3] python/qemu/qmp.py: re-raise OSError when encountered

[Philippe Mathieu-Daudé]

On 10/9/20 7:51 PM, John Snow wrote:
> Nested if conditions don't change when the exception block fires; we
> need to explicitly re-raise the error if we didn't intend to capture and
> suppress it.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   python/qemu/qmp.py | 11 ++++++-----
>   1 file changed, 6 insertions(+), 5 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

[PATCH 3/3] python/qemu/qmp.py: Fix settimeout operation

[John Snow]

We enabled callers to interface directly with settimeout, but this
reacts poorly with blocking/nonblocking operation; as they are using the
same internal mechanism.

1. Whenever we change the blocking mechanism temporarily, always set it
back to what it was afterwards.

2. Disallow callers from setting a timeout of "0", which means
Non-blocking mode. This is going to create more weird problems than
anybody wants, so just forbid it.

I opt not to coerce '0' to 'None' to maintain the principal of least
surprise in mirroring the semantics of Python's interface.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/qmp.py | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index 4969e5741cb..f64517fb0a7 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -164,16 +164,19 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
                                 retrieved or if some other error occurred.
         """
 
+        # Current timeout and blocking status
+        current_timeout = self.__sock.gettimeout()
+
         # Check for new events regardless and pull them into the cache:
         try:
-            self.__sock.setblocking(False)
+            self.__sock.settimeout(0)  # i.e. setblocking(False)
             self.__json_read()
         except OSError as err:
             # EAGAIN: No data available; not critical
             if err.errno != errno.EAGAIN:
                 raise
         finally:
-            self.__sock.setblocking(True)
+            self.__sock.settimeout(current_timeout)
 
         # Wait for new events, if needed.
         # if wait is 0.0, this means "no wait" and is also implicitly false.
@@ -187,9 +190,11 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
             except Exception as err:
                 msg = "Error while reading from socket"
                 raise QMPConnectError(msg) from err
+            finally:
+                self.__sock.settimeout(current_timeout)
+
             if ret is None:
                 raise QMPConnectError("Error while reading from socket")
-            self.__sock.settimeout(None)
 
     def __enter__(self) -> 'QEMUMonitorProtocol':
         # Implement context manager enter function.
@@ -219,7 +224,7 @@ def connect(self, negotiate: bool = True) -> Optional[QMPMessage]:
             return self.__negotiate_capabilities()
         return None
 
-    def accept(self, timeout: float = 15.0) -> QMPMessage:
+    def accept(self, timeout: Optional[float] = 15.0) -> QMPMessage:
         """
         Await connection from QMP Monitor and perform capabilities negotiation.
 
@@ -338,13 +343,19 @@ def close(self) -> None:
         if self.__sockfile:
             self.__sockfile.close()
 
-    def settimeout(self, timeout: float) -> None:
+    def settimeout(self, timeout: Optional[float]) -> None:
         """
         Set the socket timeout.
 
-        @param timeout (float): timeout in seconds, or None.
+        @param timeout (float): timeout in seconds (non-zero), or None.
         @note This is a wrap around socket.settimeout
+
+        @raise ValueError: if timeout was set to 0.
         """
+        if timeout == 0:
+            msg = "timeout cannot be 0; this engages non-blocking mode."
+            msg += " Use 'None' instead to disable timeouts."
+            raise ValueError(msg)
         self.__sock.settimeout(timeout)
 
     def get_sock_fd(self) -> int:
-- 
2.26.2

Re: [PATCH 3/3] python/qemu/qmp.py: Fix settimeout operation

[Philippe Mathieu-Daudé]

On 10/9/20 7:51 PM, John Snow wrote:
> We enabled callers to interface directly with settimeout, but this
> reacts poorly with blocking/nonblocking operation; as they are using the
> same internal mechanism.
> 
> 1. Whenever we change the blocking mechanism temporarily, always set it
> back to what it was afterwards.
> 
> 2. Disallow callers from setting a timeout of "0", which means
> Non-blocking mode. This is going to create more weird problems than
> anybody wants, so just forbid it.
> 
> I opt not to coerce '0' to 'None' to maintain the principal of least
> surprise in mirroring the semantics of Python's interface.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   python/qemu/qmp.py | 23 +++++++++++++++++------
>   1 file changed, 17 insertions(+), 6 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 0/3] python/qemu: strictly typed mypy conversion, pt3

[John Snow]

On 10/9/20 1:51 PM, John Snow wrote:
> This is actually quite short; it's already fully typed. Attached are two
> fixes for settimeout and error handling. There are actually more fixes
> that need to be made here, because use of readline() in non-blocking
> mode is actually undefined behavior, so a more thorough re-work of the
> error classes used by this library must be put on hold pending a more
> aggressive re-write.
> 
> That's a problem for later, so for now, call the initial conversion to
> the statically typed subset of python done so we can move on to adding
> the regression tests that will maintain this baseline for us.
> 
> John Snow (3):
>    python: add mypy config
>    python/qemu/qmp.py: re-raise OSError when encountered
>    python/qemu/qmp.py: Fix settimeout operation
> 
>   python/mypy.ini    |  4 ++++
>   python/qemu/qmp.py | 30 +++++++++++++++++++++---------
>   2 files changed, 25 insertions(+), 9 deletions(-)
>   create mode 100644 python/mypy.ini
> 

Based on: https://gitlab.com/jsnow/qemu/-/tree/python

Re: [PATCH 0/3] python/qemu: strictly typed mypy conversion, pt3

[John Snow]

On 10/9/20 1:51 PM, John Snow wrote:
> This is actually quite short; it's already fully typed. Attached are two
> fixes for settimeout and error handling. There are actually more fixes
> that need to be made here, because use of readline() in non-blocking
> mode is actually undefined behavior, so a more thorough re-work of the
> error classes used by this library must be put on hold pending a more
> aggressive re-write.
> 
> That's a problem for later, so for now, call the initial conversion to
> the statically typed subset of python done so we can move on to adding
> the regression tests that will maintain this baseline for us.
> 
> John Snow (3):
>    python: add mypy config
>    python/qemu/qmp.py: re-raise OSError when encountered
>    python/qemu/qmp.py: Fix settimeout operation
> 
>   python/mypy.ini    |  4 ++++
>   python/qemu/qmp.py | 30 +++++++++++++++++++++---------
>   2 files changed, 25 insertions(+), 9 deletions(-)
>   create mode 100644 python/mypy.ini
> 

Thanks -- tentatively staging to my Python branch. I intend to send a PR 
on Friday, so if there's more feedback, there's plenty of time.

https://gitlab.com/jsnow/qemu.git python
https://gitlab.com/jsnow/qemu/-/commits/python