* [PATCH] python: honour message limit when using pre-opened socket
@ 2023-03-20 10:54 Daniel P. Berrangé
2023-03-20 10:55 ` Marc-André Lureau
2023-03-20 12:20 ` Vladimir Sementsov-Ogievskiy
0 siblings, 2 replies; 6+ messages in thread
From: Daniel P. Berrangé @ 2023-03-20 10:54 UTC (permalink / raw)
To: qemu-devel
Cc: Vladimir Sementsov-Ogievskiy, Cleber Rosa, Beraldo Leal,
John Snow, Marc-André Lureau, Daniel P. Berrangé
The default message recv limit in asyncio is smaller than our needs, so
when opening connections we override it. This was done when opening a
connection using a socket address, but was missed when using a
pre-opened socket file descriptor.
This latent bug was exposed when the QEMUMachine class was enhanced to
use socketpair() when no socket address was passed by:
commit bd4c0ef409140bd1be393407c04005ac077d4574
Author: Marc-André Lureau <marcandre.lureau@redhat.com>
Date: Wed Jan 11 12:01:01 2023 +0400
python/qemu/machine: use socketpair() for QMP by default
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
python/qemu/qmp/protocol.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py
index 22e60298d2..2bd341a93e 100644
--- a/python/qemu/qmp/protocol.py
+++ b/python/qemu/qmp/protocol.py
@@ -307,7 +307,10 @@ async def open_with_socket(self, sock: socket.socket) -> None:
:raise StateError: When the `Runstate` is not `IDLE`.
"""
- self._reader, self._writer = await asyncio.open_connection(sock=sock)
+ self._reader, self._writer = await asyncio.open_connection(
+ sock=sock,
+ limit=self._limit,
+ )
self._set_state(Runstate.CONNECTING)
@upper_half
--
2.39.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH] python: honour message limit when using pre-opened socket
2023-03-20 10:54 [PATCH] python: honour message limit when using pre-opened socket Daniel P. Berrangé
@ 2023-03-20 10:55 ` Marc-André Lureau
2023-03-20 12:20 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 6+ messages in thread
From: Marc-André Lureau @ 2023-03-20 10:55 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Vladimir Sementsov-Ogievskiy, Cleber Rosa,
Beraldo Leal, John Snow
[-- Attachment #1: Type: text/plain, Size: 1642 bytes --]
On Mon, Mar 20, 2023 at 2:54 PM Daniel P. Berrangé <berrange@redhat.com>
wrote:
> The default message recv limit in asyncio is smaller than our needs, so
> when opening connections we override it. This was done when opening a
> connection using a socket address, but was missed when using a
> pre-opened socket file descriptor.
>
> This latent bug was exposed when the QEMUMachine class was enhanced to
> use socketpair() when no socket address was passed by:
>
> commit bd4c0ef409140bd1be393407c04005ac077d4574
> Author: Marc-André Lureau <marcandre.lureau@redhat.com>
> Date: Wed Jan 11 12:01:01 2023 +0400
>
> python/qemu/machine: use socketpair() for QMP by default
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>
thanks
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
> python/qemu/qmp/protocol.py | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/python/qemu/qmp/protocol.py b/python/qemu/qmp/protocol.py
> index 22e60298d2..2bd341a93e 100644
> --- a/python/qemu/qmp/protocol.py
> +++ b/python/qemu/qmp/protocol.py
> @@ -307,7 +307,10 @@ async def open_with_socket(self, sock: socket.socket)
> -> None:
>
> :raise StateError: When the `Runstate` is not `IDLE`.
> """
> - self._reader, self._writer = await
> asyncio.open_connection(sock=sock)
> + self._reader, self._writer = await asyncio.open_connection(
> + sock=sock,
> + limit=self._limit,
> + )
> self._set_state(Runstate.CONNECTING)
>
> @upper_half
> --
> 2.39.2
>
>
[-- Attachment #2: Type: text/html, Size: 2462 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] python: honour message limit when using pre-opened socket
2023-03-20 10:54 [PATCH] python: honour message limit when using pre-opened socket Daniel P. Berrangé
2023-03-20 10:55 ` Marc-André Lureau
@ 2023-03-20 12:20 ` Vladimir Sementsov-Ogievskiy
2023-03-21 0:27 ` John Snow
1 sibling, 1 reply; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2023-03-20 12:20 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Cleber Rosa, Beraldo Leal, John Snow, Marc-André Lureau
On 20.03.23 13:54, Daniel P. Berrangé wrote:
> The default message recv limit in asyncio is smaller than our needs, so
> when opening connections we override it. This was done when opening a
> connection using a socket address, but was missed when using a
> pre-opened socket file descriptor.
>
> This latent bug was exposed when the QEMUMachine class was enhanced to
> use socketpair() when no socket address was passed by:
>
> commit bd4c0ef409140bd1be393407c04005ac077d4574
> Author: Marc-André Lureau<marcandre.lureau@redhat.com>
> Date: Wed Jan 11 12:01:01 2023 +0400
>
> python/qemu/machine: use socketpair() for QMP by default
>
> Signed-off-by: Daniel P. Berrangé<berrange@redhat.com>
Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Thanks!
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] python: honour message limit when using pre-opened socket
2023-03-20 12:20 ` Vladimir Sementsov-Ogievskiy
@ 2023-03-21 0:27 ` John Snow
2023-05-15 21:14 ` John Snow
0 siblings, 1 reply; 6+ messages in thread
From: John Snow @ 2023-03-21 0:27 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Daniel P. Berrangé, qemu-devel, Cleber Rosa, Beraldo Leal,
Marc-André Lureau
On Mon, Mar 20, 2023 at 8:20 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> On 20.03.23 13:54, Daniel P. Berrangé wrote:
> > The default message recv limit in asyncio is smaller than our needs, so
> > when opening connections we override it. This was done when opening a
> > connection using a socket address, but was missed when using a
> > pre-opened socket file descriptor.
> >
> > This latent bug was exposed when the QEMUMachine class was enhanced to
> > use socketpair() when no socket address was passed by:
> >
> > commit bd4c0ef409140bd1be393407c04005ac077d4574
> > Author: Marc-André Lureau<marcandre.lureau@redhat.com>
> > Date: Wed Jan 11 12:01:01 2023 +0400
> >
> > python/qemu/machine: use socketpair() for QMP by default
> >
> > Signed-off-by: Daniel P. Berrangé<berrange@redhat.com>
>
> Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>
> Thanks!
>
> --
> Best regards,
> Vladimir
>
Thanks x3. Will stage in both places.
--js
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] python: honour message limit when using pre-opened socket
2023-03-21 0:27 ` John Snow
@ 2023-05-15 21:14 ` John Snow
2023-05-17 18:07 ` John Snow
0 siblings, 1 reply; 6+ messages in thread
From: John Snow @ 2023-05-15 21:14 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Daniel P. Berrangé, qemu-devel, Cleber Rosa, Beraldo Leal,
Marc-André Lureau
On Mon, Mar 20, 2023 at 8:27 PM John Snow <jsnow@redhat.com> wrote:
>
> On Mon, Mar 20, 2023 at 8:20 AM Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru> wrote:
> >
> > On 20.03.23 13:54, Daniel P. Berrangé wrote:
> > > The default message recv limit in asyncio is smaller than our needs, so
> > > when opening connections we override it. This was done when opening a
> > > connection using a socket address, but was missed when using a
> > > pre-opened socket file descriptor.
> > >
> > > This latent bug was exposed when the QEMUMachine class was enhanced to
> > > use socketpair() when no socket address was passed by:
> > >
> > > commit bd4c0ef409140bd1be393407c04005ac077d4574
> > > Author: Marc-André Lureau<marcandre.lureau@redhat.com>
> > > Date: Wed Jan 11 12:01:01 2023 +0400
> > >
> > > python/qemu/machine: use socketpair() for QMP by default
> > >
> > > Signed-off-by: Daniel P. Berrangé<berrange@redhat.com>
> >
> > Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> >
> > Thanks!
> >
> > --
> > Best regards,
> > Vladimir
> >
>
> Thanks x3. Will stage in both places.
>
... Realizing I let the two codebases diverge a bit more than I had
meant to allow, and the "open with socket" rewrite I made never got
brought back to qemu.git, tch. I believe this bug only exists in
qemu.git because of that. I need to backport my changes to qemu.git
and ensure that there isn't a problem with the revised code as it
exists in qemu.qmp.
I'll have to fix that *before* I drop qemu.qmp from the qemu.git repo,
which I am hoping will be soon after Paolo takes the new Python
patches that create a venv during configure time.
--js
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] python: honour message limit when using pre-opened socket
2023-05-15 21:14 ` John Snow
@ 2023-05-17 18:07 ` John Snow
0 siblings, 0 replies; 6+ messages in thread
From: John Snow @ 2023-05-17 18:07 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Daniel P. Berrangé, qemu-devel, Cleber Rosa, Beraldo Leal,
Marc-André Lureau
On Mon, May 15, 2023 at 5:14 PM John Snow <jsnow@redhat.com> wrote:
>
> On Mon, Mar 20, 2023 at 8:27 PM John Snow <jsnow@redhat.com> wrote:
> >
> > On Mon, Mar 20, 2023 at 8:20 AM Vladimir Sementsov-Ogievskiy
> > <vsementsov@yandex-team.ru> wrote:
> > >
> > > On 20.03.23 13:54, Daniel P. Berrangé wrote:
> > > > The default message recv limit in asyncio is smaller than our needs, so
> > > > when opening connections we override it. This was done when opening a
> > > > connection using a socket address, but was missed when using a
> > > > pre-opened socket file descriptor.
> > > >
> > > > This latent bug was exposed when the QEMUMachine class was enhanced to
> > > > use socketpair() when no socket address was passed by:
> > > >
> > > > commit bd4c0ef409140bd1be393407c04005ac077d4574
> > > > Author: Marc-André Lureau<marcandre.lureau@redhat.com>
> > > > Date: Wed Jan 11 12:01:01 2023 +0400
> > > >
> > > > python/qemu/machine: use socketpair() for QMP by default
> > > >
> > > > Signed-off-by: Daniel P. Berrangé<berrange@redhat.com>
> > >
> > > Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> > >
> > > Thanks!
> > >
> > > --
> > > Best regards,
> > > Vladimir
> > >
> >
> > Thanks x3. Will stage in both places.
> >
>
> ... Realizing I let the two codebases diverge a bit more than I had
> meant to allow, and the "open with socket" rewrite I made never got
> brought back to qemu.git, tch. I believe this bug only exists in
> qemu.git because of that. I need to backport my changes to qemu.git
> and ensure that there isn't a problem with the revised code as it
> exists in qemu.qmp.
>
> I'll have to fix that *before* I drop qemu.qmp from the qemu.git repo,
> which I am hoping will be soon after Paolo takes the new Python
> patches that create a venv during configure time.
>
> --js
OK, I've just sent two series:
(1) A MR for python-qemu-qmp here, which backports a few misc fixes
from qemu.git back over to python-qemu-qmp;
https://gitlab.com/qemu-project/python-qemu-qmp/-/merge_requests/23
This needs approval from at least one person. If you're interested in
becoming a "reviewer" there, let me know and I'll add you if you
aren't already.
(2) A patchset for qemu.git that backports the socket changes that
went into python-qemu-qmp.
https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg04285.html
This should fix the bug described here in this series, and is
necessary to merge before I swap the in-tree qemu.git for a vendored
qemu.qmp package.
Review welcomed, but any changes we need will need to happen on the
python-qemu-qmp repo first.
Dropping the in-tree qemu.qmp package is going to happen after the
large series Paolo just sent a PR for,
https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg04330.html
which adds a configure-time venv to the QEMU build system.
Hoping to drop qemu.qmp *this dev cycle* and release v0.0.3 for
qemu.qmp and vendor it to qemu.git as a .whl file.
Thanks,
--js
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-05-17 18:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-03-20 10:54 [PATCH] python: honour message limit when using pre-opened socket Daniel P. Berrangé
2023-03-20 10:55 ` Marc-André Lureau
2023-03-20 12:20 ` Vladimir Sementsov-Ogievskiy
2023-03-21 0:27 ` John Snow
2023-05-15 21:14 ` John Snow
2023-05-17 18:07 ` John Snow
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).