* [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 @ 2022-01-13 1:05 Jate Sujjavanich 2022-01-13 16:53 ` [bitbake-devel] " Michael Opdenacker 0 siblings, 1 reply; 5+ messages in thread From: Jate Sujjavanich @ 2022-01-13 1:05 UTC (permalink / raw) To: bitbake-devel; +Cc: Jate Sujjavanich Detect python version 3.6 and below restoring loop argument where it is still required. In 3.7 auto loop detection is available. Bitbake 1.46 is used in dunfell which requires a minimum python version of 3.5. Omitting this argument leads to a regression and hang during "Initialising tasks" at 44%. Signed-off-by: Jate Sujjavanich <jatedev@gmail.com> --- lib/hashserv/server.py | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/lib/hashserv/server.py b/lib/hashserv/server.py index 56f354bd..04a48c39 100644 --- a/lib/hashserv/server.py +++ b/lib/hashserv/server.py @@ -12,6 +12,7 @@ import math import os import signal import socket +import sys import time from . import chunkify, DEFAULT_MAX_CHUNK @@ -419,9 +420,14 @@ class Server(object): self._cleanup_socket = None def start_tcp_server(self, host, port): - self.server = self.loop.run_until_complete( - asyncio.start_server(self.handle_client, host, port) - ) + if sys.version_info[0] == 3 and sys.version_info[1] < 7: + self.server = self.loop.run_until_complete( + asyncio.start_server(self.handle_client, host, port, loop=self.loop) + ) + else: + self.server = self.loop.run_until_complete( + asyncio.start_server(self.handle_client, host, port) + ) for s in self.server.sockets: logger.info('Listening on %r' % (s.getsockname(),)) @@ -444,9 +450,14 @@ class Server(object): try: # Work around path length limits in AF_UNIX os.chdir(os.path.dirname(path)) - self.server = self.loop.run_until_complete( - asyncio.start_unix_server(self.handle_client, os.path.basename(path)) - ) + if sys.version_info[0] == 3 and sys.version_info[1] < 7: + self.server = self.loop.run_until_complete( + asyncio.start_unix_server(self.handle_client, os.path.basename(path), loop=self.loop) + ) + else: + self.server = self.loop.run_until_complete( + asyncio.start_unix_server(self.handle_client, os.path.basename(path)) + ) finally: os.chdir(cwd) -- 2.25.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 2022-01-13 1:05 [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 Jate Sujjavanich @ 2022-01-13 16:53 ` Michael Opdenacker 2022-01-13 18:04 ` Jate Sujjavanich 0 siblings, 1 reply; 5+ messages in thread From: Michael Opdenacker @ 2022-01-13 16:53 UTC (permalink / raw) To: Jate Sujjavanich, bitbake-devel Hi Jate On 1/13/22 2:05 AM, Jate Sujjavanich wrote: > Detect python version 3.6 and below restoring loop argument where > it is still required. In 3.7 auto loop detection is available. > > Bitbake 1.46 is used in dunfell which requires a minimum python version > of 3.5. Omitting this argument leads to a regression and hang during > "Initialising tasks" at 44%. Thanks for the patch! Weirdly, though, my host (Ubuntu 20.04) has Python 3.8 and still regularly has the "Initializing task" step stuck at 44% for a several minutes. Here's what strace says while this happens: futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1642092729, tv_nsec=110486000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out) futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1642092729, tv_nsec=360665000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out) futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1642092729, tv_nsec=610862000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out) futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1642092729, tv_nsec=861086000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out) futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1642092730, tv_nsec=111295000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out) ... Any clue why this can happen? Thanks Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 2022-01-13 16:53 ` [bitbake-devel] " Michael Opdenacker @ 2022-01-13 18:04 ` Jate Sujjavanich 2022-01-14 15:45 ` Michael Opdenacker [not found] ` <16CA2D972C0D44B2.13159@lists.openembedded.org> 0 siblings, 2 replies; 5+ messages in thread From: Jate Sujjavanich @ 2022-01-13 18:04 UTC (permalink / raw) To: Michael Opdenacker; +Cc: bitbake-devel, ashishm [-- Attachment #1: Type: text/plain, Size: 2672 bytes --] Hello Michael, I tested on an Ubuntu 20.04 docker container, and the code worked in that environment. It has kernel 5.10.25-linuxkit which is different from the default kernel in Ubuntu 20.04 which is 5.4. I chose to keep the loop argument for versions 3.6 and below because python first deprecated the loop argument in 3.7. Perhaps it's safer to keep specifying loop for 3.9 and below. There is one other report of the bug but it seems to be the result of the fix for 3.10 in October. So it was probably working for python 3.9 and below. Could you try the fix with the minor number changed to 10? if sys.version_info[0] == 3 and sys.version_info[1] < 10: If that works, I will update the patch. I am not familiar with the asyncio library and its system calls, so I'm hoping this works. Ashish: You reported behavior like this bug. Do you have more information on your Python and build environment? Thanks, - Jate S. On Thu, Jan 13, 2022 at 11:53 AM Michael Opdenacker < michael.opdenacker@bootlin.com> wrote: > Hi Jate > > On 1/13/22 2:05 AM, Jate Sujjavanich wrote: > > Detect python version 3.6 and below restoring loop argument where > > it is still required. In 3.7 auto loop detection is available. > > > > Bitbake 1.46 is used in dunfell which requires a minimum python version > > of 3.5. Omitting this argument leads to a regression and hang during > > "Initialising tasks" at 44%. > > > Thanks for the patch! > Weirdly, though, my host (Ubuntu 20.04) has Python 3.8 and still > regularly has the "Initializing task" step stuck at 44% for a several > minutes. > > Here's what strace says while this happens: > futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, > {tv_sec=1642092729, tv_nsec=110486000}, FUTEX_BITSET_MATCH_ANY) = -1 > ETIMEDOUT (Connection timed out) > futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, > {tv_sec=1642092729, tv_nsec=360665000}, FUTEX_BITSET_MATCH_ANY) = -1 > ETIMEDOUT (Connection timed out) > futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, > {tv_sec=1642092729, tv_nsec=610862000}, FUTEX_BITSET_MATCH_ANY) = -1 > ETIMEDOUT (Connection timed out) > futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, > {tv_sec=1642092729, tv_nsec=861086000}, FUTEX_BITSET_MATCH_ANY) = -1 > ETIMEDOUT (Connection timed out) > futex(0x277c200, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, > {tv_sec=1642092730, tv_nsec=111295000}, FUTEX_BITSET_MATCH_ANY) = -1 > ETIMEDOUT (Connection timed out) > ... > > Any clue why this can happen? > > Thanks > Michael. > > -- > Michael Opdenacker, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > > [-- Attachment #2: Type: text/html, Size: 4248 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 2022-01-13 18:04 ` Jate Sujjavanich @ 2022-01-14 15:45 ` Michael Opdenacker [not found] ` <16CA2D972C0D44B2.13159@lists.openembedded.org> 1 sibling, 0 replies; 5+ messages in thread From: Michael Opdenacker @ 2022-01-14 15:45 UTC (permalink / raw) To: Jate Sujjavanich; +Cc: bitbake-devel, ashishm Hi Jate Thanks for your advice! On 1/13/22 7:04 PM, Jate Sujjavanich wrote: > Hello Michael, > > I tested on an Ubuntu 20.04 docker container, and the code worked in > that environment. It has kernel 5.10.25-linuxkit which is different > from the default kernel in Ubuntu 20.04 which is 5.4. > > I chose to keep the loop argument for versions 3.6 and below because > python first deprecated the loop argument in 3.7. Perhaps it's safer > to keep specifying loop for 3.9 and below. There is one other report > of the bug but it seems to be the result of the fix for 3.10 in > October. So it was probably working for python 3.9 and below. > > Could you try the fix with the minor number changed to 10? > > if sys.version_info[0] == 3 and sys.version_info[1] < 10: > > If that works, I will update the patch. I am not familiar with the > asyncio library and its system calls, so I'm hoping this works. Actually, I don't have the issue on BitBake 1.46 (testing on Dunfell). Just tested. I have it with the version in master Poky, and your patch doesn't apply. Cheers, Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <16CA2D972C0D44B2.13159@lists.openembedded.org>]
* Re: [bitbake-devel] [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 [not found] ` <16CA2D972C0D44B2.13159@lists.openembedded.org> @ 2022-01-14 16:00 ` Michael Opdenacker 0 siblings, 0 replies; 5+ messages in thread From: Michael Opdenacker @ 2022-01-14 16:00 UTC (permalink / raw) To: Jate Sujjavanich; +Cc: bitbake-devel, ashishm On 1/14/22 4:45 PM, Michael Opdenacker wrote: > > Actually, I don't have the issue on BitBake 1.46 (testing on Dunfell). > Just tested. > > I have it with the version in master Poky, and your patch doesn't apply. I filed a new bug about this: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14689 Cheers Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-01-14 16:00 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-01-13 1:05 [1.46][PATCH] hashserv: specify loop for asyncio in python < 3.7 Jate Sujjavanich
2022-01-13 16:53 ` [bitbake-devel] " Michael Opdenacker
2022-01-13 18:04 ` Jate Sujjavanich
2022-01-14 15:45 ` Michael Opdenacker
[not found] ` <16CA2D972C0D44B2.13159@lists.openembedded.org>
2022-01-14 16:00 ` Michael Opdenacker
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.