[PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=845; bh=wCAz1kJFGU6x5XIcVruNfIrdFNZ+cbFWs4l3pbMJK/0=;

	b=F+Iqee6Wp0hEaYJRrNW03FBQHF+WSypPxCENwTDK/4JtMAekPNAUvho6F9wm7hHE

	s4H4t1w7oXaNXyJvJzlzTLx3HtTx5aZIpFrnM49AIEf/iwFffEnEUn3V2+Cm0RskbbM

	MLChdAN5QbJXRgWakwIVRcNkMhKRJP8Tc3IQVqGr/XWqCtfzkzKyiWB5ulqCkKaUBtK

	S9z/DSJZQWtnyuJbx8b40CSajkmkmQx707KIPH2FJKHgvS1uNSck7EzErsrTWFLIi8n

	GL13wFt3XE5avGIqKfxrLF/0RgNEyjk3Gh2B0K/W4b2uY7xI4ewtiiz9AfXsdI8hlqF

	A8zJwDXzPw==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

- Check file descriptor in ProgressBar() class init to see if it is resiz=
able.
- Optimize printing in footer update with use a StringIO buffer.
- Use 10 Hz refresh rate (FPS) for footer update.
- Cleaning and refactoring prints in code, print() functions were elimina=
ted from all loops for better performance.

Benjamin Sz=C5=91ke (4):
  bitbake: progressbar: Check resizable file descriptor.
  bitbake: knotty: Use a StringIO buffer for update footer.
  bitbake: knotty: Use 10 Hz refresh rate (FPS) for footer update.
  bitbake: knotty: print() was eliminated from all loops for better
    performance.

 lib/bb/ui/knotty.py            | 69 ++++++++++++++++++++++++----------
 lib/progressbar/progressbar.py | 25 +++++++-----
 2 files changed, 66 insertions(+), 28 deletions(-)

--=20
2.47.1.windows.2

[PATCH v2 1/4] bitbake: progressbar: Check resizable file descriptor.

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=2110; bh=FpTb7TZgCeBDZ6gBQVX9Oz9GJT0bDbIFEUtYaCztHxs=;

	b=GTkEGhTMvm6LL73h+Mrq96Z97vvy2b0TNGGFSdzDK05E7pJ9Lhh6tg4jvh8T8ipO

	hGSnvhx2M2Ko10CnODHOn+VTAli/DTWk8egU3cnUXyUsXEOkRALLLHqvWY3f5snAKaH

	Dii0cxhBcGM+fkDKAHc8YxnF34AX3t6SiUYui4YVLU9KpU/bzl9yv9uYqvKI9oQYEmM

	Ri6hzbgAy72E0nCEEdns/M1MMCJFPX7/n82z64w9ILCdqXgwZCaRknpBh8uATziT+Dk

	idhhtzVulaLNI8kdML7GhFq2aFG6CSfdvVuCY2tjSdLNR8PSXYPCuWZQ2uojixtSINp

	vCngpnaa8Q==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

- Check if given file descriptor is resizable.
- Introduce self._fd_console as a dedicated attribute of self._handle_res=
ize().

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 lib/progressbar/progressbar.py | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/lib/progressbar/progressbar.py b/lib/progressbar/progressbar=
.py
index d4da10ab7..203ff804f 100644
--- a/lib/progressbar/progressbar.py
+++ b/lib/progressbar/progressbar.py
@@ -110,19 +110,26 @@ class ProgressBar(object):
         self.widgets =3D widgets
         self.fd =3D fd
         self.left_justify =3D left_justify
+        self._fd_console =3D None
+
+        fd_num =3D fd.fileno()
=20
         self.signal_set =3D False
         if term_width is not None:
             self.term_width =3D term_width
+        elif (fd_num =3D=3D sys.stdout.fileno()) or (fd_num =3D=3D sys.s=
tderr.fileno()):
+            # Check if given file descriptor is resizable for example be=
long
+            # to a terminal/console as STDOUT or STDERR. If file descrip=
tor
+            # is resizable, let's allow to use for self._handle_resize()
+            # in a dedicated self._fd_console in order to be able to set
+            # temporarily/permanently self.fd to any StringIO or other
+            # file descriptor later.
+            self._fd_console =3D fd
+            self._handle_resize(None, None)
+            signal.signal(signal.SIGWINCH, self._handle_resize)
+            self.signal_set =3D True
         else:
-            try:
-                self._handle_resize(None, None)
-                signal.signal(signal.SIGWINCH, self._handle_resize)
-                self.signal_set =3D True
-            except (SystemExit, KeyboardInterrupt): raise
-            except Exception as e:
-                print("DEBUG 5 %s" % e)
-                self.term_width =3D self._env_size()
+            self.term_width =3D self._env_size()
=20
         self.__iterable =3D None
         self._update_widgets()
@@ -182,7 +189,7 @@ class ProgressBar(object):
     def _handle_resize(self, signum=3DNone, frame=3DNone):
         """Tries to catch resize signals sent from the terminal."""
=20
-        h, w =3D array('h', ioctl(self.fd, termios.TIOCGWINSZ, '\0' * 8)=
)[:2]
+        h, w =3D array('h', ioctl(self._fd_console, termios.TIOCGWINSZ, =
'\0' * 8))[:2]
         self.term_width =3D w
=20
=20
--=20
2.47.1.windows.2

[PATCH v2 2/4] bitbake: knotty: Use a StringIO buffer for update footer.

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=4274; bh=SjafIEl58zTmn6Ry+kbYbQ3e4/yFWWSVgI0HOiQG2po=;

	b=twEWl1DuFpZNW+V+lC01dU2289rkhKONFFBUlqaQJ9f8PH2JJYAq44R3YQG7uBQL

	dAVBLc13tCvLfiT3rfe7x3AGyGAAjdmo2qxPc4+IhwHYncC0cVqrQwm0as2N9El6K74

	sZinhZGWzMzOANHjFfmjSp1iSFJmm3YZpxM5beR1+GFWxt5Xsr3sDuU9LrbzdwB3qII

	xmWWF6s7ORm55yxJNzytadBfATsNiIy9vrtk1s8bclBbDNyLOAy6Iv9d0y4rUntsqCs

	4JevPV49rJJZBpPnrZUTPFsl2i2Yqm4YnkMj6vz1yn0Av50MH5rr42lxQYS3vIo6cGm

	H8NV7hVWTQ==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Optimize printing in footer update with use a StringIO buffer and it
prints content to terminal in a single call in every cycle.

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 lib/bb/ui/knotty.py | 31 +++++++++++++++++++++++--------
 1 file changed, 23 insertions(+), 8 deletions(-)

diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index 2fff1b366..c30bd64c7 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -10,6 +10,7 @@
=20
 from __future__ import division
=20
+import io
 import os
 import sys
 import logging
@@ -168,6 +169,9 @@ class TerminalFilter(object):
         self.lasttime =3D None
         self.quiet =3D quiet
=20
+        self._footer_buf =3D io.StringIO()
+        self._footer_lines =3D None
+
         if not self.interactive:
             return
=20
@@ -215,11 +219,11 @@ class TerminalFilter(object):
=20
     def clearFooter(self):
         if self.footer_present:
-            lines =3D self.footer_present
-            sys.stdout.buffer.write(self.curses.tparm(self.cuu, lines))
+            sys.stdout.buffer.write(self.curses.tparm(self.cuu, self._fo=
oter_lines))
             sys.stdout.buffer.write(self.curses.tparm(self.ed))
             sys.stdout.flush()
         self.footer_present =3D False
+        self._footer_lines =3D None
=20
     def elapsed(self, sec):
         hrs =3D int(sec / 3600.0)
@@ -257,6 +261,11 @@ class TerminalFilter(object):
             self.clearFooter()
         if (not self.helper.tasknumber_total or self.helper.tasknumber_c=
urrent =3D=3D self.helper.tasknumber_total) and not len(activetasks):
             return
+
+        # Clear footer buffer.
+        self._footer_buf.truncate(0)
+        self._footer_buf.seek(0)
+
         tasks =3D []
         for t in runningpids:
             start_time =3D activetasks[t].get("starttime", None)
@@ -275,6 +284,7 @@ class TerminalFilter(object):
                     else:
                         pbar =3D BBProgress("0: %s" % msg, 100, widgets=3D=
[' ', progressbar.Percentage(), ' ', progressbar.Bar(), ''], extrapos=3D5=
, resize_handler=3Dself.sigwinch_handle)
                         pbar.bouncing =3D False
+                    pbar.fd =3D self._footer_buf
                     activetasks[t]["progressbar"] =3D pbar
                 tasks.append((pbar, msg, progress, rate, start_time))
             else:
@@ -285,7 +295,7 @@ class TerminalFilter(object):
                                 "Waiting for %s running tasks to finish"=
, len(activetasks))
             if not self.quiet:
                 content +=3D ':'
-            print(content)
+            print(content, file=3Dself._footer_buf)
         else:
             scene_tasks =3D "%s of %s" % (self.helper.setscene_current, =
self.helper.setscene_total)
             cur_tasks =3D "%s of %s" % (self.helper.tasknumber_current, =
self.helper.tasknumber_total)
@@ -294,7 +304,7 @@ class TerminalFilter(object):
             if not self.quiet:
                 msg =3D "Setscene tasks: %s" % scene_tasks
                 content +=3D msg + "\n"
-                print(msg)
+                print(msg, file=3Dself._footer_buf)
=20
             if self.quiet:
                 msg =3D "Running tasks (%s, %s)" % (scene_tasks, cur_tas=
ks)
@@ -306,11 +316,12 @@ class TerminalFilter(object):
             if not self.main_progress or self.main_progress.maxval !=3D =
maxtask:
                 widgets =3D [' ', progressbar.Percentage(), ' ', progres=
sbar.Bar()]
                 self.main_progress =3D BBProgress("Running tasks", maxta=
sk, widgets=3Dwidgets, resize_handler=3Dself.sigwinch_handle)
+                self.main_progress.fd =3D self._footer_buf
                 self.main_progress.start(False)
             self.main_progress.setmessage(msg)
             progress =3D max(0, self.helper.tasknumber_current - 1)
             content +=3D self.main_progress.update(progress)
-            print('')
+            print('', file=3Dself._footer_buf)
         lines =3D self.getlines(content)
         if not self.quiet:
             for tasknum, task in enumerate(tasks[:(self.rows - 1 - lines=
)]):
@@ -326,14 +337,17 @@ class TerminalFilter(object):
                         content =3D pbar.update(progress)
                     else:
                         content =3D pbar.update(1)
-                    print('')
+                    print('', file=3Dself._footer_buf)
                 else:
                     content =3D "%s: %s" % (tasknum, task)
-                    print(content)
+                    print(content, file=3Dself._footer_buf)
                 lines =3D lines + self.getlines(content)
-        self.footer_present =3D lines
+        self._footer_lines =3D lines
+        self.footer_present =3D True
         self.lastpids =3D runningpids[:]
         self.lastcount =3D self.helper.tasknumber_current
+        # Print footer buffer.
+        print(self._footer_buf.getvalue(), end=3D'')
=20
     def getlines(self, content):
         lines =3D 0
@@ -342,6 +356,7 @@ class TerminalFilter(object):
         return lines
=20
     def finish(self):
+        self._footer_buf.close()
         if self.stdinbackup:
             fd =3D sys.stdin.fileno()
             self.termios.tcsetattr(fd, self.termios.TCSADRAIN, self.stdi=
nbackup)
--=20
2.47.1.windows.2

[PATCH v2 3/4] bitbake: knotty: Use 10 Hz refresh rate (FPS) for footer update.

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=1845; bh=EsMww1/nlvt7XQo4UmA8J5yvDu51WKjRJPQ3jZ2tbF8=;

	b=SKQsaad7N4BhQNLxvNYsIqFeZKcDejoNjb63rJ+/aAZrcVD24QhQGhI84v7GSw4Y

	A19udlJNNVojqh5V6mA1e8JrC/bCpCBAgmN2lWztSVcDCiA7AN1iAtj/s3Laobf7M9p

	LQmNWbSFz5j6sJDop84eBo6tL7ChtBcGLMdH+i/Xj/SD4dpJQT0+NsMy1/HWCJRMg/D

	pdLo+FXp9FQFS5JJ2VL1CrnA4syTEKpNmziJijgNVmnu50oZUt9wAiMoWRgqrZBv/ju

	kXYEwOKD27PW0hR/lNOlgT678y67zsqq0Auuq++1wZ0CKeb3dOgcEu9mBg5jAzPXmPU

	Qip8zau0jw==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Refresh footer in 10 Hz to avoid heavy print() flooding.

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 lib/bb/ui/knotty.py | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index c30bd64c7..ad018a182 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -128,6 +128,10 @@ class InteractConsoleLogFilter(logging.Filter):
         return True
=20
 class TerminalFilter(object):
+
+    # 10 Hz (FPS) -> 0.100 secs
+    _DEFAULT_PRINT_INTERVAL =3D 0.100
+
     rows =3D 25
     columns =3D 80
=20
@@ -166,7 +170,7 @@ class TerminalFilter(object):
         self.interactive =3D sys.stdout.isatty()
         self.footer_present =3D False
         self.lastpids =3D []
-        self.lasttime =3D None
+        self.lasttime =3D time.time()
         self.quiet =3D quiet
=20
         self._footer_buf =3D io.StringIO()
@@ -251,11 +255,23 @@ class TerminalFilter(object):
         failedtasks =3D self.helper.failed_tasks
         runningpids =3D self.helper.running_pids
         currenttime =3D time.time()
-        if not self.lasttime or (currenttime - self.lasttime > 5):
+        deltatime =3D currenttime - self.lasttime
+
+        if (deltatime > 5.0):
             self.helper.needUpdate =3D True
-            self.lasttime =3D currenttime
-        if self.footer_present and not self.helper.needUpdate:
+            need_update =3D self.helper.needUpdate
+        else:
+            # Do not let to update faster then _DEFAULT_PRINT_INTERVAL
+            # to avoid heavy print() flooding.
+            need_update =3D self.helper.needUpdate and (deltatime > self=
._DEFAULT_PRINT_INTERVAL)
+
+        if self.footer_present and (not need_update):
+            # Footer update is not need.
             return
+        else:
+            # Footer update is need and store its "lasttime" value.
+            self.lasttime =3D currenttime
+
         self.helper.needUpdate =3D False
         if self.footer_present:
             self.clearFooter()
--=20
2.47.1.windows.2

[PATCH v2 4/4] bitbake: knotty: print() was eliminated from all loops for better performance.

[egyszeregy]

	s=20181004; d=freemail.hu;

	h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding;

	l=1871; bh=tOCywO57AdcGssqoQY6VKW8CxCgsY7zpgBWvRbgEBvE=;

	b=AyzBtTiNk9uHQgoYs5u98aWhaXDE/gQykKrohGzF9Pj8oTblLdmIOrdOFsKiN7zV

	cj99R4gYoJNWhukqEJ0e9kPBomBlqyw9ZTxZTfaIZqRl3/6FpnXyKFLnDyXpX/hORQi

	ooBvyovFtrRGXdZdJfMRO3A7Mxdqhexeyci6eoeGAclwaKka3vkW3n5Ujaj449EkRAl

	ET463PuQydDVxmnEiATRCrKQKEaefweFvmDyDnJQsBIIV2HVuDqcG3jpUKReoTzcZCr

	0UAENsHZM7YL/D9tccMFKIROgiZgNsNq9ehcWf/UHKxS6f9hObDz+VmZ0dmduaPOwoA

	4ZS0lWvqkA==
Content-Transfer-Encoding: quoted-printable

From: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>

Refactoring prints, print() functions were eliminated from all loops and =
it uses
"\n".join(...) in a single print() call for better performance.

Signed-off-by: Benjamin Sz=C5=91ke <egyszeregy@freemail.hu>
---
 lib/bb/ui/knotty.py | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index ad018a182..dfc76483c 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -243,9 +243,10 @@ class TerminalFilter(object):
=20
     def keepAlive(self, t):
         if not self.cuu:
-            print("Bitbake still alive (no events for %ds). Active tasks=
:" % t)
+            msgbuf =3D ["Bitbake still alive (no events for %ds). Active=
 tasks:" % t]
             for t in self.helper.running_tasks:
-                print(t)
+                msgbuf.append(str(t))
+            print("\n".join(msgbuf))
             sys.stdout.flush()
=20
     def updateFooter(self):
@@ -378,13 +379,12 @@ class TerminalFilter(object):
             self.termios.tcsetattr(fd, self.termios.TCSADRAIN, self.stdi=
nbackup)
=20
 def print_event_log(event, includelogs, loglines, termfilter):
-    # FIXME refactor this out further
     logfile =3D event.logfile
     if logfile and os.path.exists(logfile):
         termfilter.clearFooter()
         bb.error("Logfile of failure stored in: %s" % logfile)
         if includelogs and not event.errprinted:
-            print("Log data follows:")
+            msgbuf =3D ["Log data follows:"]
             f =3D open(logfile, "r")
             lines =3D []
             while True:
@@ -397,11 +397,11 @@ def print_event_log(event, includelogs, loglines, t=
ermfilter):
                     if len(lines) > int(loglines):
                         lines.pop(0)
                 else:
-                    print('| %s' % l)
+                    msgbuf.append('| %s' % l)
             f.close()
             if lines:
-                for line in lines:
-                    print(line)
+                msgbuf.extend(lines)
+            print("\n".join(msgbuf))
=20
 def _log_settings_from_server(server, observe_only):
     # Get values of variables which control our output
--=20
2.47.1.windows.2

Re: [bitbake-devel] [PATCH v2 1/4] bitbake: progressbar: Check resizable file descriptor.

[Richard Purdie]

On Thu, 2025-01-23 at 00:24 +0100, Livius via lists.openembedded.org wrote:
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> - Check if given file descriptor is resizable.
> - Introduce self._fd_console as a dedicated attribute of self._handle_resize().
> 
> Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
> ---
>  lib/progressbar/progressbar.py | 25 ++++++++++++++++---------
>  1 file changed, 16 insertions(+), 9 deletions(-)

The existing code checked if a resize was possible and trapped the
exception if not.

The new code assumes stdout/stderr are resizeable and doesn't have an
exception if not. The commit does not say why either change is
necessary or why they're an improvement. It removes the check, contra
to what the commit message says.

Cheers,

Richard


> diff --git a/lib/progressbar/progressbar.py b/lib/progressbar/progressbar.py
> index d4da10ab7..203ff804f 100644
> --- a/lib/progressbar/progressbar.py
> +++ b/lib/progressbar/progressbar.py
> @@ -110,19 +110,26 @@ class ProgressBar(object):
>          self.widgets = widgets
>          self.fd = fd
>          self.left_justify = left_justify
> +        self._fd_console = None
> +
> +        fd_num = fd.fileno()
>  
>          self.signal_set = False
>          if term_width is not None:
>              self.term_width = term_width
> +        elif (fd_num == sys.stdout.fileno()) or (fd_num == sys.stderr.fileno()):
> +            # Check if given file descriptor is resizable for example belong
> +            # to a terminal/console as STDOUT or STDERR. If file descriptor
> +            # is resizable, let's allow to use for self._handle_resize()
> +            # in a dedicated self._fd_console in order to be able to set
> +            # temporarily/permanently self.fd to any StringIO or other
> +            # file descriptor later.
> +            self._fd_console = fd
> +            self._handle_resize(None, None)
> +            signal.signal(signal.SIGWINCH, self._handle_resize)
> +            self.signal_set = True
>          else:
> -            try:
> -                self._handle_resize(None, None)
> -                signal.signal(signal.SIGWINCH, self._handle_resize)
> -                self.signal_set = True
> -            except (SystemExit, KeyboardInterrupt): raise
> -            except Exception as e:
> -                print("DEBUG 5 %s" % e)
> -                self.term_width = self._env_size()
> +            self.term_width = self._env_size()
>  
>          self.__iterable = None
>          self._update_widgets()
> @@ -182,7 +189,7 @@ class ProgressBar(object):
>      def _handle_resize(self, signum=None, frame=None):
>          """Tries to catch resize signals sent from the terminal."""
>  
> -        h, w = array('h', ioctl(self.fd, termios.TIOCGWINSZ, '\0' * 8))[:2]
> +        h, w = array('h', ioctl(self._fd_console, termios.TIOCGWINSZ, '\0' * 8))[:2]
>          self.term_width = w
>  

Re: [bitbake-devel] [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Richard Purdie]

On Thu, 2025-01-23 at 00:24 +0100, Livius via lists.openembedded.org
wrote:
> From: Benjamin Szőke <egyszeregy@freemail.hu>
> 
> - Check file descriptor in ProgressBar() class init to see if it is
> resizable.
> - Optimize printing in footer update with use a StringIO buffer.
> - Use 10 Hz refresh rate (FPS) for footer update.
> - Cleaning and refactoring prints in code, print() functions were
> eliminated from all loops for better performance.

This series looks better, thanks. I still have concerns about the first
patch and I've replied to that one.

One question that still isn't answered is how you're measuring this
performance improvement. What issues were you seeing (apart from the
frequent console updates)?

Cheers,

Richard

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

Performance improvement is quite trivial that will happen. Not allowed to write to stdout or any unbuffered file while the code is running inside a loop in any script language or programming language.

I started to look for the performance issues because my Yocto projects started to build slowly from scarthgap release in any physical machines (only in a virtual machine it can run fast yet).
https://lists.openembedded.org/g/bitbake-devel/topic/109854844

Yesterday, I found the bug which releated to bitbake Event logger (workaround to disable it). Soon I will send a Patch v3 for footer update, but not this was the root cause, i will also send a patch for fix even logger class logging which has the real critical problem.

[-- Attachment #2: Type: text/html, Size: 848 bytes --]

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Performance improvement is quite trivial that will happen. Not allowed to write to stdout or any unbuffered file while the code is running inside a loop in any script language or programming language.

I started to look for the performance issues because my Yocto projects started to build slowly from scarthgap release in any physical machines (only in a virtual machine it can run fast yet).

https://lists.openembedded.org/g/bitbake-devel/topic/109854844

Yesterday, I found the bug which releated to bitbake Event logger (workaround to disable it). Soon I will send a Patch v3 for footer update, but not this was the root cause, i will also send a patch for fix even logger class logging which has the real critical problem.

[-- Attachment #2: Type: text/html, Size: 857 bytes --]

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Performance improvement is quite trivial that will happen. Not allowed to write to stdout or any unbuffered file while the code is running inside a loop in any script language or programming language.

I started to look for the performance issues because my Yocto projects started to build slowly from scarthgap release in any physical machines (only in a virtual machine it can run fast yet).

https://lists.openembedded.org/g/bitbake-devel/topic/109854844

Yesterday, I found the bug which releated to bitbake Event logger (workaround to disable it). Soon I will send a Patch v3 for footer update, but not this was the root cause. I will also send a patch for fix even logger class logging which has the real critical problem.

[-- Attachment #2: Type: text/html, Size: 857 bytes --]

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

Performance improvement is quite trivial that will happen. Not allowed to write to stdout or any unbuffered file while the code is running inside a loop in any script language or programming language.

I started to look for the performance issues because my Yocto projects started to build slowly from scarthgap release in any physical machines (only in a virtual machine it can run fast yet).

https://lists.openembedded.org/g/bitbake-devel/topic/109854844

Yesterday, I found the bug which releated to bitbake Event logger (workaround to disable it). Soon I will send a Patch v3 for footer update, but not this was the root cause. I will also send a patch for fix event logger class logging which has the real critical problem.

[-- Attachment #2: Type: text/html, Size: 858 bytes --]

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

PATCH v3 is ready long time ago, When it will be merged to bitbake?

[PATCH v3 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.
https://lists.openembedded.org/g/bitbake-devel/message/17083

[PATCH v3 1/4] bitbake: progressbar: Add self._fd_console to use for self._handle_resize()
https://lists.openembedded.org/g/bitbake-devel/message/17084

[PATCH v3 2/4] bitbake: knotty: Use a StringIO buffer for update footer.
https://lists.openembedded.org/g/bitbake-devel/message/17085

[PATCH v3 3/4] bitbake: knotty: Use 40 Hz refresh rate (FPS) for footer update.
https://lists.openembedded.org/g/bitbake-devel/message/17086

[PATCH v3 4/4] bitbake: knotty: print() was eliminated from all loops for better performance.
https://lists.openembedded.org/g/bitbake-devel/message/17087

[-- Attachment #2: Type: text/html, Size: 1308 bytes --]

Re: [PATCH v2 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.

[Livius]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

PATCH v3 is ready long time ago, When it will be merged to bitbake?

- [PATCH v3 0/4] bitbake: knotty/progressbar: Optimize footer update and prints.
  - https://lists.openembedded.org/g/bitbake-devel/message/17083
- [PATCH v3 1/4] bitbake: progressbar: Add self._fd_console to use for self._handle_resize()
  - https://lists.openembedded.org/g/bitbake-devel/message/17084
- [PATCH v3 2/4] bitbake: knotty: Use a StringIO buffer for update footer.
  - https://lists.openembedded.org/g/bitbake-devel/message/17085
- [PATCH v3 3/4] bitbake: knotty: Use 40 Hz refresh rate (FPS) for footer update.
  - https://lists.openembedded.org/g/bitbake-devel/message/17086
- [PATCH v3 4/4] bitbake: knotty: print() was eliminated from all loops for better performance.
  - https://lists.openembedded.org/g/bitbake-devel/message/17087

[-- Attachment #2: Type: text/html, Size: 1443 bytes --]