From: "Yann E. MORIN" <yann.morin.1998@free.fr>
To: "Raphaël Slagmolen" <raphael.slagmolen@mailfence.com>
Cc: buildroot@buildroot.org
Subject: Re: [Buildroot] [PATCH 1/1] utils/make_rich_wrapper.py: add new python script to wrap the build process into a colored TUI with progress tracking
Date: Sat, 12 Aug 2023 22:32:51 +0200 [thread overview]
Message-ID: <20230812203251.GS421096@scaer> (raw)
In-Reply-To: <1166577682.270413.1673277779708@fidget.co-bxl>
Raphaël, Al,
On 2023-01-09 16:22 +0100, Raphaël Slagmolen via buildroot spake thusly:
> As the build process can be long and hard to track, I wrote a little Python
> script that try to make it easier for users
> to see where in the process they are, what is going on, log filtering and
> more.
Thanks for this proposal, and sorry for the late reply...
We've discussed this a bit with another maintainer, and we are a bit
reluctant at carrying this script in Buildroot.
First, we cant easily review it, because it is badly line-wrapped.
Second, it looks too complicated.
Third, we already have a simpler script that just filters the output of
make, and just displays the lines with '>>>', redirecting the full log
to a file: utils/brmake; this should be enough for a beautification of
the build output.
So, we're going to reject this script.
Regards,
Yann E. MORIN.
> The script is completly autonomous and doesn't impose anything on the current
> build process so it should work without
> tampering the maintenability of the project's Makefile.
>
> The script has only two dependencies :
> - packaging (for version parsing while checking Buildroot's dependencies)
> - rich (for the colored output and layout)
>
> Features:
> - test the presence of Buildroot's dependencies, with version control and
> path validation (for `file`)
> - (optional) test the presence of Buildroot's optional dependencies but
> without blocking the process
> - can only do the checking or skip it entirely using command line option
> - filter the output of the build process while trying to detect
> fatal/error/warning/debug loglevel (may have some false positive)
> - can change the filtering level using a command line option (default on
> ERROR)
> - keep track of the amount of work to do using `make external-deps` and
> display the overall progress
> - in case the amount of work isn't correctly detected (I try to account for
> some specials steps), the progress is fixed at the end with a message
> - keep track of all the task already done and the one currently working
> - display the current task's step message (extracted from `>>> %s` output)
> - keep track of the current task progress based on the amount of steps
> done
> - display a progress in case of `wget` with extraction of filename, size,
> and downloaded from the output
>
> The script is launching `make` by itself but the user is supposed to have
> already done the configuration needed!
>
> May be added later:
> - export raw output from `make` into a log file in /tmp or elsewhere
> (command line option?)
> - add other trackable step like `wget`?
> - improve loglevel detection
> - any request from the team?!
>
> Some images can be seen on my personal server:
> - the `--help` menu: https://naheulcraft.be/nextcloud/s/aX8fW5mK8dbXwqT
> - the progress tracking:
> https://naheulcraft.be/nextcloud/s/87GK7yXQzZHydcj
> - the finished process: https://naheulcraft.be/nextcloud/s/ZPMt46WP5jib7mk
>
> Signed-off-by: Raphaël Slagmolen (Tutul) <raphael.slagmolen@mailfence.com>
> ---
> DEVELOPERS | 3 +
> utils/make_rich_wrapper.py | 660 +++++++++++++++++++++++++++++++++++++
> 2 files changed, 663 insertions(+)
> create mode 100755 utils/make_rich_wrapper.py
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 86e3f0e7b1..0c78d2f3a1 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -3159,3 +3159,6 @@ F: package/quazip/
> F: package/shapelib/
> F: package/simple-mail/
> F: package/tinc/
> +
> +N: Raphaël Slagmolen (Tutul) <raphael.slagmolen@mailfence.com>
> +F: utils/make_rich_wrapper.py
> diff --git a/utils/make_rich_wrapper.py b/utils/make_rich_wrapper.py
> new file mode 100755
> index 0000000000..152b50be65
> --- /dev/null
> +++ b/utils/make_rich_wrapper.py
> @@ -0,0 +1,660 @@
> +#!/usr/bin/env python
> +# -*- coding: utf-8 -*-
> +
> +'''
> + Date created: 2023/01/07
> + Date last modified: 2023/01/09
> + Python Version: 3.10+
> + Requirement:
> + packaging==21+
> + rich==13+
> +'''
> +
> +__author__ = 'Tutul (https://gitlab.com/Tutul)'
> +__copyright__ = 'Copyright 2023, make_rich_wrapper.py for Buildroot'
> +__license__ = 'MIT'
> +__credits__ = ['Tutul']
> +__version__ = '1.0.0'
> +__status__ = 'release' # release > test > dev
> +__maintainer__ = 'Tutul'
> +__description__ = 'simple python script that act as a wrapper around `make`
> for Buildroot using the Rich module'
> +
> +import argparse
> +import enum
> +import logging
> +import os
> +import re
> +import select
> +import shutil
> +import signal
> +import subprocess
> +from packaging import version
> +from rich import box
> +from rich.console import Console, Group
> +from rich.live import Live
> +from rich.logging import RichHandler
> +from rich.panel import Panel
> +from rich.prompt import Confirm
> +from rich.progress import (
> + BarColumn,
> + DownloadColumn,
> + MofNCompleteColumn,
> + Progress, SpinnerColumn,
> + Task,
> + TaskProgressColumn,
> + TextColumn,
> + TimeElapsedColumn,
> + TimeRemainingColumn,
> + TransferSpeedColumn
> +)
> +from rich.table import Table
> +
> +############################################################################################
> +
> +
> +class DownloadStep(enum.Enum):
> + """
> + Help to know wich downloading information is present
> + """
> + LENGTH = 0
> + OUTPUT = 1
> + PROGRESS = 2
> + DONE = 3
> +
> +############################################################################################
> +
> +
> +class State(enum.Enum):
> + """
> + Buildroot Makefile's steps
> + """
> + READY = 0
> + DOWNLOADING = 1
> + EXTRACTING = 2
> + PATCHING = 3
> + CONFIGURING = 4
> + BUILDING = 5
> + INSTALLING = 6
> + DONE = 7
> +
> + @staticmethod
> + def get_from_message(message) -> str:
> + if message.__contains__("Downloading"):
> + return State.DOWNLOADING
> + elif message.__contains__("Extracting"):
> + return State.EXTRACTING
> + elif message.__contains__("Patching"):
> + return State.PATCHING
> + elif message.__contains__("Configuring"):
> + return State.CONFIGURING
> + elif message.__contains__("Building"):
> + return State.BUILDING
> + elif message.__contains__("Installing"):
> + return State.INSTALLING
> + else:
> + return None
> +
> +############################################################################################
> +
> +
> +class Task():
> + """
> + Keep track of the current step
> + """
> +
> + def __init__(self, current_pkg_progress, step_progress,
> pkg_steps_progress, downloader):
> + self.current_pkg_progress = current_pkg_progress
> + self.step_progress = step_progress
> + self.pkg_steps_progress = pkg_steps_progress
> + self.downloader = downloader
> +
> + self.pkg_task_id = None
> + self.step_task_id = None
> + self.pkg_step_task_id = None
> + self.downloading_task_id = None
> + self.state = None
> + self.name = None
> +
> + def __del__(self):
> + if self.pkg_task_id is not None:
> + self.current_pkg_progress.stop_task(self.pkg_task_id)
> + self.pkg_task_id = None
> + if self.step_task_id is not None:
> + self.step_progress.stop_task(self.step_task_id)
> + self.step_progress.update(self.step_task_id, visible=False)
> + self.step_task_id = None
> + if self.pkg_step_task_id is not None:
> + self.pkg_steps_progress.stop_task(self.pkg_step_task_id)
> + self.pkg_steps_progress.update(self.pkg_step_task_id,
> visible=False)
> + self.pkg_step_task_id = None
> + if self.downloading_task_id is not None:
> + self.downloader.stop_task(self.downloading_task_id)
> + self.downloader.update(self.downloading_task_id, visible=False)
> + self.downloading_task_id = None
> + self.name = None
> + self.sate = State.DONE
> +
> + def update(self, new_state, new_step=None) -> None:
> + self.state = new_state
> +
> + if self.downloading_task_id is not None and State.DOWNLOADING !=
> self.state:
> + self.downloader.stop_task(self.downloading_task_id)
> + self.downloader.update(self.downloading_task_id, visible=False)
> + self.downloading_task_id = None
> +
> + if self.step_task_id is not None:
> + self.step_progress.stop_task(self.step_task_id)
> + self.step_progress.update(self.step_task_id, visible=False)
> + if new_step is not None:
> + self.step_task_id = self.step_progress.add_task("", step=new_step,
> name=self.name)
> + else:
> + self.step_task_id = None
> + if self.pkg_step_task_id is not None:
> + self.pkg_steps_progress.update(self.pkg_step_task_id,
> completed=self.state.value)
> +
> + def next(self, new_name, new_state, new_step=None) -> None:
> + if self.pkg_task_id is not None:
> + self.current_pkg_progress.stop_task(self.pkg_task_id)
> + self.current_pkg_progress.update(self.pkg_task_id,
> description="[bold green]%s done!" % self.name)
> +
> + if self.step_task_id is not None:
> + self.step_progress.stop_task(self.step_task_id)
> + self.step_progress.update(self.step_task_id, visible=False)
> +
> + if self.pkg_step_task_id is not None:
> + self.pkg_steps_progress.stop_task(self.pkg_step_task_id)
> + self.pkg_steps_progress.update(self.pkg_step_task_id,
> visible=False)
> + self.pkg_step_task_id = None
> +
> + if self.downloading_task_id is not None:
> + self.downloader.stop_task(self.downloading_task_id)
> + self.downloader.update(self.downloading_task_id, visible=False)
> + self.downloading_task_id = None
> +
> + self.name = new_name
> + self.state = new_state
> +
> + if self.state is not None:
> + self.step_task_id = self.step_progress.add_task("", step=new_step,
> name=self.name)
> + if "Finishing" == self.name:
> + self.pkg_task_id = self.current_pkg_progress.add_task("%s" %
> self.name)
> + else:
> + self.pkg_step_task_id = self.pkg_steps_progress.add_task("",
> total=State.DONE.value, name=self.name)
> + self.pkg_task_id = self.current_pkg_progress.add_task("Working on
> %s" % self.name)
> +
> +############################################################################################
> +
> +
> +def handle_task_update(current, message) -> bool:
> + """
> + Handle step update and detect task switching
> + """
> + finished = False
> + message_array = message.split(">>> ")[1].split(" ")
> +
> + name = "[b]" + message_array[0] + "[/b]"
> + if len(message_array) >= 3:
> + name = name + " " + message_array[1]
> + step = message_array[2:]
> + else:
> + step = message_array[1:]
> + if message.__contains__(">>> "):
> + name = "Finishing" # Reset for the last few tasks
> +
> + step = ' '.join(str(x) for x in step)
> + state = State.get_from_message(step)
> +
> + if current.name is None:
> + current.next(name, state, step)
> + elif name != current.name:
> + finished = True
> + current.update(State.DONE)
> + if state is None:
> + state = State.READY
> + current.next(name, state, step)
> + elif state is None:
> + state = current.state
> +
> + current.update(state, step)
> + return finished
> +
> +############################################################################################
> +
> +
> +def handle_downloading(current, message, step) -> None:
> + """
> + Handle all that is about downloading (wget only for now)
> + """
> +
> + log.debug(message)
> +
> + if current.downloading_task_id is None:
> + current.downloading_task_id = current.downloader.add_task("",
> total=None, filename="?")
> +
> + if DownloadStep.LENGTH == step:
> + current.downloader.update(current.downloading_task_id,
> total=int(message.split("Length: ")[1].split(" (")[0]))
> + elif DownloadStep.OUTPUT == step:
> + current.downloader.update(current.downloading_task_id,
> filename=message.split("output' '")[1])
> + elif DownloadStep.PROGRESS == step:
> + if message.__contains__(" ."):
> + downloaded_size = message.split(" .")[0]
> + else:
> + downloaded_size = message.split("
> ")[0]
> + downloaded_size.strip().upper().replace("O", "").replace("B",
> "").replace("I", "")
> + if downloaded_size.__contains__("K"):
> + downloaded_size = int(downloaded_size.replace("K", "")) * 1000
> + elif downloaded_size.__contains__("M"):
> + downloaded_size = int(downloaded_size.replace("M", "")) * 1000000
> + elif downloaded_size.__contains__("G"):
> + downloaded_size = int(downloaded_size.replace("G", "")) *
> 1000000000
> + elif downloaded_size.__contains__("T"):
> + downloaded_size = int(downloaded_size.replace("T", "")) *
> 1000000000000
> + current.downloader.update(current.downloading_task_id,
> completed=int(downloaded_size))
> + elif DownloadStep.DONE == step:
> + current.downloader.stop_task(current.downloading_task_id)
> + current.downloader.update(current.downloading_task_id, visible=False)
> + current.downloading_task_id = None
> +
> +############################################################################################
> +
> +
> +def which(exec_name, package=None, desired_version=None,
> version_tuple_catcher=None, path=None) -> bool:
> + """
> + Used to find if a binary is present (and usable) on the system
> + Can also check for minimal version (as long as the binary use --version)
> + And may also check if the path is the required one
> + """
> + global console
> + if package is None:
> + package = exec_name
> +
> + p = shutil.which(exec_name)
> + if p is None:
> + console.print(f" [red1]:cross_mark: [b]{package}[/b] is required
> but cannot find it in the PATH")
> + return False
> + elif path is not None and path != p:
> + console.print(
> + f" [orange_red1]:no_entry: [b]{package}[/b] is found but we
> need it to be in a specific location: {p} --> {path}")
> + return False
> + elif desired_version is not None:
> + if version_tuple_catcher is not None:
> + local_version = subprocess.run([p, "--version"],
> stdout=subprocess.PIPE, check=True,
> +
> text=True).stdout.split(version_tuple_catcher[0])[1].split(version_tuple_catcher[1])[0]
> + if version.parse(local_version) < version.parse(desired_version):
> + console.print(
> + f" [orange1]:exclamation_mark: [b]{package}[/b] is
> found but we require at least the version {desired_version} and you have the
> version {local_version}")
> + return False
> + else:
> + console.print(
> + f" [orange1]:exclamation_mark: [b]{package}[/b] is found
> but we require at least the version {desired_version}")
> + return False
> + console.print(f" [green4]:heavy_check_mark: [b]{package}[/b] is found
> and usable")
> + return True
> +
> +############################################################################################
> +
> +
> +def test_depencencies(tools_list) -> bool:
> + """
> + Simply handle calling which() from a list of tools
> + """
> + is_all_ok = True
> + for tool in tools_list:
> + if not which(tool[0], tool[1], tool[2], tool[3], tool[4]):
> + is_all_ok = False
> + return is_all_ok
> +
> +############################################################################################
> +
> +
> +# which(binary), package[binary], minimal version, version_tuple_catcher,
> required path
> +build_tools_list = [
> + ("which", None, None, None, None),
> + ("sed", None, None, None, None),
> + ("make", None, "3.81", ("GNU Make ", "\n"), None),
> + ("ld", "binutils", None, None, None),
> + ("diff", "diffutils", None, None, None),
> + ("gcc", None, "4.8", ("gcc (GCC) ", "\n"), None),
> + ("g++", None, "4.8", ("g++ (GCC) ", "\n"), None),
> + ("bash", None, None, None, None),
> + ("patch", None, None, None, None),
> + ("gzip", None, None, None, None),
> + ("bzip2", None, None, None, None),
> + ("perl", None, "5.8.7", (" (v", ") built for"), None),
> + ("tar", None, None, None, None),
> + ("cpio", None, None, None, None),
> + ("unzip", None, None, None, None),
> + ("rsync", None, None, None, None),
> + ("file", None, None, None, "/usr/bin/file"),
> + ("bc", None, None, None, None),
> + ("find", "findutils", None, None, None)
> +]
> +source_fetching_tools_list = [
> + ("wget", None, None, None, None)
> +]
> +
> +
> +def check_requirements() -> bool:
> + """
> + Manage the display and checking for the required tools, stop if any is
> missing
> + """
> + global console
> + requirements_ok = True
> + console.print("Mandatory packages:")
> + with console.status("[blue]Looking for mandatory packages...") as status:
> + console.print(" [i]Build tools:")
> + requirements_ok = test_depencencies(build_tools_list) and
> requirements_ok
> + console.print(" [i]Source fetching tools:")
> + requirements_ok = test_depencencies(source_fetching_tools_list) and
> requirements_ok
> + if not requirements_ok:
> + console.print("\nMandatory program are [u]missing[/u]")
> + console.print("you must install them before retrying!")
> + return requirements_ok
> +
> +############################################################################################
> +
> +
> +recommanded_dependencies_list = [
> + ("python", None, "2.7", ("Python ", "\n"), None)
> +]
> +configuration_interface_dependencies_list = [
> + ("ncursesw5-config", "ncurses5", None, None, None), # menuconfig
> + ("qtdiag-qt5", "qt5", None, None, None), # xconfig
> + ("glib-compile-schemas", "glib2", None, None, None), # gconfig
> + ("gtk-launch", "gtk2", None, None, None), # gconfig
> + ("glade", "glade2", None, None, None) # gconfig
> +]
> +opt_source_fetching_tools_list = [
> + ("bzr", "bazaar", None, None, None),
> + ("cvs", None, None, None, None),
> + ("git", None, None, None, None),
> + ("hg", "mercurial", None, None, None),
> + ("scp", None, None, None, None),
> + ("sftp", None, None, None, None),
> + ("svn", "subversion", None, None, None)
> +]
> +java_related_packages_list = [
> + ("javac", None, None, None, None),
> + ("jar", None, None, None, None)
> +]
> +documentation_generation_tools_list = [
> + ("asciidoc", None, "8.6.3", ("asciidoc ", "\n"), None),
> + ("w3m", None, None, None, None),
> + ("dblatex", None, None, None, None) # Only for PDF manual
> +]
> +graph_generation_tools_list = [
> + ("gvgen", "graphviz", None, None, None)
> +]
> +
> +
> +def check_optional_packages() -> bool:
> + """
> + Manage the display and checking for the optional tools, continue with a
> warning if any is missing
> + """
> + global console
> + opt_ok = True
> + console.print("Optional packages:")
> + with console.status("[blue]Looking for optional packages...") as status:
> + console.print(" [i]Recommended dependencies:")
> + opt_ok = test_depencencies(recommanded_dependencies_list) and opt_ok
> + console.print(" [i]Configuration interface dependencies:")
> + opt_ok = test_depencencies(configuration_interface_dependencies_list)
> and opt_ok
> + console.print(" [i]Source fetching tools:")
> + opt_ok = test_depencencies(opt_source_fetching_tools_list) and opt_ok
> + console.print(" [i]Java-related packages, if the Java Classpath needs
> to be built for the target system:")
> + opt_ok = test_depencencies(java_related_packages_list) and opt_ok
> + console.print(" [i]Documentation generation tools:")
> + opt_ok = test_depencencies(documentation_generation_tools_list) and
> opt_ok
> + console.print(" [i]Graph generation tools:")
> + opt_ok = test_depencencies(graph_generation_tools_list) and opt_ok
> + console.print(
> + " [gray39]:interrobang: [b]python-matplotlib[/b] cannot be
> detected automaticly. Check your distribution's repositories or [i]pip[/i]")
> + if not opt_ok:
> + console.print("\n>>> Some optional program are [u]missing[/u], some
> functionalities may not compile or be incomplete")
> + return opt_ok
> +
> +############################################################################################
> +
> +
> +# skeleton (4): host-skeleton && skeleton-init-common &&
> skeleton-init-(sysv|systemd|openrc|none) && skeleton
> +# toolcahin (2): toolchain && toolchain-buildroot
> +# gcc(+1): gcc become host-gcc-initial and host-gcc-final
> +# various (~4): host-makedevs && ifupdown-scripts && initscripts &&
> urandom-scripts
> +# +1 for the last task (finalizing host/target directory, sanitizing RPATH,
> generating root filesystems and image rootfs)
> +# BUT.... in some case this number isn't quit exact (especially the 'various'
> section)
> +extra_packages_to_count = 12
> +
> +
> +def load_all_tasks() -> None:
> + """
> + Load the external deps on witch we need to work for this run
> + if an error occure, it's probably because the user didn't do any `make
> menuconfig`
> + """
> + global total_pkgs
> + global console
> + with console.status("[bold]Loading config, requested packages and
> dependencies[/bold] (did you run `[blue]make menuconfig[/blue]`?)",
> spinner="aesthetic") as status:
> + external_deps = subprocess.run(["make", "external-deps"],
> stdout=subprocess.PIPE, check=True, text=True)
> + raw_pkgs_list = external_deps.stdout.splitlines()
> +
> + # Do some cleaning because all the patches files aren't packages
> + regex = re.compile(r'^.*\.patch.*$')
> + total_pkgs = [p for p in raw_pkgs_list if not regex.match(p)]
> +
> + console.print("In total", (len(total_pkgs) + extra_packages_to_count),
> "packages will be processed for this buildroot")
> +
> +############################################################################################
> +
> +
> +def main() -> None:
> + # Track the averall progress
> + pkgs_progress = Progress(
> + TimeElapsedColumn(),
> + TextColumn("[blue]{task.description}"),
> + BarColumn(complete_style='slate_blue1'),
> + MofNCompleteColumn(),
> + TaskProgressColumn()
> + )
> + # Track the current progress and stay after completion
> + current_pkg_progress = Progress(
> + TimeElapsedColumn(),
> + TextColumn("{task.description}")
> + )
> + # Progress bars for single pkg steps (will be hidden when step is done)
> + step_progress = Progress(
> + TextColumn(" "),
> + SpinnerColumn(),
> + TextColumn("[bold purple]{task.fields[step]}")
> + )
> + # Progress bar for current pkg (progress in steps)
> + pkg_steps_progress = Progress(
> + TextColumn("[bold cyan]Progress for pkg {task.fields[name]}"),
> + SpinnerColumn("simpleDotsScrolling"),
> + TextColumn("({task.completed} of {task.total} steps done)"),
> + )
> + # Track any downloading
> + downloader_progress = Progress(
> + TextColumn("[bold]{task.fields[filename]}", justify="right"),
> + BarColumn(bar_width=None, complete_style="dark_magenta",
> pulse_style="medium_purple"),
> + "[progress.percentage]{task.percentage:>3.1f}%",
> + "•",
> + DownloadColumn(),
> + "•",
> + TransferSpeedColumn(),
> + "•",
> + TimeRemainingColumn()
> + )
> +
> + # group of progress bars;
> + # some are always visible, others will disappear when progress is
> complete
> + working_panel = Panel(Group(current_pkg_progress, step_progress,
> pkg_steps_progress, downloader_progress), box=box.SIMPLE_HEAD)
> + progress_group = Group(
> + working_panel,
> + pkgs_progress
> + )
> +
> + pkgs_task_id = pkgs_progress.add_task("Packages", total=(len(total_pkgs) +
> extra_packages_to_count))
> + cached = None
> + overflow = None
> +
> + # use own live instance as context manager with group of progress bars,
> + # which allows for running multiple different progress bars in parallel,
> + # and dynamically showing/hiding them
> + with Live(progress_group):
> + # Current task
> + pkg = Task(current_pkg_progress=current_pkg_progress,
> step_progress=step_progress,
> + pkg_steps_progress=pkg_steps_progress,
> downloader=downloader_progress)
> +
> + # Prepare the call to `make`
> + modified_env = os.environ.copy()
> + modified_env["LANG"] = "C"
> + log.debug(f"Using modified encironment: {modified_env}")
> + process = subprocess.Popen(
> + 'make',
> + env=modified_env,
> + stdout=subprocess.PIPE,
> + stderr=subprocess.STDOUT,
> + text=True,
> + close_fds=True)
> +
> + # Loop as long as the process is running
> + while process.poll() is None:
> + # Wait for something to read from stdout or stderr
> + _, _, _ = select.select([process.stdout.fileno()], [], [])
> + read = process.stdout.readline()
> +
> + # Detect each step
> + if read.__contains__("[7m>>>"):
> + if handle_task_update(pkg, read):
> + pkgs_progress.advance(pkgs_task_id, advance=1)
> + # Handles all WGET downloading steps
> + elif re.search("Length: [0-9]+ \([0-9ioObBkKmMgGtT.,]+\)", read):
> + handle_downloading(pkg, read, DownloadStep.LENGTH)
> + elif re.search("wget -.*output' '.*'", read):
> + handle_downloading(pkg, read, DownloadStep.OUTPUT)
> + elif re.search("[0-9kKmMgGtTioObB.,]+ [. ]+ [0-9]+%", read):
> + handle_downloading(pkg, read, DownloadStep.PROGRESS)
> + elif re.search("[0-9-/pPaAmM ]+ \([0-9., kKmMgGtTbBoOiI\/]+s\) -
> .* saved", read):
> + handle_downloading(pkg, read, DownloadStep.DONE)
> + # Handles auto-detect for logging level
> + elif re.search("[\] ]{1}(EXCEPTION|EMERGENCY|EMERG)[\]: ]{1}",
> read.upper()):
> + log.exception(read)
> + elif re.search("[\] ]{1}(CRITICAL|CRIT|FATAL|ALERT)[\]: ]{1}",
> read.upper()):
> + log.critical(read)
> + elif re.search("[\] ]{1}(ERROR|ERR)[\]: ]{1}", read.upper()):
> + log.error(read)
> + elif re.search("[\] ]{1}(WARNING|WARN)[\]: ]{1}", read.upper()):
> + log.warning(read)
> + elif re.search("[\] ]{1}(DEBUG|TRACE)[\]: ]{1}", read.upper()):
> + log.debug(read)
> + elif read:
> + log.info(read)
> +
> + # Clean the "current" progress bar
> + del pkg
> +
> + # We may need to fix this progress depending of the untrackable tasks
> that may be badly accounted for earlier
> + pkgs_task = [t for t in pkgs_progress.tasks if t.id ==
> pkgs_task_id][0]
> + if pkgs_task.completed < pkgs_task.total:
> + cached = pkgs_task.total - pkgs_task.completed
> + pkgs_progress.update(pkgs_task_id, completed=(pkgs_task.total),
> advance=1)
> + elif pkgs_task.completed > pkgs_task.total:
> + overflow = pkgs_task.completed - pkgs_task.total
> + pkgs_progress.update(pkgs_task_id, total=pkgs_task.completed,
> completed=(pkgs_task.completed), advance=1)
> + pkgs_progress.refresh()
> + if cached is not None:
> + console.print(f"[spring_green3][b]{cached}[/b] pkg(s) loaded from
> [i]cache")
> + if overflow is not None:
> + console.print(f"[deep_pink2 b]{overflow} unacounted pkg(s) added to
> our workload")
> + console.print("Work is done :relieved_face:")
> +
> +############################################################################################
> +
> +
> +def handle_sigint(signum, frame) -> None:
> + console.print("Interrupted by the user")
> + quit()
> +
> +
> +signal.signal(signal.SIGINT, handle_sigint)
> +
> +############################################################################################
> +
> +if __name__ == '__main__':
> + parser = argparse.ArgumentParser(
> + prog="make_rich_wrapper.py",
> + description=f"{__description__}",
> + epilog=f"{__copyright__}, {__author__}, license: {__license__}"
> + )
> + parser.add_argument('-l', '--loglevel',
> + help='set the verbosity level for the logs printed on
> the screen',
> + type=str,
> + choices=["all", "debug", "info", "warn", "error",
> "critical"],
> + default="error")
> + group = parser.add_mutually_exclusive_group()
> + group.add_argument('-f', '--fast',
> + help='disable the requirements and optionnal check
> before running',
> + action='store_true')
> + group.add_argument('--only-check',
> + help='only do the requirements and optionnal check and
> exit',
> + action='store_true')
> + parser.add_argument('--check-optional',
> + help='enable checking for optional ressources and
> exit',
> + action='store_true')
> + parser.add_argument('--version',
> + help='output version information and exit',
> + action='version',
> + version=f'%(prog)s {__version__} - {__status__}')
> + parser.add_argument('--about',
> + help='output about information and exit',
> + action='store_true')
> +
> + arguments = parser.parse_args()
> +
> + if arguments.about:
> + console = Console(highlight=False)
> + console.print(f"{__copyright__}\nCreated by {__author__}, license:
> {__license__}\n\n{__description__}\n")
> + if "release" == __status__:
> + console.print(f"[green b]{__status__}")
> + elif "test" == __status__:
> + console.print(f"[cyan]{__status__}")
> + elif "dev" == __status__:
> + console.print(f"[purple i]{__status__}")
> + else:
> + console.print(f"[red u]{__status__}")
> + console.print(f"Version: {__version__}\nMaintainer:
> {__maintainer__}\n\nCredits: {__credits__}")
> + quit()
> +
> + # Initialise the rich console for printing
> + console = Console()
> +
> + # Initialing the rich log handler
> + logging.basicConfig(
> + level=arguments.loglevel.upper().replace("ALL",
> "NOTSET").replace("WARN", "WARNING"),
> + format="%(message)s",
> + datefmt="[%X]",
> + handlers=[RichHandler(rich_tracebacks=True, show_path=False)]
> + )
> + log = logging.getLogger("rich")
> +
> + if not arguments.fast:
> + ok = check_requirements()
> + if not arguments.only_check:
> + if ok:
> + console.rule(style="green")
> + else:
> + quit()
> +
> + if not arguments.fast and arguments.check_optional:
> + ok = check_optional_packages()
> + if not arguments.only_check:
> + if ok:
> + console.rule(style="blue")
> + else:
> + console.rule(style="magenta")
> +
> + if not arguments.only_check:
> + load_all_tasks()
> + if not arguments.fast and not Confirm.ask("Proceed? (it may take some
> times)", console=console, default="y"):
> + quit()
> + console.print("\n\n")
> + main()
> --
> 2.39.0
>
> ----
> De: raphael.slagmolen@mailfence.com
> À: buildroot@buildroot.org, raphael.slagmolen@mailfence.com
> 9 janv. 2023 15:57:48
>
> ----
> De: raphael.slagmolen@mailfence.com
> À: buildroot@buildroot.org
> 9 janv. 2023 16:22:52
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
prev parent reply other threads:[~2023-08-12 20:33 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-09 15:22 [Buildroot] [PATCH 1/1] utils/make_rich_wrapper.py: add new python script to wrap the build process into a colored TUI with progress tracking Raphaël Slagmolen via buildroot
2023-08-12 20:32 ` Yann E. MORIN [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230812203251.GS421096@scaer \
--to=yann.morin.1998@free.fr \
--cc=buildroot@buildroot.org \
--cc=raphael.slagmolen@mailfence.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox