From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9CAF3C43334 for ; Sat, 23 Jul 2022 16:11:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 40A0D40920; Sat, 23 Jul 2022 16:11:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 40A0D40920 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lNAZ9SyYLjpF; Sat, 23 Jul 2022 16:11:26 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp2.osuosl.org (Postfix) with ESMTP id 3B10D402B1; Sat, 23 Jul 2022 16:11:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3B10D402B1 Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 8E5311BF271 for ; Sat, 23 Jul 2022 16:11:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 63F554018E for ; Sat, 23 Jul 2022 16:11:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 63F554018E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BR330i-k_qXM for ; Sat, 23 Jul 2022 16:11:21 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3F94E4031D Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3F94E4031D for ; Sat, 23 Jul 2022 16:11:21 +0000 (UTC) Received: by mail-wm1-x32f.google.com with SMTP id w8-20020a05600c014800b003a32e89bc4eso4003093wmm.5 for ; Sat, 23 Jul 2022 09:11:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:organization:in-reply-to :content-transfer-encoding; bh=u/9Avj+qTTxie6ALcW9gh+6LmqKI2mypPNKFdm4Ju+o=; b=N9LFP2DOY+Yp1SmFaCogb7KSdHKE8O+N8pkdQM6UhGBn+XP/hwjPhbWrwkFq9gdOYg Z6tMk9NTZRGncU1m1TPpw4J1ME/kuAJhxeZWD0eX8s/3sHuLqFk8kZlvXboZ77KNEmt+ r0WJcqh0CNS66srhFIo8A6weO+YI6+KV7CwAm/1vfxAhTAZh5q8nybU6yw27pCWPiRbG Jc2jhDuWu27rsMwqiy3jqN5BU6nZhJqmKnj1pv7Jwz/eko+zJoKB7PGR4P3jxBXJdYvr be57gHvdJcEiOCAb23ITcANMCF9FYTfnvAjGYPZlabjdhqIVqKlY7uhucYYqm4IjXXbp hD4Q== X-Gm-Message-State: AJIora+JTGUJyFFA6hck5guePyyhxsZWs5QlABQmFOoR+ejHx4NXjeYT WcMV10mL+GJXxoZnt+/PA7/EF8KUtzf77w== X-Google-Smtp-Source: AGRyM1vmtmWyx4alZnGt9Ewrx4kZYlsfQ6MMis3VijSXRSbTY9qd+x+R/hkOD/ldZf4M5uiXylWBlQ== X-Received: by 2002:a1c:e90a:0:b0:3a3:10d1:cece with SMTP id q10-20020a1ce90a000000b003a310d1cecemr15387545wmc.92.1658592678998; Sat, 23 Jul 2022 09:11:18 -0700 (PDT) Received: from ?IPV6:2a01:cb19:8acf:5600:3b0f:2669:24db:51d0? ([2a01:cb19:8acf:5600:3b0f:2669:24db:51d0]) by smtp.gmail.com with ESMTPSA id q12-20020adff50c000000b0021e5757fb4csm6535628wro.88.2022.07.23.09.11.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 23 Jul 2022 09:11:18 -0700 (PDT) Message-ID: Date: Sat, 23 Jul 2022 18:11:16 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Content-Language: en-GB To: Sen Hastings , buildroot@buildroot.org References: <20220722191558.809011-1-sen@phobosdpl.com> From: Arnout Vandecappelle Organization: Essensium/Mind In-Reply-To: <20220722191558.809011-1-sen@phobosdpl.com> X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; h=message-id:date:mime-version:user-agent:subject:content-language:to :references:from:organization:in-reply-to:content-transfer-encoding; bh=u/9Avj+qTTxie6ALcW9gh+6LmqKI2mypPNKFdm4Ju+o=; b=UJ68ht53Ix83ORHJ7YDWhd79Lwlk9to+egtnUL0mjMeHDgxZQ+vqpEu3bslFfMFD/i h/QR8/PKCKvJWUTqfxSW7A4NwZNVnedr5FAfYAah6vKEuF4oFMU9iieBkr0gZGsx5ZVO xbkQi8Xv1JcNnr+B07qmM0dGRnCyoWUFG32noWD+5LTdjhzog0LRr8OdAhp2My7uDZXH NXJbtKVy8waXeKSN7M0TiLJwmFHgEuhEb+uzghfIIf9FOPXqB8kZJvJ2QHVzHa7kyIpf rxt8hNoU+oKeBek3Nqsm7UA6ro2CXeYBD2YyD8U/WnapG9zAnr9cJDUjgIKf6fTBo45M mlWw== X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=mind.be header.i=@mind.be header.a=rsa-sha256 header.s=google header.b=UJ68ht53 Subject: Re: [Buildroot] [PATCH v3] support/scripts/pkg-stats: migrate to CSS grid and inline javascript X-BeenThere: buildroot@buildroot.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: buildroot-bounces@buildroot.org Sender: "buildroot" On 22/07/2022 21:15, Sen Hastings wrote: > This migrates pkg-stats.html from html tables to CSS grid, allowing > the use of newer, simpler javascript that is short enough to be > inlined, instead of relying on externally hosted javascript. > > Javascript sorting function was rewritten from scratch in ~55 lines, > short enough to be inlined directly in the html. > > Tables were redone in CSS grid, but with care taken to mimic existing > "look and feel" of prevous implementation, albeit with slightly > better responsive behavior and default styling characteristics. > > Column labels are now "sticky" and stay stuck to the top of the > viewport as you scroll down the page. > > Also, css was rewritten in fewer lines and table elements were changed > to divs (for grid support). I don't really understand any of this, but I trust you did it correctly. The output looks OK at least. Applied to master, thanks. Regards, Arnout > > Other small misc fixes include quoted hrefs and document language > declarations to make the w3c html validator happy. > > Signed-off-by: Sen Hastings > > --- > v1->v2: > * added new entry in DEVELOPERS as part of same commit > * sent patch with UTF-8 instead of base64 transfer encoding > v2->v3: > * rewrote sortGrid to remove all UTF-8 characters. > * also sortGrid is now slightly cleaner and only uses arrays. > * patch *hopefully* now us-ascii and will *not* be base64 encoded > --- > DEVELOPERS | 3 + > support/scripts/pkg-stats | 405 +++++++++++++++++++++----------------- > 2 files changed, 226 insertions(+), 182 deletions(-) > > diff --git a/DEVELOPERS b/DEVELOPERS > index 5c3c24ff7a..0246f80713 100644 > --- a/DEVELOPERS > +++ b/DEVELOPERS > @@ -2565,6 +2565,9 @@ F: package/libbson/ > F: package/lua-resty-http/ > F: package/mpir/ > > +N: Sen Hastings > +F: support/scripts/pkg-stats > + > N: Sergey Bobrenok > F: package/sdbus-cpp/ > > diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats > index f67be0063f..6dc206d2bc 100755 > --- a/support/scripts/pkg-stats > +++ b/support/scripts/pkg-stats > @@ -718,89 +718,113 @@ def calculate_stats(packages): > > > html_header = """ > + > + > > - > - > + > Statistics of Buildroot packages > + > > > + > + > Results
> >

> @@ -808,13 +832,13 @@ td.cve_ignored { > > > html_footer = """ > - > > + > > """ > > @@ -841,73 +865,87 @@ def boolean_str(b): > > > def dump_html_pkg(f, pkg): > - f.write(" \n") > - f.write(" %s\n" % pkg.path) > - > + f.write( f'
+ class=\"package data {pkg.name}\">{pkg.path}
\n') > # Patch count > - td_class = ["centered"] > + data_field_id = f'patch_count_{pkg.name}' > + div_class = ["centered patch_count data"] > + div_class.append(pkg.name) > if pkg.patch_count == 0: > - td_class.append("nopatches") > + div_class.append("nopatches") > elif pkg.patch_count < 5: > - td_class.append("somepatches") > + div_class.append("somepatches") > else: > - td_class.append("lotsofpatches") > - f.write(" %s\n" % > - (" ".join(td_class), str(pkg.patch_count))) > + div_class.append("lotsofpatches") > + f.write( f'
+ \">{str(pkg.patch_count)}
\n') > > # Infrastructure > + data_field_id = f'infrastructure_{pkg.name}' > infra = infra_str(pkg.infras) > - td_class = ["centered"] > + div_class = ["centered infrastructure data"] > + div_class.append(pkg.name) > if infra == "Unknown": > - td_class.append("wrong") > + div_class.append("wrong") > else: > - td_class.append("correct") > - f.write(" %s\n" % > - (" ".join(td_class), infra_str(pkg.infras))) > + div_class.append("correct") > + f.write( f'
+ \">{infra_str(pkg.infras)}
\n') > > # License > - td_class = ["centered"] > + data_field_id = f'license_{pkg.name}' > + div_class = ["centered license data"] > + div_class.append(pkg.name) > if pkg.is_status_ok('license'): > - td_class.append("correct") > + div_class.append("correct") > else: > - td_class.append("wrong") > - f.write(" %s\n" % > - (" ".join(td_class), boolean_str(pkg.is_status_ok('license')))) > + div_class.append("wrong") > + f.write(f'
+ \">{boolean_str(pkg.is_status_ok("license"))}
\n') > > # License files > - td_class = ["centered"] > + data_field_id = f'license_files_{pkg.name}' > + div_class = ["centered license_files data"] > + div_class.append(pkg.name) > if pkg.is_status_ok('license-files'): > - td_class.append("correct") > + div_class.append("correct") > else: > - td_class.append("wrong") > - f.write(" %s\n" % > - (" ".join(td_class), boolean_str(pkg.is_status_ok('license-files')))) > + div_class.append("wrong") > + f.write(f'
+ \">{boolean_str(pkg.is_status_ok("license-files"))}
\n') > > # Hash > - td_class = ["centered"] > + data_field_id = f'hash_file_{pkg.name}' > + div_class = ["centered hash_file data"] > + div_class.append(pkg.name) > if pkg.is_status_ok('hash'): > - td_class.append("correct") > + div_class.append("correct") > else: > - td_class.append("wrong") > - f.write(" %s\n" % > - (" ".join(td_class), boolean_str(pkg.is_status_ok('hash')))) > + div_class.append("wrong") > + f.write(f'
+ \">{boolean_str(pkg.is_status_ok("hash"))}
\n') > > # Current version > + data_field_id = f'current_version_{pkg.name}' > if len(pkg.current_version) > 20: > current_version = pkg.current_version[:20] + "..." > else: > current_version = pkg.current_version > - f.write(" %s\n" % current_version) > + f.write(f'
+ class=\"centered current_version data {pkg.name}\">{current_version}
\n') > > # Latest version > + data_field_id = f'latest_version_{pkg.name}' > + div_class.append(pkg.name) > + div_class.append("latest_version data") > if pkg.latest_version['status'] == RM_API_STATUS_ERROR: > - td_class.append("version-error") > + div_class.append("version-error") > if pkg.latest_version['version'] is None: > - td_class.append("version-unknown") > + div_class.append("version-unknown") > elif pkg.latest_version['version'] != pkg.current_version: > - td_class.append("version-needs-update") > + div_class.append("version-needs-update") > else: > - td_class.append("version-good") > + div_class.append("version-good") > > if pkg.latest_version['status'] == RM_API_STATUS_ERROR: > latest_version_text = "Error" > @@ -927,74 +965,81 @@ def dump_html_pkg(f, pkg): > else: > latest_version_text += "found by guess" > > - f.write(" %s\n" % > - (" ".join(td_class), latest_version_text)) > + f.write(f'
{latest_version_text}
\n') > > # Warnings > - td_class = ["centered"] > + data_field_id = f'warnings_{pkg.name}' > + div_class = ["centered warnings data"] > + div_class.append(pkg.name) > if pkg.warnings == 0: > - td_class.append("correct") > + div_class.append("correct") > else: > - td_class.append("wrong") > - f.write(" %d\n" % > - (" ".join(td_class), pkg.warnings)) > + div_class.append("wrong") > + f.write(f'
{pkg.warnings}
\n') > > # URL status > - td_class = ["centered"] > + data_field_id = f'upstream_url_{pkg.name}' > + div_class = ["centered upstream_url data"] > + div_class.append(pkg.name) > url_str = pkg.status['url'][1] > if pkg.status['url'][0] in ("error", "warning"): > - td_class.append("missing_url") > + div_class.append("missing_url") > if pkg.status['url'][0] == "error": > - td_class.append("invalid_url") > - url_str = "%s" % (pkg.url, pkg.status['url'][1]) > + div_class.append("invalid_url") > + url_str = "%s" % (pkg.url, pkg.status['url'][1]) > else: > - td_class.append("good_url") > - url_str = "Link" % pkg.url > - f.write(" %s\n" % > - (" ".join(td_class), url_str)) > + div_class.append("good_url") > + url_str = "Link" % pkg.url > + f.write(f'
{url_str}
\n') > > # CVEs > - td_class = ["centered"] > + data_field_id = f'cves_{pkg.name}' > + div_class = ["centered cves data"] > + div_class.append(pkg.name) > if pkg.is_status_ok("cve"): > - td_class.append("cve-ok") > + div_class.append("cve-ok") > elif pkg.is_status_error("cve"): > - td_class.append("cve-nok") > + div_class.append("cve-nok") > elif pkg.is_status_na("cve") and not pkg.is_actual_package: > - td_class.append("cve-ok") > + div_class.append("cve-ok") > else: > - td_class.append("cve-unknown") > - f.write(" \n" % " ".join(td_class)) > + div_class.append("cve-unknown") > + f.write(f'
\n') > if pkg.is_status_error("cve"): > for cve in pkg.cves: > - f.write(" %s
\n" % (cve, cve)) > + f.write(" %s
\n" % (cve, cve)) > for cve in pkg.unsure_cves: > - f.write(" %s (unsure)
\n" % (cve, cve)) > + f.write(" %s (unsure)
\n" % (cve, cve)) > elif pkg.is_status_na("cve"): > f.write(" %s" % pkg.status['cve'][1]) > else: > f.write(" N/A\n") > - f.write(" \n") > + f.write("
\n") > > # CVEs Ignored > - td_class = ["centered"] > + data_field_id = f'ignored_cves_{pkg.name}' > + div_class = ["centered data ignored_cves"] > + div_class.append(pkg.name) > if pkg.ignored_cves: > - td_class.append("cve_ignored") > - f.write(" \n" % " ".join(td_class)) > + div_class.append("cve_ignored") > + f.write(f'
\n') > for ignored_cve in pkg.ignored_cves: > - f.write(" %s
\n" % (ignored_cve, ignored_cve)) > - f.write(" \n") > + f.write(" %s
\n" % (ignored_cve, ignored_cve)) > + f.write("
\n") > > # CPE ID > - td_class = ["left"] > + data_field_id = f'cpe_id_{pkg.name}' > + div_class = ["left cpe_id data"] > + div_class.append(pkg.name) > if pkg.is_status_ok("cpe"): > - td_class.append("cpe-ok") > + div_class.append("cpe-ok") > elif pkg.is_status_error("cpe"): > - td_class.append("cpe-nok") > + div_class.append("cpe-nok") > elif pkg.is_status_na("cpe") and not pkg.is_actual_package: > - td_class.append("cpe-ok") > + div_class.append("cpe-ok") > else: > - td_class.append("cpe-unknown") > - f.write(" \n" % " ".join(td_class)) > + div_class.append("cpe-unknown") > + f.write(f'
\n') > if pkg.cpeid: > f.write(" %s\n" % pkg.cpeid) > if not pkg.is_status_ok("cpe"): > @@ -1008,79 +1053,75 @@ def dump_html_pkg(f, pkg): > else: > f.write(" %s\n" % pkg.status['cpe'][1]) > > - f.write(" \n") > - > - f.write(" \n") > + f.write("
\n") > > > def dump_html_all_pkgs(f, packages): > f.write(""" > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > - > +
> +
Package
> +
Patch count
> +
Infrastructure
> +
License
> +
License files
> +
Hash file
> +
Current version
> +
Latest version
> +
Warnings
> +
Upstream URL
> +
CVEs
> +
CVEs Ignored
> +
CPE ID
> """) > for pkg in sorted(packages): > dump_html_pkg(f, pkg) > - f.write("
PackagePatch countInfrastructureLicenseLicense filesHash fileCurrent versionLatest versionWarningsUpstream URLCVEsCVEs IgnoredCPE ID
") > + f.write("") > > > def dump_html_stats(f, stats): > f.write("\n") > - f.write("\n") > + f.write("
\n") > infras = [infra[6:] for infra in stats.keys() if infra.startswith("infra-")] > for infra in infras: > - f.write("
\n" % > + f.write("
Packages using the %s infrastructure
%s
\n" % > (infra, stats["infra-%s" % infra])) > - f.write(" \n" % > + f.write("
Packages having license information
%s
\n" % > stats["license"]) > - f.write(" \n" % > + f.write("
Packages not having license information
%s
\n" % > stats["no-license"]) > - f.write(" \n" % > + f.write("
Packages having license files information
%s
\n" % > stats["license-files"]) > - f.write(" \n" % > + f.write("
Packages not having license files information
%s
\n" % > stats["no-license-files"]) > - f.write(" \n" % > + f.write("
Packages having a hash file
%s
\n" % > stats["hash"]) > - f.write(" \n" % > + f.write("
Packages not having a hash file
%s
\n" % > stats["no-hash"]) > - f.write(" \n" % > + f.write("
Total number of patches
%s
\n" % > stats["patches"]) > - f.write("\n" % > + f.write("
Packages having a mapping on release-monitoring.org
%s
\n" % > stats["rmo-mapping"]) > - f.write("\n" % > + f.write("
Packages lacking a mapping on release-monitoring.org
%s
\n" % > stats["rmo-no-mapping"]) > - f.write("\n" % > + f.write("
Packages that are up-to-date
%s
\n" % > stats["version-uptodate"]) > - f.write("\n" % > + f.write("
Packages that are not up-to-date
%s
\n" % > stats["version-not-uptodate"]) > - f.write("\n" % > + f.write("
Packages with no known upstream version
%s
\n" % > stats["version-unknown"]) > - f.write("\n" % > + f.write("
Packages affected by CVEs
%s
\n" % > stats["pkg-cves"]) > - f.write("\n" % > + f.write("
Total number of CVEs affecting all packages
%s
\n" % > stats["total-cves"]) > - f.write("\n" % > + f.write("
Packages affected by unsure CVEs
%s
\n" % > stats["pkg-unsure-cves"]) > - f.write("\n" % > + f.write("
Total number of unsure CVEs affecting all packages
%s
\n" % > stats["total-unsure-cves"]) > - f.write("\n" % > + f.write("
Packages with CPE ID
%s
\n" % > stats["cpe-id"]) > - f.write("\n" % > + f.write("
Packages without CPE ID
%s
\n" % > stats["no-cpe-id"]) > - f.write("
Packages using the %s infrastructure%s
Packages having license information%s
Packages not having license information%s
Packages having license files information%s
Packages not having license files information%s
Packages having a hash file%s
Packages not having a hash file%s
Total number of patches%s
Packages having a mapping on release-monitoring.org%s
Packages lacking a mapping on release-monitoring.org%s
Packages that are up-to-date%s
Packages that are not up-to-date%s
Packages with no known upstream version%s
Packages affected by CVEs%s
Total number of CVEs affecting all packages%s
Packages affected by unsure CVEs%s
Total number of unsure CVEs affecting all packages%s
Packages with CPE ID%s
Packages without CPE ID%s
\n") > + f.write("\n") > > > def dump_html_gen_info(f, date, commit): _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot