[Buildroot] [PATCH 0/2] support/scripts/pkg-stats: fix row/column ordering issues

[Buildroot] [PATCH 0/2] support/scripts/pkg-stats: fix row/column ordering issues

[Sen Hastings]

This patchset is essentially a revision of [3/4] and republishing of [2/4]
from the previous patchset, but is published as a new patchset.

This is because I'm not sure whether you are supposed to rev patches that
have already been committed.

Sen Hastings (2):
  support/scripts/pkg-stats: fix rows with duplicate package class names
  support/scripts/pkg-stats: fixed sortGrid() performance

 support/scripts/pkg-stats | 169 ++++++++++++++++++++------------------
 1 file changed, 91 insertions(+), 78 deletions(-)

-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] support/scripts/pkg-stats: fix duplicate package class names across rows

[Sen Hastings]

Like all good problems, disparate pieces work together to create
a "synergistically" hairy mess.

The sortGrid() overhaul highlighted a flaw in pkg-stats allowing
for duplicate package class names across rows.

As an example,

boot/barebox/barebox.mk and boot/barebox/barebox/barebox.mk
both get the classname ._barebox and so sortGrid() sticks them on
the same line giving a table with a vestigal row sticking out
of the right side like some kind of appendage.

Also I neglected to add a "_" to the current version column's cells
pkgname class so instead of "._pkgname" we had ".pkgname" and so
the cells were not collected properly as part of the row.

These issues explain the formatting weirdness.

package classnames are now ".path_to_package_makefile" without suffix
(.mk) (so ._boot_barebox_barebox and ._boot_barebox_barebox_barebox
instead of ._barebox) in order to guarantee uniqueness.

and what was *accidentally*
class="centered current_version data .barebox" is now
class="centered current_version data ._boot_barebox_barebox"
just like *all* the other cells in the row. :p

Signed-off-by: Sen Hastings <sen@phobosdpl.com>
---
 support/scripts/pkg-stats | 53 ++++++++++++++++++++-------------------
 1 file changed, 27 insertions(+), 26 deletions(-)

diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index 9c37506cd4..d3ec65edbb 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -885,12 +885,13 @@ def boolean_str(b):
 
 
 def dump_html_pkg(f, pkg):
-    f.write(f'<div id=\"package__{pkg.name}\" \
-        class=\"package data _{pkg.name}\">{pkg.path}</div>\n')
+    pkg_css_class = pkg.path.replace("/", "_")[:-3]
+    f.write(f'<div id=\"package__{pkg_css_class}\" \
+        class=\"package data _{pkg_css_class}\">{pkg.path}</div>\n')
     # Patch count
-    data_field_id = f'patch_count__{pkg.name}'
+    data_field_id = f'patch_count__{pkg_css_class}'
     div_class = ["centered patch_count data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.patch_count == 0:
         div_class.append("nopatches")
     elif pkg.patch_count < 5:
@@ -901,10 +902,10 @@ def dump_html_pkg(f, pkg):
         \">{str(pkg.patch_count)}</div>\n')
 
     # Infrastructure
-    data_field_id = f'infrastructure__{pkg.name}'
+    data_field_id = f'infrastructure__{pkg_css_class}'
     infra = infra_str(pkg.infras)
     div_class = ["centered infrastructure data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if infra == "Unknown":
         div_class.append("wrong")
     else:
@@ -913,9 +914,9 @@ def dump_html_pkg(f, pkg):
         \">{infra_str(pkg.infras)}</div>\n')
 
     # License
-    data_field_id = f'license__{pkg.name}'
+    data_field_id = f'license__{pkg_css_class}'
     div_class = ["centered license data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.is_status_ok('license'):
         div_class.append("correct")
     else:
@@ -924,9 +925,9 @@ def dump_html_pkg(f, pkg):
         \">{boolean_str(pkg.is_status_ok("license"))}</div>\n')
 
     # License files
-    data_field_id = f'license_files__{pkg.name}'
+    data_field_id = f'license_files__{pkg_css_class}'
     div_class = ["centered license_files data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.is_status_ok('license-files'):
         div_class.append("correct")
     else:
@@ -935,9 +936,9 @@ def dump_html_pkg(f, pkg):
         \">{boolean_str(pkg.is_status_ok("license-files"))}</div>\n')
 
     # Hash
-    data_field_id = f'hash_file__{pkg.name}'
+    data_field_id = f'hash_file__{pkg_css_class}'
     div_class = ["centered hash_file data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.is_status_ok('hash'):
         div_class.append("correct")
     else:
@@ -946,17 +947,17 @@ def dump_html_pkg(f, pkg):
         \">{boolean_str(pkg.is_status_ok("hash"))}</div>\n')
 
     # Current version
-    data_field_id = f'current_version__{pkg.name}'
+    data_field_id = f'current_version__{pkg_css_class}'
     if len(pkg.current_version) > 20:
         current_version = pkg.current_version[:20] + "..."
     else:
         current_version = pkg.current_version
     f.write(f'  <div id=\"{data_field_id}\" \
-        class=\"centered current_version data {pkg.name}\">{current_version}</div>\n')
+        class=\"centered current_version data _{pkg_css_class}\">{current_version}</div>\n')
 
     # Latest version
-    data_field_id = f'latest_version__{pkg.name}'
-    div_class.append(f'_{pkg.name}')
+    data_field_id = f'latest_version__{pkg_css_class}'
+    div_class.append(f'_{pkg_css_class}')
     div_class.append("latest_version data")
     if pkg.latest_version['status'] == RM_API_STATUS_ERROR:
         div_class.append("version-error")
@@ -988,9 +989,9 @@ def dump_html_pkg(f, pkg):
     f.write(f'  <div id=\"{data_field_id}\" class=\"{" ".join(div_class)}\">{latest_version_text}</div>\n')
 
     # Warnings
-    data_field_id = f'warnings__{pkg.name}'
+    data_field_id = f'warnings__{pkg_css_class}'
     div_class = ["centered warnings data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.warnings == 0:
         div_class.append("correct")
     else:
@@ -998,9 +999,9 @@ def dump_html_pkg(f, pkg):
     f.write(f'  <div id=\"{data_field_id}\" class=\"{" ".join(div_class)}\">{pkg.warnings}</div>\n')
 
     # URL status
-    data_field_id = f'upstream_url__{pkg.name}'
+    data_field_id = f'upstream_url__{pkg_css_class}'
     div_class = ["centered upstream_url data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     url_str = pkg.status['url'][1]
     if pkg.status['url'][0] in ("error", "warning"):
         div_class.append("missing_url")
@@ -1013,9 +1014,9 @@ def dump_html_pkg(f, pkg):
     f.write(f'  <div id=\"{data_field_id}\" class=\"{" ".join(div_class)}\">{url_str}</div>\n')
 
     # CVEs
-    data_field_id = f'cves__{pkg.name}'
+    data_field_id = f'cves__{pkg_css_class}'
     div_class = ["centered cves data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.is_status_ok("cve"):
         div_class.append("cve-ok")
     elif pkg.is_status_error("cve"):
@@ -1037,9 +1038,9 @@ def dump_html_pkg(f, pkg):
     f.write("  </div>\n")
 
     # CVEs Ignored
-    data_field_id = f'ignored_cves__{pkg.name}'
+    data_field_id = f'ignored_cves__{pkg_css_class}'
     div_class = ["centered data ignored_cves"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.ignored_cves:
         div_class.append("cve_ignored")
     f.write(f'  <div id=\"{data_field_id}\" class=\"{" ".join(div_class)}\">\n')
@@ -1048,9 +1049,9 @@ def dump_html_pkg(f, pkg):
     f.write("  </div>\n")
 
     # CPE ID
-    data_field_id = f'cpe_id__{pkg.name}'
+    data_field_id = f'cpe_id__{pkg_css_class}'
     div_class = ["left cpe_id data"]
-    div_class.append(f'_{pkg.name}')
+    div_class.append(f'_{pkg_css_class}')
     if pkg.is_status_ok("cpe"):
         div_class.append("cpe-ok")
     elif pkg.is_status_error("cpe"):
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] support/scripts/pkg-stats: fixed sortGrid() performance

[Sen Hastings]

sortGrid() has been rewritten to dynamically generate stylesheets with
explicit grid-row properties to re-order the rows, instead of removing
and reinserting the cells.
Performance *should* now be comperable to sorttable.js.

Signed-off-by: Sen Hastings <sen@phobosdpl.com>
---
 support/scripts/pkg-stats | 116 +++++++++++++++++++++-----------------
 1 file changed, 64 insertions(+), 52 deletions(-)

diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index d3ec65edbb..d795131cef 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -744,60 +744,72 @@ html_header = """
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <script>
+const triangleUp = String.fromCodePoint(32, 9652);
+const triangleDown = String.fromCodePoint(32, 9662);
+var lastColumnName = false;
+
 function sortGrid(sortLabel){
+	let i = 0;
 	let pkgSortArray = [], sortedPkgArray = [], pkgStringSortArray = [], pkgNumSortArray = [];
-	let columnValues = Array.from(document.getElementsByClassName(sortLabel));
-
-	columnValues.shift();
-	columnValues.forEach((listing) => {
-		let sortArr = [];
-		sortArr[0] = listing.id.replace(sortLabel+"_", "");
-		if (!listing.innerText){
-			sortArr[1] = -1;
-		} else {
-			sortArr[1] = listing.innerText;
-		};
-		pkgSortArray.push(sortArr);
-	})
-	pkgSortArray.forEach((listing) => {
-		if ( isNaN(parseInt(listing[1], 10)) ){
-			pkgStringSortArray.push(listing);
-		} else {
-			listing[1] = parseFloat(listing[1]);
-			pkgNumSortArray.push(listing);
-		};
-	})
-	sortedStringPkgArray = pkgStringSortArray.sort(function(a, b) {
-		const nameA = a[1].toUpperCase(); // ignore upper and lowercase
-		const nameB = b[1].toUpperCase(); // ignore upper and lowercase
-		if (nameA < nameB) { return -1;	};
-		if (nameA > nameB) { return 1; };
-		return 0;   // names must be equal
-	});
-	sortedNumPkgArray = pkgNumSortArray.sort(function(a, b) {
-		return a[1] - b[1];
-	});
-
-	let triangleUp = String.fromCodePoint(32, 9652);
-	let triangleDown = String.fromCodePoint(32, 9662);
-	let columnName = document.getElementById(sortLabel);
-
-	if (columnName.lastElementChild.innerText == triangleDown) {
-		columnName.lastElementChild.innerText = triangleUp;
-		sortedStringPkgArray.reverse();
-		sortedNumPkgArray.reverse();
-		sortedPkgArray = sortedNumPkgArray.concat(sortedStringPkgArray)
-	} else {
-		columnName.lastElementChild.innerText = triangleDown;
-		sortedPkgArray = sortedStringPkgArray.concat(sortedNumPkgArray)
-	}
-
-	sortedPkgArray.forEach((listing) => {
-		let row = Array.from(document.getElementsByClassName(listing[0]));
-		let packageGrid = document.getElementById("package-grid");
-		row.forEach((element) => { packageGrid.append(element)});
-	})
-}
+	const columnValues = Array.from(document.getElementsByClassName(sortLabel));
+	const columnName = document.getElementById(sortLabel);
+	let lastStyle = document.getElementById("sort-css");
+
+	if (lastStyle){
+		lastStyle.disable = true;
+		lastStyle.remove();
+	};
+	const styleElement = document.createElement('style');
+	styleElement.id = "sort-css";
+	document.head.appendChild(styleElement);
+	const styleSheet = styleElement.sheet;
+
+        columnValues.shift();
+        columnValues.forEach((listing) => {
+                let sortArr = [];
+                sortArr[0] = listing.id.replace(sortLabel+"_", "");
+                if (!listing.innerText){
+                        sortArr[1] = -1;
+                } else {
+                        sortArr[1] = listing.innerText;
+                };
+                pkgSortArray.push(sortArr);
+        });
+        pkgSortArray.forEach((listing) => {
+                if ( isNaN(parseInt(listing[1], 10)) ){
+                        pkgStringSortArray.push(listing);
+                } else {
+                        listing[1] = parseFloat(listing[1]);
+                        pkgNumSortArray.push(listing);
+                };
+        });
+
+        let sortedStringPkgArray = pkgStringSortArray.sort((a, b) => {
+                if (a[1].toUpperCase() < b[1].toUpperCase()) { return -1; };
+                if (a[1].toUpperCase() > b[1].toUpperCase()) { return 1; };
+                return 0;
+        });
+        let sortedNumPkgArray = pkgNumSortArray.sort((a, b) => a[1] - b[1]);
+
+        if (columnName.lastElementChild.innerText == triangleDown) {
+                columnName.lastElementChild.innerText = triangleUp;
+                sortedStringPkgArray.reverse();
+                sortedNumPkgArray.reverse();
+                sortedPkgArray = sortedNumPkgArray.concat(sortedStringPkgArray);
+        } else {
+                columnName.lastElementChild.innerText = triangleDown;
+                sortedPkgArray = sortedStringPkgArray.concat(sortedNumPkgArray);
+        };
+
+	if (lastColumnName && lastColumnName != columnName){lastColumnName.lastElementChild.innerText = ""};
+	lastColumnName = columnName;
+	sortedPkgArray.unshift(["label"]);
+        sortedPkgArray.forEach((listing) => {
+		i++;
+		let rule = "." + listing[0] + " { grid-row: " + i + "; }";
+		styleSheet.insertRule(rule);
+        });
+};
 </script>
 
 <style>
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2] support/scripts/pkg-stats: fix row/column ordering issues

[Thomas Petazzoni via buildroot]

Hello Sen,

On Wed, 27 Jul 2022 21:14:42 -0500
Sen Hastings <sen@phobosdpl.com> wrote:

> This is because I'm not sure whether you are supposed to rev patches that
> have already been committed.

Nope, you are expected to do exactly what you did: rebase your
remaining contributions on the latest master, so that you resend only
the remaining of the work that has not been merged so far.

> Sen Hastings (2):
>   support/scripts/pkg-stats: fix rows with duplicate package class names
>   support/scripts/pkg-stats: fixed sortGrid() performance

Both applied, thanks a lot for the quick turnaround!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot