From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D07A1287513 for ; Mon, 22 Sep 2025 09:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758533821; cv=none; b=SB9hvTpx1rdCkRVB8a5H1jFouYSze0573c2MTUX2fbgAkdsHo4K44+2pa4fXXxC7+Lk6NK6gmcdhqgb+hnb06EZH2gAFPWgeGrFJVw2SgrWDfpU2Ng30KrMQNOh6GI5AcFtnutRC+ulZZB2Mt06s47XFpnQGSnSo5CPvRn4F7yQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758533821; c=relaxed/simple; bh=mttlSoCnmHOGIXFsnrMaFobSgpdAveh+Xmr2h7OjHg8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MhYr5MGs2zmrKQTrHArr8PKQ4z36BvOB/mdk7i+9z16z4A61RUpGIgda6DWpD4Oy8wdZIaNkgNpO7WXrKkUk1lHUm09M5sWhNthrP5PAIqZZ9125A3fGw2UkVP9BSd0yw8L+S+0NpknwamNGiJnl0DtvkwOzm8lm1AVkk3114eo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=VBXQrK11; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="VBXQrK11" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description; bh=FzIbiosxAVR/GVVEn8JqkU6hgcFwK5AnI0EJL1PEL8E=; b=VBXQrK11BDbwfi5+ltNFqos2jv dR+vejwclE9zkErcyDhTduw+yfO+inpPz6Z+PAAx1rzsKGwW3rqOe2gRvvcyu8sVc3XmQ0ZVm0YrY aVASlZ4qam8OJ+fU/z/hOK+EpFpJQqwQbqeLhVrgLef4KCnLPyiK7LbCOPjULwFHFKtRfEPN1w3at 3nwH2v3TLnBPOqJwkaZWV8u8ERQLSJD7CxlyLryrmW/Upao8I6xxt3KdhXwjP0NX2sk6/AkW30fN6 S353vMmJLclvPQWenUavY4f6hLvfIwS2rApxFZ0pwTGuuKY3mHPYQX9zFieDvh3ANWFTY0xgiwmaY FaTX9qjg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1v0cyg-00000009uEf-0g7y; Mon, 22 Sep 2025 09:36:58 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH 13/13] pynfs: add visualization support for test results Date: Mon, 22 Sep 2025 02:36:55 -0700 Message-ID: <20250922093656.2361016-14-mcgrof@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250922093656.2361016-1-mcgrof@kernel.org> References: <20250922093656.2361016-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Add 'make pynfs-visualize' target to generate comprehensive HTML reports with PNG charts for pynfs test results. This makes it easy to understand test outcomes at a glance and share results. Features: - HTML report with test summaries, statistics, and detailed results - PNG charts showing test distribution (pie and bar charts) - Comparison charts when multiple NFS versions are tested - Automatic kernel version detection from results directory - Self-contained output directory for easy transfer via scp The visualization script generates: - index.html: Main report with interactive tabs - pynfs-v4_0-results.png: NFS v4.0 test charts - pynfs-v4_1-results.png: NFS v4.1 test charts - pynfs-vblock-results.png: pNFS block layout charts - pynfs-comparison.png: Side-by-side version comparison Usage: make pynfs-visualize # Auto-detect kernel make pynfs-visualize LAST_KERNEL= # Specific kernel Output is generated in: workflows/pynfs/results//html/ Generated-by: Claude AI Signed-off-by: Luis Chamberlain --- scripts/workflows/pynfs/visualize_results.py | 1014 ++++++++++++++++++ workflows/pynfs/Makefile | 17 +- 2 files changed, 1030 insertions(+), 1 deletion(-) create mode 100755 scripts/workflows/pynfs/visualize_results.py diff --git a/scripts/workflows/pynfs/visualize_results.py b/scripts/workflows/pynfs/visualize_results.py new file mode 100755 index 00000000..15b0089b --- /dev/null +++ b/scripts/workflows/pynfs/visualize_results.py @@ -0,0 +1,1014 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +""" +Generate HTML visualization report for pynfs test results with charts and summaries. +Creates both an HTML report and PNG chart files. +""" + +import json +import os +import sys +import argparse +from pathlib import Path +from datetime import datetime +import re + +# Try to import matplotlib for PNG generation +try: + import matplotlib + + matplotlib.use("Agg") # Use non-interactive backend + import matplotlib.pyplot as plt + import matplotlib.patches as mpatches + + MATPLOTLIB_AVAILABLE = True +except ImportError: + MATPLOTLIB_AVAILABLE = False + print("Warning: matplotlib not available, PNG charts will not be generated") + print("Install with: pip3 install matplotlib") + + +def load_json_results(filepath): + """Load and parse a JSON result file.""" + try: + with open(filepath, "r") as f: + return json.load(f) + except Exception as e: + print(f"Error loading {filepath}: {e}") + return None + + +def categorize_tests(testcases): + """Categorize tests by their class/module.""" + categories = {} + for test in testcases: + classname = test.get("classname", "unknown") + if classname not in categories: + categories[classname] = { + "passed": [], + "failed": [], + "skipped": [], + "error": [], + } + + if test.get("skipped"): + categories[classname]["skipped"].append(test) + elif test.get("failure"): + categories[classname]["failed"].append(test) + elif test.get("error"): + categories[classname]["error"].append(test) + else: + categories[classname]["passed"].append(test) + + return categories + + +def generate_png_charts(charts, output_dir): + """Generate PNG charts using matplotlib.""" + if not MATPLOTLIB_AVAILABLE: + return [] + + png_files = [] + + # Set up the style + plt.style.use("seaborn-v0_8-darkgrid") + + for chart in charts: + version = chart["version"] + + # Create figure with subplots + fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 6)) + fig.suptitle( + f'PyNFS {version.upper()} Test Results - Kernel {chart.get("kernel", "Unknown")}', + fontsize=16, + fontweight="bold", + ) + + # Pie chart + sizes = [chart["passed"], chart["failed"], chart["errors"], chart["skipped"]] + labels = ["Passed", "Failed", "Errors", "Skipped"] + colors = ["#48bb78", "#f56565", "#ed8936", "#a0aec0"] + explode = (0.05, 0.1, 0.1, 0) # Explode failed and error slices + + # Only show non-zero values + non_zero_sizes = [] + non_zero_labels = [] + non_zero_colors = [] + non_zero_explode = [] + for i, size in enumerate(sizes): + if size > 0: + non_zero_sizes.append(size) + non_zero_labels.append(f"{labels[i]}: {size}") + non_zero_colors.append(colors[i]) + non_zero_explode.append(explode[i]) + + ax1.pie( + non_zero_sizes, + explode=non_zero_explode, + labels=non_zero_labels, + colors=non_zero_colors, + autopct="%1.1f%%", + startangle=90, + shadow=True, + ) + ax1.set_title("Test Distribution") + + # Bar chart + ax2.bar(labels, sizes, color=colors, edgecolor="black", linewidth=1.5) + ax2.set_ylabel("Number of Tests", fontweight="bold") + ax2.set_title("Test Counts") + ax2.grid(axis="y", alpha=0.3) + + # Add text annotations on bars + for i, (label, value) in enumerate(zip(labels, sizes)): + ax2.text( + i, + value + max(sizes) * 0.01, + str(value), + ha="center", + va="bottom", + fontweight="bold", + ) + + # Add summary statistics + total = chart["total"] + pass_rate = chart["pass_rate"] + fig.text( + 0.5, + 0.02, + f"Total Tests: {total} | Pass Rate: {pass_rate}%", + ha="center", + fontsize=12, + fontweight="bold", + bbox=dict(boxstyle="round", facecolor="wheat", alpha=0.5), + ) + + plt.tight_layout() + + # Save the figure + png_filename = f'pynfs-{version.replace(".", "_")}-results.png' + png_path = output_dir / png_filename + plt.savefig(png_path, dpi=150, bbox_inches="tight") + plt.close() + + png_files.append(png_filename) + print(f" Generated: {png_path}") + + # Generate a summary chart comparing all versions + if len(charts) > 1: + fig, axes = plt.subplots(2, 2, figsize=(14, 10)) + fig.suptitle("PyNFS Test Results Comparison", fontsize=18, fontweight="bold") + + # Prepare data + versions = [c["version"].upper() for c in charts] + passed = [c["passed"] for c in charts] + failed = [c["failed"] for c in charts] + errors = [c["errors"] for c in charts] + skipped = [c["skipped"] for c in charts] + pass_rates = [c["pass_rate"] for c in charts] + + x = range(len(versions)) + width = 0.2 + + # Grouped bar chart + ax = axes[0, 0] + ax.bar( + [i - width * 1.5 for i in x], passed, width, label="Passed", color="#48bb78" + ) + ax.bar( + [i - width * 0.5 for i in x], failed, width, label="Failed", color="#f56565" + ) + ax.bar( + [i + width * 0.5 for i in x], errors, width, label="Errors", color="#ed8936" + ) + ax.bar( + [i + width * 1.5 for i in x], + skipped, + width, + label="Skipped", + color="#a0aec0", + ) + ax.set_xlabel("Version") + ax.set_ylabel("Number of Tests") + ax.set_title("Test Results by Version") + ax.set_xticks(x) + ax.set_xticklabels(versions) + ax.legend() + ax.grid(axis="y", alpha=0.3) + + # Pass rate comparison + ax = axes[0, 1] + bars = ax.bar( + versions, + pass_rates, + color=[ + "#48bb78" if p >= 90 else "#ed8936" if p >= 70 else "#f56565" + for p in pass_rates + ], + ) + ax.set_ylabel("Pass Rate (%)") + ax.set_title("Pass Rate Comparison") + ax.set_ylim(0, 105) + ax.grid(axis="y", alpha=0.3) + + # Add value labels on bars + for bar, rate in zip(bars, pass_rates): + height = bar.get_height() + ax.text( + bar.get_x() + bar.get_width() / 2.0, + height + 1, + f"{rate:.1f}%", + ha="center", + va="bottom", + fontweight="bold", + ) + + # Stacked bar chart + ax = axes[1, 0] + ax.bar(versions, passed, label="Passed", color="#48bb78") + ax.bar(versions, failed, bottom=passed, label="Failed", color="#f56565") + ax.bar( + versions, + errors, + bottom=[p + f for p, f in zip(passed, failed)], + label="Errors", + color="#ed8936", + ) + ax.bar( + versions, + skipped, + bottom=[p + f + e for p, f, e in zip(passed, failed, errors)], + label="Skipped", + color="#a0aec0", + ) + ax.set_ylabel("Number of Tests") + ax.set_title("Stacked Test Results") + ax.legend() + ax.grid(axis="y", alpha=0.3) + + # Summary table + ax = axes[1, 1] + ax.axis("tight") + ax.axis("off") + + table_data = [ + ["Version", "Total", "Passed", "Failed", "Errors", "Skipped", "Pass Rate"] + ] + for c in charts: + table_data.append( + [ + c["version"].upper(), + str(c["total"]), + str(c["passed"]), + str(c["failed"]), + str(c["errors"]), + str(c["skipped"]), + f"{c['pass_rate']}%", + ] + ) + + table = ax.table(cellText=table_data, loc="center", cellLoc="center") + table.auto_set_font_size(False) + table.set_fontsize(10) + table.scale(1.2, 1.5) + + # Style the header row + for i in range(7): + table[(0, i)].set_facecolor("#4a5568") + table[(0, i)].set_text_props(weight="bold", color="white") + + # Color code the cells + for i in range(1, len(table_data)): + # Pass rate column + pass_rate = float(table_data[i][6].strip("%")) + if pass_rate >= 90: + table[(i, 6)].set_facecolor("#c6f6d5") + elif pass_rate >= 70: + table[(i, 6)].set_facecolor("#feebc8") + else: + table[(i, 6)].set_facecolor("#fed7d7") + + plt.tight_layout() + + # Save comparison chart + comparison_path = output_dir / "pynfs-comparison.png" + plt.savefig(comparison_path, dpi=150, bbox_inches="tight") + plt.close() + + png_files.append("pynfs-comparison.png") + print(f" Generated: {comparison_path}") + + return png_files + + +def generate_chart_data(results, kernel_version): + """Generate data for charts.""" + charts = [] + for version, data in results.items(): + if not data: + continue + + total = data.get("tests", 0) + passed = ( + total + - data.get("failures", 0) + - data.get("errors", 0) + - data.get("skipped", 0) + ) + failed = data.get("failures", 0) + errors = data.get("errors", 0) + skipped = data.get("skipped", 0) + + charts.append( + { + "version": version, + "kernel": kernel_version, + "total": total, + "passed": passed, + "failed": failed, + "errors": errors, + "skipped": skipped, + "pass_rate": round((passed / total * 100) if total > 0 else 0, 2), + } + ) + + return charts + + +def generate_html_report(results_dir, kernel_version): + """Generate the main HTML report with embedded charts and links to PNG files.""" + results = {} + + # Load all JSON files for this kernel version + for json_file in Path(results_dir).glob(f"{kernel_version}*.json"): + # Extract version from filename (e.g., v4.0, v4.1, vblock) + match = re.search(r"-v(4\.[01]|block)\.json$", str(json_file)) + if match: + version = "v" + match.group(1) + results[version] = load_json_results(json_file) + + if not results: + print(f"No results found for kernel {kernel_version}") + return None, [] + + # Generate chart data + charts = generate_chart_data(results, kernel_version) + + # Create output directory for HTML and PNGs + output_dir = Path(results_dir) / "html" + output_dir.mkdir(parents=True, exist_ok=True) + + # Generate PNG charts + png_files = generate_png_charts(charts, output_dir) + + # Generate detailed test results + detailed_results = {} + for version, data in results.items(): + if data and "testcase" in data: + detailed_results[version] = categorize_tests(data["testcase"]) + + # Create HTML content + html_content = f""" + + + + + PyNFS Test Results - {kernel_version} + + + + +
+
+

๐Ÿงช PyNFS Test Results

+
Kernel Version: {kernel_version}
+
Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
+""" + + # Add PNG download links if any were generated + if png_files: + html_content += """ +
+

๐Ÿ“Š Download Charts:

+""" + for png_file in png_files: + html_content += ( + f' {png_file}\n' + ) + html_content += """
+""" + + html_content += """ +
+ +
+""" + + # Add summary cards for each version + for chart in charts: + version = chart["version"] + png_file = f'pynfs-{version.replace(".", "_")}-results.png' + + html_content += f""" +
+
NFS {version.upper()} Results
+
+
+
Total Tests
+
{chart['total']}
+
+
+
Pass Rate
+
{chart['pass_rate']}%
+
+
+
Passed
+
{chart['passed']}
+
+
+
Failed
+
{chart['failed']}
+
+
+
Errors
+
{chart['errors']}
+
+
+
Skipped
+
{chart['skipped']}
+
+
+
+
+
+""" + + # Add PNG preview if available + if png_file in png_files: + html_content += f""" +
+ + {version.upper()} Results Chart + +
+""" + else: + # Fallback to JavaScript chart + html_content += f""" +
+ +
+""" + + html_content += """ +
+""" + + html_content += """ +
+""" + + # Add comparison chart preview if available + if "pynfs-comparison.png" in png_files: + html_content += """ +
+

Test Results Comparison

+
+ + PyNFS Comparison Chart + +
+
+""" + + html_content += """ +
+

Detailed Test Results

+
+""" + + # Add tabs for each version + first = True + for version in detailed_results.keys(): + active = "active" if first else "" + html_content += f""" + +""" + first = False + + html_content += """ +
+""" + + # Add tab content for each version + first = True + for version, categories in detailed_results.items(): + active = "active" if first else "" + html_content += f""" +
+""" + + # Sort categories by name + for category_name in sorted(categories.keys()): + category = categories[category_name] + total_in_category = ( + len(category["passed"]) + + len(category["failed"]) + + len(category["skipped"]) + + len(category["error"]) + ) + + if total_in_category == 0: + continue + + html_content += f""" +
+
+ {category_name} ({total_in_category} tests) +
+
+""" + + # Add passed tests + for test in sorted(category["passed"], key=lambda x: x.get("name", "")): + html_content += f""" +
+ {test.get('name', 'Unknown')} + {test.get('code', '')} +
+""" + + # Add failed tests + for test in sorted(category["failed"], key=lambda x: x.get("name", "")): + html_content += f""" +
+ {test.get('name', 'Unknown')} + {test.get('code', '')} +
+""" + + # Add error tests + for test in sorted(category["error"], key=lambda x: x.get("name", "")): + html_content += f""" +
+ {test.get('name', 'Unknown')} + {test.get('code', '')} +
+""" + + # Add skipped tests (collapsed by default) + if category["skipped"]: + html_content += f""" +
+ + Skipped Tests ({len(category['skipped'])}) + +
+""" + for test in sorted( + category["skipped"], key=lambda x: x.get("name", "") + ): + html_content += f""" +
+ {test.get('name', 'Unknown')} + {test.get('code', '')} +
+""" + html_content += """ +
+
+""" + + html_content += """ +
+
+""" + + html_content += """ +
+""" + first = False + + html_content += """ +
+ +
+ Generated by kdevops pynfs-visualize | ๐Ÿค– Generated with Claude Code +
+
+ + + + +""" + + return html_content, png_files + + +def main(): + parser = argparse.ArgumentParser( + description="Generate HTML visualization for pynfs results" + ) + parser.add_argument("results_dir", help="Path to results directory") + parser.add_argument("kernel_version", help="Kernel version string") + parser.add_argument("--output", "-o", help="Output HTML file path") + + args = parser.parse_args() + + # Generate the HTML report and PNG charts + html_content, png_files = generate_html_report( + args.results_dir, args.kernel_version + ) + + if not html_content: + sys.exit(1) + + # Determine output path + if args.output: + output_path = Path(args.output) + output_path.parent.mkdir(parents=True, exist_ok=True) + else: + output_dir = Path(args.results_dir) / "html" + output_dir.mkdir(parents=True, exist_ok=True) + output_path = output_dir / "index.html" + + # Write the HTML file + with open(output_path, "w") as f: + f.write(html_content) + + print(f"โœ… HTML report generated: {output_path}") + + if png_files: + print(f"โœ… Generated {len(png_files)} PNG charts in: {output_path.parent}") + elif MATPLOTLIB_AVAILABLE: + print("โš ๏ธ No PNG charts generated (no data)") + else: + print("โš ๏ธ PNG charts not generated (matplotlib not installed)") + print(" Install with: pip3 install matplotlib") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/workflows/pynfs/Makefile b/workflows/pynfs/Makefile index e0da0cf5..29b0feac 100644 --- a/workflows/pynfs/Makefile +++ b/workflows/pynfs/Makefile @@ -13,7 +13,7 @@ WORKFLOW_ARGS += $(PYNFS_ARGS) ifndef LAST_KERNEL -LAST_KERNEL := $(shell cat workflows/pynfs/results/last-kernel.txt 2>/dev/null) +LAST_KERNEL := $(shell cat workflows/pynfs/results/last-kernel.txt 2>/dev/null || ls -1dt workflows/pynfs/results/*/ 2>/dev/null | grep -v "last-run" | head -1 | xargs -r basename) endif ifeq ($(LAST_KERNEL), $(shell cat workflows/pynfs/results/last-kernel.txt 2>/dev/null)) @@ -76,10 +76,25 @@ pynfs-show-results: | xargs $(XARGS_ARGS) \ | sed '$${/^$$/d;}' +pynfs-visualize: + $(Q)if [ ! -d "workflows/pynfs/results/$(LAST_KERNEL)" ]; then \ + echo "Error: No results found for kernel $(LAST_KERNEL)"; \ + echo "Available kernels:"; \ + ls -1 workflows/pynfs/results/ | grep -v last; \ + exit 1; \ + fi + $(Q)echo "Generating HTML visualization for kernel $(LAST_KERNEL)..." + $(Q)python3 scripts/workflows/pynfs/visualize_results.py \ + workflows/pynfs/results/$(LAST_KERNEL) \ + $(LAST_KERNEL) \ + --output workflows/pynfs/results/$(LAST_KERNEL)/html/index.html + $(Q)echo "โœ… Visualization complete: workflows/pynfs/results/$(LAST_KERNEL)/html/index.html" + pynfs-help-menu: @echo "pynfs options:" @echo "pynfs - Git clone pynfs, build and install it" @echo "pynfs-{baseline,dev} - Run the pynfs test on baseline or dev hosts and collect results" + @echo "pynfs-visualize - Generate HTML visualization of test results" @echo "" HELP_TARGETS += pynfs-help-menu -- 2.51.0