From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CC1F2853E3 for ; Tue, 12 Aug 2025 00:42:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754959375; cv=none; b=aHb3633prFGNOrPJAzMlOHSSBIroAXj9Bky4sjrvWjHBhGr8MuBH88as+6hGPgH5aexMfAfGpb+CgtgddS0KBXtIusazPLO5XUy1Bwh8Pbjw1hNitPg5Oj+d5pVRYuC/7AcJDAr47W2hjs8mnUapuUEqJf7kRN+IlqOFL1InuVM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754959375; c=relaxed/simple; bh=M3FUdRGuare1tX9768HlSkyasnjR7vkcS9gFED3r0E0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=RPFonK6nUHP2VAtLFKXtQA7tDmqRP57ZZvdKgrINWG0xVRllCebzs/QosVeehFTBvje1q7wNYMnJY4Cb7r4RJ+Zv/zC/jhvOsdSjm8/RKnBiNALm6Ztwt1czaBPXBUsTJt2WfTA8EQshKBUbgi3oTAwR3fK+F4Altkh0htcGH3o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=i0UkvHIV; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i0UkvHIV" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=7C08fE0fGsDb+jUUgR1MGfaQVD59zBziR0vBAAbJ3oI=; b=i0UkvHIVDQpydjoIAHtzD8Sj3q LWqCrVMSHHHXz74EUnQFx09H6k5GZ6HwpOawwC5j4BlV24GaOqkifkJ/WbAhtDSCMvgUjm++/AS49 B4dyIt+4HA7IT2GVqhy2mhvafF5HX9nXDBkRuJ4hY8HN9UYfozqjAQBGaqSd3ucL7CFWDSvje48fg aPr+RPvT9M4x/uCOUCQNggMsrApIeGVfGRHKo+WgVTciKVW1TiDRBOlZmkq+jDVsOKnG98Hlg1/qb zJfW6Jc9UURzDwFAvaVZt6CvxOvL9OHAuFqw9BJtow9kru7VP+S25pUgzwBI1+ntZz4wpCAW+aeq4 sTYpkHpw==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uld6L-00000009T2T-30yc; Tue, 12 Aug 2025 00:42:53 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH 0/2] mmtests-compare and minor fix Date: Mon, 11 Aug 2025 17:42:49 -0700 Message-ID: <20250812004252.2256571-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain While running mmtests I realized the html graphs were not being generaed. So kick that off and also try to enhance graphing. We can fine tune this later with time. The goal is to easily be able to do AB testing with mmtests against different kernels. Also a small fix for gen_hosts is provided due to a patch from Daniel which forgot to update the kdevops_hosts in that file. Luis Chamberlain (2): gen_hosts: fix inventory path for dedicated mmtests workflow mmtests: add AB testing and comparison support .gitignore | 2 + defconfigs/mmtests-ab-testing | 22 + defconfigs/mmtests-ab-testing-thpcompact | 31 + playbooks/mmtests-compare.yml | 5 + playbooks/mmtests.yml | 2 +- playbooks/roles/gen_hosts/tasks/main.yml | 2 +- .../roles/gen_hosts/templates/mmtests.j2 | 10 + playbooks/roles/mmtests/tasks/main.yaml | 58 +- .../roles/mmtests_compare/defaults/main.yml | 6 + .../mmtests_compare/files/apply_patch.sh | 32 + .../files/embed_graphs_in_html.sh | 100 +++ .../mmtests_compare/files/generate_graphs.sh | 148 +++++ .../files/generate_html_with_graphs.sh | 89 +++ .../mmtests_compare/files/run_comparison.sh | 58 ++ .../roles/mmtests_compare/tasks/main.yml | 472 ++++++++++++++ .../templates/comparison_report.html.j2 | 414 ++++++++++++ scripts/generate_mmtests_graphs.py | 598 ++++++++++++++++++ workflows/mmtests/Makefile | 19 +- ...fined-array-reference-when-no-operat.patch | 46 ++ ...act-fix-library-order-in-gcc-command.patch | 33 + 20 files changed, 2139 insertions(+), 8 deletions(-) create mode 100644 defconfigs/mmtests-ab-testing create mode 100644 defconfigs/mmtests-ab-testing-thpcompact create mode 100644 playbooks/mmtests-compare.yml create mode 100644 playbooks/roles/mmtests_compare/defaults/main.yml create mode 100755 playbooks/roles/mmtests_compare/files/apply_patch.sh create mode 100755 playbooks/roles/mmtests_compare/files/embed_graphs_in_html.sh create mode 100755 playbooks/roles/mmtests_compare/files/generate_graphs.sh create mode 100755 playbooks/roles/mmtests_compare/files/generate_html_with_graphs.sh create mode 100755 playbooks/roles/mmtests_compare/files/run_comparison.sh create mode 100644 playbooks/roles/mmtests_compare/tasks/main.yml create mode 100644 playbooks/roles/mmtests_compare/templates/comparison_report.html.j2 create mode 100644 scripts/generate_mmtests_graphs.py create mode 100644 workflows/mmtests/fixes/0001-compare-Fix-undefined-array-reference-when-no-operat.patch create mode 100644 workflows/mmtests/fixes/0002-thpcompact-fix-library-order-in-gcc-command.patch -- 2.47.2