From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFF46749C for ; Thu, 4 Sep 2025 00:15:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756944924; cv=none; b=j7b9WyUeaINlWZM0phuhAUy7wfxAnxEaCwVfXk36wYHiVue/ESZjMqsZax39H3ZDNmjxyPjU4XRsPtz2U21gErAWOiekP1kRKxBTPmFEkLAV7Mrhu804BJ1hRPjXUun6J/HLnSsVO3/HR03+1/j8Epm58pagJIujAb/8ESIH2pg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756944924; c=relaxed/simple; bh=tjIxIfeQ84Eiqhem8xPnTAPdLasnPVF1gZqLJto7JV4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=fBDWF7WGiR84ZO7pkJt9YQ/EXLOL+FboZ3lTTGKtuAHzndQMj8W/V+kT4KBSHJUWQvFDhSTV0JJaiwtAOsQCEyCBtExwUbze4kScEDPimMXoh4rzrPR13jgSY5AeEGFZPfmBclDsDUQt/XGFz1o7jRsYF1kr7rCihI6Bnk145qc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=rcNzcdnu; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="rcNzcdnu" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=cL1OqRwekUUUb8OYKoCX8Z76AVpcrpueJ3Ma6Ys0axI=; b=rcNzcdnuVMM8yXvz7GMGxQn6mu n6B+hsjiod72JJ13Mr5RbKtNeYZPiICjpDY88ABxPgrRY7Z4Ouly3ExLpk/Jb13O36OFdDfycKQjB 86kCq3hvfC1rwd0dOwZ3FhFOSW++IJoDDJIMMlKgdJzM2HxkxPjhMiDlM5D5bwqj5eGzH83FvMk/F rGt1ZXkyY5cA8//DeZ0gn5R3zitmvqXsCEd6xQ4Zez9dN0AQMa3HS3SyRRwLuaRzlHwLbHxUysCHD wgF3SGHwa3DjmOOFBg7iRA04UVnvNmaiIG/AhSosQIoVjKcdYnPkXBELieGJjMAhNB213fXVwbsIK +UgzG+Qg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1utxdK-00000007xkX-0mSP; Thu, 04 Sep 2025 00:15:22 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: hui81.qi@samsung.com, kundan.kumar@samsung.com, Luis Chamberlain Subject: [PATCH v4 0/3] declared hosts support Date: Wed, 3 Sep 2025 17:15:16 -0700 Message-ID: <20250904001521.1897970-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Changes on this v4: - remove KDEVOPS_DECLARED_HOSTS_PYTHON_INTERPRETER as suggested by Daniel Gomez. Luis Chamberlain (3): gen_hosts: use kdevops_workflow_name directly for template selection declared_hosts: add support for pre-existing infrastructure minio: add MinIO Warp S3 benchmarking with declared hosts support .gitignore | 2 + Makefile | 7 + defconfigs/minio-warp | 40 + defconfigs/minio-warp-ab | 41 + defconfigs/minio-warp-btrfs | 35 + defconfigs/minio-warp-declared-hosts | 56 ++ defconfigs/minio-warp-multifs | 74 ++ defconfigs/minio-warp-storage | 65 ++ defconfigs/minio-warp-xfs | 53 ++ defconfigs/minio-warp-xfs-16k | 53 ++ defconfigs/minio-warp-xfs-lbs | 65 ++ kconfigs/Kconfig.bringup | 8 + kconfigs/Kconfig.declared_hosts | 58 ++ kconfigs/workflows/Kconfig | 51 ++ kconfigs/workflows/Kconfig.data_partition | 13 +- playbooks/create_data_partition.yml | 2 + playbooks/minio.yml | 53 ++ playbooks/roles/ai_setup/tasks/main.yml | 40 +- .../create_data_partition/defaults/main.yml | 1 + playbooks/roles/devconfig/defaults/main.yml | 2 + playbooks/roles/devconfig/tasks/main.yml | 25 + playbooks/roles/gen_hosts/defaults/main.yml | 1 + playbooks/roles/gen_hosts/tasks/main.yml | 315 ++----- playbooks/roles/gen_hosts/templates/hosts.j2 | 242 +---- .../roles/gen_hosts/templates/workflows/ai.j2 | 99 ++ .../gen_hosts/templates/workflows/blktests.j2 | 58 ++ .../gen_hosts/templates/workflows/cxl.j2 | 7 + .../templates/workflows/declared-hosts.j2 | 239 +++++ .../templates/workflows/fio-tests.j2 | 38 + .../gen_hosts/templates/workflows/fstests.j2 | 72 ++ .../gen_hosts/templates/workflows/gitr.j2 | 41 + .../gen_hosts/templates/workflows/linux.j2 | 110 +++ .../gen_hosts/templates/workflows/ltp.j2 | 41 + .../gen_hosts/templates/workflows/minio.j2 | 173 ++++ .../gen_hosts/templates/workflows/mix.j2 | 62 ++ .../gen_hosts/templates/workflows/mmtests.j2 | 77 ++ .../gen_hosts/templates/workflows/nfstest.j2 | 41 + .../gen_hosts/templates/workflows/pynfs.j2 | 7 + .../templates/workflows/reboot-limit.j2 | 33 + .../templates/workflows/selftests.j2 | 53 ++ .../gen_hosts/templates/workflows/sysbench.j2 | 53 ++ playbooks/roles/gen_nodes/tasks/main.yml | 116 +++ playbooks/roles/minio_destroy/tasks/main.yml | 34 + playbooks/roles/minio_install/tasks/main.yml | 61 ++ playbooks/roles/minio_results/tasks/main.yml | 86 ++ playbooks/roles/minio_setup/defaults/main.yml | 16 + playbooks/roles/minio_setup/tasks/main.yml | 100 ++ .../roles/minio_uninstall/tasks/main.yml | 17 + playbooks/roles/minio_warp_run/tasks/main.yml | 249 +++++ .../templates/warp_config.json.j2 | 14 + workflows/Makefile | 4 + workflows/ai/Makefile | 3 - workflows/blktests/Makefile | 3 - workflows/cxl/Makefile | 2 - workflows/demos/reboot-limit/Kconfig | 5 + workflows/fio-tests/Makefile | 3 - workflows/fstests/Makefile | 3 - workflows/gitr/Makefile | 3 - workflows/linux/Makefile | 1 - workflows/ltp/Makefile | 3 - workflows/minio/Kconfig | 23 + workflows/minio/Kconfig.docker | 66 ++ workflows/minio/Kconfig.storage | 364 ++++++++ workflows/minio/Kconfig.warp | 141 +++ workflows/minio/Makefile | 76 ++ .../minio/scripts/analyze_warp_results.py | 858 ++++++++++++++++++ .../minio/scripts/generate_warp_report.py | 404 +++++++++ .../minio/scripts/run_benchmark_suite.sh | 116 +++ workflows/mmtests/Makefile | 3 - workflows/nfstest/Makefile | 3 - workflows/pynfs/Makefile | 3 - workflows/selftests/Makefile | 3 - workflows/sysbench/Makefile | 3 - 73 files changed, 4738 insertions(+), 554 deletions(-) create mode 100644 defconfigs/minio-warp create mode 100644 defconfigs/minio-warp-ab create mode 100644 defconfigs/minio-warp-btrfs create mode 100644 defconfigs/minio-warp-declared-hosts create mode 100644 defconfigs/minio-warp-multifs create mode 100644 defconfigs/minio-warp-storage create mode 100644 defconfigs/minio-warp-xfs create mode 100644 defconfigs/minio-warp-xfs-16k create mode 100644 defconfigs/minio-warp-xfs-lbs create mode 100644 kconfigs/Kconfig.declared_hosts create mode 100644 playbooks/minio.yml create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/linux.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/minio.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mix.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 create mode 100644 playbooks/roles/minio_destroy/tasks/main.yml create mode 100644 playbooks/roles/minio_install/tasks/main.yml create mode 100644 playbooks/roles/minio_results/tasks/main.yml create mode 100644 playbooks/roles/minio_setup/defaults/main.yml create mode 100644 playbooks/roles/minio_setup/tasks/main.yml create mode 100644 playbooks/roles/minio_uninstall/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/templates/warp_config.json.j2 create mode 100644 workflows/minio/Kconfig create mode 100644 workflows/minio/Kconfig.docker create mode 100644 workflows/minio/Kconfig.storage create mode 100644 workflows/minio/Kconfig.warp create mode 100644 workflows/minio/Makefile create mode 100755 workflows/minio/scripts/analyze_warp_results.py create mode 100755 workflows/minio/scripts/generate_warp_report.py create mode 100755 workflows/minio/scripts/run_benchmark_suite.sh -- 2.50.1