From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 723112EAB6C for ; Tue, 2 Sep 2025 23:53:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756857241; cv=none; b=QCX4JNvh0Qe7Nr5oOosFkWluJIMUU8i1LQ8qNuXuHtfAuMD/cWYygYbeD8H1zwQFjlIYJHju+Q8XDZOxT2CVacBOgBWmwKLEUoGpWJ2gw8YhnJ9dwfYCfUQU4UIIO7rgP2/6EPEEUyXe1dYSa2Ah5GiMCDzEOWS2jac7ttX8ke4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756857241; c=relaxed/simple; bh=L8VnIbvq0rEi0Xo3UfpC0RaF+S9QdbW+nmtxnKdaAwc=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=MqzVRkeI3WkM1m56k4ASGVdEQkg6Gduvpb2cqPkYgC/kWnQTl5ppLZTESpCqIJ5edsdSBMRretNMyPjdW3Pcf7odFUIO3Rt9GLeKD2qEfexmKF6lypyYrDdtzmGseedoPMr3wxrIca9Ewq2s9DSWnZvDhocJV7yfAJBzN3+sBIw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=JBRJ1DDp; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JBRJ1DDp" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=EEj2I3pKtJA7sUmiM1FGkzW/knjSvTV/lHKmP16jgP4=; b=JBRJ1DDp5wlNxvTH0GXsTosin1 r7RE2+9rg7ZPNeNMHvD9reA9hZrmTsCexUWYd1dZ24UNkgSAWBoOAkGGH7K/4dyMOB4JcLv6G9R0/ maHKeNARjyxQ2K8jXWSMQkTnSbu6DAxl9k3R4+wSuu8UM0xJmQRd8EKn11nnMYlxwWpnLNKOiCDk9 lJeAZUQsq/5qSp0XesVWVn+FL8PB8U3x5b+ZHn2W4raJnYyGF0BJjd3YGylpfWw+w8xb+dr7MxEGd 0YYffMnpu9Wlvr8vCm9GfMFppRZN+b3hxmJCNFbfYgTilMVU2QVBdZX8SmVOo5exGnGc/Fr3jL8DI oVSn04lA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1utap4-00000002gJH-2hWo; Tue, 02 Sep 2025 23:53:58 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: hui81.qi@samsung.com, kundan.kumar@samsung.com, Luis Chamberlain Subject: [PATCH v3 0/3] declared hosts support Date: Tue, 2 Sep 2025 16:53:52 -0700 Message-ID: <20250902235356.639271-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain This v3 now goes tested, and I simplfied the workflow so we just put a stop gap at the data partition on the role instead of adding tons of check on the tasks on the create_data_partition role in different tasks. This demos the DECLARE_HOSTS feature with minio warp, a new simple workflow we can test for S3 with minio. Luis Chamberlain (3): gen_hosts: use kdevops_workflow_name directly for template selection declared_hosts: add support for pre-existing infrastructure minio: add MinIO Warp S3 benchmarking with declared hosts support .gitignore | 2 + Makefile | 7 + defconfigs/minio-warp | 40 + defconfigs/minio-warp-ab | 41 + defconfigs/minio-warp-btrfs | 35 + defconfigs/minio-warp-declared-hosts | 56 ++ defconfigs/minio-warp-multifs | 74 ++ defconfigs/minio-warp-storage | 65 ++ defconfigs/minio-warp-xfs | 53 ++ defconfigs/minio-warp-xfs-16k | 53 ++ defconfigs/minio-warp-xfs-lbs | 65 ++ kconfigs/Kconfig.bringup | 8 + kconfigs/Kconfig.declared_hosts | 71 ++ kconfigs/workflows/Kconfig | 51 ++ kconfigs/workflows/Kconfig.data_partition | 13 +- playbooks/create_data_partition.yml | 2 + playbooks/minio.yml | 53 ++ playbooks/roles/ai_setup/tasks/main.yml | 40 +- .../create_data_partition/defaults/main.yml | 1 + playbooks/roles/devconfig/defaults/main.yml | 2 + playbooks/roles/devconfig/tasks/main.yml | 25 + playbooks/roles/gen_hosts/defaults/main.yml | 1 + playbooks/roles/gen_hosts/tasks/main.yml | 315 ++----- playbooks/roles/gen_hosts/templates/hosts.j2 | 242 +---- .../roles/gen_hosts/templates/workflows/ai.j2 | 99 ++ .../gen_hosts/templates/workflows/blktests.j2 | 58 ++ .../gen_hosts/templates/workflows/cxl.j2 | 7 + .../templates/workflows/declared-hosts.j2 | 239 +++++ .../templates/workflows/fio-tests.j2 | 38 + .../gen_hosts/templates/workflows/fstests.j2 | 72 ++ .../gen_hosts/templates/workflows/gitr.j2 | 41 + .../gen_hosts/templates/workflows/linux.j2 | 110 +++ .../gen_hosts/templates/workflows/ltp.j2 | 41 + .../gen_hosts/templates/workflows/minio.j2 | 173 ++++ .../gen_hosts/templates/workflows/mix.j2 | 62 ++ .../gen_hosts/templates/workflows/mmtests.j2 | 77 ++ .../gen_hosts/templates/workflows/nfstest.j2 | 41 + .../gen_hosts/templates/workflows/pynfs.j2 | 7 + .../templates/workflows/reboot-limit.j2 | 33 + .../templates/workflows/selftests.j2 | 53 ++ .../gen_hosts/templates/workflows/sysbench.j2 | 53 ++ playbooks/roles/gen_nodes/tasks/main.yml | 116 +++ playbooks/roles/minio_destroy/tasks/main.yml | 34 + playbooks/roles/minio_install/tasks/main.yml | 61 ++ playbooks/roles/minio_results/tasks/main.yml | 86 ++ playbooks/roles/minio_setup/defaults/main.yml | 16 + playbooks/roles/minio_setup/tasks/main.yml | 100 ++ .../roles/minio_uninstall/tasks/main.yml | 17 + playbooks/roles/minio_warp_run/tasks/main.yml | 249 +++++ .../templates/warp_config.json.j2 | 14 + workflows/Makefile | 4 + workflows/ai/Makefile | 3 - workflows/blktests/Makefile | 3 - workflows/cxl/Makefile | 2 - workflows/demos/reboot-limit/Kconfig | 5 + workflows/fio-tests/Makefile | 3 - workflows/fstests/Makefile | 3 - workflows/gitr/Makefile | 3 - workflows/linux/Makefile | 1 - workflows/ltp/Makefile | 3 - workflows/minio/Kconfig | 23 + workflows/minio/Kconfig.docker | 66 ++ workflows/minio/Kconfig.storage | 364 ++++++++ workflows/minio/Kconfig.warp | 141 +++ workflows/minio/Makefile | 76 ++ .../minio/scripts/analyze_warp_results.py | 858 ++++++++++++++++++ .../minio/scripts/generate_warp_report.py | 404 +++++++++ .../minio/scripts/run_benchmark_suite.sh | 116 +++ workflows/mmtests/Makefile | 3 - workflows/nfstest/Makefile | 3 - workflows/pynfs/Makefile | 3 - workflows/selftests/Makefile | 3 - workflows/sysbench/Makefile | 3 - 73 files changed, 4751 insertions(+), 554 deletions(-) create mode 100644 defconfigs/minio-warp create mode 100644 defconfigs/minio-warp-ab create mode 100644 defconfigs/minio-warp-btrfs create mode 100644 defconfigs/minio-warp-declared-hosts create mode 100644 defconfigs/minio-warp-multifs create mode 100644 defconfigs/minio-warp-storage create mode 100644 defconfigs/minio-warp-xfs create mode 100644 defconfigs/minio-warp-xfs-16k create mode 100644 defconfigs/minio-warp-xfs-lbs create mode 100644 kconfigs/Kconfig.declared_hosts create mode 100644 playbooks/minio.yml create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/linux.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/minio.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mix.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 create mode 100644 playbooks/roles/minio_destroy/tasks/main.yml create mode 100644 playbooks/roles/minio_install/tasks/main.yml create mode 100644 playbooks/roles/minio_results/tasks/main.yml create mode 100644 playbooks/roles/minio_setup/defaults/main.yml create mode 100644 playbooks/roles/minio_setup/tasks/main.yml create mode 100644 playbooks/roles/minio_uninstall/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/templates/warp_config.json.j2 create mode 100644 workflows/minio/Kconfig create mode 100644 workflows/minio/Kconfig.docker create mode 100644 workflows/minio/Kconfig.storage create mode 100644 workflows/minio/Kconfig.warp create mode 100644 workflows/minio/Makefile create mode 100755 workflows/minio/scripts/analyze_warp_results.py create mode 100755 workflows/minio/scripts/generate_warp_report.py create mode 100755 workflows/minio/scripts/run_benchmark_suite.sh -- 2.50.1