From: Luis Chamberlain <mcgrof@kernel.org>
To: Chuck Lever <cel@kernel.org>, Daniel Gomez <da.gomez@kruces.com>,
kdevops@lists.linux.dev
Cc: hui81.qi@samsung.com, kundan.kumar@samsung.com,
Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v4 2/3] declared_hosts: add support for pre-existing infrastructure
Date: Wed, 3 Sep 2025 17:15:18 -0700 [thread overview]
Message-ID: <20250904001521.1897970-3-mcgrof@kernel.org> (raw)
In-Reply-To: <20250904001521.1897970-1-mcgrof@kernel.org>
This adds support for using pre-existing infrastructure (bare metal servers,
pre-provisioned VMs, cloud instances) that users already have SSH access to,
bypassing the kdevops bringup process.
We borrow the DECLARE_* foo practice from the Linux kernel to ensure the
user will declare the hosts they already have set up with:
make DECLARE_HOSTS="foo bar" defconfig-foo
or
make DECLARE_HOSTS="foo bar" menuconfig
We just skip the data partition setup, at the role level. The user
is encouraged to set DATA_PATH if they want something other than /data/
to be expected to be used. The onus is on them to ensure that the
DATA_PATH works for the user the the host is configured to ssh access
to already.
Currently no workflows are fully supported with declared hosts.
Each workflow requires individual review and testing to ensure proper
operation with pre-existing infrastructure before being enabled.
Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
Makefile | 7 +
kconfigs/Kconfig.bringup | 8 +
kconfigs/Kconfig.declared_hosts | 58 +++++
kconfigs/workflows/Kconfig | 23 ++
kconfigs/workflows/Kconfig.data_partition | 13 +-
playbooks/create_data_partition.yml | 2 +
.../create_data_partition/defaults/main.yml | 1 +
playbooks/roles/devconfig/defaults/main.yml | 2 +
playbooks/roles/devconfig/tasks/main.yml | 25 ++
playbooks/roles/gen_hosts/defaults/main.yml | 1 +
playbooks/roles/gen_hosts/tasks/main.yml | 17 ++
playbooks/roles/gen_hosts/templates/hosts.j2 | 6 +
.../templates/workflows/declared-hosts.j2 | 239 ++++++++++++++++++
13 files changed, 401 insertions(+), 1 deletion(-)
create mode 100644 kconfigs/Kconfig.declared_hosts
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
diff --git a/Makefile b/Makefile
index 38147009..ad744613 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,13 @@ export KDEVOPS_NODES :=
export PYTHONUNBUFFERED=1
export TOPDIR=./
export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+
+# Export CLI override variables for Kconfig to detect them
+# Note: We accept DECLARE_HOSTS but export as DECLARED_HOSTS for consistency
+ifdef DECLARE_HOSTS
+export DECLARED_HOSTS := $(DECLARE_HOSTS)
+endif
+
include scripts/refs.Makefile
KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
diff --git a/kconfigs/Kconfig.bringup b/kconfigs/Kconfig.bringup
index 78d01249..0b1c8805 100644
--- a/kconfigs/Kconfig.bringup
+++ b/kconfigs/Kconfig.bringup
@@ -13,6 +13,13 @@ config CLOUD_INITIALIZED
bool
default $(shell, test -f .cloud.initialized && echo y || echo n) = "y"
+# CLI override detection for DECLARED_HOSTS which should enable SKIP_BRINGUP
+config SKIP_BRINGUP_SET_BY_CLI
+ bool
+ default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS)
+ select SKIP_BRINGUP
+ select KDEVOPS_USE_DECLARED_HOSTS
+
choice
prompt "Node bring up method"
default TERRAFORM if CLOUD_INITIALIZED
@@ -85,6 +92,7 @@ config LIBVIRT
source "kconfigs/Kconfig.guestfs"
source "kconfigs/Kconfig.nixos"
source "terraform/Kconfig"
+source "kconfigs/Kconfig.declared_hosts"
if LIBVIRT
source "kconfigs/Kconfig.libvirt"
endif
diff --git a/kconfigs/Kconfig.declared_hosts b/kconfigs/Kconfig.declared_hosts
new file mode 100644
index 00000000..7522b362
--- /dev/null
+++ b/kconfigs/Kconfig.declared_hosts
@@ -0,0 +1,58 @@
+# Configuration for declared hosts that skip bringup process
+
+config KDEVOPS_USE_DECLARED_HOSTS
+ bool "Use declared hosts (skip bringup process)"
+ select WORKFLOW_INFER_USER_AND_GROUP
+ output yaml
+ help
+ Enable this option to use pre-existing hosts that you have already
+ configured with SSH access. This is useful for:
+
+ * Bare metal systems
+ * Pre-provisioned VMs or cloud instances
+ * Systems managed by other infrastructure tools
+
+ When this option is enabled:
+ - SSH keys will not be generated (assumes you already have access)
+ - Bringup and teardown operations will be skipped
+ - User and group settings will be inferred from the target hosts
+ - You must provide the list of hosts in KDEVOPS_DECLARED_HOSTS
+
+ This option automatically:
+ - Selects WORKFLOW_INFER_USER_AND_GROUP to detect the correct
+ user and group on the target systems
+ - Assumes SSH access is already configured
+
+if KDEVOPS_USE_DECLARED_HOSTS
+
+config KDEVOPS_DECLARED_HOSTS
+ string "List of declared hosts"
+ output yaml
+ default "$(shell, echo ${DECLARED_HOSTS})"
+ default ""
+ help
+ Provide a list of hostnames or IP addresses for the pre-existing
+ systems you want to use. These hosts must already be accessible
+ via SSH with the appropriate keys configured.
+
+ Format: Space or comma-separated list
+ Example: "host1 host2 host3" or "host1,host2,host3"
+
+ These hosts will be used directly without any bringup process.
+ Make sure you have:
+ - SSH access configured
+ - Required packages installed
+ - Appropriate user permissions
+
+config KDEVOPS_DECLARED_HOSTS_GROUP_VARS
+ bool "Apply group variables to declared hosts"
+ default y
+ output yaml
+ help
+ When enabled, kdevops will apply the appropriate group variables
+ to the declared hosts based on the selected workflow.
+
+ This ensures that declared hosts receive the same configuration
+ variables as dynamically provisioned hosts would.
+
+endif # KDEVOPS_USE_DECLARED_HOSTS
diff --git a/kconfigs/workflows/Kconfig b/kconfigs/workflows/Kconfig
index de279b48..30d4fc5e 100644
--- a/kconfigs/workflows/Kconfig
+++ b/kconfigs/workflows/Kconfig
@@ -124,6 +124,7 @@ choice
config KDEVOPS_WORKFLOW_DEDICATE_FSTESTS
bool "fstests"
select KDEVOPS_WORKFLOW_ENABLE_FSTESTS
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
help
This will dedicate your configuration only to fstests.
@@ -139,12 +140,14 @@ config KDEVOPS_WORKFLOW_DEDICATE_FSTESTS
config KDEVOPS_WORKFLOW_DEDICATE_BLKTESTS
bool "blktests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_BLKTESTS
help
This will dedicate your configuration only to blktests.
config KDEVOPS_WORKFLOW_DEDICATE_CXL
bool "cxl"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_CXL
help
This will dedicate your configuration only to cxl work.
@@ -159,6 +162,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_CXL
config KDEVOPS_WORKFLOW_DEDICATE_PYNFS
bool "pynfs"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_PYNFS
help
This will dedicate your configuration only to running pynfs
@@ -166,6 +170,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_PYNFS
config KDEVOPS_WORKFLOW_DEDICATE_SELFTESTS
bool "Linux kernel selftests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_SELFTESTS
help
This will dedicate your configuration only to Linux kernel
@@ -174,6 +179,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_SELFTESTS
config KDEVOPS_WORKFLOW_DEDICATE_GITR
bool "gitr"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_GITR
help
This will dedicate your configuration to running only the
@@ -181,6 +187,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_GITR
config KDEVOPS_WORKFLOW_DEDICATE_LTP
bool "ltp"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_LTP
help
This will dedicate your configuration to running only the
@@ -188,6 +195,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_LTP
config KDEVOPS_WORKFLOW_DEDICATE_NFSTEST
bool "nfstest"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_NFSTEST
help
This will dedicate your configuration to running only the
@@ -195,6 +203,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_NFSTEST
config KDEVOPS_WORKFLOW_DEDICATE_SYSBENCH
bool "sysbench"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_SYSBENCH
help
This will dedicate your configuration to running only the
@@ -202,6 +211,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_SYSBENCH
config KDEVOPS_WORKFLOW_DEDICATE_MMTESTS
bool "mmtests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_MMTESTS
help
This will dedicate your configuration to running only the
@@ -209,6 +219,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_MMTESTS
config KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS
bool "fio-tests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
help
This will dedicate your configuration to running only the
@@ -216,6 +227,7 @@ config KDEVOPS_WORKFLOW_DEDICATE_FIO_TESTS
config KDEVOPS_WORKFLOW_DEDICATE_AI
bool "ai"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_AI
help
This will dedicate your configuration to running only the
@@ -264,6 +276,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_FSTESTS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_BLKTESTS
bool "blktests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_BLKTESTS
help
Select this option if you are doing block layer development and want
@@ -272,6 +285,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_BLKTESTS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_CXL
bool "cxl"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_CXL
help
Select this option if you are doing cxl development and testing.
@@ -285,6 +299,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_CXL
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_PYNFS
bool "pynfs"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_PYNFS
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -294,6 +309,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_PYNFS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_SELFTESTS
bool "Linux kernel selftest"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_SELFTESTS
help
Select this option if you are doing Linux kernel developent and
@@ -301,6 +317,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_SELFTESTS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_GITR
bool "gitr"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_GITR
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -309,6 +326,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_GITR
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_LTP
bool "ltp"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_LTP
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -317,6 +335,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_LTP
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_NFSTEST
bool "nfstest"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_NFSTEST
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -325,6 +344,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_NFSTEST
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_SYSBENCH
bool "sysbench"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_SYSBENCH
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -333,6 +353,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_SYSBENCH
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_MMTESTS
bool "mmtests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_MMTESTS
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -341,6 +362,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_MMTESTS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_FIO_TESTS
bool "fio-tests"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_FIO_TESTS
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
@@ -349,6 +371,7 @@ config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_FIO_TESTS
config KDEVOPS_WORKFLOW_NOT_DEDICATED_ENABLE_AI
bool "ai"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
select KDEVOPS_WORKFLOW_ENABLE_AI
depends on LIBVIRT || TERRAFORM_PRIVATE_NET
help
diff --git a/kconfigs/workflows/Kconfig.data_partition b/kconfigs/workflows/Kconfig.data_partition
index 6b17cddf..847d4dbd 100644
--- a/kconfigs/workflows/Kconfig.data_partition
+++ b/kconfigs/workflows/Kconfig.data_partition
@@ -1,5 +1,10 @@
+config WORKFLOW_DATA_PATH_SET_BY_CLI
+ bool
+ default $(shell, scripts/check-cli-set-var.sh DATA_PATH)
+
config WORKFLOW_DATA_DEVICE_ENABLE_CUSTOM
bool "Enable custom device to use to create the workflow data parition"
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
help
Enable this if you want to override the default data device.
Typically we have enough heuristics with kconfig to get this right
@@ -29,6 +34,7 @@ config WORKFLOW_DATA_DEVICE_CUSTOM
config WORKFLOW_DATA_DEVICE
string
+ depends on !KDEVOPS_USE_DECLARED_HOSTS
default "/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_kdevops0" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_NVME
default "/dev/disk/by-id/virtio-kdevops0" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO
default "/dev/disk/by-id/ata-QEMU_HARDDISK_kdevops0" if LIBVIRT && LIBVIRT_EXTRA_STORAGE_DRIVE_IDE
@@ -37,16 +43,21 @@ config WORKFLOW_DATA_DEVICE
default TERRAFORM_AWS_DATA_VOLUME_DEVICE_FILE_NAME if TERRAFORM_AWS
default TERRAFORM_OCI_DATA_VOLUME_DEVICE_FILE_NAME if TERRAFORM_OCI
default WORKFLOW_DATA_DEVICE_CUSTOM if WORKFLOW_DATA_DEVICE_ENABLE_CUSTOM
+ default "" if KDEVOPS_USE_DECLARED_HOSTS
config WORKFLOW_DATA_PATH
string "Directory path to place data for workflows"
- default "/data"
+ default "$(shell, echo ${DATA_PATH})" if WORKFLOW_DATA_PATH_SET_BY_CLI
+ default "/data" if !WORKFLOW_DATA_PATH_SET_BY_CLI
help
The data workflow is kept then in a location other than your default
user home directory. Use this option to specify the path which we
will use to place your workflow data. This will be the mount point
for the new worfklow data partition created.
+ When using declared hosts, this path should already exist on the
+ target systems.
+
config WORKFLOW_INFER_USER_AND_GROUP
bool "Infer user and group to use for the workflow data partition"
default y
diff --git a/playbooks/create_data_partition.yml b/playbooks/create_data_partition.yml
index b180cf4b..4336bbff 100644
--- a/playbooks/create_data_partition.yml
+++ b/playbooks/create_data_partition.yml
@@ -3,3 +3,5 @@
hosts: baseline:dev
roles:
- role: create_data_partition
+ when:
+ - kdevops_use_declared_hosts|bool
diff --git a/playbooks/roles/create_data_partition/defaults/main.yml b/playbooks/roles/create_data_partition/defaults/main.yml
index 0afbb52e..c2d05bb4 100644
--- a/playbooks/roles/create_data_partition/defaults/main.yml
+++ b/playbooks/roles/create_data_partition/defaults/main.yml
@@ -1,2 +1,3 @@
---
kdevops_enable_terraform: false
+kdevops_use_declared_hosts: false
diff --git a/playbooks/roles/devconfig/defaults/main.yml b/playbooks/roles/devconfig/defaults/main.yml
index 98dce312..c5c06e01 100644
--- a/playbooks/roles/devconfig/defaults/main.yml
+++ b/playbooks/roles/devconfig/defaults/main.yml
@@ -57,3 +57,5 @@ kdevops_enable_guestfs: false
guestfs_copy_sources_from_host_to_guest: false
distro_debian_has_hop1_sources: false
unattended_upgrades_installed: false
+workflow_infer_user_and_group: false
+kdevops_use_declared_hosts: false
diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index fccd1fcf..ae16a698 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -17,6 +17,31 @@
ansible.builtin.setup:
tags: always
+# For declared hosts, infer user and group from the target systems
+- name: Infer user on declared hosts
+ ansible.builtin.command: "whoami"
+ register: declared_host_user
+ when:
+ - kdevops_use_declared_hosts
+ - workflow_infer_user_and_group
+ changed_when: false
+
+- name: Infer group on declared hosts
+ ansible.builtin.command: "id -g -n"
+ register: declared_host_group
+ when:
+ - kdevops_use_declared_hosts
+ - workflow_infer_user_and_group
+ changed_when: false
+
+- name: Set inferred user and group for declared hosts
+ set_fact:
+ data_user: "{{ declared_host_user.stdout | default(data_user) }}"
+ data_group: "{{ declared_host_group.stdout | default(data_group) }}"
+ when:
+ - kdevops_use_declared_hosts
+ - workflow_infer_user_and_group
+
# Update /etc/hostname first so the change gets picked up by the reboot
# that occurs during the distro-specific tasks
diff --git a/playbooks/roles/gen_hosts/defaults/main.yml b/playbooks/roles/gen_hosts/defaults/main.yml
index 4a7515f9..b0b59542 100644
--- a/playbooks/roles/gen_hosts/defaults/main.yml
+++ b/playbooks/roles/gen_hosts/defaults/main.yml
@@ -31,6 +31,7 @@ kdevops_workflow_enable_fio_tests: false
kdevops_workflow_enable_mmtests: false
kdevops_workflow_enable_ai: false
workflows_reboot_limit: false
+kdevops_use_declared_hosts: false
is_fstests: false
fstests_fstyp: "bogus"
diff --git a/playbooks/roles/gen_hosts/tasks/main.yml b/playbooks/roles/gen_hosts/tasks/main.yml
index 518064ed..d44566ad 100644
--- a/playbooks/roles/gen_hosts/tasks/main.yml
+++ b/playbooks/roles/gen_hosts/tasks/main.yml
@@ -10,6 +10,19 @@
skip: true
tags: vars
+- name: Parse declared hosts list when using declared hosts
+ set_fact:
+ kdevops_declared_hosts: >-
+ {%- if kdevops_declared_hosts is string -%}
+ {{ (kdevops_declared_hosts | default('')) | regex_replace(',', ' ') | split() }}
+ {%- else -%}
+ {{ kdevops_declared_hosts }}
+ {%- endif -%}
+ when:
+ - kdevops_use_declared_hosts
+ - kdevops_declared_hosts is defined
+ tags: vars
+
- name: Get our user
ansible.builtin.command: "whoami"
register: my_user
@@ -75,6 +88,7 @@
when:
- is_fstests
- ansible_hosts_template.stat.exists
+ - not kdevops_use_declared_hosts
- name: Infer enabled blktests test section types
ansible.builtin.set_fact:
@@ -89,6 +103,7 @@
- kdevops_workflows_dedicated_workflow
- kdevops_workflow_enable_blktests
- ansible_hosts_template.stat.exists
+ - not kdevops_use_declared_hosts
- name: Debug inferring block test types
ansible.builtin.debug:
@@ -112,6 +127,7 @@
- kdevops_workflows_dedicated_workflow
- kdevops_workflow_enable_selftests
- ansible_hosts_template.stat.exists
+ - not kdevops_use_declared_hosts
- name: Collect dynamically supported filesystems
vars:
@@ -155,6 +171,7 @@
- kdevops_workflows_dedicated_workflow
- kdevops_workflow_enable_mmtests
- ansible_hosts_template.stat.exists
+ - not kdevops_use_declared_hosts
- name: Load AI nodes configuration for multi-filesystem setup
include_vars:
diff --git a/playbooks/roles/gen_hosts/templates/hosts.j2 b/playbooks/roles/gen_hosts/templates/hosts.j2
index be0378db..2ec18de8 100644
--- a/playbooks/roles/gen_hosts/templates/hosts.j2
+++ b/playbooks/roles/gen_hosts/templates/hosts.j2
@@ -5,4 +5,10 @@ proper identation. We don't need identation for the ansible hosts file.
Each workflow which has its own custom ansible host file generated should use
its own jinja2 template file and define its own ansible task for its generation.
#}
+{% if kdevops_declared_hosts is defined and kdevops_declared_hosts %}
+{# Use declared hosts that skip bringup process - for bare metal or pre-existing infrastructure #}
+{% include 'workflows/declared-hosts.j2' %}
+{% else %}
+{# Include workflow-specific template dynamically based on workflow name #}
{% include 'workflows/' + kdevops_workflow_name + '.j2' %}
+{% endif %}
diff --git a/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
new file mode 100644
index 00000000..7d7975c7
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
@@ -0,0 +1,239 @@
+{# Template for declared hosts that skip bringup process #}
+{# This template is used when users have pre-existing infrastructure like:
+ - Bare metal servers
+ - Pre-provisioned VMs
+ - Cloud instances managed outside of kdevops
+ - Any hosts with existing SSH access
+
+ The hosts are provided via kdevops_declared_hosts variable which contains
+ a list of hostnames/IPs that are already accessible via SSH.
+#}
+{# Parse declared hosts if it's a string #}
+{% if kdevops_declared_hosts is string %}
+{% set parsed_hosts = kdevops_declared_hosts | regex_replace(',', ' ') | split() %}
+{% else %}
+{% set parsed_hosts = kdevops_declared_hosts %}
+{% endif %}
+[all]
+localhost ansible_connection=local
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_workflows_dedicated_workflow %}
+{# For workflows, organize hosts into baseline/dev groups for A/B testing #}
+{% if kdevops_baseline_and_dev %}
+[baseline]
+{# Odd-numbered hosts become baseline nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is odd %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[dev]
+{# Even-numbered hosts become dev nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is even %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% else %}
+{# Without A/B testing, all hosts are baseline #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{# Add workflow-specific groups based on enabled workflow #}
+{% if workflows_reboot_limit %}
+[reboot-limit]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[reboot-limit:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fio_tests %}
+[fio_tests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fio_tests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fstests %}
+[fstests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# Add per-section groups if needed #}
+{% for section in fstests_enabled_test_types|default([]) %}
+[fstests_{{ section | replace('-', '_') }}]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endfor %}
+
+{% elif kdevops_workflow_enable_blktests %}
+[blktests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[blktests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_selftests %}
+[selftests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[selftests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_mmtests %}
+[mmtests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[mmtests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_sysbench %}
+[sysbench]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[sysbench:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ai|default(false)|bool %}
+[ai]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ai:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_minio|default(false)|bool %}
+[minio]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[minio:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_cxl %}
+[cxl]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[cxl:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_pynfs %}
+[pynfs]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[pynfs:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_gitr %}
+[gitr]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[gitr:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ltp %}
+[ltp]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ltp:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_nfstest %}
+[nfstest]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[nfstest:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{% else %}
+{# Non-workflow setup - just use baseline group #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# For non-dedicated workflows (mix mode), add fstests group if enabled #}
+{% if kdevops_workflow_enable_fstests %}
+[fstests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# Add per-section groups if needed #}
+{% for section in fstests_enabled_test_types|default([]) %}
+[fstests_{{ section | replace('-', '_') }}]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endfor %}
+{% endif %}
+
+{% endif %}
+
+[service]
+{# Service nodes are typically not needed for declared hosts #}
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
--
2.50.1
next prev parent reply other threads:[~2025-09-04 0:15 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-04 0:15 [PATCH v4 0/3] declared hosts support Luis Chamberlain
2025-09-04 0:15 ` [PATCH v4 1/3] gen_hosts: use kdevops_workflow_name directly for template selection Luis Chamberlain
2025-09-04 6:05 ` Daniel Gomez
2025-09-04 9:10 ` Luis Chamberlain
2025-09-04 0:15 ` Luis Chamberlain [this message]
2025-09-04 0:15 ` [PATCH v4 3/3] minio: add MinIO Warp S3 benchmarking with declared hosts support Luis Chamberlain
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250904001521.1897970-3-mcgrof@kernel.org \
--to=mcgrof@kernel.org \
--cc=cel@kernel.org \
--cc=da.gomez@kruces.com \
--cc=hui81.qi@samsung.com \
--cc=kdevops@lists.linux.dev \
--cc=kundan.kumar@samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox