[PATCH 0/2] declared hosts support

kdevops.lists.linux.dev archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/2] declared hosts support
@ 2025-08-29 10:31 Luis Chamberlain
  2025-08-29 10:31 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain
  2025-08-29 10:31 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
  0 siblings, 2 replies; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:31 UTC (permalink / raw)
  To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain

It should be easy for us to add bare metal support because well, the
biggest pain is bring up. If bringup and ssh is already configured,
then all we have to do is just skip a bunch of things. So the question
to ask, is how do we generate the hosts file? The easiest interface
I could think of is to allow you to specify it at make menuconfig time
or defconfig time with a DECLARE_HOSTS= specifier with either a space
or comma separated lists of hosts.

I tested this with a new workflow I am testing, minio WARP testing, but I
also tested this simply with a random host I can ssh to with just:

  make defconfig-xfs_reflink_4k DECLARE_HOSTS=foo ; make AV=2
  make uname

And that works. For my other workflow I am testing I got full test
restults and mkfs'd a drive. And so each workflow needs a bit of love
for further DECLARE_HOSTS support. Things like the data partition, and
so on. I suspect we can just specify some of these in the command line
as CLI variables.

But this goes with just basic raw support. Maybe we only enable
workflows for KDEVOPS_USE_DECLARED_HOSTS if we declare the workflow
as supporting it (maybe select SUPPORTS_DECLARED_HOSTS)?

Luis Chamberlain (2):
  gen_hosts: refactor monolithic hosts.j2 template into per-workflow
    templates
  declared_hosts: add support for pre-existing infrastructure

 Makefile                                      |   7 +
 kconfigs/Kconfig.bringup                      |   7 +
 kconfigs/Kconfig.declared_hosts               |  65 +++++
 playbooks/roles/devconfig/tasks/main.yml      |  60 ++++
 playbooks/roles/gen_hosts/tasks/main.yml      |  40 ++-
 playbooks/roles/gen_hosts/templates/hosts.j2  | 264 +++---------------
 .../roles/gen_hosts/templates/workflows/ai.j2 |  99 +++++++
 .../gen_hosts/templates/workflows/blktests.j2 |  58 ++++
 .../gen_hosts/templates/workflows/cxl.j2      |   7 +
 .../templates/workflows/declared-hosts.j2     | 219 +++++++++++++++
 .../gen_hosts/templates/workflows/default.j2  |  62 ++++
 .../templates/workflows/fio-tests.j2          |  38 +++
 .../gen_hosts/templates/workflows/fstests.j2  |  72 +++++
 .../gen_hosts/templates/workflows/gitr.j2     |  41 +++
 .../gen_hosts/templates/workflows/ltp.j2      |  41 +++
 .../gen_hosts/templates/workflows/mmtests.j2  |  77 +++++
 .../gen_hosts/templates/workflows/nfstest.j2  |  41 +++
 .../gen_hosts/templates/workflows/pynfs.j2    |   7 +
 .../templates/workflows/reboot-limit.j2       |  33 +++
 .../templates/workflows/selftests.j2          |  53 ++++
 .../gen_hosts/templates/workflows/sysbench.j2 |  53 ++++
 playbooks/roles/gen_nodes/tasks/main.yml      |   5 +
 workflows/ai/Makefile                         |   3 +-
 workflows/blktests/Makefile                   |   3 +-
 workflows/cxl/Makefile                        |   3 +-
 workflows/fio-tests/Makefile                  |   3 +-
 workflows/fstests/Makefile                    |   3 +-
 workflows/gitr/Makefile                       |   3 +-
 workflows/linux/Makefile                      |   3 +-
 workflows/ltp/Makefile                        |   3 +-
 workflows/mmtests/Makefile                    |   3 +-
 workflows/nfstest/Makefile                    |   3 +-
 workflows/pynfs/Makefile                      |   3 +-
 workflows/selftests/Makefile                  |   3 +-
 workflows/sysbench/Makefile                   |   3 +-
 35 files changed, 1143 insertions(+), 245 deletions(-)
 create mode 100644 kconfigs/Kconfig.declared_hosts
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2

-- 
2.50.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates
  2025-08-29 10:31 [PATCH 0/2] declared hosts support Luis Chamberlain
@ 2025-08-29 10:31 ` Luis Chamberlain
  2025-08-29 10:36   ` Luis Chamberlain
  2025-08-29 10:31 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
  1 sibling, 1 reply; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:31 UTC (permalink / raw)
  To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain

The hosts.j2 template had become unwieldy spaghetti code with all workflows
mixed together in a single file, making it difficult to read, maintain, and
extend. This was particularly problematic when attempting to add new features.

This refactoring:
- Splits the monolithic hosts.j2 into individual workflow templates under
  playbooks/roles/gen_hosts/templates/workflows/
- Creates dedicated templates for each workflow (fstests, blktests, selftests,
  mmtests, sysbench, ai, fio-tests, etc.)
- Converts the main hosts.j2 to use Jinja2 includes for cleaner organization
- Maintains exact same functionality with improved maintainability
- Removes workflow-specific template overrides from all workflow Makefiles
  as they now all use the unified hosts.j2 template

Benefits:
- Each workflow template is now self-contained and easy to understand
- Adding new workflows no longer requires navigating through hundreds of
  lines of unrelated configuration
- Template logic is easier to debug and test
- All workflows now use a consistent template system

This is a pure refactoring with no functional changes. The generated hosts
files remain identical to the previous implementation.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 playbooks/roles/gen_hosts/tasks/main.yml      |  40 ++-
 playbooks/roles/gen_hosts/templates/hosts.j2  | 264 +++---------------
 .../roles/gen_hosts/templates/workflows/ai.j2 |  99 +++++++
 .../gen_hosts/templates/workflows/blktests.j2 |  58 ++++
 .../gen_hosts/templates/workflows/cxl.j2      |   7 +
 .../gen_hosts/templates/workflows/default.j2  |  62 ++++
 .../templates/workflows/fio-tests.j2          |  38 +++
 .../gen_hosts/templates/workflows/fstests.j2  |  72 +++++
 .../gen_hosts/templates/workflows/gitr.j2     |  41 +++
 .../gen_hosts/templates/workflows/ltp.j2      |  41 +++
 .../gen_hosts/templates/workflows/mmtests.j2  |  77 +++++
 .../gen_hosts/templates/workflows/nfstest.j2  |  41 +++
 .../gen_hosts/templates/workflows/pynfs.j2    |   7 +
 .../templates/workflows/reboot-limit.j2       |  33 +++
 .../templates/workflows/selftests.j2          |  53 ++++
 .../gen_hosts/templates/workflows/sysbench.j2 |  53 ++++
 workflows/ai/Makefile                         |   3 +-
 workflows/blktests/Makefile                   |   3 +-
 workflows/cxl/Makefile                        |   3 +-
 workflows/fio-tests/Makefile                  |   3 +-
 workflows/fstests/Makefile                    |   3 +-
 workflows/gitr/Makefile                       |   3 +-
 workflows/linux/Makefile                      |   3 +-
 workflows/ltp/Makefile                        |   3 +-
 workflows/mmtests/Makefile                    |   3 +-
 workflows/nfstest/Makefile                    |   3 +-
 workflows/pynfs/Makefile                      |   3 +-
 workflows/selftests/Makefile                  |   3 +-
 workflows/sysbench/Makefile                   |   3 +-
 29 files changed, 780 insertions(+), 245 deletions(-)
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2

diff --git a/playbooks/roles/gen_hosts/tasks/main.yml b/playbooks/roles/gen_hosts/tasks/main.yml
index fb63629a..657fd22b 100644
--- a/playbooks/roles/gen_hosts/tasks/main.yml
+++ b/playbooks/roles/gen_hosts/tasks/main.yml
@@ -10,6 +10,7 @@
       skip: true
   tags: vars
 
+
 - name: Get our user
   ansible.builtin.command: "whoami"
   register: my_user
@@ -39,6 +40,21 @@
     path: "{{ kdevops_hosts_template_full_path }}"
   register: ansible_hosts_template
 
+- name: Generate the Ansible hosts file for declared hosts configuration
+  tags: ['hosts']
+  ansible.builtin.template:
+    src: "{{ kdevops_hosts_template }}"
+    dest: "{{ ansible_cfg_inventory }}"
+    force: true
+    trim_blocks: True
+    lstrip_blocks: True
+    mode: '0644'
+  when:
+    - kdevops_use_declared_hosts|default(false)|bool
+    - kdevops_declared_hosts is defined
+    - kdevops_declared_hosts|length > 0
+    - ansible_hosts_template.stat.exists
+
 - name: Set fstests config file variable for {{ fstests_fstyp }}
   ansible.builtin.set_fact:
     is_fstests: true
@@ -67,6 +83,7 @@
   when:
     - bootlinux_builder
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file
   tags: ["hosts"]
@@ -80,6 +97,7 @@
     - not kdevops_workflows_dedicated_workflow
     - ansible_hosts_template.stat.exists
     - not kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file for NixOS
   tags: ['hosts']
@@ -93,6 +111,7 @@
     - not kdevops_workflows_dedicated_workflow
     - ansible_hosts_template.stat.exists
     - kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Update Ansible inventory access modification time so make sees it updated
   ansible.builtin.file:
@@ -112,6 +131,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_cxl
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file for dedicated pynfs work
   tags: ["hosts"]
@@ -125,6 +145,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_pynfs
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file for dedicated gitr workflow
   tags: ["hosts"]
@@ -140,6 +161,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_gitr
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate an Ansible inventory file for a dedicated ltp workflow
   tags: ["hosts"]
@@ -155,6 +177,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_ltp
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file for dedicated nfstest workflow
   tags: ["hosts"]
@@ -170,6 +193,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_nfstest
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Set empty enabled test types list for fstests
   tags: ["hosts"]
@@ -178,6 +202,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Check which fstests test types are enabled
   register: fstests_enabled_test_types_reg
@@ -202,6 +227,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Now expand the list of enabled fstests for valid configuration sections
   tags: ["hosts"]
@@ -213,6 +239,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
     - item.changed
 
 - name: Generate the Ansible inventory file for a dedicated fstests setup
@@ -233,6 +260,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Infer enabled blktests test section types
   ansible.builtin.set_fact:
@@ -247,6 +275,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_blktests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Debug inferring block test types
   ansible.builtin.debug:
@@ -269,6 +298,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_blktests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Infer enabled selftests test section types
   ansible.builtin.set_fact:
@@ -283,6 +313,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_selftests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible inventory file for a dedicated selftests setup
   tags: ["hosts"]
@@ -296,6 +327,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_selftests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Collect dynamically supported filesystems
   vars:
@@ -338,6 +370,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_sysbench
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 
 - name: Generate the Ansible hosts file for a dedicated fio-tests setup
@@ -354,6 +387,7 @@
     - kdevops_workflow_enable_fio_tests
     - ansible_hosts_template.stat.exists
     - not kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 
 - name: Infer enabled mmtests test types
@@ -369,6 +403,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_mmtests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible hosts file for a dedicated mmtests setup
   tags: ["hosts"]
@@ -382,6 +417,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_mmtests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Generate the Ansible hosts file for a dedicated reboot-limit setup
   tags: ["hosts"]
@@ -395,6 +431,7 @@
     - kdevops_workflows_dedicated_workflow
     - workflows_reboot_limit
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Load AI nodes configuration for multi-filesystem setup
   include_vars:
@@ -426,8 +463,9 @@
     mode: '0644'
   when:
     - kdevops_workflows_dedicated_workflow
-    - kdevops_workflow_enable_ai
+    - kdevops_workflow_enable_ai|default(false)|bool
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts|default(false)|bool
 
 - name: Verify if final host file exists
   ansible.builtin.stat:
diff --git a/playbooks/roles/gen_hosts/templates/hosts.j2 b/playbooks/roles/gen_hosts/templates/hosts.j2
index 0e896481..2f2f5d41 100644
--- a/playbooks/roles/gen_hosts/templates/hosts.j2
+++ b/playbooks/roles/gen_hosts/templates/hosts.j2
@@ -5,242 +5,44 @@ proper identation. We don't need identation for the ansible hosts file.
 Each workflow which has its own custom ansible host file generated should use
 its own jinja2 template file and define its own ansible task for its generation.
 #}
-{% if kdevops_workflows_dedicated_workflow %}
+{% if kdevops_declared_hosts is defined and kdevops_declared_hosts %}
+{# Use declared hosts that skip bringup process - for bare metal or pre-existing infrastructure #}
+{% include 'workflows/declared-hosts.j2' %}
+{% elif kdevops_workflows_dedicated_workflow %}
+{# Include workflow-specific template based on enabled workflow. Each workflow has its own
+   template file under workflows/ directory for better maintainability and modularity #}
 {% if workflows_reboot_limit %}
-[all]
-localhost ansible_connection=local
-{{ kdevops_host_prefix }}-reboot-limit
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-reboot-limit-dev
-{% endif %}
-
-[all:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-[baseline]
-{{ kdevops_host_prefix }}-reboot-limit
-
-[baseline:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% if kdevops_baseline_and_dev %}
-[dev]
-{{ kdevops_host_prefix }}-reboot-limit-dev
-
-[dev:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% endif %}
-[reboot-limit]
-{{ kdevops_host_prefix }}-reboot-limit
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-reboot-limit-dev
-{% endif %}
-
-[reboot-limit:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% include 'workflows/reboot-limit.j2' %}
 {% elif kdevops_workflow_enable_fio_tests %}
-[all]
-localhost ansible_connection=local
-{{ kdevops_host_prefix }}-fio-tests
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-fio-tests-dev
-{% endif %}
-
-[all:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-[baseline]
-{{ kdevops_host_prefix }}-fio-tests
-
-[baseline:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% if kdevops_baseline_and_dev %}
-[dev]
-{{ kdevops_host_prefix }}-fio-tests-dev
-
-[dev:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% endif %}
-[fio_tests]
-{{ kdevops_host_prefix }}-fio-tests
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-fio-tests-dev
-{% endif %}
-
-[fio_tests:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-[service]
-
-[service:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-{% elif kdevops_workflow_enable_ai %}
-{% if ai_enable_multifs_testing|default(false)|bool %}
-{# Multi-filesystem section-based hosts #}
-[all]
-localhost ansible_connection=local
-{% for node in all_generic_nodes %}
-{{ node }}
-{% endfor %}
-
-[all:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-[baseline]
-{% for node in all_generic_nodes %}
-{% if not node.endswith('-dev') %}
-{{ node }}
-{% endif %}
-{% endfor %}
-
-[baseline:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% if kdevops_baseline_and_dev %}
-[dev]
-{% for node in all_generic_nodes %}
-{% if node.endswith('-dev') %}
-{{ node }}
-{% endif %}
-{% endfor %}
-
-[dev:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% endif %}
-[ai]
-{% for node in all_generic_nodes %}
-{{ node }}
-{% endfor %}
-
-[ai:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{# Individual section groups for multi-filesystem testing #}
-{% set section_names = [] %}
-{% for node in all_generic_nodes %}
-{% if not node.endswith('-dev') %}
-{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %}
-{% if section != kdevops_host_prefix + '-ai' %}
-{% if section_names.append(section) %}{% endif %}
-{% endif %}
-{% endif %}
-{% endfor %}
-
-{% for section in section_names %}
-[ai_{{ section | replace('-', '_') }}]
-{{ kdevops_host_prefix }}-ai-{{ section }}
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-ai-{{ section }}-dev
-{% endif %}
-
-[ai_{{ section | replace('-', '_') }}:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% endfor %}
-{% else %}
-{# Single filesystem hosts (original behavior) #}
-[all]
-localhost ansible_connection=local
-{{ kdevops_host_prefix }}-ai
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-ai-dev
-{% endif %}
-
-[all:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-[baseline]
-{{ kdevops_host_prefix }}-ai
-
-[baseline:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% if kdevops_baseline_and_dev %}
-[dev]
-{{ kdevops_host_prefix }}-ai-dev
-
-[dev:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-
-{% endif %}
-[ai]
-{{ kdevops_host_prefix }}-ai
-{% if kdevops_baseline_and_dev %}
-{{ kdevops_host_prefix }}-ai-dev
-{% endif %}
-
-[ai:vars]
-ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
-{% endif %}
+{% include 'workflows/fio-tests.j2' %}
+{% elif kdevops_workflow_enable_fstests %}
+{% include 'workflows/fstests.j2' %}
+{% elif kdevops_workflow_enable_blktests %}
+{% include 'workflows/blktests.j2' %}
+{% elif kdevops_workflow_enable_selftests %}
+{% include 'workflows/selftests.j2' %}
+{% elif kdevops_workflow_enable_mmtests %}
+{% include 'workflows/mmtests.j2' %}
+{% elif kdevops_workflow_enable_sysbench %}
+{% include 'workflows/sysbench.j2' %}
+{% elif kdevops_workflow_enable_cxl %}
+{% include 'workflows/cxl.j2' %}
+{% elif kdevops_workflow_enable_pynfs %}
+{% include 'workflows/pynfs.j2' %}
+{% elif kdevops_workflow_enable_gitr %}
+{% include 'workflows/gitr.j2' %}
+{% elif kdevops_workflow_enable_ltp %}
+{% include 'workflows/ltp.j2' %}
+{% elif kdevops_workflow_enable_nfstest %}
+{% include 'workflows/nfstest.j2' %}
+{% elif kdevops_workflow_enable_ai|default(false)|bool %}
+{% include 'workflows/ai.j2' %}
 {% else %}
 [all]
 localhost ansible_connection=local
 write-your-own-template-for-your-workflow-and-task
 {% endif %}
 {% else %}
-[all]
-localhost ansible_connection=local
-{% if kdevops_enable_nixos|default(false) %}
-{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3
-{% else %}
-{{ kdevops_host_prefix }}
-{% endif %}
-{% if kdevops_baseline_and_dev == True %}
-{% if kdevops_enable_nixos|default(false) %}
-{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3
-{% else %}
-{{ kdevops_host_prefix }}-dev
-{% endif %}
-{% endif %}
-{% if kdevops_enable_iscsi %}
-{{ kdevops_host_prefix }}-iscsi
-{% endif %}
-{% if kdevops_nfsd_enable %}
-{{ kdevops_host_prefix }}-nfsd
-{% endif %}
-[all:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-[baseline]
-{% if kdevops_enable_nixos|default(false) %}
-{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3
-{% else %}
-{{ kdevops_host_prefix }}
-{% endif %}
-[baseline:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-[dev]
-{% if kdevops_baseline_and_dev %}
-{% if kdevops_enable_nixos|default(false) %}
-{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3
-{% else %}
-{{ kdevops_host_prefix }}-dev
-{% endif %}
-{% endif %}
-[dev:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-{% if kdevops_enable_iscsi %}
-[iscsi]
-{{ kdevops_host_prefix }}-iscsi
-[iscsi:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-{% endif %}
-{% if kdevops_nfsd_enable %}
-[nfsd]
-{{ kdevops_host_prefix }}-nfsd
-[nfsd:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-{% endif %}
-[service]
-{% if kdevops_enable_iscsi %}
-{{ kdevops_host_prefix }}-iscsi
-{% endif %}
-{% if kdevops_nfsd_enable %}
-{{ kdevops_host_prefix }}-nfsd
-{% endif %}
-[service:vars]
-ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-{% endif %}
+{# Default non-workflow template #}
+{% include 'workflows/default.j2' %}
+{% endif %}
\ No newline at end of file
diff --git a/playbooks/roles/gen_hosts/templates/workflows/ai.j2 b/playbooks/roles/gen_hosts/templates/workflows/ai.j2
new file mode 100644
index 00000000..d0914436
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/ai.j2
@@ -0,0 +1,99 @@
+{# Workflow template for AI #}
+{% if ai_enable_multifs_testing|default(false)|bool %}
+{# Multi-filesystem section-based hosts #}
+[all]
+localhost ansible_connection=local
+{% for node in all_generic_nodes %}
+{{ node }}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for node in all_generic_nodes %}
+{% if not node.endswith('-dev') %}
+{{ node }}
+{% endif %}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for node in all_generic_nodes %}
+{% if node.endswith('-dev') %}
+{{ node }}
+{% endif %}
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[ai]
+{% for node in all_generic_nodes %}
+{{ node }}
+{% endfor %}
+
+[ai:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# Individual section groups for multi-filesystem testing #}
+{% set section_names = [] %}
+{% for node in all_generic_nodes %}
+{% if not node.endswith('-dev') %}
+{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %}
+{% if section != kdevops_host_prefix + '-ai' %}
+{% if section_names.append(section) %}{% endif %}
+{% endif %}
+{% endif %}
+{% endfor %}
+
+{% for section in section_names %}
+[ai_{{ section | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-ai-{{ section }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-ai-{{ section }}-dev
+{% endif %}
+
+[ai_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
+{% else %}
+{# Single filesystem hosts (original behavior) #}
+[all]
+localhost ansible_connection=local
+{{ kdevops_host_prefix }}-ai
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-ai-dev
+{% endif %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{{ kdevops_host_prefix }}-ai
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{{ kdevops_host_prefix }}-ai-dev
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[ai]
+{{ kdevops_host_prefix }}-ai
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-ai-dev
+{% endif %}
+
+[ai:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
diff --git a/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2
new file mode 100644
index 00000000..eea3eef3
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2
@@ -0,0 +1,58 @@
+{# Workflow template for blktests #}
+[all]
+localhost ansible_connection=local
+{% for test_type in blktests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for test_type in blktests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for test_type in blktests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[blktests]
+{% for test_type in blktests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+
+[blktests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% for test_type in blktests_enabled_test_types %}
+[blktests_{{ test_type | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+
+[blktests_{{ test_type | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
+
+[service]
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2
new file mode 100644
index 00000000..53790f29
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2
@@ -0,0 +1,7 @@
+{# Workflow template for CXL #}
+[all]
+localhost ansible_connection=local
+write-your-own-template-for-cxl-workflow
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/default.j2 b/playbooks/roles/gen_hosts/templates/workflows/default.j2
new file mode 100644
index 00000000..86619309
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/default.j2
@@ -0,0 +1,62 @@
+{# Default template for non-workflow setups #}
+[all]
+localhost ansible_connection=local
+{% if kdevops_enable_nixos|default(false) %}
+{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3
+{% else %}
+{{ kdevops_host_prefix }}
+{% endif %}
+{% if kdevops_baseline_and_dev == True %}
+{% if kdevops_enable_nixos|default(false) %}
+{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3
+{% else %}
+{{ kdevops_host_prefix }}-dev
+{% endif %}
+{% endif %}
+{% if kdevops_enable_iscsi %}
+{{ kdevops_host_prefix }}-iscsi
+{% endif %}
+{% if kdevops_nfsd_enable %}
+{{ kdevops_host_prefix }}-nfsd
+{% endif %}
+[all:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[baseline]
+{% if kdevops_enable_nixos|default(false) %}
+{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3
+{% else %}
+{{ kdevops_host_prefix }}
+{% endif %}
+[baseline:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[dev]
+{% if kdevops_baseline_and_dev %}
+{% if kdevops_enable_nixos|default(false) %}
+{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3
+{% else %}
+{{ kdevops_host_prefix }}-dev
+{% endif %}
+{% endif %}
+[dev:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+{% if kdevops_enable_iscsi %}
+[iscsi]
+{{ kdevops_host_prefix }}-iscsi
+[iscsi:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+{% endif %}
+{% if kdevops_nfsd_enable %}
+[nfsd]
+{{ kdevops_host_prefix }}-nfsd
+[nfsd:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+{% endif %}
+[service]
+{% if kdevops_enable_iscsi %}
+{{ kdevops_host_prefix }}-iscsi
+{% endif %}
+{% if kdevops_nfsd_enable %}
+{{ kdevops_host_prefix }}-nfsd
+{% endif %}
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
new file mode 100644
index 00000000..548941a0
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
@@ -0,0 +1,38 @@
+{# Workflow template for fio-tests #}
+[all]
+localhost ansible_connection=local
+{{ kdevops_host_prefix }}-fio-tests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-fio-tests-dev
+{% endif %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{{ kdevops_host_prefix }}-fio-tests
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{{ kdevops_host_prefix }}-fio-tests-dev
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[fio_tests]
+{{ kdevops_host_prefix }}-fio-tests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-fio-tests-dev
+{% endif %}
+
+[fio_tests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[service]
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2
new file mode 100644
index 00000000..600d125f
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2
@@ -0,0 +1,72 @@
+{# Workflow template for fstests #}
+[all]
+localhost ansible_connection=local
+{% for node_section in fstests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ node_section }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ node_section }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for node_section in fstests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ node_section }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for node_section in fstests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ node_section }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[fstests]
+{% for node_section in fstests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ node_section }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ node_section }}-dev
+{% endif %}
+{% endfor %}
+
+[fstests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% for section in sections %}
+[fstests_{{ section | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-{{ section }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ section }}-dev
+{% endif %}
+
+[fstests_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
+
+[nfsd]
+{% if kdevops_nfsd_enable %}
+{{ kdevops_host_prefix }}-nfsd
+{% endif %}
+
+[nfsd:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[service]
+{% if kdevops_enable_iscsi %}
+{{ kdevops_host_prefix }}-iscsi
+{% endif %}
+{% if kdevops_nfsd_enable %}
+{{ kdevops_host_prefix }}-nfsd
+{% endif %}
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2
new file mode 100644
index 00000000..86ee9326
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2
@@ -0,0 +1,41 @@
+{# Workflow template for gitr #}
+[all]
+localhost ansible_connection=local
+{% for host in gitr_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for host in gitr_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for host in gitr_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[gitr]
+{% for host in gitr_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[gitr:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2
new file mode 100644
index 00000000..fb120828
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2
@@ -0,0 +1,41 @@
+{# Workflow template for ltp #}
+[all]
+localhost ansible_connection=local
+{% for host in ltp_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for host in ltp_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for host in ltp_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[ltp]
+{% for host in ltp_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[ltp:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
new file mode 100644
index 00000000..d796cbe6
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
@@ -0,0 +1,77 @@
+{# Workflow template for mmtests #}
+[all]
+localhost ansible_connection=local
+{% if mmtests_enabled_test_types %}
+{% for test_type in mmtests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+{% else %}
+{{ kdevops_host_prefix }}-mmtests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-mmtests-dev
+{% endif %}
+{% endif %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% if mmtests_enabled_test_types %}
+{% for test_type in mmtests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% endfor %}
+{% else %}
+{{ kdevops_host_prefix }}-mmtests
+{% endif %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% if mmtests_enabled_test_types %}
+{% for test_type in mmtests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endfor %}
+{% else %}
+{{ kdevops_host_prefix }}-mmtests-dev
+{% endif %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[mmtests]
+{% if mmtests_enabled_test_types %}
+{% for test_type in mmtests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+{% else %}
+{{ kdevops_host_prefix }}-mmtests
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-mmtests-dev
+{% endif %}
+{% endif %}
+
+[mmtests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if mmtests_enabled_test_types %}
+{% for test_type in mmtests_enabled_test_types %}
+[mmtests_{{ test_type | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+
+[mmtests_{{ test_type | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
+{% endif %}
diff --git a/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
new file mode 100644
index 00000000..34aa7dfe
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
@@ -0,0 +1,41 @@
+{# Workflow template for nfstest #}
+[all]
+localhost ansible_connection=local
+{% for host in nfstest_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for host in nfstest_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for host in nfstest_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[nfstest]
+{% for host in nfstest_enabled_hosts %}
+{{ kdevops_host_prefix }}-{{ host }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ host }}-dev
+{% endif %}
+{% endfor %}
+
+[nfstest:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
new file mode 100644
index 00000000..6145b1a0
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
@@ -0,0 +1,7 @@
+{# Workflow template for pynfs #}
+[all]
+localhost ansible_connection=local
+write-your-own-template-for-pynfs-workflow
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
new file mode 100644
index 00000000..07bd6f80
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
@@ -0,0 +1,33 @@
+{# Workflow template for reboot-limit #}
+[all]
+localhost ansible_connection=local
+{{ kdevops_host_prefix }}-reboot-limit
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-reboot-limit-dev
+{% endif %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{{ kdevops_host_prefix }}-reboot-limit
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{{ kdevops_host_prefix }}-reboot-limit-dev
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[reboot-limit]
+{{ kdevops_host_prefix }}-reboot-limit
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-reboot-limit-dev
+{% endif %}
+
+[reboot-limit:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2
new file mode 100644
index 00000000..4ef598c7
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2
@@ -0,0 +1,53 @@
+{# Workflow template for selftests #}
+[all]
+localhost ansible_connection=local
+{% for test_type in selftests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for test_type in selftests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for test_type in selftests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[selftests]
+{% for test_type in selftests_enabled_test_types %}
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+{% endfor %}
+
+[selftests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% for test_type in selftests_enabled_test_types %}
+[selftests_{{ test_type | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-{{ test_type }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test_type }}-dev
+{% endif %}
+
+[selftests_{{ test_type | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
diff --git a/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2
new file mode 100644
index 00000000..34f22a83
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2
@@ -0,0 +1,53 @@
+{# Workflow template for sysbench #}
+[all]
+localhost ansible_connection=local
+{% for test in enabled_sysbench_tests %}
+{{ kdevops_host_prefix }}-{{ test }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test }}-dev
+{% endif %}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[baseline]
+{% for test in enabled_sysbench_tests %}
+{{ kdevops_host_prefix }}-{{ test }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_baseline_and_dev %}
+[dev]
+{% for test in enabled_sysbench_tests %}
+{{ kdevops_host_prefix }}-{{ test }}-dev
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endif %}
+[sysbench]
+{% for test in enabled_sysbench_tests %}
+{{ kdevops_host_prefix }}-{{ test }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test }}-dev
+{% endif %}
+{% endfor %}
+
+[sysbench:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% for test in enabled_sysbench_tests %}
+[sysbench_{{ test | replace('-', '_') }}]
+{{ kdevops_host_prefix }}-{{ test }}
+{% if kdevops_baseline_and_dev %}
+{{ kdevops_host_prefix }}-{{ test }}-dev
+{% endif %}
+
+[sysbench_{{ test | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% endfor %}
diff --git a/workflows/ai/Makefile b/workflows/ai/Makefile
index 1c297edd..27dea382 100644
--- a/workflows/ai/Makefile
+++ b/workflows/ai/Makefile
@@ -4,7 +4,8 @@ PHONY += ai-tests ai-tests-baseline ai-tests-dev
 PHONY += ai-tests-results
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 endif
 
 export AI_DATA_TARGET := $(subst ",,$(CONFIG_AI_BENCHMARK_RESULTS_DIR))
diff --git a/workflows/blktests/Makefile b/workflows/blktests/Makefile
index 11cd9a65..97649026 100644
--- a/workflows/blktests/Makefile
+++ b/workflows/blktests/Makefile
@@ -12,7 +12,8 @@ ID=$(shell id -u)
 BLKTESTS_ARGS	:=
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := blktests.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := blktests.j2
 endif
 
 BLKTESTS_GIT:=$(subst ",,$(CONFIG_BLKTESTS_GIT))
diff --git a/workflows/cxl/Makefile b/workflows/cxl/Makefile
index bb130057..feabe197 100644
--- a/workflows/cxl/Makefile
+++ b/workflows/cxl/Makefile
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: copyleft-next-0.3.1
 
-export KDEVOPS_HOSTS_TEMPLATE := cxl.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := cxl.j2
 
 CXL_ARGS :=
 CXL_ARGS += ndctl_git='$(subst ",,$(CONFIG_NDCTL_GIT))'
diff --git a/workflows/fio-tests/Makefile b/workflows/fio-tests/Makefile
index 5eb2ccbd..20a5d192 100644
--- a/workflows/fio-tests/Makefile
+++ b/workflows/fio-tests/Makefile
@@ -1,5 +1,6 @@
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2
 endif
 
 fio-tests:
diff --git a/workflows/fstests/Makefile b/workflows/fstests/Makefile
index d2e5c636..989c564b 100644
--- a/workflows/fstests/Makefile
+++ b/workflows/fstests/Makefile
@@ -9,7 +9,8 @@ FSTESTS_BASELINE_EXTRA :=
 export FSTYP:=$(subst ",,$(CONFIG_FSTESTS_FSTYP))
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := fstests.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := fstests.j2
 endif
 
 FSTESTS_ARGS += fstests_fstyp='$(FSTYP)'
diff --git a/workflows/gitr/Makefile b/workflows/gitr/Makefile
index c685395a..eadcdac1 100644
--- a/workflows/gitr/Makefile
+++ b/workflows/gitr/Makefile
@@ -1,5 +1,6 @@
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := gitr.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := gitr.j2
 endif
 
 GITR_MNT:=$(subst ",,$(CONFIG_GITR_MNT))
diff --git a/workflows/linux/Makefile b/workflows/linux/Makefile
index 30b123f9..60d755b3 100644
--- a/workflows/linux/Makefile
+++ b/workflows/linux/Makefile
@@ -14,7 +14,8 @@ TREE_CONFIG:=config-$(TREE_REF)-pure-iomap
 endif
 
 ifeq (y,$(CONFIG_BOOTLINUX_BUILDER))
-KDEVOPS_HOSTS_TEMPLATE=builder.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# KDEVOPS_HOSTS_TEMPLATE=builder.j2
 endif
 
 # Describes the Linux clone
diff --git a/workflows/ltp/Makefile b/workflows/ltp/Makefile
index 767465bc..b3c3319b 100644
--- a/workflows/ltp/Makefile
+++ b/workflows/ltp/Makefile
@@ -1,5 +1,6 @@
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := ltp.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := ltp.j2
 endif
 
 LTP_REPO:=$(subst ",,$(CONFIG_LTP_REPO))
diff --git a/workflows/mmtests/Makefile b/workflows/mmtests/Makefile
index b65d256b..186dc8cb 100644
--- a/workflows/mmtests/Makefile
+++ b/workflows/mmtests/Makefile
@@ -1,7 +1,8 @@
 MMTESTS_ARGS	:=
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2
 endif
 
 mmtests:
diff --git a/workflows/nfstest/Makefile b/workflows/nfstest/Makefile
index bbfd3f64..bc880b6d 100644
--- a/workflows/nfstest/Makefile
+++ b/workflows/nfstest/Makefile
@@ -1,5 +1,6 @@
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2
 endif # CONFIG_WORKFLOWS_DEDICATED_WORKFLOW
 
 ifeq (y,$(CONFIG_NFSTEST_USE_KDEVOPS_NFSD))
diff --git a/workflows/pynfs/Makefile b/workflows/pynfs/Makefile
index 2f3ff97b..b7bb12bb 100644
--- a/workflows/pynfs/Makefile
+++ b/workflows/pynfs/Makefile
@@ -1,5 +1,6 @@
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2
 endif
 
 PYNFS_GIT:=$(subst ",,$(CONFIG_PYNFS_GIT))
diff --git a/workflows/selftests/Makefile b/workflows/selftests/Makefile
index d3b7044c..4ec691f3 100644
--- a/workflows/selftests/Makefile
+++ b/workflows/selftests/Makefile
@@ -3,7 +3,8 @@
 SELFTESTS_ARGS :=
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := selftests.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := selftests.j2
 endif
 
 SELFTESTS_DYNAMIC_RUNTIME_VARS := "kdevops_run_selftests": True
diff --git a/workflows/sysbench/Makefile b/workflows/sysbench/Makefile
index daf7bc75..5f484691 100644
--- a/workflows/sysbench/Makefile
+++ b/workflows/sysbench/Makefile
@@ -1,7 +1,8 @@
 PHONY += sysbench sysbench-test sysbench-telemetry sysbench-help-menu
 
 ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW))
-export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2
+# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates
+# export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2
 endif
 
 TAGS_SYSBENCH_RUN := db_start
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure
  2025-08-29 10:31 [PATCH 0/2] declared hosts support Luis Chamberlain
  2025-08-29 10:31 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain
@ 2025-08-29 10:31 ` Luis Chamberlain
  1 sibling, 0 replies; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:31 UTC (permalink / raw)
  To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain

This adds support for using pre-existing infrastructure (bare metal
servers, pre-provisioned VMs) that already have SSH access configured.
This is useful for environments where kdevops bringup/teardown is not
desired or possible.

Key features:
- New DECLARE_HOSTS CLI parameter to specify existing hosts
- Automatic detection of CLI override to enable declared hosts mode
- SKIP_BRINGUP is automatically selected when using declared hosts
- Hosts can be specified as space or comma-separated list
- For A/B testing workflows, odd-numbered hosts become baseline nodes
  and even-numbered hosts become dev nodes
- New Kconfig.declared_hosts for configuration options
- New declared-hosts.j2 template for generating inventory

Usage:
  make defconfig-xfs_reflink_4k DECLARE_HOSTS='server1 server2 server3'
  make  # Will skip bringup and use the declared hosts

The implementation assumes SSH access is already configured in the user's
~/.ssh/config or via SSH keys. No SSH configuration is generated by kdevops
when using declared hosts.

This feature integrates cleanly with the refactored template system,
using the same hosts.j2 entry point with a dedicated declared-hosts.j2
template for this mode.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 Makefile                                      |   7 +
 kconfigs/Kconfig.bringup                      |   7 +
 kconfigs/Kconfig.declared_hosts               |  65 ++++++
 playbooks/roles/devconfig/tasks/main.yml      |  60 +++++
 .../templates/workflows/declared-hosts.j2     | 219 ++++++++++++++++++
 playbooks/roles/gen_nodes/tasks/main.yml      |   5 +
 6 files changed, 363 insertions(+)
 create mode 100644 kconfigs/Kconfig.declared_hosts
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2

diff --git a/Makefile b/Makefile
index 83c67340..fcb626ae 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,13 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+
+# Export CLI override variables for Kconfig to detect them
+# Note: We accept DECLARE_HOSTS but export as DECLARED_HOSTS for consistency
+ifdef DECLARE_HOSTS
+export DECLARED_HOSTS := $(DECLARE_HOSTS)
+endif
+
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
diff --git a/kconfigs/Kconfig.bringup b/kconfigs/Kconfig.bringup
index 8caf07be..f7f9735a 100644
--- a/kconfigs/Kconfig.bringup
+++ b/kconfigs/Kconfig.bringup
@@ -9,8 +9,14 @@ config KDEVOPS_ENABLE_NIXOS
 	bool
 	output yaml
 
+# CLI override detection for DECLARED_HOSTS which should enable SKIP_BRINGUP
+config SKIP_BRINGUP_SET_BY_CLI
+	bool
+	default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS)
+
 choice
 	prompt "Node bring up method"
+	default SKIP_BRINGUP if SKIP_BRINGUP_SET_BY_CLI
 	default GUESTFS
 
 config GUESTFS
@@ -80,6 +86,7 @@ config LIBVIRT
 source "kconfigs/Kconfig.guestfs"
 source "kconfigs/Kconfig.nixos"
 source "terraform/Kconfig"
+source "kconfigs/Kconfig.declared_hosts"
 if LIBVIRT
 source "kconfigs/Kconfig.libvirt"
 endif
diff --git a/kconfigs/Kconfig.declared_hosts b/kconfigs/Kconfig.declared_hosts
new file mode 100644
index 00000000..56b56845
--- /dev/null
+++ b/kconfigs/Kconfig.declared_hosts
@@ -0,0 +1,65 @@
+# Configuration for declared hosts that skip bringup process
+
+# CLI override support for DECLARED_HOSTS
+config KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	bool
+	output yaml
+	default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS)
+
+config KDEVOPS_USE_DECLARED_HOSTS
+	bool "Use declared hosts (skip bringup process)"
+	depends on SKIP_BRINGUP || KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	select WORKFLOW_INFER_USER_AND_GROUP
+	output yaml
+	default y if SKIP_BRINGUP
+	default y if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	help
+	  Enable this option to use pre-existing hosts that you have already
+	  configured with SSH access. This is useful for:
+
+	  * Bare metal systems
+	  * Pre-provisioned VMs or cloud instances
+	  * Systems managed by other infrastructure tools
+
+	  When this option is enabled:
+	  - SSH keys will not be generated (assumes you already have access)
+	  - Bringup and teardown operations will be skipped
+	  - User and group settings will be inferred from the target hosts
+	  - You must provide the list of hosts in KDEVOPS_DECLARED_HOSTS
+
+	  This option automatically:
+	  - Selects WORKFLOW_INFER_USER_AND_GROUP to detect the correct
+	    user and group on the target systems
+	  - Assumes SSH access is already configured
+
+if KDEVOPS_USE_DECLARED_HOSTS
+
+config KDEVOPS_DECLARED_HOSTS
+	string "List of declared hosts (space or comma separated)"
+	output yaml
+	default $(shell, ./scripts/append-makefile-vars.sh $(DECLARED_HOSTS)) if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	default ""
+	help
+	  Provide a space or comma-separated list of hostnames or IP addresses
+	  that are already accessible via SSH.
+
+	  Examples:
+	  - Single host: "server1.example.com"
+	  - Multiple hosts: "server1.example.com server2.example.com"
+	  - IP addresses: "192.168.1.10 192.168.1.11"
+	  - Mixed: "server1.example.com 10.0.0.5 server2.local"
+
+	  For A/B testing workflows with baseline/dev configurations, you can
+	  provide an even number of hosts. The odd-numbered hosts will be used
+	  as baseline nodes and even-numbered hosts as dev nodes.
+
+	  Requirements:
+	  - SSH access must already be configured for these hosts
+	  - The user running kdevops must have passwordless SSH access
+	  - The hosts should have the required dependencies installed
+
+	  Note: kdevops assumes you have already configured SSH access to these
+	  hosts in your ~/.ssh/config or have appropriate SSH keys set up.
+	  Ansible will use your existing SSH configuration to connect.
+
+endif # KDEVOPS_USE_DECLARED_HOSTS
diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index fccd1fcf..0d69f5fd 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -17,6 +17,66 @@
   ansible.builtin.setup:
   tags: always
 
+- name: Determine the actual user's home directory when using inferred user
+  block:
+    - name: Get the current username
+      ansible.builtin.command: whoami
+      register: current_user
+      changed_when: false
+
+    - name: Get home directory for the current user
+      ansible.builtin.shell: |
+        # When workflow_infer_user_and_group is enabled, we need to use the actual SSH user
+        # not the hardcoded 'kdevops' user from group_vars/all
+        actual_user="{{ current_user.stdout }}"
+        if [ -n "$actual_user" ]; then
+          getent passwd "$actual_user" | cut -d: -f6
+        else
+          echo "/root"
+        fi
+      register: inferred_home_dir
+      changed_when: false
+
+    - name: Set data_user and data_home_dir based on inferred user
+      set_fact:
+        data_user: "{{ current_user.stdout }}"
+        data_home_dir: "{{ inferred_home_dir.stdout }}"
+        dev_gitconfig_dest: "{{ inferred_home_dir.stdout }}/.gitconfig"
+        dev_bash_config: "{{ inferred_home_dir.stdout }}/.bashrc"
+        dev_bash_config_hacks_dest: "{{ inferred_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}"
+  when: workflow_infer_user_and_group | default(false) | bool
+
+# Also handle the case when workflow_infer_user_and_group is false but we're using declared hosts
+# In this case, we should still use the SSH user's actual home directory
+- name: Check if using declared hosts without infer setting
+  block:
+    - name: Get the current username for declared hosts
+      ansible.builtin.command: whoami
+      register: current_user_declared
+      changed_when: false
+
+    - name: Get home directory for declared hosts user
+      ansible.builtin.shell: |
+        actual_user="{{ current_user_declared.stdout }}"
+        if [ -n "$actual_user" ]; then
+          getent passwd "$actual_user" | cut -d: -f6
+        else
+          echo "/root"
+        fi
+      register: declared_home_dir
+      changed_when: false
+
+    - name: Override data paths for declared hosts
+      set_fact:
+        data_user: "{{ current_user_declared.stdout }}"
+        data_home_dir: "{{ declared_home_dir.stdout }}"
+        dev_gitconfig_dest: "{{ declared_home_dir.stdout }}/.gitconfig"
+        dev_bash_config: "{{ declared_home_dir.stdout }}/.bashrc"
+        dev_bash_config_hacks_dest: "{{ declared_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}"
+  when:
+    - kdevops_use_declared_hosts | default(false) | bool
+    - not (workflow_infer_user_and_group | default(false) | bool)
+
 # Update /etc/hostname first so the change gets picked up by the reboot
 # that occurs during the distro-specific tasks
 
diff --git a/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
new file mode 100644
index 00000000..61b51885
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
@@ -0,0 +1,219 @@
+{# Template for declared hosts that skip bringup process #}
+{# This template is used when users have pre-existing infrastructure like:
+   - Bare metal servers
+   - Pre-provisioned VMs
+   - Cloud instances managed outside of kdevops
+   - Any hosts with existing SSH access
+
+   The hosts are provided via kdevops_declared_hosts variable which contains
+   a list of hostnames/IPs that are already accessible via SSH.
+#}
+{# Parse declared hosts - handle both string and list formats #}
+{% if kdevops_declared_hosts is string %}
+{% set parsed_hosts = kdevops_declared_hosts | regex_replace(',', ' ') | split() %}
+{% elif kdevops_declared_hosts is iterable and kdevops_declared_hosts is not string %}
+{% set parsed_hosts = kdevops_declared_hosts %}
+{% else %}
+{% set parsed_hosts = [kdevops_declared_hosts] %}
+{% endif %}
+[all]
+localhost ansible_connection=local
+{# Loop through parsed hosts #}
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_workflows_dedicated_workflow %}
+{# For workflows, organize hosts into baseline/dev groups for A/B testing #}
+{% if kdevops_baseline_and_dev %}
+[baseline]
+{# Odd-numbered hosts become baseline nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is odd %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[dev]
+{# Even-numbered hosts become dev nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is even %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% else %}
+{# Without A/B testing, all hosts are baseline #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{# Add workflow-specific groups based on enabled workflow #}
+{% if workflows_reboot_limit %}
+[reboot-limit]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[reboot-limit:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fio_tests %}
+[fio_tests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fio_tests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fstests %}
+[fstests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# Add per-section groups if needed #}
+{% for section in sections|default([]) %}
+[fstests_{{ section | replace('-', '_') }}]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endfor %}
+
+{% elif kdevops_workflow_enable_blktests %}
+[blktests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[blktests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_selftests %}
+[selftests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[selftests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_mmtests %}
+[mmtests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[mmtests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_sysbench %}
+[sysbench]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[sysbench:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ai|default(false)|bool %}
+[ai]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ai:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_minio|default(false)|bool %}
+[minio]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[minio:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_cxl %}
+[cxl]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[cxl:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_pynfs %}
+[pynfs]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[pynfs:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_gitr %}
+[gitr]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[gitr:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ltp %}
+[ltp]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ltp:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_nfstest %}
+[nfstest]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[nfstest:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{% else %}
+{# Non-workflow setup - just use baseline group #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+[service]
+{# Service nodes are typically not needed for declared hosts #}
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_nodes/tasks/main.yml b/playbooks/roles/gen_nodes/tasks/main.yml
index b1a1946f..0fa778bf 100644
--- a/playbooks/roles/gen_nodes/tasks/main.yml
+++ b/playbooks/roles/gen_nodes/tasks/main.yml
@@ -20,6 +20,11 @@
   register: my_group
   changed_when: false
 
+- name: Skip node generation if using declared hosts
+  meta: end_play
+  when:
+    - kdevops_use_declared_hosts|default(false)|bool
+
 - name: Create guestfs directory
   ansible.builtin.file:
     path: "{{ guestfs_path }}"
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates
  2025-08-29 10:31 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain
@ 2025-08-29 10:36   ` Luis Chamberlain
  0 siblings, 0 replies; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:36 UTC (permalink / raw)
  To: Chuck Lever, Daniel Gomez, kdevops

On Fri, Aug 29, 2025 at 03:31:01AM -0700, Luis Chamberlain wrote:
> @@ -67,6 +83,7 @@
>    when:
>      - bootlinux_builder
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file
>    tags: ["hosts"]
> @@ -80,6 +97,7 @@
>      - not kdevops_workflows_dedicated_workflow
>      - ansible_hosts_template.stat.exists
>      - not kdevops_enable_nixos|default(false)|bool
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file for NixOS
>    tags: ['hosts']
> @@ -93,6 +111,7 @@
>      - not kdevops_workflows_dedicated_workflow
>      - ansible_hosts_template.stat.exists
>      - kdevops_enable_nixos|default(false)|bool
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Update Ansible inventory access modification time so make sees it updated
>    ansible.builtin.file:
> @@ -112,6 +131,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_cxl
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file for dedicated pynfs work
>    tags: ["hosts"]
> @@ -125,6 +145,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_pynfs
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file for dedicated gitr workflow
>    tags: ["hosts"]
> @@ -140,6 +161,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_gitr
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate an Ansible inventory file for a dedicated ltp workflow
>    tags: ["hosts"]
> @@ -155,6 +177,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_ltp
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file for dedicated nfstest workflow
>    tags: ["hosts"]
> @@ -170,6 +193,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_nfstest
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Set empty enabled test types list for fstests
>    tags: ["hosts"]
> @@ -178,6 +202,7 @@
>    when:
>      - is_fstests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Check which fstests test types are enabled
>    register: fstests_enabled_test_types_reg
> @@ -202,6 +227,7 @@
>    when:
>      - is_fstests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Now expand the list of enabled fstests for valid configuration sections
>    tags: ["hosts"]
> @@ -213,6 +239,7 @@
>    when:
>      - is_fstests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>      - item.changed
>  
>  - name: Generate the Ansible inventory file for a dedicated fstests setup
> @@ -233,6 +260,7 @@
>    when:
>      - is_fstests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Infer enabled blktests test section types
>    ansible.builtin.set_fact:
> @@ -247,6 +275,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_blktests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Debug inferring block test types
>    ansible.builtin.debug:
> @@ -269,6 +298,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_blktests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Infer enabled selftests test section types
>    ansible.builtin.set_fact:
> @@ -283,6 +313,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_selftests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible inventory file for a dedicated selftests setup
>    tags: ["hosts"]
> @@ -296,6 +327,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_selftests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Collect dynamically supported filesystems
>    vars:
> @@ -338,6 +370,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_sysbench
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  
>  - name: Generate the Ansible hosts file for a dedicated fio-tests setup
> @@ -354,6 +387,7 @@
>      - kdevops_workflow_enable_fio_tests
>      - ansible_hosts_template.stat.exists
>      - not kdevops_enable_nixos|default(false)|bool
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  
>  - name: Infer enabled mmtests test types
> @@ -369,6 +403,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_mmtests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible hosts file for a dedicated mmtests setup
>    tags: ["hosts"]
> @@ -382,6 +417,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - kdevops_workflow_enable_mmtests
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Generate the Ansible hosts file for a dedicated reboot-limit setup
>    tags: ["hosts"]
> @@ -395,6 +431,7 @@
>      - kdevops_workflows_dedicated_workflow
>      - workflows_reboot_limit
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  
>  - name: Load AI nodes configuration for multi-filesystem setup
>    include_vars:
> @@ -426,8 +463,9 @@
>      mode: '0644'
>    when:
>      - kdevops_workflows_dedicated_workflow
> -    - kdevops_workflow_enable_ai
> +    - kdevops_workflow_enable_ai|default(false)|bool
>      - ansible_hosts_template.stat.exists
> +    - not kdevops_use_declared_hosts|default(false)|bool
>  

Sorry, these hunks should have been part of the second patch.

 Luis

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure
  2025-08-29 10:56 [PATCH 0/2] declared hosts support Luis Chamberlain
@ 2025-08-29 10:56 ` Luis Chamberlain
  0 siblings, 0 replies; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:56 UTC (permalink / raw)
  To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain

This adds support for using pre-existing infrastructure (bare metal
servers, pre-provisioned VMs) that already have SSH access configured.
This is useful for environments where kdevops bringup/teardown is not
desired or possible.

Key features:
- New DECLARE_HOSTS CLI parameter to specify existing hosts
- Automatic detection of CLI override to enable declared hosts mode
- SKIP_BRINGUP is automatically selected when using declared hosts
- Hosts can be specified as space or comma-separated list
- For A/B testing workflows, odd-numbered hosts become baseline nodes
  and even-numbered hosts become dev nodes
- New Kconfig.declared_hosts for configuration options
- New declared-hosts.j2 template for generating inventory

Usage:
  make defconfig-xfs_reflink_4k DECLARE_HOSTS='server1 server2 server3'
  make  # Will skip bringup and use the declared hosts

The implementation assumes SSH access is already configured in the user's
~/.ssh/config or via SSH keys. No SSH configuration is generated by kdevops
when using declared hosts.

This feature integrates cleanly with the refactored template system,
using the same hosts.j2 entry point with a dedicated declared-hosts.j2
template for this mode.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 Makefile                                      |   7 +
 kconfigs/Kconfig.bringup                      |   7 +
 kconfigs/Kconfig.declared_hosts               |  65 ++++++
 playbooks/roles/devconfig/defaults/main.yml   |   1 +
 playbooks/roles/devconfig/tasks/main.yml      |  60 +++++
 playbooks/roles/gen_hosts/defaults/main.yml   |   1 +
 playbooks/roles/gen_hosts/tasks/main.yml      |  40 +++-
 .../templates/workflows/declared-hosts.j2     | 219 ++++++++++++++++++
 playbooks/roles/gen_nodes/tasks/main.yml      |   5 +
 9 files changed, 404 insertions(+), 1 deletion(-)
 create mode 100644 kconfigs/Kconfig.declared_hosts
 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2

diff --git a/Makefile b/Makefile
index 83c67340..fcb626ae 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,13 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+
+# Export CLI override variables for Kconfig to detect them
+# Note: We accept DECLARE_HOSTS but export as DECLARED_HOSTS for consistency
+ifdef DECLARE_HOSTS
+export DECLARED_HOSTS := $(DECLARE_HOSTS)
+endif
+
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
diff --git a/kconfigs/Kconfig.bringup b/kconfigs/Kconfig.bringup
index 8caf07be..f7f9735a 100644
--- a/kconfigs/Kconfig.bringup
+++ b/kconfigs/Kconfig.bringup
@@ -9,8 +9,14 @@ config KDEVOPS_ENABLE_NIXOS
 	bool
 	output yaml
 
+# CLI override detection for DECLARED_HOSTS which should enable SKIP_BRINGUP
+config SKIP_BRINGUP_SET_BY_CLI
+	bool
+	default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS)
+
 choice
 	prompt "Node bring up method"
+	default SKIP_BRINGUP if SKIP_BRINGUP_SET_BY_CLI
 	default GUESTFS
 
 config GUESTFS
@@ -80,6 +86,7 @@ config LIBVIRT
 source "kconfigs/Kconfig.guestfs"
 source "kconfigs/Kconfig.nixos"
 source "terraform/Kconfig"
+source "kconfigs/Kconfig.declared_hosts"
 if LIBVIRT
 source "kconfigs/Kconfig.libvirt"
 endif
diff --git a/kconfigs/Kconfig.declared_hosts b/kconfigs/Kconfig.declared_hosts
new file mode 100644
index 00000000..56b56845
--- /dev/null
+++ b/kconfigs/Kconfig.declared_hosts
@@ -0,0 +1,65 @@
+# Configuration for declared hosts that skip bringup process
+
+# CLI override support for DECLARED_HOSTS
+config KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	bool
+	output yaml
+	default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS)
+
+config KDEVOPS_USE_DECLARED_HOSTS
+	bool "Use declared hosts (skip bringup process)"
+	depends on SKIP_BRINGUP || KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	select WORKFLOW_INFER_USER_AND_GROUP
+	output yaml
+	default y if SKIP_BRINGUP
+	default y if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	help
+	  Enable this option to use pre-existing hosts that you have already
+	  configured with SSH access. This is useful for:
+
+	  * Bare metal systems
+	  * Pre-provisioned VMs or cloud instances
+	  * Systems managed by other infrastructure tools
+
+	  When this option is enabled:
+	  - SSH keys will not be generated (assumes you already have access)
+	  - Bringup and teardown operations will be skipped
+	  - User and group settings will be inferred from the target hosts
+	  - You must provide the list of hosts in KDEVOPS_DECLARED_HOSTS
+
+	  This option automatically:
+	  - Selects WORKFLOW_INFER_USER_AND_GROUP to detect the correct
+	    user and group on the target systems
+	  - Assumes SSH access is already configured
+
+if KDEVOPS_USE_DECLARED_HOSTS
+
+config KDEVOPS_DECLARED_HOSTS
+	string "List of declared hosts (space or comma separated)"
+	output yaml
+	default $(shell, ./scripts/append-makefile-vars.sh $(DECLARED_HOSTS)) if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI
+	default ""
+	help
+	  Provide a space or comma-separated list of hostnames or IP addresses
+	  that are already accessible via SSH.
+
+	  Examples:
+	  - Single host: "server1.example.com"
+	  - Multiple hosts: "server1.example.com server2.example.com"
+	  - IP addresses: "192.168.1.10 192.168.1.11"
+	  - Mixed: "server1.example.com 10.0.0.5 server2.local"
+
+	  For A/B testing workflows with baseline/dev configurations, you can
+	  provide an even number of hosts. The odd-numbered hosts will be used
+	  as baseline nodes and even-numbered hosts as dev nodes.
+
+	  Requirements:
+	  - SSH access must already be configured for these hosts
+	  - The user running kdevops must have passwordless SSH access
+	  - The hosts should have the required dependencies installed
+
+	  Note: kdevops assumes you have already configured SSH access to these
+	  hosts in your ~/.ssh/config or have appropriate SSH keys set up.
+	  Ansible will use your existing SSH configuration to connect.
+
+endif # KDEVOPS_USE_DECLARED_HOSTS
diff --git a/playbooks/roles/devconfig/defaults/main.yml b/playbooks/roles/devconfig/defaults/main.yml
index 98dce312..62e72ffa 100644
--- a/playbooks/roles/devconfig/defaults/main.yml
+++ b/playbooks/roles/devconfig/defaults/main.yml
@@ -57,3 +57,4 @@ kdevops_enable_guestfs: false
 guestfs_copy_sources_from_host_to_guest: false
 distro_debian_has_hop1_sources: false
 unattended_upgrades_installed: false
+kdevops_use_declared_hosts: false
diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index fccd1fcf..e93c6078 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -17,6 +17,66 @@
   ansible.builtin.setup:
   tags: always
 
+- name: Determine the actual user's home directory when using inferred user
+  block:
+    - name: Get the current username
+      ansible.builtin.command: whoami
+      register: current_user
+      changed_when: false
+
+    - name: Get home directory for the current user
+      ansible.builtin.shell: |
+        # When workflow_infer_user_and_group is enabled, we need to use the actual SSH user
+        # not the hardcoded 'kdevops' user from group_vars/all
+        actual_user="{{ current_user.stdout }}"
+        if [ -n "$actual_user" ]; then
+          getent passwd "$actual_user" | cut -d: -f6
+        else
+          echo "/root"
+        fi
+      register: inferred_home_dir
+      changed_when: false
+
+    - name: Set data_user and data_home_dir based on inferred user
+      set_fact:
+        data_user: "{{ current_user.stdout }}"
+        data_home_dir: "{{ inferred_home_dir.stdout }}"
+        dev_gitconfig_dest: "{{ inferred_home_dir.stdout }}/.gitconfig"
+        dev_bash_config: "{{ inferred_home_dir.stdout }}/.bashrc"
+        dev_bash_config_hacks_dest: "{{ inferred_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}"
+  when: workflow_infer_user_and_group | default(false) | bool
+
+# Also handle the case when workflow_infer_user_and_group is false but we're using declared hosts
+# In this case, we should still use the SSH user's actual home directory
+- name: Check if using declared hosts without infer setting
+  block:
+    - name: Get the current username for declared hosts
+      ansible.builtin.command: whoami
+      register: current_user_declared
+      changed_when: false
+
+    - name: Get home directory for declared hosts user
+      ansible.builtin.shell: |
+        actual_user="{{ current_user_declared.stdout }}"
+        if [ -n "$actual_user" ]; then
+          getent passwd "$actual_user" | cut -d: -f6
+        else
+          echo "/root"
+        fi
+      register: declared_home_dir
+      changed_when: false
+
+    - name: Override data paths for declared hosts
+      set_fact:
+        data_user: "{{ current_user_declared.stdout }}"
+        data_home_dir: "{{ declared_home_dir.stdout }}"
+        dev_gitconfig_dest: "{{ declared_home_dir.stdout }}/.gitconfig"
+        dev_bash_config: "{{ declared_home_dir.stdout }}/.bashrc"
+        dev_bash_config_hacks_dest: "{{ declared_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}"
+  when:
+    - kdevops_use_declared_hosts
+    - not (workflow_infer_user_and_group | default(false) | bool)
+
 # Update /etc/hostname first so the change gets picked up by the reboot
 # that occurs during the distro-specific tasks
 
diff --git a/playbooks/roles/gen_hosts/defaults/main.yml b/playbooks/roles/gen_hosts/defaults/main.yml
index 4a7515f9..027cbe44 100644
--- a/playbooks/roles/gen_hosts/defaults/main.yml
+++ b/playbooks/roles/gen_hosts/defaults/main.yml
@@ -20,6 +20,7 @@ kdevops_workflows_dedicated_workflow: false
 kdevops_workflow_enable_fstests: false
 kdevops_workflow_enable_blktests: false
 kdevops_workflow_enable_selftests: false
+kdevops_use_declared_hosts: false
 
 kdevops_workflow_enable_cxl: false
 kdevops_workflow_enable_pynfs: false
diff --git a/playbooks/roles/gen_hosts/tasks/main.yml b/playbooks/roles/gen_hosts/tasks/main.yml
index fb63629a..b475e2b0 100644
--- a/playbooks/roles/gen_hosts/tasks/main.yml
+++ b/playbooks/roles/gen_hosts/tasks/main.yml
@@ -10,6 +10,7 @@
       skip: true
   tags: vars
 
+
 - name: Get our user
   ansible.builtin.command: "whoami"
   register: my_user
@@ -39,6 +40,21 @@
     path: "{{ kdevops_hosts_template_full_path }}"
   register: ansible_hosts_template
 
+- name: Generate the Ansible hosts file for declared hosts configuration
+  tags: ['hosts']
+  ansible.builtin.template:
+    src: "{{ kdevops_hosts_template }}"
+    dest: "{{ ansible_cfg_inventory }}"
+    force: true
+    trim_blocks: True
+    lstrip_blocks: True
+    mode: '0644'
+  when:
+    - kdevops_use_declared_hosts
+    - kdevops_declared_hosts is defined
+    - kdevops_declared_hosts|length > 0
+    - ansible_hosts_template.stat.exists
+
 - name: Set fstests config file variable for {{ fstests_fstyp }}
   ansible.builtin.set_fact:
     is_fstests: true
@@ -67,6 +83,7 @@
   when:
     - bootlinux_builder
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file
   tags: ["hosts"]
@@ -80,6 +97,7 @@
     - not kdevops_workflows_dedicated_workflow
     - ansible_hosts_template.stat.exists
     - not kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file for NixOS
   tags: ['hosts']
@@ -93,6 +111,7 @@
     - not kdevops_workflows_dedicated_workflow
     - ansible_hosts_template.stat.exists
     - kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts
 
 - name: Update Ansible inventory access modification time so make sees it updated
   ansible.builtin.file:
@@ -112,6 +131,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_cxl
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file for dedicated pynfs work
   tags: ["hosts"]
@@ -125,6 +145,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_pynfs
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file for dedicated gitr workflow
   tags: ["hosts"]
@@ -140,6 +161,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_gitr
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate an Ansible inventory file for a dedicated ltp workflow
   tags: ["hosts"]
@@ -155,6 +177,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_ltp
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file for dedicated nfstest workflow
   tags: ["hosts"]
@@ -170,6 +193,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_nfstest
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Set empty enabled test types list for fstests
   tags: ["hosts"]
@@ -178,6 +202,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Check which fstests test types are enabled
   register: fstests_enabled_test_types_reg
@@ -202,6 +227,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Now expand the list of enabled fstests for valid configuration sections
   tags: ["hosts"]
@@ -213,6 +239,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
     - item.changed
 
 - name: Generate the Ansible inventory file for a dedicated fstests setup
@@ -233,6 +260,7 @@
   when:
     - is_fstests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Infer enabled blktests test section types
   ansible.builtin.set_fact:
@@ -247,6 +275,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_blktests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Debug inferring block test types
   ansible.builtin.debug:
@@ -269,6 +298,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_blktests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Infer enabled selftests test section types
   ansible.builtin.set_fact:
@@ -283,6 +313,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_selftests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible inventory file for a dedicated selftests setup
   tags: ["hosts"]
@@ -296,6 +327,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_selftests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Collect dynamically supported filesystems
   vars:
@@ -338,6 +370,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_sysbench
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 
 - name: Generate the Ansible hosts file for a dedicated fio-tests setup
@@ -354,6 +387,7 @@
     - kdevops_workflow_enable_fio_tests
     - ansible_hosts_template.stat.exists
     - not kdevops_enable_nixos|default(false)|bool
+    - not kdevops_use_declared_hosts
 
 
 - name: Infer enabled mmtests test types
@@ -369,6 +403,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_mmtests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible hosts file for a dedicated mmtests setup
   tags: ["hosts"]
@@ -382,6 +417,7 @@
     - kdevops_workflows_dedicated_workflow
     - kdevops_workflow_enable_mmtests
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Generate the Ansible hosts file for a dedicated reboot-limit setup
   tags: ["hosts"]
@@ -395,6 +431,7 @@
     - kdevops_workflows_dedicated_workflow
     - workflows_reboot_limit
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Load AI nodes configuration for multi-filesystem setup
   include_vars:
@@ -426,8 +463,9 @@
     mode: '0644'
   when:
     - kdevops_workflows_dedicated_workflow
-    - kdevops_workflow_enable_ai
+    - kdevops_workflow_enable_ai|default(false)|bool
     - ansible_hosts_template.stat.exists
+    - not kdevops_use_declared_hosts
 
 - name: Verify if final host file exists
   ansible.builtin.stat:
diff --git a/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
new file mode 100644
index 00000000..61b51885
--- /dev/null
+++ b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
@@ -0,0 +1,219 @@
+{# Template for declared hosts that skip bringup process #}
+{# This template is used when users have pre-existing infrastructure like:
+   - Bare metal servers
+   - Pre-provisioned VMs
+   - Cloud instances managed outside of kdevops
+   - Any hosts with existing SSH access
+
+   The hosts are provided via kdevops_declared_hosts variable which contains
+   a list of hostnames/IPs that are already accessible via SSH.
+#}
+{# Parse declared hosts - handle both string and list formats #}
+{% if kdevops_declared_hosts is string %}
+{% set parsed_hosts = kdevops_declared_hosts | regex_replace(',', ' ') | split() %}
+{% elif kdevops_declared_hosts is iterable and kdevops_declared_hosts is not string %}
+{% set parsed_hosts = kdevops_declared_hosts %}
+{% else %}
+{% set parsed_hosts = [kdevops_declared_hosts] %}
+{% endif %}
+[all]
+localhost ansible_connection=local
+{# Loop through parsed hosts #}
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[all:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% if kdevops_workflows_dedicated_workflow %}
+{# For workflows, organize hosts into baseline/dev groups for A/B testing #}
+{% if kdevops_baseline_and_dev %}
+[baseline]
+{# Odd-numbered hosts become baseline nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is odd %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+[dev]
+{# Even-numbered hosts become dev nodes #}
+{% for host in parsed_hosts %}
+{% if loop.index is even %}
+{{ host }}
+{% endif %}
+{% endfor %}
+
+[dev:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% else %}
+{# Without A/B testing, all hosts are baseline #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{# Add workflow-specific groups based on enabled workflow #}
+{% if workflows_reboot_limit %}
+[reboot-limit]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[reboot-limit:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fio_tests %}
+[fio_tests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fio_tests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_fstests %}
+[fstests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{# Add per-section groups if needed #}
+{% for section in sections|default([]) %}
+[fstests_{{ section | replace('-', '_') }}]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[fstests_{{ section | replace('-', '_') }}:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endfor %}
+
+{% elif kdevops_workflow_enable_blktests %}
+[blktests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[blktests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_selftests %}
+[selftests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[selftests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_mmtests %}
+[mmtests]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[mmtests:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_sysbench %}
+[sysbench]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[sysbench:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ai|default(false)|bool %}
+[ai]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ai:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_minio|default(false)|bool %}
+[minio]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[minio:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_cxl %}
+[cxl]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[cxl:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_pynfs %}
+[pynfs]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[pynfs:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_gitr %}
+[gitr]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[gitr:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_ltp %}
+[ltp]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[ltp:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+
+{% elif kdevops_workflow_enable_nfstest %}
+[nfstest]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[nfstest:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+{% else %}
+{# Non-workflow setup - just use baseline group #}
+[baseline]
+{% for host in parsed_hosts %}
+{{ host }}
+{% endfor %}
+
+[baseline:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
+{% endif %}
+
+[service]
+{# Service nodes are typically not needed for declared hosts #}
+
+[service:vars]
+ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_nodes/tasks/main.yml b/playbooks/roles/gen_nodes/tasks/main.yml
index b1a1946f..0fa778bf 100644
--- a/playbooks/roles/gen_nodes/tasks/main.yml
+++ b/playbooks/roles/gen_nodes/tasks/main.yml
@@ -20,6 +20,11 @@
   register: my_group
   changed_when: false
 
+- name: Skip node generation if using declared hosts
+  meta: end_play
+  when:
+    - kdevops_use_declared_hosts|default(false)|bool
+
 - name: Create guestfs directory
   ansible.builtin.file:
     path: "{{ guestfs_path }}"
-- 
2.50.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-08-29 10:56 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-29 10:31 [PATCH 0/2] declared hosts support Luis Chamberlain
2025-08-29 10:31 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain
2025-08-29 10:36   ` Luis Chamberlain
2025-08-29 10:31 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
  -- strict thread matches above, loose matches on Subject: below --
2025-08-29 10:56 [PATCH 0/2] declared hosts support Luis Chamberlain
2025-08-29 10:56 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).