* [PATCH 0/2] declared hosts support
@ 2025-08-29 10:56 Luis Chamberlain
2025-08-29 10:56 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain
2025-08-29 10:56 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
0 siblings, 2 replies; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:56 UTC (permalink / raw)
To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain
This v2, just moves the hunks which were left on v1 to v2 where they
were supposed to go and also simplfies the logic by declaring the bools
as false originally on the roles that use them.
Luis Chamberlain (2):
gen_hosts: refactor monolithic hosts.j2 template into per-workflow
templates
declared_hosts: add support for pre-existing infrastructure
Makefile | 7 +
kconfigs/Kconfig.bringup | 7 +
kconfigs/Kconfig.declared_hosts | 65 +++++
playbooks/roles/devconfig/defaults/main.yml | 1 +
playbooks/roles/devconfig/tasks/main.yml | 60 ++++
playbooks/roles/gen_hosts/defaults/main.yml | 1 +
playbooks/roles/gen_hosts/tasks/main.yml | 40 ++-
playbooks/roles/gen_hosts/templates/hosts.j2 | 264 +++---------------
.../roles/gen_hosts/templates/workflows/ai.j2 | 99 +++++++
.../gen_hosts/templates/workflows/blktests.j2 | 58 ++++
.../gen_hosts/templates/workflows/cxl.j2 | 7 +
.../templates/workflows/declared-hosts.j2 | 219 +++++++++++++++
.../gen_hosts/templates/workflows/default.j2 | 62 ++++
.../templates/workflows/fio-tests.j2 | 38 +++
.../gen_hosts/templates/workflows/fstests.j2 | 72 +++++
.../gen_hosts/templates/workflows/gitr.j2 | 41 +++
.../gen_hosts/templates/workflows/ltp.j2 | 41 +++
.../gen_hosts/templates/workflows/mmtests.j2 | 77 +++++
.../gen_hosts/templates/workflows/nfstest.j2 | 41 +++
.../gen_hosts/templates/workflows/pynfs.j2 | 7 +
.../templates/workflows/reboot-limit.j2 | 33 +++
.../templates/workflows/selftests.j2 | 53 ++++
.../gen_hosts/templates/workflows/sysbench.j2 | 53 ++++
playbooks/roles/gen_nodes/tasks/main.yml | 5 +
workflows/ai/Makefile | 3 +-
workflows/blktests/Makefile | 3 +-
workflows/cxl/Makefile | 3 +-
workflows/fio-tests/Makefile | 3 +-
workflows/fstests/Makefile | 3 +-
workflows/gitr/Makefile | 3 +-
workflows/linux/Makefile | 3 +-
workflows/ltp/Makefile | 3 +-
workflows/mmtests/Makefile | 3 +-
workflows/nfstest/Makefile | 3 +-
workflows/pynfs/Makefile | 3 +-
workflows/selftests/Makefile | 3 +-
workflows/sysbench/Makefile | 3 +-
37 files changed, 1145 insertions(+), 245 deletions(-)
create mode 100644 kconfigs/Kconfig.declared_hosts
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2
--
2.50.1
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates 2025-08-29 10:56 [PATCH 0/2] declared hosts support Luis Chamberlain @ 2025-08-29 10:56 ` Luis Chamberlain 2025-08-29 16:42 ` Chuck Lever 2025-08-29 10:56 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain 1 sibling, 1 reply; 5+ messages in thread From: Luis Chamberlain @ 2025-08-29 10:56 UTC (permalink / raw) To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain The hosts.j2 template had become unwieldy spaghetti code with all workflows mixed together in a single file, making it difficult to read, maintain, and extend. This was particularly problematic when attempting to add new features. This refactoring: - Splits the monolithic hosts.j2 into individual workflow templates under playbooks/roles/gen_hosts/templates/workflows/ - Creates dedicated templates for each workflow (fstests, blktests, selftests, mmtests, sysbench, ai, fio-tests, etc.) - Converts the main hosts.j2 to use Jinja2 includes for cleaner organization - Maintains exact same functionality with improved maintainability - Removes workflow-specific template overrides from all workflow Makefiles as they now all use the unified hosts.j2 template Benefits: - Each workflow template is now self-contained and easy to understand - Adding new workflows no longer requires navigating through hundreds of lines of unrelated configuration - Template logic is easier to debug and test - All workflows now use a consistent template system This is a pure refactoring with no functional changes. The generated hosts files remain identical to the previous implementation. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- playbooks/roles/gen_hosts/templates/hosts.j2 | 264 +++--------------- .../roles/gen_hosts/templates/workflows/ai.j2 | 99 +++++++ .../gen_hosts/templates/workflows/blktests.j2 | 58 ++++ .../gen_hosts/templates/workflows/cxl.j2 | 7 + .../gen_hosts/templates/workflows/default.j2 | 62 ++++ .../templates/workflows/fio-tests.j2 | 38 +++ .../gen_hosts/templates/workflows/fstests.j2 | 72 +++++ .../gen_hosts/templates/workflows/gitr.j2 | 41 +++ .../gen_hosts/templates/workflows/ltp.j2 | 41 +++ .../gen_hosts/templates/workflows/mmtests.j2 | 77 +++++ .../gen_hosts/templates/workflows/nfstest.j2 | 41 +++ .../gen_hosts/templates/workflows/pynfs.j2 | 7 + .../templates/workflows/reboot-limit.j2 | 33 +++ .../templates/workflows/selftests.j2 | 53 ++++ .../gen_hosts/templates/workflows/sysbench.j2 | 53 ++++ workflows/ai/Makefile | 3 +- workflows/blktests/Makefile | 3 +- workflows/cxl/Makefile | 3 +- workflows/fio-tests/Makefile | 3 +- workflows/fstests/Makefile | 3 +- workflows/gitr/Makefile | 3 +- workflows/linux/Makefile | 3 +- workflows/ltp/Makefile | 3 +- workflows/mmtests/Makefile | 3 +- workflows/nfstest/Makefile | 3 +- workflows/pynfs/Makefile | 3 +- workflows/selftests/Makefile | 3 +- workflows/sysbench/Makefile | 3 +- 28 files changed, 741 insertions(+), 244 deletions(-) create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 diff --git a/playbooks/roles/gen_hosts/templates/hosts.j2 b/playbooks/roles/gen_hosts/templates/hosts.j2 index 0e896481..2f2f5d41 100644 --- a/playbooks/roles/gen_hosts/templates/hosts.j2 +++ b/playbooks/roles/gen_hosts/templates/hosts.j2 @@ -5,242 +5,44 @@ proper identation. We don't need identation for the ansible hosts file. Each workflow which has its own custom ansible host file generated should use its own jinja2 template file and define its own ansible task for its generation. #} -{% if kdevops_workflows_dedicated_workflow %} +{% if kdevops_declared_hosts is defined and kdevops_declared_hosts %} +{# Use declared hosts that skip bringup process - for bare metal or pre-existing infrastructure #} +{% include 'workflows/declared-hosts.j2' %} +{% elif kdevops_workflows_dedicated_workflow %} +{# Include workflow-specific template based on enabled workflow. Each workflow has its own + template file under workflows/ directory for better maintainability and modularity #} {% if workflows_reboot_limit %} -[all] -localhost ansible_connection=local -{{ kdevops_host_prefix }}-reboot-limit -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-reboot-limit-dev -{% endif %} - -[all:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -[baseline] -{{ kdevops_host_prefix }}-reboot-limit - -[baseline:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% if kdevops_baseline_and_dev %} -[dev] -{{ kdevops_host_prefix }}-reboot-limit-dev - -[dev:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% endif %} -[reboot-limit] -{{ kdevops_host_prefix }}-reboot-limit -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-reboot-limit-dev -{% endif %} - -[reboot-limit:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% include 'workflows/reboot-limit.j2' %} {% elif kdevops_workflow_enable_fio_tests %} -[all] -localhost ansible_connection=local -{{ kdevops_host_prefix }}-fio-tests -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-fio-tests-dev -{% endif %} - -[all:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -[baseline] -{{ kdevops_host_prefix }}-fio-tests - -[baseline:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% if kdevops_baseline_and_dev %} -[dev] -{{ kdevops_host_prefix }}-fio-tests-dev - -[dev:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% endif %} -[fio_tests] -{{ kdevops_host_prefix }}-fio-tests -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-fio-tests-dev -{% endif %} - -[fio_tests:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -[service] - -[service:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% elif kdevops_workflow_enable_ai %} -{% if ai_enable_multifs_testing|default(false)|bool %} -{# Multi-filesystem section-based hosts #} -[all] -localhost ansible_connection=local -{% for node in all_generic_nodes %} -{{ node }} -{% endfor %} - -[all:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -[baseline] -{% for node in all_generic_nodes %} -{% if not node.endswith('-dev') %} -{{ node }} -{% endif %} -{% endfor %} - -[baseline:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% if kdevops_baseline_and_dev %} -[dev] -{% for node in all_generic_nodes %} -{% if node.endswith('-dev') %} -{{ node }} -{% endif %} -{% endfor %} - -[dev:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% endif %} -[ai] -{% for node in all_generic_nodes %} -{{ node }} -{% endfor %} - -[ai:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{# Individual section groups for multi-filesystem testing #} -{% set section_names = [] %} -{% for node in all_generic_nodes %} -{% if not node.endswith('-dev') %} -{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %} -{% if section != kdevops_host_prefix + '-ai' %} -{% if section_names.append(section) %}{% endif %} -{% endif %} -{% endif %} -{% endfor %} - -{% for section in section_names %} -[ai_{{ section | replace('-', '_') }}] -{{ kdevops_host_prefix }}-ai-{{ section }} -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-ai-{{ section }}-dev -{% endif %} - -[ai_{{ section | replace('-', '_') }}:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% endfor %} -{% else %} -{# Single filesystem hosts (original behavior) #} -[all] -localhost ansible_connection=local -{{ kdevops_host_prefix }}-ai -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-ai-dev -{% endif %} - -[all:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -[baseline] -{{ kdevops_host_prefix }}-ai - -[baseline:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% if kdevops_baseline_and_dev %} -[dev] -{{ kdevops_host_prefix }}-ai-dev - -[dev:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% endif %} -[ai] -{{ kdevops_host_prefix }}-ai -{% if kdevops_baseline_and_dev %} -{{ kdevops_host_prefix }}-ai-dev -{% endif %} - -[ai:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} +{% include 'workflows/fio-tests.j2' %} +{% elif kdevops_workflow_enable_fstests %} +{% include 'workflows/fstests.j2' %} +{% elif kdevops_workflow_enable_blktests %} +{% include 'workflows/blktests.j2' %} +{% elif kdevops_workflow_enable_selftests %} +{% include 'workflows/selftests.j2' %} +{% elif kdevops_workflow_enable_mmtests %} +{% include 'workflows/mmtests.j2' %} +{% elif kdevops_workflow_enable_sysbench %} +{% include 'workflows/sysbench.j2' %} +{% elif kdevops_workflow_enable_cxl %} +{% include 'workflows/cxl.j2' %} +{% elif kdevops_workflow_enable_pynfs %} +{% include 'workflows/pynfs.j2' %} +{% elif kdevops_workflow_enable_gitr %} +{% include 'workflows/gitr.j2' %} +{% elif kdevops_workflow_enable_ltp %} +{% include 'workflows/ltp.j2' %} +{% elif kdevops_workflow_enable_nfstest %} +{% include 'workflows/nfstest.j2' %} +{% elif kdevops_workflow_enable_ai|default(false)|bool %} +{% include 'workflows/ai.j2' %} {% else %} [all] localhost ansible_connection=local write-your-own-template-for-your-workflow-and-task {% endif %} {% else %} -[all] -localhost ansible_connection=local -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }} -{% endif %} -{% if kdevops_baseline_and_dev == True %} -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }}-dev -{% endif %} -{% endif %} -{% if kdevops_enable_iscsi %} -{{ kdevops_host_prefix }}-iscsi -{% endif %} -{% if kdevops_nfsd_enable %} -{{ kdevops_host_prefix }}-nfsd -{% endif %} -[all:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -[baseline] -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }} -{% endif %} -[baseline:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -[dev] -{% if kdevops_baseline_and_dev %} -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }}-dev -{% endif %} -{% endif %} -[dev:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_enable_iscsi %} -[iscsi] -{{ kdevops_host_prefix }}-iscsi -[iscsi:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} -{% if kdevops_nfsd_enable %} -[nfsd] -{{ kdevops_host_prefix }}-nfsd -[nfsd:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} -[service] -{% if kdevops_enable_iscsi %} -{{ kdevops_host_prefix }}-iscsi -{% endif %} -{% if kdevops_nfsd_enable %} -{{ kdevops_host_prefix }}-nfsd -{% endif %} -[service:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} +{# Default non-workflow template #} +{% include 'workflows/default.j2' %} +{% endif %} \ No newline at end of file diff --git a/playbooks/roles/gen_hosts/templates/workflows/ai.j2 b/playbooks/roles/gen_hosts/templates/workflows/ai.j2 new file mode 100644 index 00000000..d0914436 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/ai.j2 @@ -0,0 +1,99 @@ +{# Workflow template for AI #} +{% if ai_enable_multifs_testing|default(false)|bool %} +{# Multi-filesystem section-based hosts #} +[all] +localhost ansible_connection=local +{% for node in all_generic_nodes %} +{{ node }} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for node in all_generic_nodes %} +{% if not node.endswith('-dev') %} +{{ node }} +{% endif %} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for node in all_generic_nodes %} +{% if node.endswith('-dev') %} +{{ node }} +{% endif %} +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[ai] +{% for node in all_generic_nodes %} +{{ node }} +{% endfor %} + +[ai:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{# Individual section groups for multi-filesystem testing #} +{% set section_names = [] %} +{% for node in all_generic_nodes %} +{% if not node.endswith('-dev') %} +{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %} +{% if section != kdevops_host_prefix + '-ai' %} +{% if section_names.append(section) %}{% endif %} +{% endif %} +{% endif %} +{% endfor %} + +{% for section in section_names %} +[ai_{{ section | replace('-', '_') }}] +{{ kdevops_host_prefix }}-ai-{{ section }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-ai-{{ section }}-dev +{% endif %} + +[ai_{{ section | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} +{% else %} +{# Single filesystem hosts (original behavior) #} +[all] +localhost ansible_connection=local +{{ kdevops_host_prefix }}-ai +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-ai-dev +{% endif %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{{ kdevops_host_prefix }}-ai + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{{ kdevops_host_prefix }}-ai-dev + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[ai] +{{ kdevops_host_prefix }}-ai +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-ai-dev +{% endif %} + +[ai:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} diff --git a/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 new file mode 100644 index 00000000..eea3eef3 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 @@ -0,0 +1,58 @@ +{# Workflow template for blktests #} +[all] +localhost ansible_connection=local +{% for test_type in blktests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for test_type in blktests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for test_type in blktests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[blktests] +{% for test_type in blktests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} + +[blktests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% for test_type in blktests_enabled_test_types %} +[blktests_{{ test_type | replace('-', '_') }}] +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} + +[blktests_{{ test_type | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} + +[service] + +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 new file mode 100644 index 00000000..53790f29 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 @@ -0,0 +1,7 @@ +{# Workflow template for CXL #} +[all] +localhost ansible_connection=local +write-your-own-template-for-cxl-workflow + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/default.j2 b/playbooks/roles/gen_hosts/templates/workflows/default.j2 new file mode 100644 index 00000000..86619309 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/default.j2 @@ -0,0 +1,62 @@ +{# Default template for non-workflow setups #} +[all] +localhost ansible_connection=local +{% if kdevops_enable_nixos|default(false) %} +{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 +{% else %} +{{ kdevops_host_prefix }} +{% endif %} +{% if kdevops_baseline_and_dev == True %} +{% if kdevops_enable_nixos|default(false) %} +{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 +{% else %} +{{ kdevops_host_prefix }}-dev +{% endif %} +{% endif %} +{% if kdevops_enable_iscsi %} +{{ kdevops_host_prefix }}-iscsi +{% endif %} +{% if kdevops_nfsd_enable %} +{{ kdevops_host_prefix }}-nfsd +{% endif %} +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +[baseline] +{% if kdevops_enable_nixos|default(false) %} +{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 +{% else %} +{{ kdevops_host_prefix }} +{% endif %} +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +[dev] +{% if kdevops_baseline_and_dev %} +{% if kdevops_enable_nixos|default(false) %} +{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 +{% else %} +{{ kdevops_host_prefix }}-dev +{% endif %} +{% endif %} +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% if kdevops_enable_iscsi %} +[iscsi] +{{ kdevops_host_prefix }}-iscsi +[iscsi:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} +{% if kdevops_nfsd_enable %} +[nfsd] +{{ kdevops_host_prefix }}-nfsd +[nfsd:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} +[service] +{% if kdevops_enable_iscsi %} +{{ kdevops_host_prefix }}-iscsi +{% endif %} +{% if kdevops_nfsd_enable %} +{{ kdevops_host_prefix }}-nfsd +{% endif %} +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 new file mode 100644 index 00000000..548941a0 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 @@ -0,0 +1,38 @@ +{# Workflow template for fio-tests #} +[all] +localhost ansible_connection=local +{{ kdevops_host_prefix }}-fio-tests +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-fio-tests-dev +{% endif %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{{ kdevops_host_prefix }}-fio-tests + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{{ kdevops_host_prefix }}-fio-tests-dev + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[fio_tests] +{{ kdevops_host_prefix }}-fio-tests +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-fio-tests-dev +{% endif %} + +[fio_tests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[service] + +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 new file mode 100644 index 00000000..600d125f --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 @@ -0,0 +1,72 @@ +{# Workflow template for fstests #} +[all] +localhost ansible_connection=local +{% for node_section in fstests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ node_section }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ node_section }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for node_section in fstests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ node_section }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for node_section in fstests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ node_section }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[fstests] +{% for node_section in fstests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ node_section }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ node_section }}-dev +{% endif %} +{% endfor %} + +[fstests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% for section in sections %} +[fstests_{{ section | replace('-', '_') }}] +{{ kdevops_host_prefix }}-{{ section }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ section }}-dev +{% endif %} + +[fstests_{{ section | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} + +[nfsd] +{% if kdevops_nfsd_enable %} +{{ kdevops_host_prefix }}-nfsd +{% endif %} + +[nfsd:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[service] +{% if kdevops_enable_iscsi %} +{{ kdevops_host_prefix }}-iscsi +{% endif %} +{% if kdevops_nfsd_enable %} +{{ kdevops_host_prefix }}-nfsd +{% endif %} + +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 new file mode 100644 index 00000000..86ee9326 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 @@ -0,0 +1,41 @@ +{# Workflow template for gitr #} +[all] +localhost ansible_connection=local +{% for host in gitr_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for host in gitr_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for host in gitr_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[gitr] +{% for host in gitr_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[gitr:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 new file mode 100644 index 00000000..fb120828 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 @@ -0,0 +1,41 @@ +{# Workflow template for ltp #} +[all] +localhost ansible_connection=local +{% for host in ltp_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for host in ltp_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for host in ltp_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[ltp] +{% for host in ltp_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[ltp:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 new file mode 100644 index 00000000..d796cbe6 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 @@ -0,0 +1,77 @@ +{# Workflow template for mmtests #} +[all] +localhost ansible_connection=local +{% if mmtests_enabled_test_types %} +{% for test_type in mmtests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} +{% else %} +{{ kdevops_host_prefix }}-mmtests +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-mmtests-dev +{% endif %} +{% endif %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% if mmtests_enabled_test_types %} +{% for test_type in mmtests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% endfor %} +{% else %} +{{ kdevops_host_prefix }}-mmtests +{% endif %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% if mmtests_enabled_test_types %} +{% for test_type in mmtests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endfor %} +{% else %} +{{ kdevops_host_prefix }}-mmtests-dev +{% endif %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[mmtests] +{% if mmtests_enabled_test_types %} +{% for test_type in mmtests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} +{% else %} +{{ kdevops_host_prefix }}-mmtests +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-mmtests-dev +{% endif %} +{% endif %} + +[mmtests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if mmtests_enabled_test_types %} +{% for test_type in mmtests_enabled_test_types %} +[mmtests_{{ test_type | replace('-', '_') }}] +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} + +[mmtests_{{ test_type | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} +{% endif %} diff --git a/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 new file mode 100644 index 00000000..34aa7dfe --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 @@ -0,0 +1,41 @@ +{# Workflow template for nfstest #} +[all] +localhost ansible_connection=local +{% for host in nfstest_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for host in nfstest_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for host in nfstest_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[nfstest] +{% for host in nfstest_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[nfstest:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 new file mode 100644 index 00000000..6145b1a0 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 @@ -0,0 +1,7 @@ +{# Workflow template for pynfs #} +[all] +localhost ansible_connection=local +write-your-own-template-for-pynfs-workflow + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 new file mode 100644 index 00000000..07bd6f80 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 @@ -0,0 +1,33 @@ +{# Workflow template for reboot-limit #} +[all] +localhost ansible_connection=local +{{ kdevops_host_prefix }}-reboot-limit +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-reboot-limit-dev +{% endif %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{{ kdevops_host_prefix }}-reboot-limit + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{{ kdevops_host_prefix }}-reboot-limit-dev + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[reboot-limit] +{{ kdevops_host_prefix }}-reboot-limit +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-reboot-limit-dev +{% endif %} + +[reboot-limit:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 new file mode 100644 index 00000000..4ef598c7 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 @@ -0,0 +1,53 @@ +{# Workflow template for selftests #} +[all] +localhost ansible_connection=local +{% for test_type in selftests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for test_type in selftests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for test_type in selftests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[selftests] +{% for test_type in selftests_enabled_test_types %} +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} +{% endfor %} + +[selftests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% for test_type in selftests_enabled_test_types %} +[selftests_{{ test_type | replace('-', '_') }}] +{{ kdevops_host_prefix }}-{{ test_type }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test_type }}-dev +{% endif %} + +[selftests_{{ test_type | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} diff --git a/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 new file mode 100644 index 00000000..34f22a83 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 @@ -0,0 +1,53 @@ +{# Workflow template for sysbench #} +[all] +localhost ansible_connection=local +{% for test in enabled_sysbench_tests %} +{{ kdevops_host_prefix }}-{{ test }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test }}-dev +{% endif %} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for test in enabled_sysbench_tests %} +{{ kdevops_host_prefix }}-{{ test }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_baseline_and_dev %} +[dev] +{% for test in enabled_sysbench_tests %} +{{ kdevops_host_prefix }}-{{ test }}-dev +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endif %} +[sysbench] +{% for test in enabled_sysbench_tests %} +{{ kdevops_host_prefix }}-{{ test }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test }}-dev +{% endif %} +{% endfor %} + +[sysbench:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% for test in enabled_sysbench_tests %} +[sysbench_{{ test | replace('-', '_') }}] +{{ kdevops_host_prefix }}-{{ test }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ test }}-dev +{% endif %} + +[sysbench_{{ test | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% endfor %} diff --git a/workflows/ai/Makefile b/workflows/ai/Makefile index 1c297edd..27dea382 100644 --- a/workflows/ai/Makefile +++ b/workflows/ai/Makefile @@ -4,7 +4,8 @@ PHONY += ai-tests ai-tests-baseline ai-tests-dev PHONY += ai-tests-results ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := hosts.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := hosts.j2 endif export AI_DATA_TARGET := $(subst ",,$(CONFIG_AI_BENCHMARK_RESULTS_DIR)) diff --git a/workflows/blktests/Makefile b/workflows/blktests/Makefile index 11cd9a65..97649026 100644 --- a/workflows/blktests/Makefile +++ b/workflows/blktests/Makefile @@ -12,7 +12,8 @@ ID=$(shell id -u) BLKTESTS_ARGS := ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := blktests.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := blktests.j2 endif BLKTESTS_GIT:=$(subst ",,$(CONFIG_BLKTESTS_GIT)) diff --git a/workflows/cxl/Makefile b/workflows/cxl/Makefile index bb130057..feabe197 100644 --- a/workflows/cxl/Makefile +++ b/workflows/cxl/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: copyleft-next-0.3.1 -export KDEVOPS_HOSTS_TEMPLATE := cxl.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := cxl.j2 CXL_ARGS := CXL_ARGS += ndctl_git='$(subst ",,$(CONFIG_NDCTL_GIT))' diff --git a/workflows/fio-tests/Makefile b/workflows/fio-tests/Makefile index 5eb2ccbd..20a5d192 100644 --- a/workflows/fio-tests/Makefile +++ b/workflows/fio-tests/Makefile @@ -1,5 +1,6 @@ ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2 endif fio-tests: diff --git a/workflows/fstests/Makefile b/workflows/fstests/Makefile index d2e5c636..989c564b 100644 --- a/workflows/fstests/Makefile +++ b/workflows/fstests/Makefile @@ -9,7 +9,8 @@ FSTESTS_BASELINE_EXTRA := export FSTYP:=$(subst ",,$(CONFIG_FSTESTS_FSTYP)) ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := fstests.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := fstests.j2 endif FSTESTS_ARGS += fstests_fstyp='$(FSTYP)' diff --git a/workflows/gitr/Makefile b/workflows/gitr/Makefile index c685395a..eadcdac1 100644 --- a/workflows/gitr/Makefile +++ b/workflows/gitr/Makefile @@ -1,5 +1,6 @@ ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := gitr.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := gitr.j2 endif GITR_MNT:=$(subst ",,$(CONFIG_GITR_MNT)) diff --git a/workflows/linux/Makefile b/workflows/linux/Makefile index 30b123f9..60d755b3 100644 --- a/workflows/linux/Makefile +++ b/workflows/linux/Makefile @@ -14,7 +14,8 @@ TREE_CONFIG:=config-$(TREE_REF)-pure-iomap endif ifeq (y,$(CONFIG_BOOTLINUX_BUILDER)) -KDEVOPS_HOSTS_TEMPLATE=builder.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# KDEVOPS_HOSTS_TEMPLATE=builder.j2 endif # Describes the Linux clone diff --git a/workflows/ltp/Makefile b/workflows/ltp/Makefile index 767465bc..b3c3319b 100644 --- a/workflows/ltp/Makefile +++ b/workflows/ltp/Makefile @@ -1,5 +1,6 @@ ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := ltp.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := ltp.j2 endif LTP_REPO:=$(subst ",,$(CONFIG_LTP_REPO)) diff --git a/workflows/mmtests/Makefile b/workflows/mmtests/Makefile index b65d256b..186dc8cb 100644 --- a/workflows/mmtests/Makefile +++ b/workflows/mmtests/Makefile @@ -1,7 +1,8 @@ MMTESTS_ARGS := ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2 endif mmtests: diff --git a/workflows/nfstest/Makefile b/workflows/nfstest/Makefile index bbfd3f64..bc880b6d 100644 --- a/workflows/nfstest/Makefile +++ b/workflows/nfstest/Makefile @@ -1,5 +1,6 @@ ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2 endif # CONFIG_WORKFLOWS_DEDICATED_WORKFLOW ifeq (y,$(CONFIG_NFSTEST_USE_KDEVOPS_NFSD)) diff --git a/workflows/pynfs/Makefile b/workflows/pynfs/Makefile index 2f3ff97b..b7bb12bb 100644 --- a/workflows/pynfs/Makefile +++ b/workflows/pynfs/Makefile @@ -1,5 +1,6 @@ ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2 endif PYNFS_GIT:=$(subst ",,$(CONFIG_PYNFS_GIT)) diff --git a/workflows/selftests/Makefile b/workflows/selftests/Makefile index d3b7044c..4ec691f3 100644 --- a/workflows/selftests/Makefile +++ b/workflows/selftests/Makefile @@ -3,7 +3,8 @@ SELFTESTS_ARGS := ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := selftests.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := selftests.j2 endif SELFTESTS_DYNAMIC_RUNTIME_VARS := "kdevops_run_selftests": True diff --git a/workflows/sysbench/Makefile b/workflows/sysbench/Makefile index daf7bc75..5f484691 100644 --- a/workflows/sysbench/Makefile +++ b/workflows/sysbench/Makefile @@ -1,7 +1,8 @@ PHONY += sysbench sysbench-test sysbench-telemetry sysbench-help-menu ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) -export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2 +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates +# export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2 endif TAGS_SYSBENCH_RUN := db_start -- 2.50.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates 2025-08-29 10:56 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain @ 2025-08-29 16:42 ` Chuck Lever 0 siblings, 0 replies; 5+ messages in thread From: Chuck Lever @ 2025-08-29 16:42 UTC (permalink / raw) To: Luis Chamberlain, Daniel Gomez, kdevops On 8/29/25 6:56 AM, Luis Chamberlain wrote: > The hosts.j2 template had become unwieldy spaghetti code with all workflows > mixed together in a single file, making it difficult to read, maintain, and > extend. This was particularly problematic when attempting to add new features. > > This refactoring: > - Splits the monolithic hosts.j2 into individual workflow templates under > playbooks/roles/gen_hosts/templates/workflows/ > - Creates dedicated templates for each workflow (fstests, blktests, selftests, > mmtests, sysbench, ai, fio-tests, etc.) > - Converts the main hosts.j2 to use Jinja2 includes for cleaner organization > - Maintains exact same functionality with improved maintainability > - Removes workflow-specific template overrides from all workflow Makefiles > as they now all use the unified hosts.j2 template > > Benefits: > - Each workflow template is now self-contained and easy to understand > - Adding new workflows no longer requires navigating through hundreds of > lines of unrelated configuration > - Template logic is easier to debug and test > - All workflows now use a consistent template system > > This is a pure refactoring with no functional changes. The generated hosts > files remain identical to the previous implementation. > > Generated-by: Claude AI > Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> > --- > playbooks/roles/gen_hosts/templates/hosts.j2 | 264 +++--------------- > .../roles/gen_hosts/templates/workflows/ai.j2 | 99 +++++++ > .../gen_hosts/templates/workflows/blktests.j2 | 58 ++++ > .../gen_hosts/templates/workflows/cxl.j2 | 7 + > .../gen_hosts/templates/workflows/default.j2 | 62 ++++ > .../templates/workflows/fio-tests.j2 | 38 +++ > .../gen_hosts/templates/workflows/fstests.j2 | 72 +++++ > .../gen_hosts/templates/workflows/gitr.j2 | 41 +++ > .../gen_hosts/templates/workflows/ltp.j2 | 41 +++ > .../gen_hosts/templates/workflows/mmtests.j2 | 77 +++++ > .../gen_hosts/templates/workflows/nfstest.j2 | 41 +++ > .../gen_hosts/templates/workflows/pynfs.j2 | 7 + > .../templates/workflows/reboot-limit.j2 | 33 +++ > .../templates/workflows/selftests.j2 | 53 ++++ > .../gen_hosts/templates/workflows/sysbench.j2 | 53 ++++ > workflows/ai/Makefile | 3 +- > workflows/blktests/Makefile | 3 +- > workflows/cxl/Makefile | 3 +- > workflows/fio-tests/Makefile | 3 +- > workflows/fstests/Makefile | 3 +- > workflows/gitr/Makefile | 3 +- > workflows/linux/Makefile | 3 +- > workflows/ltp/Makefile | 3 +- > workflows/mmtests/Makefile | 3 +- > workflows/nfstest/Makefile | 3 +- > workflows/pynfs/Makefile | 3 +- > workflows/selftests/Makefile | 3 +- > workflows/sysbench/Makefile | 3 +- > 28 files changed, 741 insertions(+), 244 deletions(-) > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2 > create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 > > diff --git a/playbooks/roles/gen_hosts/templates/hosts.j2 b/playbooks/roles/gen_hosts/templates/hosts.j2 > index 0e896481..2f2f5d41 100644 > --- a/playbooks/roles/gen_hosts/templates/hosts.j2 > +++ b/playbooks/roles/gen_hosts/templates/hosts.j2 > @@ -5,242 +5,44 @@ proper identation. We don't need identation for the ansible hosts file. > Each workflow which has its own custom ansible host file generated should use > its own jinja2 template file and define its own ansible task for its generation. > #} > -{% if kdevops_workflows_dedicated_workflow %} > +{% if kdevops_declared_hosts is defined and kdevops_declared_hosts %} > +{# Use declared hosts that skip bringup process - for bare metal or pre-existing infrastructure #} > +{% include 'workflows/declared-hosts.j2' %} > +{% elif kdevops_workflows_dedicated_workflow %} > +{# Include workflow-specific template based on enabled workflow. Each workflow has its own > + template file under workflows/ directory for better maintainability and modularity #} > {% if workflows_reboot_limit %} > -[all] > -localhost ansible_connection=local > -{{ kdevops_host_prefix }}-reboot-limit > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-reboot-limit-dev > -{% endif %} > - > -[all:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -[baseline] > -{{ kdevops_host_prefix }}-reboot-limit > - > -[baseline:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% if kdevops_baseline_and_dev %} > -[dev] > -{{ kdevops_host_prefix }}-reboot-limit-dev > - > -[dev:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% endif %} > -[reboot-limit] > -{{ kdevops_host_prefix }}-reboot-limit > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-reboot-limit-dev > -{% endif %} > - > -[reboot-limit:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +{% include 'workflows/reboot-limit.j2' %} > {% elif kdevops_workflow_enable_fio_tests %} > -[all] > -localhost ansible_connection=local > -{{ kdevops_host_prefix }}-fio-tests > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-fio-tests-dev > -{% endif %} > - > -[all:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -[baseline] > -{{ kdevops_host_prefix }}-fio-tests > - > -[baseline:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% if kdevops_baseline_and_dev %} > -[dev] > -{{ kdevops_host_prefix }}-fio-tests-dev > - > -[dev:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% endif %} > -[fio_tests] > -{{ kdevops_host_prefix }}-fio-tests > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-fio-tests-dev > -{% endif %} > - > -[fio_tests:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -[service] > - > -[service:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% elif kdevops_workflow_enable_ai %} > -{% if ai_enable_multifs_testing|default(false)|bool %} > -{# Multi-filesystem section-based hosts #} > -[all] > -localhost ansible_connection=local > -{% for node in all_generic_nodes %} > -{{ node }} > -{% endfor %} > - > -[all:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -[baseline] > -{% for node in all_generic_nodes %} > -{% if not node.endswith('-dev') %} > -{{ node }} > -{% endif %} > -{% endfor %} > - > -[baseline:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% if kdevops_baseline_and_dev %} > -[dev] > -{% for node in all_generic_nodes %} > -{% if node.endswith('-dev') %} > -{{ node }} > -{% endif %} > -{% endfor %} > - > -[dev:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% endif %} > -[ai] > -{% for node in all_generic_nodes %} > -{{ node }} > -{% endfor %} > - > -[ai:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{# Individual section groups for multi-filesystem testing #} > -{% set section_names = [] %} > -{% for node in all_generic_nodes %} > -{% if not node.endswith('-dev') %} > -{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %} > -{% if section != kdevops_host_prefix + '-ai' %} > -{% if section_names.append(section) %}{% endif %} > -{% endif %} > -{% endif %} > -{% endfor %} > - > -{% for section in section_names %} > -[ai_{{ section | replace('-', '_') }}] > -{{ kdevops_host_prefix }}-ai-{{ section }} > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-ai-{{ section }}-dev > -{% endif %} > - > -[ai_{{ section | replace('-', '_') }}:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% endfor %} > -{% else %} > -{# Single filesystem hosts (original behavior) #} > -[all] > -localhost ansible_connection=local > -{{ kdevops_host_prefix }}-ai > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-ai-dev > -{% endif %} > - > -[all:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -[baseline] > -{{ kdevops_host_prefix }}-ai > - > -[baseline:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% if kdevops_baseline_and_dev %} > -[dev] > -{{ kdevops_host_prefix }}-ai-dev > - > -[dev:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > - > -{% endif %} > -[ai] > -{{ kdevops_host_prefix }}-ai > -{% if kdevops_baseline_and_dev %} > -{{ kdevops_host_prefix }}-ai-dev > -{% endif %} > - > -[ai:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% endif %} > +{% include 'workflows/fio-tests.j2' %} > +{% elif kdevops_workflow_enable_fstests %} > +{% include 'workflows/fstests.j2' %} > +{% elif kdevops_workflow_enable_blktests %} > +{% include 'workflows/blktests.j2' %} > +{% elif kdevops_workflow_enable_selftests %} > +{% include 'workflows/selftests.j2' %} > +{% elif kdevops_workflow_enable_mmtests %} > +{% include 'workflows/mmtests.j2' %} > +{% elif kdevops_workflow_enable_sysbench %} > +{% include 'workflows/sysbench.j2' %} > +{% elif kdevops_workflow_enable_cxl %} > +{% include 'workflows/cxl.j2' %} > +{% elif kdevops_workflow_enable_pynfs %} > +{% include 'workflows/pynfs.j2' %} > +{% elif kdevops_workflow_enable_gitr %} > +{% include 'workflows/gitr.j2' %} > +{% elif kdevops_workflow_enable_ltp %} > +{% include 'workflows/ltp.j2' %} > +{% elif kdevops_workflow_enable_nfstest %} > +{% include 'workflows/nfstest.j2' %} > +{% elif kdevops_workflow_enable_ai|default(false)|bool %} > +{% include 'workflows/ai.j2' %} Seems like a smarter approach here would be to have the gen_hosts playbook's "ansible.builtin.template" step select the .j2 file to templatize. Just construct the template's pathname based on the name of the workflow. Then you don't have to add a new "if" arm when adding a new workflow template. > {% else %} > [all] > localhost ansible_connection=local > write-your-own-template-for-your-workflow-and-task > {% endif %} > {% else %} > -[all] > -localhost ansible_connection=local > -{% if kdevops_enable_nixos|default(false) %} > -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 > -{% else %} > -{{ kdevops_host_prefix }} > -{% endif %} > -{% if kdevops_baseline_and_dev == True %} > -{% if kdevops_enable_nixos|default(false) %} > -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 > -{% else %} > -{{ kdevops_host_prefix }}-dev > -{% endif %} > -{% endif %} > -{% if kdevops_enable_iscsi %} > -{{ kdevops_host_prefix }}-iscsi > -{% endif %} > -{% if kdevops_nfsd_enable %} > -{{ kdevops_host_prefix }}-nfsd > -{% endif %} > -[all:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -[baseline] > -{% if kdevops_enable_nixos|default(false) %} > -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 > -{% else %} > -{{ kdevops_host_prefix }} > -{% endif %} > -[baseline:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -[dev] > -{% if kdevops_baseline_and_dev %} > -{% if kdevops_enable_nixos|default(false) %} > -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 > -{% else %} > -{{ kdevops_host_prefix }}-dev > -{% endif %} > -{% endif %} > -[dev:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% if kdevops_enable_iscsi %} > -[iscsi] > -{{ kdevops_host_prefix }}-iscsi > -[iscsi:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% endif %} > -{% if kdevops_nfsd_enable %} > -[nfsd] > -{{ kdevops_host_prefix }}-nfsd > -[nfsd:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% endif %} > -[service] > -{% if kdevops_enable_iscsi %} > -{{ kdevops_host_prefix }}-iscsi > -{% endif %} > -{% if kdevops_nfsd_enable %} > -{{ kdevops_host_prefix }}-nfsd > -{% endif %} > -[service:vars] > -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > -{% endif %} > +{# Default non-workflow template #} > +{% include 'workflows/default.j2' %} > +{% endif %} > \ No newline at end of file > diff --git a/playbooks/roles/gen_hosts/templates/workflows/ai.j2 b/playbooks/roles/gen_hosts/templates/workflows/ai.j2 > new file mode 100644 > index 00000000..d0914436 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/ai.j2 > @@ -0,0 +1,99 @@ > +{# Workflow template for AI #} > +{% if ai_enable_multifs_testing|default(false)|bool %} > +{# Multi-filesystem section-based hosts #} > +[all] > +localhost ansible_connection=local > +{% for node in all_generic_nodes %} > +{{ node }} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for node in all_generic_nodes %} > +{% if not node.endswith('-dev') %} > +{{ node }} > +{% endif %} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for node in all_generic_nodes %} > +{% if node.endswith('-dev') %} > +{{ node }} > +{% endif %} > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[ai] > +{% for node in all_generic_nodes %} > +{{ node }} > +{% endfor %} > + > +[ai:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{# Individual section groups for multi-filesystem testing #} > +{% set section_names = [] %} > +{% for node in all_generic_nodes %} > +{% if not node.endswith('-dev') %} > +{% set section = node.replace(kdevops_host_prefix + '-ai-', '') %} > +{% if section != kdevops_host_prefix + '-ai' %} > +{% if section_names.append(section) %}{% endif %} > +{% endif %} > +{% endif %} > +{% endfor %} > + > +{% for section in section_names %} > +[ai_{{ section | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-ai-{{ section }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-ai-{{ section }}-dev > +{% endif %} > + > +[ai_{{ section | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > +{% else %} > +{# Single filesystem hosts (original behavior) #} > +[all] > +localhost ansible_connection=local > +{{ kdevops_host_prefix }}-ai > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-ai-dev > +{% endif %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{{ kdevops_host_prefix }}-ai > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{{ kdevops_host_prefix }}-ai-dev > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[ai] > +{{ kdevops_host_prefix }}-ai > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-ai-dev > +{% endif %} > + > +[ai:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +{% endif %} > diff --git a/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 > new file mode 100644 > index 00000000..eea3eef3 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/blktests.j2 > @@ -0,0 +1,58 @@ > +{# Workflow template for blktests #} > +[all] > +localhost ansible_connection=local > +{% for test_type in blktests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for test_type in blktests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for test_type in blktests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[blktests] > +{% for test_type in blktests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > + > +[blktests:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% for test_type in blktests_enabled_test_types %} > +[blktests_{{ test_type | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > + > +[blktests_{{ test_type | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > + > +[service] > + > +[service:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 > new file mode 100644 > index 00000000..53790f29 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/cxl.j2 > @@ -0,0 +1,7 @@ > +{# Workflow template for CXL #} > +[all] > +localhost ansible_connection=local > +write-your-own-template-for-cxl-workflow > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/default.j2 b/playbooks/roles/gen_hosts/templates/workflows/default.j2 > new file mode 100644 > index 00000000..86619309 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/default.j2 > @@ -0,0 +1,62 @@ > +{# Default template for non-workflow setups #} > +[all] > +localhost ansible_connection=local > +{% if kdevops_enable_nixos|default(false) %} > +{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 > +{% else %} > +{{ kdevops_host_prefix }} > +{% endif %} > +{% if kdevops_baseline_and_dev == True %} > +{% if kdevops_enable_nixos|default(false) %} > +{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 > +{% else %} > +{{ kdevops_host_prefix }}-dev > +{% endif %} > +{% endif %} > +{% if kdevops_enable_iscsi %} > +{{ kdevops_host_prefix }}-iscsi > +{% endif %} > +{% if kdevops_nfsd_enable %} > +{{ kdevops_host_prefix }}-nfsd > +{% endif %} > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +[baseline] > +{% if kdevops_enable_nixos|default(false) %} > +{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 > +{% else %} > +{{ kdevops_host_prefix }} > +{% endif %} > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +[dev] > +{% if kdevops_baseline_and_dev %} > +{% if kdevops_enable_nixos|default(false) %} > +{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 > +{% else %} > +{{ kdevops_host_prefix }}-dev > +{% endif %} > +{% endif %} > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +{% if kdevops_enable_iscsi %} > +[iscsi] > +{{ kdevops_host_prefix }}-iscsi > +[iscsi:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +{% endif %} > +{% if kdevops_nfsd_enable %} > +[nfsd] > +{{ kdevops_host_prefix }}-nfsd > +[nfsd:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > +{% endif %} > +[service] > +{% if kdevops_enable_iscsi %} > +{{ kdevops_host_prefix }}-iscsi > +{% endif %} > +{% if kdevops_nfsd_enable %} > +{{ kdevops_host_prefix }}-nfsd > +{% endif %} > +[service:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 > new file mode 100644 > index 00000000..548941a0 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 > @@ -0,0 +1,38 @@ > +{# Workflow template for fio-tests #} > +[all] > +localhost ansible_connection=local > +{{ kdevops_host_prefix }}-fio-tests > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-fio-tests-dev > +{% endif %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{{ kdevops_host_prefix }}-fio-tests > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{{ kdevops_host_prefix }}-fio-tests-dev > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[fio_tests] > +{{ kdevops_host_prefix }}-fio-tests > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-fio-tests-dev > +{% endif %} > + > +[fio_tests:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[service] > + > +[service:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 > new file mode 100644 > index 00000000..600d125f > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/fstests.j2 > @@ -0,0 +1,72 @@ > +{# Workflow template for fstests #} > +[all] > +localhost ansible_connection=local > +{% for node_section in fstests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ node_section }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ node_section }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for node_section in fstests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ node_section }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for node_section in fstests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ node_section }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[fstests] > +{% for node_section in fstests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ node_section }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ node_section }}-dev > +{% endif %} > +{% endfor %} > + > +[fstests:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% for section in sections %} > +[fstests_{{ section | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-{{ section }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ section }}-dev > +{% endif %} > + > +[fstests_{{ section | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > + > +[nfsd] > +{% if kdevops_nfsd_enable %} > +{{ kdevops_host_prefix }}-nfsd > +{% endif %} > + > +[nfsd:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[service] > +{% if kdevops_enable_iscsi %} > +{{ kdevops_host_prefix }}-iscsi > +{% endif %} > +{% if kdevops_nfsd_enable %} > +{{ kdevops_host_prefix }}-nfsd > +{% endif %} > + > +[service:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 > new file mode 100644 > index 00000000..86ee9326 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/gitr.j2 > @@ -0,0 +1,41 @@ > +{# Workflow template for gitr #} > +[all] > +localhost ansible_connection=local > +{% for host in gitr_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for host in gitr_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for host in gitr_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[gitr] > +{% for host in gitr_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[gitr:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 > new file mode 100644 > index 00000000..fb120828 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/ltp.j2 > @@ -0,0 +1,41 @@ > +{# Workflow template for ltp #} > +[all] > +localhost ansible_connection=local > +{% for host in ltp_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for host in ltp_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for host in ltp_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[ltp] > +{% for host in ltp_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[ltp:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 > new file mode 100644 > index 00000000..d796cbe6 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 > @@ -0,0 +1,77 @@ > +{# Workflow template for mmtests #} > +[all] > +localhost ansible_connection=local > +{% if mmtests_enabled_test_types %} > +{% for test_type in mmtests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > +{% else %} > +{{ kdevops_host_prefix }}-mmtests > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-mmtests-dev > +{% endif %} > +{% endif %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% if mmtests_enabled_test_types %} > +{% for test_type in mmtests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% endfor %} > +{% else %} > +{{ kdevops_host_prefix }}-mmtests > +{% endif %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% if mmtests_enabled_test_types %} > +{% for test_type in mmtests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endfor %} > +{% else %} > +{{ kdevops_host_prefix }}-mmtests-dev > +{% endif %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[mmtests] > +{% if mmtests_enabled_test_types %} > +{% for test_type in mmtests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > +{% else %} > +{{ kdevops_host_prefix }}-mmtests > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-mmtests-dev > +{% endif %} > +{% endif %} > + > +[mmtests:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if mmtests_enabled_test_types %} > +{% for test_type in mmtests_enabled_test_types %} > +[mmtests_{{ test_type | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > + > +[mmtests_{{ test_type | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > +{% endif %} > diff --git a/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 > new file mode 100644 > index 00000000..34aa7dfe > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 > @@ -0,0 +1,41 @@ > +{# Workflow template for nfstest #} > +[all] > +localhost ansible_connection=local > +{% for host in nfstest_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for host in nfstest_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for host in nfstest_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[nfstest] > +{% for host in nfstest_enabled_hosts %} > +{{ kdevops_host_prefix }}-{{ host }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ host }}-dev > +{% endif %} > +{% endfor %} > + > +[nfstest:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 > new file mode 100644 > index 00000000..6145b1a0 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 > @@ -0,0 +1,7 @@ > +{# Workflow template for pynfs #} > +[all] > +localhost ansible_connection=local > +write-your-own-template-for-pynfs-workflow > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 > new file mode 100644 > index 00000000..07bd6f80 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 > @@ -0,0 +1,33 @@ > +{# Workflow template for reboot-limit #} > +[all] > +localhost ansible_connection=local > +{{ kdevops_host_prefix }}-reboot-limit > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-reboot-limit-dev > +{% endif %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{{ kdevops_host_prefix }}-reboot-limit > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{{ kdevops_host_prefix }}-reboot-limit-dev > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[reboot-limit] > +{{ kdevops_host_prefix }}-reboot-limit > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-reboot-limit-dev > +{% endif %} > + > +[reboot-limit:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > diff --git a/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 > new file mode 100644 > index 00000000..4ef598c7 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/selftests.j2 > @@ -0,0 +1,53 @@ > +{# Workflow template for selftests #} > +[all] > +localhost ansible_connection=local > +{% for test_type in selftests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for test_type in selftests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for test_type in selftests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[selftests] > +{% for test_type in selftests_enabled_test_types %} > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > +{% endfor %} > + > +[selftests:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% for test_type in selftests_enabled_test_types %} > +[selftests_{{ test_type | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-{{ test_type }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test_type }}-dev > +{% endif %} > + > +[selftests_{{ test_type | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > diff --git a/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 > new file mode 100644 > index 00000000..34f22a83 > --- /dev/null > +++ b/playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 > @@ -0,0 +1,53 @@ > +{# Workflow template for sysbench #} > +[all] > +localhost ansible_connection=local > +{% for test in enabled_sysbench_tests %} > +{{ kdevops_host_prefix }}-{{ test }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test }}-dev > +{% endif %} > +{% endfor %} > + > +[all:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +[baseline] > +{% for test in enabled_sysbench_tests %} > +{{ kdevops_host_prefix }}-{{ test }} > +{% endfor %} > + > +[baseline:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% if kdevops_baseline_and_dev %} > +[dev] > +{% for test in enabled_sysbench_tests %} > +{{ kdevops_host_prefix }}-{{ test }}-dev > +{% endfor %} > + > +[dev:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endif %} > +[sysbench] > +{% for test in enabled_sysbench_tests %} > +{{ kdevops_host_prefix }}-{{ test }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test }}-dev > +{% endif %} > +{% endfor %} > + > +[sysbench:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% for test in enabled_sysbench_tests %} > +[sysbench_{{ test | replace('-', '_') }}] > +{{ kdevops_host_prefix }}-{{ test }} > +{% if kdevops_baseline_and_dev %} > +{{ kdevops_host_prefix }}-{{ test }}-dev > +{% endif %} > + > +[sysbench_{{ test | replace('-', '_') }}:vars] > +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" > + > +{% endfor %} > diff --git a/workflows/ai/Makefile b/workflows/ai/Makefile > index 1c297edd..27dea382 100644 > --- a/workflows/ai/Makefile > +++ b/workflows/ai/Makefile > @@ -4,7 +4,8 @@ PHONY += ai-tests ai-tests-baseline ai-tests-dev > PHONY += ai-tests-results > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := hosts.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := hosts.j2 This comment seems like Claude is "showing his work". I'm not sure there is a long-term value in keeping these comments, and they might end up becoming technical debt over time. > endif > > export AI_DATA_TARGET := $(subst ",,$(CONFIG_AI_BENCHMARK_RESULTS_DIR)) > diff --git a/workflows/blktests/Makefile b/workflows/blktests/Makefile > index 11cd9a65..97649026 100644 > --- a/workflows/blktests/Makefile > +++ b/workflows/blktests/Makefile > @@ -12,7 +12,8 @@ ID=$(shell id -u) > BLKTESTS_ARGS := > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := blktests.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := blktests.j2 > endif > > BLKTESTS_GIT:=$(subst ",,$(CONFIG_BLKTESTS_GIT)) > diff --git a/workflows/cxl/Makefile b/workflows/cxl/Makefile > index bb130057..feabe197 100644 > --- a/workflows/cxl/Makefile > +++ b/workflows/cxl/Makefile > @@ -1,6 +1,7 @@ > # SPDX-License-Identifier: copyleft-next-0.3.1 > > -export KDEVOPS_HOSTS_TEMPLATE := cxl.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := cxl.j2 > > CXL_ARGS := > CXL_ARGS += ndctl_git='$(subst ",,$(CONFIG_NDCTL_GIT))' > diff --git a/workflows/fio-tests/Makefile b/workflows/fio-tests/Makefile > index 5eb2ccbd..20a5d192 100644 > --- a/workflows/fio-tests/Makefile > +++ b/workflows/fio-tests/Makefile > @@ -1,5 +1,6 @@ > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := fio-tests.j2 > endif > > fio-tests: > diff --git a/workflows/fstests/Makefile b/workflows/fstests/Makefile > index d2e5c636..989c564b 100644 > --- a/workflows/fstests/Makefile > +++ b/workflows/fstests/Makefile > @@ -9,7 +9,8 @@ FSTESTS_BASELINE_EXTRA := > export FSTYP:=$(subst ",,$(CONFIG_FSTESTS_FSTYP)) > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := fstests.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := fstests.j2 > endif > > FSTESTS_ARGS += fstests_fstyp='$(FSTYP)' > diff --git a/workflows/gitr/Makefile b/workflows/gitr/Makefile > index c685395a..eadcdac1 100644 > --- a/workflows/gitr/Makefile > +++ b/workflows/gitr/Makefile > @@ -1,5 +1,6 @@ > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := gitr.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := gitr.j2 > endif > > GITR_MNT:=$(subst ",,$(CONFIG_GITR_MNT)) > diff --git a/workflows/linux/Makefile b/workflows/linux/Makefile > index 30b123f9..60d755b3 100644 > --- a/workflows/linux/Makefile > +++ b/workflows/linux/Makefile > @@ -14,7 +14,8 @@ TREE_CONFIG:=config-$(TREE_REF)-pure-iomap > endif > > ifeq (y,$(CONFIG_BOOTLINUX_BUILDER)) > -KDEVOPS_HOSTS_TEMPLATE=builder.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# KDEVOPS_HOSTS_TEMPLATE=builder.j2 > endif > > # Describes the Linux clone > diff --git a/workflows/ltp/Makefile b/workflows/ltp/Makefile > index 767465bc..b3c3319b 100644 > --- a/workflows/ltp/Makefile > +++ b/workflows/ltp/Makefile > @@ -1,5 +1,6 @@ > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := ltp.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := ltp.j2 > endif > > LTP_REPO:=$(subst ",,$(CONFIG_LTP_REPO)) > diff --git a/workflows/mmtests/Makefile b/workflows/mmtests/Makefile > index b65d256b..186dc8cb 100644 > --- a/workflows/mmtests/Makefile > +++ b/workflows/mmtests/Makefile > @@ -1,7 +1,8 @@ > MMTESTS_ARGS := > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := mmtests.j2 > endif > > mmtests: > diff --git a/workflows/nfstest/Makefile b/workflows/nfstest/Makefile > index bbfd3f64..bc880b6d 100644 > --- a/workflows/nfstest/Makefile > +++ b/workflows/nfstest/Makefile > @@ -1,5 +1,6 @@ > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := nfstest.j2 > endif # CONFIG_WORKFLOWS_DEDICATED_WORKFLOW > > ifeq (y,$(CONFIG_NFSTEST_USE_KDEVOPS_NFSD)) > diff --git a/workflows/pynfs/Makefile b/workflows/pynfs/Makefile > index 2f3ff97b..b7bb12bb 100644 > --- a/workflows/pynfs/Makefile > +++ b/workflows/pynfs/Makefile > @@ -1,5 +1,6 @@ > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := pynfs.j2 > endif > > PYNFS_GIT:=$(subst ",,$(CONFIG_PYNFS_GIT)) > diff --git a/workflows/selftests/Makefile b/workflows/selftests/Makefile > index d3b7044c..4ec691f3 100644 > --- a/workflows/selftests/Makefile > +++ b/workflows/selftests/Makefile > @@ -3,7 +3,8 @@ > SELFTESTS_ARGS := > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := selftests.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := selftests.j2 > endif > > SELFTESTS_DYNAMIC_RUNTIME_VARS := "kdevops_run_selftests": True > diff --git a/workflows/sysbench/Makefile b/workflows/sysbench/Makefile > index daf7bc75..5f484691 100644 > --- a/workflows/sysbench/Makefile > +++ b/workflows/sysbench/Makefile > @@ -1,7 +1,8 @@ > PHONY += sysbench sysbench-test sysbench-telemetry sysbench-help-menu > > ifeq (y,$(CONFIG_WORKFLOWS_DEDICATED_WORKFLOW)) > -export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2 > +# After refactoring, all workflows use hosts.j2 which includes workflow-specific templates > +# export KDEVOPS_HOSTS_TEMPLATE := sysbench.j2 > endif > > TAGS_SYSBENCH_RUN := db_start -- Chuck Lever ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure 2025-08-29 10:56 [PATCH 0/2] declared hosts support Luis Chamberlain 2025-08-29 10:56 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain @ 2025-08-29 10:56 ` Luis Chamberlain 1 sibling, 0 replies; 5+ messages in thread From: Luis Chamberlain @ 2025-08-29 10:56 UTC (permalink / raw) To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain This adds support for using pre-existing infrastructure (bare metal servers, pre-provisioned VMs) that already have SSH access configured. This is useful for environments where kdevops bringup/teardown is not desired or possible. Key features: - New DECLARE_HOSTS CLI parameter to specify existing hosts - Automatic detection of CLI override to enable declared hosts mode - SKIP_BRINGUP is automatically selected when using declared hosts - Hosts can be specified as space or comma-separated list - For A/B testing workflows, odd-numbered hosts become baseline nodes and even-numbered hosts become dev nodes - New Kconfig.declared_hosts for configuration options - New declared-hosts.j2 template for generating inventory Usage: make defconfig-xfs_reflink_4k DECLARE_HOSTS='server1 server2 server3' make # Will skip bringup and use the declared hosts The implementation assumes SSH access is already configured in the user's ~/.ssh/config or via SSH keys. No SSH configuration is generated by kdevops when using declared hosts. This feature integrates cleanly with the refactored template system, using the same hosts.j2 entry point with a dedicated declared-hosts.j2 template for this mode. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- Makefile | 7 + kconfigs/Kconfig.bringup | 7 + kconfigs/Kconfig.declared_hosts | 65 ++++++ playbooks/roles/devconfig/defaults/main.yml | 1 + playbooks/roles/devconfig/tasks/main.yml | 60 +++++ playbooks/roles/gen_hosts/defaults/main.yml | 1 + playbooks/roles/gen_hosts/tasks/main.yml | 40 +++- .../templates/workflows/declared-hosts.j2 | 219 ++++++++++++++++++ playbooks/roles/gen_nodes/tasks/main.yml | 5 + 9 files changed, 404 insertions(+), 1 deletion(-) create mode 100644 kconfigs/Kconfig.declared_hosts create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 diff --git a/Makefile b/Makefile index 83c67340..fcb626ae 100644 --- a/Makefile +++ b/Makefile @@ -19,6 +19,13 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) + +# Export CLI override variables for Kconfig to detect them +# Note: We accept DECLARE_HOSTS but export as DECLARED_HOSTS for consistency +ifdef DECLARE_HOSTS +export DECLARED_HOSTS := $(DECLARE_HOSTS) +endif + include scripts/refs.Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates diff --git a/kconfigs/Kconfig.bringup b/kconfigs/Kconfig.bringup index 8caf07be..f7f9735a 100644 --- a/kconfigs/Kconfig.bringup +++ b/kconfigs/Kconfig.bringup @@ -9,8 +9,14 @@ config KDEVOPS_ENABLE_NIXOS bool output yaml +# CLI override detection for DECLARED_HOSTS which should enable SKIP_BRINGUP +config SKIP_BRINGUP_SET_BY_CLI + bool + default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS) + choice prompt "Node bring up method" + default SKIP_BRINGUP if SKIP_BRINGUP_SET_BY_CLI default GUESTFS config GUESTFS @@ -80,6 +86,7 @@ config LIBVIRT source "kconfigs/Kconfig.guestfs" source "kconfigs/Kconfig.nixos" source "terraform/Kconfig" +source "kconfigs/Kconfig.declared_hosts" if LIBVIRT source "kconfigs/Kconfig.libvirt" endif diff --git a/kconfigs/Kconfig.declared_hosts b/kconfigs/Kconfig.declared_hosts new file mode 100644 index 00000000..56b56845 --- /dev/null +++ b/kconfigs/Kconfig.declared_hosts @@ -0,0 +1,65 @@ +# Configuration for declared hosts that skip bringup process + +# CLI override support for DECLARED_HOSTS +config KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + bool + output yaml + default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS) + +config KDEVOPS_USE_DECLARED_HOSTS + bool "Use declared hosts (skip bringup process)" + depends on SKIP_BRINGUP || KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + select WORKFLOW_INFER_USER_AND_GROUP + output yaml + default y if SKIP_BRINGUP + default y if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + help + Enable this option to use pre-existing hosts that you have already + configured with SSH access. This is useful for: + + * Bare metal systems + * Pre-provisioned VMs or cloud instances + * Systems managed by other infrastructure tools + + When this option is enabled: + - SSH keys will not be generated (assumes you already have access) + - Bringup and teardown operations will be skipped + - User and group settings will be inferred from the target hosts + - You must provide the list of hosts in KDEVOPS_DECLARED_HOSTS + + This option automatically: + - Selects WORKFLOW_INFER_USER_AND_GROUP to detect the correct + user and group on the target systems + - Assumes SSH access is already configured + +if KDEVOPS_USE_DECLARED_HOSTS + +config KDEVOPS_DECLARED_HOSTS + string "List of declared hosts (space or comma separated)" + output yaml + default $(shell, ./scripts/append-makefile-vars.sh $(DECLARED_HOSTS)) if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + default "" + help + Provide a space or comma-separated list of hostnames or IP addresses + that are already accessible via SSH. + + Examples: + - Single host: "server1.example.com" + - Multiple hosts: "server1.example.com server2.example.com" + - IP addresses: "192.168.1.10 192.168.1.11" + - Mixed: "server1.example.com 10.0.0.5 server2.local" + + For A/B testing workflows with baseline/dev configurations, you can + provide an even number of hosts. The odd-numbered hosts will be used + as baseline nodes and even-numbered hosts as dev nodes. + + Requirements: + - SSH access must already be configured for these hosts + - The user running kdevops must have passwordless SSH access + - The hosts should have the required dependencies installed + + Note: kdevops assumes you have already configured SSH access to these + hosts in your ~/.ssh/config or have appropriate SSH keys set up. + Ansible will use your existing SSH configuration to connect. + +endif # KDEVOPS_USE_DECLARED_HOSTS diff --git a/playbooks/roles/devconfig/defaults/main.yml b/playbooks/roles/devconfig/defaults/main.yml index 98dce312..62e72ffa 100644 --- a/playbooks/roles/devconfig/defaults/main.yml +++ b/playbooks/roles/devconfig/defaults/main.yml @@ -57,3 +57,4 @@ kdevops_enable_guestfs: false guestfs_copy_sources_from_host_to_guest: false distro_debian_has_hop1_sources: false unattended_upgrades_installed: false +kdevops_use_declared_hosts: false diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml index fccd1fcf..e93c6078 100644 --- a/playbooks/roles/devconfig/tasks/main.yml +++ b/playbooks/roles/devconfig/tasks/main.yml @@ -17,6 +17,66 @@ ansible.builtin.setup: tags: always +- name: Determine the actual user's home directory when using inferred user + block: + - name: Get the current username + ansible.builtin.command: whoami + register: current_user + changed_when: false + + - name: Get home directory for the current user + ansible.builtin.shell: | + # When workflow_infer_user_and_group is enabled, we need to use the actual SSH user + # not the hardcoded 'kdevops' user from group_vars/all + actual_user="{{ current_user.stdout }}" + if [ -n "$actual_user" ]; then + getent passwd "$actual_user" | cut -d: -f6 + else + echo "/root" + fi + register: inferred_home_dir + changed_when: false + + - name: Set data_user and data_home_dir based on inferred user + set_fact: + data_user: "{{ current_user.stdout }}" + data_home_dir: "{{ inferred_home_dir.stdout }}" + dev_gitconfig_dest: "{{ inferred_home_dir.stdout }}/.gitconfig" + dev_bash_config: "{{ inferred_home_dir.stdout }}/.bashrc" + dev_bash_config_hacks_dest: "{{ inferred_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}" + when: workflow_infer_user_and_group | default(false) | bool + +# Also handle the case when workflow_infer_user_and_group is false but we're using declared hosts +# In this case, we should still use the SSH user's actual home directory +- name: Check if using declared hosts without infer setting + block: + - name: Get the current username for declared hosts + ansible.builtin.command: whoami + register: current_user_declared + changed_when: false + + - name: Get home directory for declared hosts user + ansible.builtin.shell: | + actual_user="{{ current_user_declared.stdout }}" + if [ -n "$actual_user" ]; then + getent passwd "$actual_user" | cut -d: -f6 + else + echo "/root" + fi + register: declared_home_dir + changed_when: false + + - name: Override data paths for declared hosts + set_fact: + data_user: "{{ current_user_declared.stdout }}" + data_home_dir: "{{ declared_home_dir.stdout }}" + dev_gitconfig_dest: "{{ declared_home_dir.stdout }}/.gitconfig" + dev_bash_config: "{{ declared_home_dir.stdout }}/.bashrc" + dev_bash_config_hacks_dest: "{{ declared_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}" + when: + - kdevops_use_declared_hosts + - not (workflow_infer_user_and_group | default(false) | bool) + # Update /etc/hostname first so the change gets picked up by the reboot # that occurs during the distro-specific tasks diff --git a/playbooks/roles/gen_hosts/defaults/main.yml b/playbooks/roles/gen_hosts/defaults/main.yml index 4a7515f9..027cbe44 100644 --- a/playbooks/roles/gen_hosts/defaults/main.yml +++ b/playbooks/roles/gen_hosts/defaults/main.yml @@ -20,6 +20,7 @@ kdevops_workflows_dedicated_workflow: false kdevops_workflow_enable_fstests: false kdevops_workflow_enable_blktests: false kdevops_workflow_enable_selftests: false +kdevops_use_declared_hosts: false kdevops_workflow_enable_cxl: false kdevops_workflow_enable_pynfs: false diff --git a/playbooks/roles/gen_hosts/tasks/main.yml b/playbooks/roles/gen_hosts/tasks/main.yml index fb63629a..b475e2b0 100644 --- a/playbooks/roles/gen_hosts/tasks/main.yml +++ b/playbooks/roles/gen_hosts/tasks/main.yml @@ -10,6 +10,7 @@ skip: true tags: vars + - name: Get our user ansible.builtin.command: "whoami" register: my_user @@ -39,6 +40,21 @@ path: "{{ kdevops_hosts_template_full_path }}" register: ansible_hosts_template +- name: Generate the Ansible hosts file for declared hosts configuration + tags: ['hosts'] + ansible.builtin.template: + src: "{{ kdevops_hosts_template }}" + dest: "{{ ansible_cfg_inventory }}" + force: true + trim_blocks: True + lstrip_blocks: True + mode: '0644' + when: + - kdevops_use_declared_hosts + - kdevops_declared_hosts is defined + - kdevops_declared_hosts|length > 0 + - ansible_hosts_template.stat.exists + - name: Set fstests config file variable for {{ fstests_fstyp }} ansible.builtin.set_fact: is_fstests: true @@ -67,6 +83,7 @@ when: - bootlinux_builder - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file tags: ["hosts"] @@ -80,6 +97,7 @@ - not kdevops_workflows_dedicated_workflow - ansible_hosts_template.stat.exists - not kdevops_enable_nixos|default(false)|bool + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file for NixOS tags: ['hosts'] @@ -93,6 +111,7 @@ - not kdevops_workflows_dedicated_workflow - ansible_hosts_template.stat.exists - kdevops_enable_nixos|default(false)|bool + - not kdevops_use_declared_hosts - name: Update Ansible inventory access modification time so make sees it updated ansible.builtin.file: @@ -112,6 +131,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_cxl - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file for dedicated pynfs work tags: ["hosts"] @@ -125,6 +145,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_pynfs - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file for dedicated gitr workflow tags: ["hosts"] @@ -140,6 +161,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_gitr - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate an Ansible inventory file for a dedicated ltp workflow tags: ["hosts"] @@ -155,6 +177,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_ltp - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file for dedicated nfstest workflow tags: ["hosts"] @@ -170,6 +193,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_nfstest - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Set empty enabled test types list for fstests tags: ["hosts"] @@ -178,6 +202,7 @@ when: - is_fstests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Check which fstests test types are enabled register: fstests_enabled_test_types_reg @@ -202,6 +227,7 @@ when: - is_fstests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Now expand the list of enabled fstests for valid configuration sections tags: ["hosts"] @@ -213,6 +239,7 @@ when: - is_fstests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - item.changed - name: Generate the Ansible inventory file for a dedicated fstests setup @@ -233,6 +260,7 @@ when: - is_fstests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Infer enabled blktests test section types ansible.builtin.set_fact: @@ -247,6 +275,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_blktests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Debug inferring block test types ansible.builtin.debug: @@ -269,6 +298,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_blktests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Infer enabled selftests test section types ansible.builtin.set_fact: @@ -283,6 +313,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_selftests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible inventory file for a dedicated selftests setup tags: ["hosts"] @@ -296,6 +327,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_selftests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Collect dynamically supported filesystems vars: @@ -338,6 +370,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_sysbench - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible hosts file for a dedicated fio-tests setup @@ -354,6 +387,7 @@ - kdevops_workflow_enable_fio_tests - ansible_hosts_template.stat.exists - not kdevops_enable_nixos|default(false)|bool + - not kdevops_use_declared_hosts - name: Infer enabled mmtests test types @@ -369,6 +403,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_mmtests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible hosts file for a dedicated mmtests setup tags: ["hosts"] @@ -382,6 +417,7 @@ - kdevops_workflows_dedicated_workflow - kdevops_workflow_enable_mmtests - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Generate the Ansible hosts file for a dedicated reboot-limit setup tags: ["hosts"] @@ -395,6 +431,7 @@ - kdevops_workflows_dedicated_workflow - workflows_reboot_limit - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Load AI nodes configuration for multi-filesystem setup include_vars: @@ -426,8 +463,9 @@ mode: '0644' when: - kdevops_workflows_dedicated_workflow - - kdevops_workflow_enable_ai + - kdevops_workflow_enable_ai|default(false)|bool - ansible_hosts_template.stat.exists + - not kdevops_use_declared_hosts - name: Verify if final host file exists ansible.builtin.stat: diff --git a/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 new file mode 100644 index 00000000..61b51885 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 @@ -0,0 +1,219 @@ +{# Template for declared hosts that skip bringup process #} +{# This template is used when users have pre-existing infrastructure like: + - Bare metal servers + - Pre-provisioned VMs + - Cloud instances managed outside of kdevops + - Any hosts with existing SSH access + + The hosts are provided via kdevops_declared_hosts variable which contains + a list of hostnames/IPs that are already accessible via SSH. +#} +{# Parse declared hosts - handle both string and list formats #} +{% if kdevops_declared_hosts is string %} +{% set parsed_hosts = kdevops_declared_hosts | regex_replace(',', ' ') | split() %} +{% elif kdevops_declared_hosts is iterable and kdevops_declared_hosts is not string %} +{% set parsed_hosts = kdevops_declared_hosts %} +{% else %} +{% set parsed_hosts = [kdevops_declared_hosts] %} +{% endif %} +[all] +localhost ansible_connection=local +{# Loop through parsed hosts #} +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_workflows_dedicated_workflow %} +{# For workflows, organize hosts into baseline/dev groups for A/B testing #} +{% if kdevops_baseline_and_dev %} +[baseline] +{# Odd-numbered hosts become baseline nodes #} +{% for host in parsed_hosts %} +{% if loop.index is odd %} +{{ host }} +{% endif %} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[dev] +{# Even-numbered hosts become dev nodes #} +{% for host in parsed_hosts %} +{% if loop.index is even %} +{{ host }} +{% endif %} +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% else %} +{# Without A/B testing, all hosts are baseline #} +[baseline] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +{# Add workflow-specific groups based on enabled workflow #} +{% if workflows_reboot_limit %} +[reboot-limit] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[reboot-limit:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_fio_tests %} +[fio_tests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fio_tests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_fstests %} +[fstests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fstests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{# Add per-section groups if needed #} +{% for section in sections|default([]) %} +[fstests_{{ section | replace('-', '_') }}] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fstests_{{ section | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endfor %} + +{% elif kdevops_workflow_enable_blktests %} +[blktests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[blktests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_selftests %} +[selftests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[selftests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_mmtests %} +[mmtests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[mmtests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_sysbench %} +[sysbench] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[sysbench:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_ai|default(false)|bool %} +[ai] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[ai:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_minio|default(false)|bool %} +[minio] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[minio:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_cxl %} +[cxl] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[cxl:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_pynfs %} +[pynfs] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[pynfs:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_gitr %} +[gitr] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[gitr:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_ltp %} +[ltp] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[ltp:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_nfstest %} +[nfstest] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[nfstest:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +{% else %} +{# Non-workflow setup - just use baseline group #} +[baseline] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +[service] +{# Service nodes are typically not needed for declared hosts #} + +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_nodes/tasks/main.yml b/playbooks/roles/gen_nodes/tasks/main.yml index b1a1946f..0fa778bf 100644 --- a/playbooks/roles/gen_nodes/tasks/main.yml +++ b/playbooks/roles/gen_nodes/tasks/main.yml @@ -20,6 +20,11 @@ register: my_group changed_when: false +- name: Skip node generation if using declared hosts + meta: end_play + when: + - kdevops_use_declared_hosts|default(false)|bool + - name: Create guestfs directory ansible.builtin.file: path: "{{ guestfs_path }}" -- 2.50.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 0/2] declared hosts support
@ 2025-08-29 10:31 Luis Chamberlain
2025-08-29 10:31 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
0 siblings, 1 reply; 5+ messages in thread
From: Luis Chamberlain @ 2025-08-29 10:31 UTC (permalink / raw)
To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain
It should be easy for us to add bare metal support because well, the
biggest pain is bring up. If bringup and ssh is already configured,
then all we have to do is just skip a bunch of things. So the question
to ask, is how do we generate the hosts file? The easiest interface
I could think of is to allow you to specify it at make menuconfig time
or defconfig time with a DECLARE_HOSTS= specifier with either a space
or comma separated lists of hosts.
I tested this with a new workflow I am testing, minio WARP testing, but I
also tested this simply with a random host I can ssh to with just:
make defconfig-xfs_reflink_4k DECLARE_HOSTS=foo ; make AV=2
make uname
And that works. For my other workflow I am testing I got full test
restults and mkfs'd a drive. And so each workflow needs a bit of love
for further DECLARE_HOSTS support. Things like the data partition, and
so on. I suspect we can just specify some of these in the command line
as CLI variables.
But this goes with just basic raw support. Maybe we only enable
workflows for KDEVOPS_USE_DECLARED_HOSTS if we declare the workflow
as supporting it (maybe select SUPPORTS_DECLARED_HOSTS)?
Luis Chamberlain (2):
gen_hosts: refactor monolithic hosts.j2 template into per-workflow
templates
declared_hosts: add support for pre-existing infrastructure
Makefile | 7 +
kconfigs/Kconfig.bringup | 7 +
kconfigs/Kconfig.declared_hosts | 65 +++++
playbooks/roles/devconfig/tasks/main.yml | 60 ++++
playbooks/roles/gen_hosts/tasks/main.yml | 40 ++-
playbooks/roles/gen_hosts/templates/hosts.j2 | 264 +++---------------
.../roles/gen_hosts/templates/workflows/ai.j2 | 99 +++++++
.../gen_hosts/templates/workflows/blktests.j2 | 58 ++++
.../gen_hosts/templates/workflows/cxl.j2 | 7 +
.../templates/workflows/declared-hosts.j2 | 219 +++++++++++++++
.../gen_hosts/templates/workflows/default.j2 | 62 ++++
.../templates/workflows/fio-tests.j2 | 38 +++
.../gen_hosts/templates/workflows/fstests.j2 | 72 +++++
.../gen_hosts/templates/workflows/gitr.j2 | 41 +++
.../gen_hosts/templates/workflows/ltp.j2 | 41 +++
.../gen_hosts/templates/workflows/mmtests.j2 | 77 +++++
.../gen_hosts/templates/workflows/nfstest.j2 | 41 +++
.../gen_hosts/templates/workflows/pynfs.j2 | 7 +
.../templates/workflows/reboot-limit.j2 | 33 +++
.../templates/workflows/selftests.j2 | 53 ++++
.../gen_hosts/templates/workflows/sysbench.j2 | 53 ++++
playbooks/roles/gen_nodes/tasks/main.yml | 5 +
workflows/ai/Makefile | 3 +-
workflows/blktests/Makefile | 3 +-
workflows/cxl/Makefile | 3 +-
workflows/fio-tests/Makefile | 3 +-
workflows/fstests/Makefile | 3 +-
workflows/gitr/Makefile | 3 +-
workflows/linux/Makefile | 3 +-
workflows/ltp/Makefile | 3 +-
workflows/mmtests/Makefile | 3 +-
workflows/nfstest/Makefile | 3 +-
workflows/pynfs/Makefile | 3 +-
workflows/selftests/Makefile | 3 +-
workflows/sysbench/Makefile | 3 +-
35 files changed, 1143 insertions(+), 245 deletions(-)
create mode 100644 kconfigs/Kconfig.declared_hosts
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/default.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2
create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2
--
2.50.1
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure 2025-08-29 10:31 [PATCH 0/2] declared hosts support Luis Chamberlain @ 2025-08-29 10:31 ` Luis Chamberlain 0 siblings, 0 replies; 5+ messages in thread From: Luis Chamberlain @ 2025-08-29 10:31 UTC (permalink / raw) To: Chuck Lever, Daniel Gomez, kdevops; +Cc: Luis Chamberlain This adds support for using pre-existing infrastructure (bare metal servers, pre-provisioned VMs) that already have SSH access configured. This is useful for environments where kdevops bringup/teardown is not desired or possible. Key features: - New DECLARE_HOSTS CLI parameter to specify existing hosts - Automatic detection of CLI override to enable declared hosts mode - SKIP_BRINGUP is automatically selected when using declared hosts - Hosts can be specified as space or comma-separated list - For A/B testing workflows, odd-numbered hosts become baseline nodes and even-numbered hosts become dev nodes - New Kconfig.declared_hosts for configuration options - New declared-hosts.j2 template for generating inventory Usage: make defconfig-xfs_reflink_4k DECLARE_HOSTS='server1 server2 server3' make # Will skip bringup and use the declared hosts The implementation assumes SSH access is already configured in the user's ~/.ssh/config or via SSH keys. No SSH configuration is generated by kdevops when using declared hosts. This feature integrates cleanly with the refactored template system, using the same hosts.j2 entry point with a dedicated declared-hosts.j2 template for this mode. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- Makefile | 7 + kconfigs/Kconfig.bringup | 7 + kconfigs/Kconfig.declared_hosts | 65 ++++++ playbooks/roles/devconfig/tasks/main.yml | 60 +++++ .../templates/workflows/declared-hosts.j2 | 219 ++++++++++++++++++ playbooks/roles/gen_nodes/tasks/main.yml | 5 + 6 files changed, 363 insertions(+) create mode 100644 kconfigs/Kconfig.declared_hosts create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 diff --git a/Makefile b/Makefile index 83c67340..fcb626ae 100644 --- a/Makefile +++ b/Makefile @@ -19,6 +19,13 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) + +# Export CLI override variables for Kconfig to detect them +# Note: We accept DECLARE_HOSTS but export as DECLARED_HOSTS for consistency +ifdef DECLARE_HOSTS +export DECLARED_HOSTS := $(DECLARE_HOSTS) +endif + include scripts/refs.Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates diff --git a/kconfigs/Kconfig.bringup b/kconfigs/Kconfig.bringup index 8caf07be..f7f9735a 100644 --- a/kconfigs/Kconfig.bringup +++ b/kconfigs/Kconfig.bringup @@ -9,8 +9,14 @@ config KDEVOPS_ENABLE_NIXOS bool output yaml +# CLI override detection for DECLARED_HOSTS which should enable SKIP_BRINGUP +config SKIP_BRINGUP_SET_BY_CLI + bool + default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS) + choice prompt "Node bring up method" + default SKIP_BRINGUP if SKIP_BRINGUP_SET_BY_CLI default GUESTFS config GUESTFS @@ -80,6 +86,7 @@ config LIBVIRT source "kconfigs/Kconfig.guestfs" source "kconfigs/Kconfig.nixos" source "terraform/Kconfig" +source "kconfigs/Kconfig.declared_hosts" if LIBVIRT source "kconfigs/Kconfig.libvirt" endif diff --git a/kconfigs/Kconfig.declared_hosts b/kconfigs/Kconfig.declared_hosts new file mode 100644 index 00000000..56b56845 --- /dev/null +++ b/kconfigs/Kconfig.declared_hosts @@ -0,0 +1,65 @@ +# Configuration for declared hosts that skip bringup process + +# CLI override support for DECLARED_HOSTS +config KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + bool + output yaml + default $(shell, scripts/check-cli-set-var.sh DECLARED_HOSTS) + +config KDEVOPS_USE_DECLARED_HOSTS + bool "Use declared hosts (skip bringup process)" + depends on SKIP_BRINGUP || KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + select WORKFLOW_INFER_USER_AND_GROUP + output yaml + default y if SKIP_BRINGUP + default y if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + help + Enable this option to use pre-existing hosts that you have already + configured with SSH access. This is useful for: + + * Bare metal systems + * Pre-provisioned VMs or cloud instances + * Systems managed by other infrastructure tools + + When this option is enabled: + - SSH keys will not be generated (assumes you already have access) + - Bringup and teardown operations will be skipped + - User and group settings will be inferred from the target hosts + - You must provide the list of hosts in KDEVOPS_DECLARED_HOSTS + + This option automatically: + - Selects WORKFLOW_INFER_USER_AND_GROUP to detect the correct + user and group on the target systems + - Assumes SSH access is already configured + +if KDEVOPS_USE_DECLARED_HOSTS + +config KDEVOPS_DECLARED_HOSTS + string "List of declared hosts (space or comma separated)" + output yaml + default $(shell, ./scripts/append-makefile-vars.sh $(DECLARED_HOSTS)) if KDEVOPS_DECLARED_HOSTS_SET_BY_CLI + default "" + help + Provide a space or comma-separated list of hostnames or IP addresses + that are already accessible via SSH. + + Examples: + - Single host: "server1.example.com" + - Multiple hosts: "server1.example.com server2.example.com" + - IP addresses: "192.168.1.10 192.168.1.11" + - Mixed: "server1.example.com 10.0.0.5 server2.local" + + For A/B testing workflows with baseline/dev configurations, you can + provide an even number of hosts. The odd-numbered hosts will be used + as baseline nodes and even-numbered hosts as dev nodes. + + Requirements: + - SSH access must already be configured for these hosts + - The user running kdevops must have passwordless SSH access + - The hosts should have the required dependencies installed + + Note: kdevops assumes you have already configured SSH access to these + hosts in your ~/.ssh/config or have appropriate SSH keys set up. + Ansible will use your existing SSH configuration to connect. + +endif # KDEVOPS_USE_DECLARED_HOSTS diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml index fccd1fcf..0d69f5fd 100644 --- a/playbooks/roles/devconfig/tasks/main.yml +++ b/playbooks/roles/devconfig/tasks/main.yml @@ -17,6 +17,66 @@ ansible.builtin.setup: tags: always +- name: Determine the actual user's home directory when using inferred user + block: + - name: Get the current username + ansible.builtin.command: whoami + register: current_user + changed_when: false + + - name: Get home directory for the current user + ansible.builtin.shell: | + # When workflow_infer_user_and_group is enabled, we need to use the actual SSH user + # not the hardcoded 'kdevops' user from group_vars/all + actual_user="{{ current_user.stdout }}" + if [ -n "$actual_user" ]; then + getent passwd "$actual_user" | cut -d: -f6 + else + echo "/root" + fi + register: inferred_home_dir + changed_when: false + + - name: Set data_user and data_home_dir based on inferred user + set_fact: + data_user: "{{ current_user.stdout }}" + data_home_dir: "{{ inferred_home_dir.stdout }}" + dev_gitconfig_dest: "{{ inferred_home_dir.stdout }}/.gitconfig" + dev_bash_config: "{{ inferred_home_dir.stdout }}/.bashrc" + dev_bash_config_hacks_dest: "{{ inferred_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}" + when: workflow_infer_user_and_group | default(false) | bool + +# Also handle the case when workflow_infer_user_and_group is false but we're using declared hosts +# In this case, we should still use the SSH user's actual home directory +- name: Check if using declared hosts without infer setting + block: + - name: Get the current username for declared hosts + ansible.builtin.command: whoami + register: current_user_declared + changed_when: false + + - name: Get home directory for declared hosts user + ansible.builtin.shell: | + actual_user="{{ current_user_declared.stdout }}" + if [ -n "$actual_user" ]; then + getent passwd "$actual_user" | cut -d: -f6 + else + echo "/root" + fi + register: declared_home_dir + changed_when: false + + - name: Override data paths for declared hosts + set_fact: + data_user: "{{ current_user_declared.stdout }}" + data_home_dir: "{{ declared_home_dir.stdout }}" + dev_gitconfig_dest: "{{ declared_home_dir.stdout }}/.gitconfig" + dev_bash_config: "{{ declared_home_dir.stdout }}/.bashrc" + dev_bash_config_hacks_dest: "{{ declared_home_dir.stdout }}/.{{ dev_bash_config_hacks_name }}" + when: + - kdevops_use_declared_hosts | default(false) | bool + - not (workflow_infer_user_and_group | default(false) | bool) + # Update /etc/hostname first so the change gets picked up by the reboot # that occurs during the distro-specific tasks diff --git a/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 new file mode 100644 index 00000000..61b51885 --- /dev/null +++ b/playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 @@ -0,0 +1,219 @@ +{# Template for declared hosts that skip bringup process #} +{# This template is used when users have pre-existing infrastructure like: + - Bare metal servers + - Pre-provisioned VMs + - Cloud instances managed outside of kdevops + - Any hosts with existing SSH access + + The hosts are provided via kdevops_declared_hosts variable which contains + a list of hostnames/IPs that are already accessible via SSH. +#} +{# Parse declared hosts - handle both string and list formats #} +{% if kdevops_declared_hosts is string %} +{% set parsed_hosts = kdevops_declared_hosts | regex_replace(',', ' ') | split() %} +{% elif kdevops_declared_hosts is iterable and kdevops_declared_hosts is not string %} +{% set parsed_hosts = kdevops_declared_hosts %} +{% else %} +{% set parsed_hosts = [kdevops_declared_hosts] %} +{% endif %} +[all] +localhost ansible_connection=local +{# Loop through parsed hosts #} +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[all:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% if kdevops_workflows_dedicated_workflow %} +{# For workflows, organize hosts into baseline/dev groups for A/B testing #} +{% if kdevops_baseline_and_dev %} +[baseline] +{# Odd-numbered hosts become baseline nodes #} +{% for host in parsed_hosts %} +{% if loop.index is odd %} +{{ host }} +{% endif %} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[dev] +{# Even-numbered hosts become dev nodes #} +{% for host in parsed_hosts %} +{% if loop.index is even %} +{{ host }} +{% endif %} +{% endfor %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% else %} +{# Without A/B testing, all hosts are baseline #} +[baseline] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +{# Add workflow-specific groups based on enabled workflow #} +{% if workflows_reboot_limit %} +[reboot-limit] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[reboot-limit:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_fio_tests %} +[fio_tests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fio_tests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_fstests %} +[fstests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fstests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{# Add per-section groups if needed #} +{% for section in sections|default([]) %} +[fstests_{{ section | replace('-', '_') }}] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[fstests_{{ section | replace('-', '_') }}:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endfor %} + +{% elif kdevops_workflow_enable_blktests %} +[blktests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[blktests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_selftests %} +[selftests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[selftests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_mmtests %} +[mmtests] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[mmtests:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_sysbench %} +[sysbench] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[sysbench:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_ai|default(false)|bool %} +[ai] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[ai:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_minio|default(false)|bool %} +[minio] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[minio:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_cxl %} +[cxl] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[cxl:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_pynfs %} +[pynfs] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[pynfs:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_gitr %} +[gitr] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[gitr:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_ltp %} +[ltp] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[ltp:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +{% elif kdevops_workflow_enable_nfstest %} +[nfstest] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[nfstest:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +{% else %} +{# Non-workflow setup - just use baseline group #} +[baseline] +{% for host in parsed_hosts %} +{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" +{% endif %} + +[service] +{# Service nodes are typically not needed for declared hosts #} + +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" diff --git a/playbooks/roles/gen_nodes/tasks/main.yml b/playbooks/roles/gen_nodes/tasks/main.yml index b1a1946f..0fa778bf 100644 --- a/playbooks/roles/gen_nodes/tasks/main.yml +++ b/playbooks/roles/gen_nodes/tasks/main.yml @@ -20,6 +20,11 @@ register: my_group changed_when: false +- name: Skip node generation if using declared hosts + meta: end_play + when: + - kdevops_use_declared_hosts|default(false)|bool + - name: Create guestfs directory ansible.builtin.file: path: "{{ guestfs_path }}" -- 2.50.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2025-08-29 16:42 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-08-29 10:56 [PATCH 0/2] declared hosts support Luis Chamberlain 2025-08-29 10:56 ` [PATCH 1/2] gen_hosts: refactor monolithic hosts.j2 template into per-workflow templates Luis Chamberlain 2025-08-29 16:42 ` Chuck Lever 2025-08-29 10:56 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain -- strict thread matches above, loose matches on Subject: below -- 2025-08-29 10:31 [PATCH 0/2] declared hosts support Luis Chamberlain 2025-08-29 10:31 ` [PATCH 2/2] declared_hosts: add support for pre-existing infrastructure Luis Chamberlain
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox