[PATCH v3 0/6] Replace terraform update_ssh_config module

[PATCH v3 0/6] Replace terraform update_ssh_config module

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Luis suggested merging the way guestfs and terraform handle ssh
host config. This series is a step in that direction.

guestfs provisioning is currently implemented nearly entirely by
shell scripts. That makes it somewhat high risk to re-write both
terraform and guestfs to use shared infrastructure at the same time.
So instead of building infrastructure that guestfs and terraform can
share, I'm leaving guestfs out of the equation for the moment. All
this series does is make terraform ssh configuration behave more
like guestfs.

Subsequent work can replace the current set of guestfs scripts with
Ansible playbooks, and then more common ssh infrastructure can be
introduced that both provisioning methods can utilize.

Change since v2:
- Move terraform bringup/destroy into an Ansible playbook
- Move terraform's ssh configuration into that playbook
- Tested with AWS, Azure, and OCI

Changes since RFC:
- Split the RFC series into smaller patches for easier review
- The "Include" directive now follows CONFIG_KDEVOPS_SSH_CONFIG
- Various bug fixes and optimizations

Chuck Lever (6):
  terraform: Replace scripts/*_terraform.sh with an Ansible playbook
  ssh.Makefile: Define a kdevops_ssh_config variable
  terraform: Clean up ssh configuration during "make destroy"
  terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}
  terraform: "make mrproper" should remove terraform/*/.terraform
  terraform: Remove the terrraform update_ssh_config module

 Makefile                                      |  1 +
 kconfigs/Kconfig.ssh                          | 19 +++-
 playbooks/roles/terraform/defaults/main.yml   |  3 +
 playbooks/roles/terraform/tasks/main.yml      | 86 +++++++++++++++++++
 .../roles/terraform/templates/ssh_config.j2   | 15 ++++
 playbooks/terraform.yml                       |  5 ++
 scripts/bringup_terraform.sh                  | 12 ---
 scripts/destroy_terraform.sh                  | 10 ---
 scripts/ssh.Makefile                          | 10 ++-
 scripts/terraform.Makefile                    | 35 ++++++--
 terraform/aws/output.tf                       | 30 ++-----
 terraform/aws/update_ssh_config.tf            |  1 -
 terraform/aws/update_ssh_config_use.tf        | 12 ---
 terraform/azure/output.tf                     | 22 ++---
 terraform/azure/update_ssh_config.tf          |  1 -
 terraform/azure/update_ssh_config_use.tf      |  4 -
 terraform/gce/output.tf                       | 31 ++-----
 terraform/gce/update_ssh_config.tf            |  1 -
 terraform/gce/update_ssh_config_use.tf        |  4 -
 terraform/oci/output.tf                       |  9 ++
 terraform/oci/update_ssh_config.tf            |  1 -
 terraform/oci/update_ssh_config_use.tf        |  8 --
 terraform/openstack/output.tf                 |  9 ++
 terraform/openstack/update_ssh_config.tf      |  1 -
 terraform/openstack/update_ssh_config_use.tf  |  4 -
 terraform/update_ssh_config.tf                | 17 ----
 26 files changed, 198 insertions(+), 153 deletions(-)
 create mode 100644 playbooks/roles/terraform/defaults/main.yml
 create mode 100644 playbooks/roles/terraform/tasks/main.yml
 create mode 100644 playbooks/roles/terraform/templates/ssh_config.j2
 create mode 100644 playbooks/terraform.yml
 delete mode 100755 scripts/bringup_terraform.sh
 delete mode 100755 scripts/destroy_terraform.sh
 delete mode 120000 terraform/aws/update_ssh_config.tf
 delete mode 100644 terraform/aws/update_ssh_config_use.tf
 delete mode 120000 terraform/azure/update_ssh_config.tf
 delete mode 100644 terraform/azure/update_ssh_config_use.tf
 delete mode 120000 terraform/gce/update_ssh_config.tf
 delete mode 100644 terraform/gce/update_ssh_config_use.tf
 create mode 100644 terraform/oci/output.tf
 delete mode 120000 terraform/oci/update_ssh_config.tf
 delete mode 100644 terraform/oci/update_ssh_config_use.tf
 delete mode 120000 terraform/openstack/update_ssh_config.tf
 delete mode 100644 terraform/openstack/update_ssh_config_use.tf
 delete mode 100644 terraform/update_ssh_config.tf

-- 
2.48.1

[PATCH v3 1/6] terraform: Replace scripts/*_terraform.sh with an Ansible playbook

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Refactor: Combine separate terraform scripts into one playbook,
adopting declarative infrastructure-as-code mechanisms rather than
shell scripts. This also makes it simple to iterate over all defined
target nodes -- that's Ansible's bread and butter.

Terraform-specific Ansible-based ssh configuration can now easily
be introduced.

Note that the terraform actions are now silent unless an error
occurs.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/terraform/tasks/main.yml | 18 ++++++++++++++++++
 playbooks/terraform.yml                  |  5 +++++
 scripts/bringup_terraform.sh             | 12 ------------
 scripts/destroy_terraform.sh             | 10 ----------
 scripts/terraform.Makefile               | 12 ++++++++++--
 5 files changed, 33 insertions(+), 24 deletions(-)
 create mode 100644 playbooks/roles/terraform/tasks/main.yml
 create mode 100644 playbooks/terraform.yml
 delete mode 100755 scripts/bringup_terraform.sh
 delete mode 100755 scripts/destroy_terraform.sh

diff --git a/playbooks/roles/terraform/tasks/main.yml b/playbooks/roles/terraform/tasks/main.yml
new file mode 100644
index 000000000000..e328ea7bc0b2
--- /dev/null
+++ b/playbooks/roles/terraform/tasks/main.yml
@@ -0,0 +1,18 @@
+---
+- name: Bring up terraform resources
+  community.general.terraform:
+    project_path: "{{ topdir_path }}/terraform/{{ kdevops_terraform_provider }}"
+    state: present
+    force_init: true
+  tags:
+    - bringup
+
+- name: Destroy terraform resources
+  delegate_to: localhost
+  run_once: true
+  community.general.terraform:
+    project_path: "{{ topdir_path }}/terraform/{{ kdevops_terraform_provider }}"
+    state: absent
+    force_init: true
+  tags:
+    - destroy
diff --git a/playbooks/terraform.yml b/playbooks/terraform.yml
new file mode 100644
index 000000000000..374a76fb0ae7
--- /dev/null
+++ b/playbooks/terraform.yml
@@ -0,0 +1,5 @@
+---
+- hosts: all
+  gather_facts: false
+  roles:
+    - role: terraform
diff --git a/scripts/bringup_terraform.sh b/scripts/bringup_terraform.sh
deleted file mode 100755
index 427962ac6eb9..000000000000
--- a/scripts/bringup_terraform.sh
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: copyleft-next-0.3.1
-
-set -e
-
-source ${TOPDIR}/.config
-source ${TOPDIR}/scripts/lib.sh
-
-cd terraform/${KDEVOPS_CLOUD_PROVIDER}
-terraform init
-terraform plan
-terraform apply -auto-approve
diff --git a/scripts/destroy_terraform.sh b/scripts/destroy_terraform.sh
deleted file mode 100755
index 58d467a40c07..000000000000
--- a/scripts/destroy_terraform.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: copyleft-next-0.3.1
-
-set -e
-
-source ${TOPDIR}/.config
-source ${TOPDIR}/scripts/lib.sh
-
-cd terraform/${KDEVOPS_CLOUD_PROVIDER}
-terraform destroy -auto-approve
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index 888d3af88e3e..2436571a4aac 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -164,10 +164,18 @@ endif # CONFIG_TERRAFORM_SSH_CONFIG_GENKEY
 ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS)
 
 bringup_terraform:
-	$(Q)$(TOPDIR)/scripts/bringup_terraform.sh
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		--connection=local --inventory localhost, \
+		playbooks/terraform.yml --tags bringup \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3'
 
 destroy_terraform:
-	$(Q)$(TOPDIR)/scripts/destroy_terraform.sh
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		--connection=local -i $(KDEVOPS_HOSTFILE) \
+		playbooks/terraform.yml --tags destroy \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3'
 	$(Q)rm -f $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
 $(KDEVOPS_TFVARS): $(KDEVOPS_TFVARS_TEMPLATE) .config
-- 
2.48.1

[PATCH v3 2/6] ssh.Makefile: Define a kdevops_ssh_config variable

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Ensure that all provisioning scripts use the same pathname for the
ssh config file that stores ssh entries for kdevops target nodes
(which I'm referring to as the "ephemeral ssh config file").

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 kconfigs/Kconfig.ssh | 19 ++++++++++++++++---
 scripts/ssh.Makefile | 10 +++++++++-
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/kconfigs/Kconfig.ssh b/kconfigs/Kconfig.ssh
index 66ab3ce6f6ea..3b85568e19b4 100644
--- a/kconfigs/Kconfig.ssh
+++ b/kconfigs/Kconfig.ssh
@@ -18,11 +18,24 @@ config KDEVOPS_SSH_CONFIG_UPDATE
 if KDEVOPS_SSH_CONFIG_UPDATE
 
 config KDEVOPS_SSH_CONFIG
-	string "The ssh configuration file to update for systems created"
+	string "The controller's ssh configuration file"
 	default "~/.ssh/config"
 	help
-	  The ssh configuration file we should update, for each of the systems
-	  created.
+	  The pathname of the Ansible controller's ssh configuration file.
+	  kdevops may add an Include directive to this file that refers to a
+	  separate file where target node ssh configuration is managed.
+
+config KDEVOPS_SSH_CONFIG_PREFIX
+	string "Pathname prefix to an Ansible-managed ssh config file"
+	output yaml
+	default "~/.ssh/config_kdevops_"
+	help
+	  The pathname of a file into which Ansible inserts ssh configuration
+	  information for the target nodes during the "make bringup" step. On
+	  "make destroy" this file is destroyed.
+
+	  This file is kept in the controller's .ssh directory by default,
+	  but can be moved to other locations if desired.
 
 config KDEVOPS_SSH_CONFIG_UPDATE_STRICT
 	bool "Use strict configuration settings when adding each host"
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index 3ee9437b1b4c..d18a03e094ee 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -8,8 +8,16 @@ endif
 
 ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
-ANSIBLE_EXTRA_ARGS += sshconfig=$(CONFIG_KDEVOPS_SSH_CONFIG)
+ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
+
+ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
+SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
+else
+SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
 endif
+ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(shell scripts/append-makefile-vars.sh $(CONFIG_KDEVOPS_SSH_CONFIG_PREFIX) $(SSH_CONFIG_SUFFIX))
+
+endif # CONFIG_KDEVOPS_SSH_CONFIG_UPDATE
 
 PHONY += remove-ssh-key
 remove-ssh-key:
-- 
2.48.1

[PATCH v3 3/6] terraform: Clean up ssh configuration during "make destroy"

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/terraform/tasks/main.yml | 9 +++++++++
 scripts/terraform.Makefile               | 2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/playbooks/roles/terraform/tasks/main.yml b/playbooks/roles/terraform/tasks/main.yml
index e328ea7bc0b2..31c97fdb78b1 100644
--- a/playbooks/roles/terraform/tasks/main.yml
+++ b/playbooks/roles/terraform/tasks/main.yml
@@ -16,3 +16,12 @@
     force_init: true
   tags:
     - destroy
+
+- name: Remove the ephemeral ssh config file on the control host
+  delegate_to: localhost
+  run_once: true
+  ansible.builtin.file:
+    path: "{{ kdevops_ssh_config }}"
+    state: absent
+  tags:
+    - destroy
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index 2436571a4aac..f22538721a55 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -176,7 +176,7 @@ destroy_terraform:
 		playbooks/terraform.yml --tags destroy \
 		--extra-vars=@./extra_vars.yaml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
-	$(Q)rm -f $(KDEVOPS_PROVISIONED_DEVCONFIG)
+	$(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
 $(KDEVOPS_TFVARS): $(KDEVOPS_TFVARS_TEMPLATE) .config
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-- 
2.48.1

[PATCH v3 4/6] terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The fixed update_ssh_config module is still not removing ssh Host
configuration information with "make destroy".

Also, we want to have more control over how the control host's
ssh config is managed. Updating a separately maintained terraform
module is getting awkward.

Replace the independent terraform module that handles ssh
configuration with tasks in a playbook that operate the same as
guestfs: the host config is stuffed into a common file under ~/.ssh
that is included in ~/.ssh/config, and then is easily located and
deleted by "make destroy".

Suggested-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/terraform/defaults/main.yml   |  3 +
 playbooks/roles/terraform/tasks/main.yml      | 59 +++++++++++++++++++
 .../roles/terraform/templates/ssh_config.j2   | 15 +++++
 scripts/terraform.Makefile                    | 21 +++++--
 terraform/aws/output.tf                       |  7 +++
 terraform/azure/output.tf                     |  8 +++
 terraform/gce/output.tf                       |  8 +++
 terraform/oci/output.tf                       |  9 +++
 terraform/openstack/output.tf                 |  7 +++
 9 files changed, 131 insertions(+), 6 deletions(-)
 create mode 100644 playbooks/roles/terraform/defaults/main.yml
 create mode 100644 playbooks/roles/terraform/templates/ssh_config.j2
 create mode 100644 terraform/oci/output.tf

diff --git a/playbooks/roles/terraform/defaults/main.yml b/playbooks/roles/terraform/defaults/main.yml
new file mode 100644
index 000000000000..f3ef9b18c6cd
--- /dev/null
+++ b/playbooks/roles/terraform/defaults/main.yml
@@ -0,0 +1,3 @@
+---
+ssh_config_kexalgorithms: ""
+kdevops_terraform_ssh_config_genkey_overwrite: false
diff --git a/playbooks/roles/terraform/tasks/main.yml b/playbooks/roles/terraform/tasks/main.yml
index 31c97fdb78b1..272ebf93a355 100644
--- a/playbooks/roles/terraform/tasks/main.yml
+++ b/playbooks/roles/terraform/tasks/main.yml
@@ -7,6 +7,65 @@
   tags:
     - bringup
 
+- name: Retrieve the public_ip_map from terraform
+  delegate_to: localhost
+  run_once: true
+  ansible.builtin.command:
+    chdir: "{{ topdir_path }}/terraform/{{ kdevops_terraform_provider }}"
+    cmd: "terraform output -json public_ip_map"
+  register: terraform_output
+  changed_when: false
+  when:
+    - kdevops_terraform_ssh_config_genkey_overwrite|bool
+  tags:
+    - ssh
+
+- name: Convert the retrieved public_ip_map into a dictionary
+  delegate_to: localhost
+  run_once: true
+  ansible.builtin.set_fact:
+    public_ip_map: "{{ terraform_output.stdout | from_json }}"
+  when:
+    - kdevops_terraform_ssh_config_genkey_overwrite|bool
+  tags:
+    - ssh
+
+- name: Add each target node's ssh Host entry on the control host
+  delegate_to: localhost
+  throttle: 1
+  ansible.builtin.blockinfile:
+    block: "{{ lookup('template', 'ssh_config.j2') }}"
+    create: true
+    dest: "{{ kdevops_ssh_config }}"
+    insertafter: "EOF"
+    marker: "# {mark} host configuration for {{ inventory_hostname }}"
+    mode: "u=rw,g=r,o=r"
+  when:
+    - kdevops_terraform_ssh_config_genkey_overwrite|bool
+  tags:
+    - ssh
+
+- name: Ensure the Include directive is present on the controller
+  delegate_to: localhost
+  run_once: true
+  ansible.builtin.blockinfile:
+    path: "{{ sshconfig }}"
+    insertbefore: BOF
+    append_newline: true
+    marker: "# {mark} Managed by kdevops"
+    mode: "u=rw,g=r,o=r"
+    block: "Include {{ kdevops_ssh_config_prefix }}*"
+  when:
+    - kdevops_terraform_ssh_config_genkey_overwrite|bool
+  tags:
+    - ssh
+
+- name: Wait for target nodes to become reachable
+  ansible.builtin.wait_for_connection:
+    timeout: 60
+  tags:
+    - ssh
+
 - name: Destroy terraform resources
   delegate_to: localhost
   run_once: true
diff --git a/playbooks/roles/terraform/templates/ssh_config.j2 b/playbooks/roles/terraform/templates/ssh_config.j2
new file mode 100644
index 000000000000..d9b057b13da8
--- /dev/null
+++ b/playbooks/roles/terraform/templates/ssh_config.j2
@@ -0,0 +1,15 @@
+Host {{ inventory_hostname }} {{ public_ip_map[inventory_hostname] }}
+	HostName {{ public_ip_map[inventory_hostname] }}
+	User {{ kdevops_terraform_ssh_config_user }}
+	Port 22
+	IdentityFile {{ kdevops_terraform_ssh_config_privkey_file }}
+{% if ssh_config_kexalgorithms %}
+	KexAlgorithms {{ ssh_config_kexalgorithms }}
+{% endif %}
+{% if kdevops_terraform_ssh_config_update_strict %}
+	UserKnownHostsFile /dev/null
+	StrictHostKeyChecking no
+	PasswordAuthentication no
+	IdentitiesOnly yes
+	LogLevel FATAL
+{% endif %}
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index f22538721a55..6543da89a17f 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -25,9 +25,10 @@ KDEVOPS_NODES_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/terraform_nodes.tf.
 KDEVOPS_NODES :=		terraform/$(KDEVOPS_CLOUD_PROVIDER)/nodes.tf
 
 TERRAFORM_EXTRA_VARS += kdevops_enable_terraform='True'
-
 TERRAFORM_EXTRA_VARS += kdevops_terraform_provider='$(KDEVOPS_CLOUD_PROVIDER)'
 
+export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD)
+
 TFVARS_TEMPLATE_DIR=playbooks/roles/gen_tfvars/templates
 TFVARS_FILE_NAME=terraform.tfvars
 TFVARS_FILE_POSTFIX=$(TFVARS_FILE_NAME).j2
@@ -145,17 +146,17 @@ endif
 
 endif # CONFIG_KDEVOPS_SSH_CONFIG_UPDATE
 
-TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_pubkey_file='$(subst ",,$(CONFIG_TERRAFORM_SSH_CONFIG_PUBKEY_FILE))'
-TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_user='$(subst ",,$(CONFIG_TERRAFORM_SSH_CONFIG_USER))'
+export KDEVOPS_SSH_PUBKEY:=$(shell realpath $(subst ",,$(CONFIG_TERRAFORM_SSH_CONFIG_PUBKEY_FILE)))
+TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_pubkey_file='$(KDEVOPS_SSH_PUBKEY)'
+TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_user='$(SSH_CONFIG_USER)'
 
 ifeq (y,$(CONFIG_TERRAFORM_SSH_CONFIG_GENKEY))
-export KDEVOPS_SSH_PUBKEY:=$(subst ",,$(CONFIG_TERRAFORM_SSH_CONFIG_PUBKEY_FILE))
-# We have to do shell expansion. Oh, life is so hard.
-export KDEVOPS_SSH_PUBKEY:=$(subst ~,$(HOME),$(KDEVOPS_SSH_PUBKEY))
 export KDEVOPS_SSH_PRIVKEY:=$(basename $(KDEVOPS_SSH_PUBKEY))
+TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_privkey_file='$(KDEVOPS_SSH_PRIVKEY)'
 
 ifeq (y,$(CONFIG_TERRAFORM_SSH_CONFIG_GENKEY_OVERWRITE))
 DEFAULT_DEPS += remove-ssh-key
+TERRAFORM_EXTRA_VARS += kdevops_terraform_ssh_config_genkey_overwrite='True'
 endif
 
 DEFAULT_DEPS += $(KDEVOPS_SSH_PRIVKEY)
@@ -170,6 +171,14 @@ bringup_terraform:
 		--extra-vars=@./extra_vars.yaml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
 
+$(KDEVOPS_PROVISIONED_SSH):
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		-i $(KDEVOPS_HOSTFILE) \
+		playbooks/terraform.yml --tags ssh \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3'
+	$(Q)touch $(KDEVOPS_PROVISIONED_SSH)
+
 destroy_terraform:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
 		--connection=local -i $(KDEVOPS_HOSTFILE) \
diff --git a/terraform/aws/output.tf b/terraform/aws/output.tf
index 6ff195be2515..cb8cab4afcdd 100644
--- a/terraform/aws/output.tf
+++ b/terraform/aws/output.tf
@@ -25,3 +25,10 @@ output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
 
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], aws_eip.kdevops_eip[*].public_ip)}"
+}
diff --git a/terraform/azure/output.tf b/terraform/azure/output.tf
index 5a2654970011..22dfa2f0736d 100644
--- a/terraform/azure/output.tf
+++ b/terraform/azure/output.tf
@@ -37,3 +37,11 @@ data "null_data_source" "group_hostnames_and_ips" {
 output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], azurerm_public_ip.kdevops_publicip[*].ip_address)}"
+}
diff --git a/terraform/gce/output.tf b/terraform/gce/output.tf
index 7b96c829173b..b95667cc7efd 100644
--- a/terraform/gce/output.tf
+++ b/terraform/gce/output.tf
@@ -24,3 +24,11 @@ data "null_data_source" "group_hostnames_and_ips" {
 output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], local.ipv4s[*])}"
+}
diff --git a/terraform/oci/output.tf b/terraform/oci/output.tf
new file mode 100644
index 000000000000..91ef37101c6b
--- /dev/null
+++ b/terraform/oci/output.tf
@@ -0,0 +1,9 @@
+# All generic output goes here
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], oci_core_instance.kdevops_instance.*.public_ip)}"
+}
diff --git a/terraform/openstack/output.tf b/terraform/openstack/output.tf
index 148343561ae5..aff44d1b45f9 100644
--- a/terraform/openstack/output.tf
+++ b/terraform/openstack/output.tf
@@ -16,3 +16,10 @@ output "kdevops_hosts_and_ipv4" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
 
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], openstack_compute_instance_v2.kdevops_instances[*].access_ip_v4)}"
+}
-- 
2.48.1

[PATCH v3 5/6] terraform: "make mrproper" should remove terraform/*/.terraform

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

"make mrproper" should clear this directory out so that the next
kdevops run can retrieve up-to-date modules, as needed.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index a0441b2c7bb8..e0e5d8f962d2 100644
--- a/Makefile
+++ b/Makefile
@@ -252,6 +252,7 @@ mrproper:
 	$(Q)$(MAKE) -f scripts/build.Makefile $@
 	$(Q)rm -f $(KDEVOPS_DEPCHECK)
 	$(Q)rm -f terraform/*/terraform.tfvars
+	$(Q)rm -rf terraform/*/.terraform
 	$(Q)rm -f $(KDEVOPS_NODES)
 	$(Q)rm -f $(KDEVOPS_HOSTFILE) $(KDEVOPS_MRPROPER)
 	$(Q)rm -f .config .config.old extra_vars.yaml $(KCONFIG_YAMLCFG)
-- 
2.48.1

[PATCH v3 6/6] terraform: Remove the terrraform update_ssh_config module

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Unhook terraform's update_ssh_config module now that kdevops handles
this step.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/aws/output.tf                      | 25 ------------------
 terraform/aws/update_ssh_config.tf           |  1 -
 terraform/aws/update_ssh_config_use.tf       | 12 ---------
 terraform/azure/output.tf                    | 18 -------------
 terraform/azure/update_ssh_config.tf         |  1 -
 terraform/azure/update_ssh_config_use.tf     |  4 ---
 terraform/gce/output.tf                      | 27 +-------------------
 terraform/gce/update_ssh_config.tf           |  1 -
 terraform/gce/update_ssh_config_use.tf       |  4 ---
 terraform/oci/update_ssh_config.tf           |  1 -
 terraform/oci/update_ssh_config_use.tf       |  8 ------
 terraform/openstack/output.tf                |  2 ++
 terraform/openstack/update_ssh_config.tf     |  1 -
 terraform/openstack/update_ssh_config_use.tf |  4 ---
 terraform/update_ssh_config.tf               | 17 ------------
 15 files changed, 3 insertions(+), 123 deletions(-)
 delete mode 120000 terraform/aws/update_ssh_config.tf
 delete mode 100644 terraform/aws/update_ssh_config_use.tf
 delete mode 120000 terraform/azure/update_ssh_config.tf
 delete mode 100644 terraform/azure/update_ssh_config_use.tf
 delete mode 120000 terraform/gce/update_ssh_config.tf
 delete mode 100644 terraform/gce/update_ssh_config_use.tf
 delete mode 120000 terraform/oci/update_ssh_config.tf
 delete mode 100644 terraform/oci/update_ssh_config_use.tf
 delete mode 120000 terraform/openstack/update_ssh_config.tf
 delete mode 100644 terraform/openstack/update_ssh_config_use.tf
 delete mode 100644 terraform/update_ssh_config.tf

diff --git a/terraform/aws/output.tf b/terraform/aws/output.tf
index cb8cab4afcdd..83a85a388055 100644
--- a/terraform/aws/output.tf
+++ b/terraform/aws/output.tf
@@ -1,30 +1,5 @@
 # All generic output goes here
 
-locals {
-  ssh_key_i = format(
-    " %s%s ",
-    var.ssh_config_pubkey_file != "" ? "-i " : "",
-    var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "",
-  )
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    value = format(
-      "%30s  :  ssh %s@%s %s ",
-      element(var.kdevops_nodes, count.index),
-      var.ssh_config_user,
-      element(aws_eip.kdevops_eip.*.public_ip, count.index),
-      local.ssh_key_i,
-    )
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
-
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
 # Each map entry contains the node's hostname and public IP address.
diff --git a/terraform/aws/update_ssh_config.tf b/terraform/aws/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/aws/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/aws/update_ssh_config_use.tf b/terraform/aws/update_ssh_config_use.tf
deleted file mode 100644
index c33d9b6a2ec4..000000000000
--- a/terraform/aws/update_ssh_config_use.tf
+++ /dev/null
@@ -1,12 +0,0 @@
-locals {
-  all_tags    = aws_instance.kdevops_instance.*.tags
-  shorthosts = [
-    for tags in local.all_tags :
-    format("%s", lookup(tags, "Name"))
-  ]
-  all_ipv4s = aws_eip.kdevops_eip.*.public_ip
-  ipv4s = [
-    for ip in local.all_ipv4s :
-    ip == "" ? "0.0.0.0" : ip
-  ]
-}
diff --git a/terraform/azure/output.tf b/terraform/azure/output.tf
index 22dfa2f0736d..ef5e3eca6883 100644
--- a/terraform/azure/output.tf
+++ b/terraform/azure/output.tf
@@ -20,24 +20,6 @@ output "kdevops_public_ip_addresses" {
   value = data.azurerm_public_ip.public_ips.*.ip_address
 }
 
-locals {
-  ssh_key_i = format(" %s%s ", var.ssh_config_pubkey_file != "" ? "-i " : "", var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "")
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    # In theory using "${self.triggers["name"]}" and "${self.triggersp["ip"]}"
-    # would be nice but it is not supported in this context, only in the
-    # provisioner and connection contexts.
-    value = "${format("%30s  :  ssh %s@%s %s ", element(azurerm_linux_virtual_machine.kdevops_vm.*.name, count.index), var.ssh_config_user, element(azurerm_public_ip.kdevops_publicip.*.ip_address, count.index), local.ssh_key_i)}"
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
-
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
 # Each map entry contains the node's hostname and public IP address.
diff --git a/terraform/azure/update_ssh_config.tf b/terraform/azure/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/azure/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/azure/update_ssh_config_use.tf b/terraform/azure/update_ssh_config_use.tf
deleted file mode 100644
index 9b91206a66b1..000000000000
--- a/terraform/azure/update_ssh_config_use.tf
+++ /dev/null
@@ -1,4 +0,0 @@
-locals {
-  shorthosts  = azurerm_linux_virtual_machine.kdevops_vm.*.name
-  ipv4s       = data.azurerm_public_ip.public_ips.*.ip_address
-}
diff --git a/terraform/gce/output.tf b/terraform/gce/output.tf
index b95667cc7efd..470617f700ee 100644
--- a/terraform/gce/output.tf
+++ b/terraform/gce/output.tf
@@ -1,29 +1,4 @@
-locals {
-  ssh_key_i          = format(" %s%s ", var.ssh_config_pubkey_file != "" ? "-i " : "", var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "")
-  network_interfaces = google_compute_instance.kdevops_instances.*.network_interface
-  access_configs = [
-    for net_interface in local.network_interfaces :
-    net_interface[0].access_config
-  ]
-  ipv4s = [
-    for access_config in local.access_configs :
-    access_config[0].nat_ip
-  ]
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    # In theory using "${self.triggers["name"]}" and "${self.triggersp["ip"]}"
-    # would be nice but it is not supported in this context, only in the
-    # provisioner and connection contexts.
-    value = "${format("%30s  :  ssh %s@%s %s ", element(google_compute_instance.kdevops_instances.*.name, count.index), var.ssh_config_user, element(local.ipv4s, count.index), local.ssh_key_i)}"
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
+# All generic output goes here
 
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
diff --git a/terraform/gce/update_ssh_config.tf b/terraform/gce/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/gce/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/gce/update_ssh_config_use.tf b/terraform/gce/update_ssh_config_use.tf
deleted file mode 100644
index 6eb008695576..000000000000
--- a/terraform/gce/update_ssh_config_use.tf
+++ /dev/null
@@ -1,4 +0,0 @@
-locals {
-  shorthosts  = google_compute_instance.kdevops_instances.*.name
-  all_ipv4s   = local.ipv4s
-}
diff --git a/terraform/oci/update_ssh_config.tf b/terraform/oci/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/oci/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/oci/update_ssh_config_use.tf b/terraform/oci/update_ssh_config_use.tf
deleted file mode 100644
index f4b2519b515b..000000000000
--- a/terraform/oci/update_ssh_config_use.tf
+++ /dev/null
@@ -1,8 +0,0 @@
-locals {
-  shorthosts	= oci_core_instance.kdevops_instance.*.display_name
-  ipv4s		= (
-    var.oci_assign_public_ip == "false" ?
-    oci_core_instance.kdevops_instance.*.private_ip :
-    oci_core_instance.kdevops_instance.*.public_ip
-  )
-}
diff --git a/terraform/openstack/output.tf b/terraform/openstack/output.tf
index aff44d1b45f9..2d60cc46c030 100644
--- a/terraform/openstack/output.tf
+++ b/terraform/openstack/output.tf
@@ -1,3 +1,5 @@
+# All generic output goes here
+
 data "null_data_source" "group_hostnames_and_ips" {
   count = local.kdevops_num_boxes
   inputs = {
diff --git a/terraform/openstack/update_ssh_config.tf b/terraform/openstack/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/openstack/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/openstack/update_ssh_config_use.tf b/terraform/openstack/update_ssh_config_use.tf
deleted file mode 100644
index 9d4ceb31d9fb..000000000000
--- a/terraform/openstack/update_ssh_config_use.tf
+++ /dev/null
@@ -1,4 +0,0 @@
-locals {
-  shorthosts  = openstack_compute_instance_v2.kdevops_instances.*.name
-  ipv4s       = openstack_compute_instance_v2.kdevops_instances.*.access_ip_v4
-}
diff --git a/terraform/update_ssh_config.tf b/terraform/update_ssh_config.tf
deleted file mode 100644
index 03f0cbed424a..000000000000
--- a/terraform/update_ssh_config.tf
+++ /dev/null
@@ -1,17 +0,0 @@
-module "ssh_config_update_host_entries" {
-  source  = "linux-kdevops/add-host-ssh-config/kdevops"
-  version = "3.0.0"
-
-  ssh_config               = var.ssh_config
-  update_ssh_config_enable = var.ssh_config_update
-  cmd                      = "update"
-  shorthosts               = join(",", slice(local.shorthosts, 0, local.kdevops_num_boxes))
-  hostnames                = join(",", slice(local.ipv4s, 0, local.kdevops_num_boxes))
-  ports                    = "22"
-  user                     = var.ssh_config_user == "" ? "" : var.ssh_config_user
-  id                       = replace(var.ssh_config_pubkey_file, ".pub", "")
-  strict                   = var.ssh_config_use_strict_settings ? "true" : ""
-  use_backup               = !var.ssh_config_backup || var.ssh_config == "/dev/null" ? "" : "true"
-  backup_postfix           = "kdevops"
-  kexalgorithms            = var.ssh_config_kexalgorithms == "" ? "" : var.ssh_config_kexalgorithms
-}
-- 
2.48.1