[PATCH v2 00/12] Replace terraform update_ssh_config module

[PATCH v2 00/12] Replace terraform update_ssh_config module

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Luis suggested merging the way guestfs and terraform handle ssh
host config. Luis, please let me know if I've grossly misunderstood
your suggestion.

Vagrant is deprecated, so it will not be modified by these changes.
The two remaining virtualization mechanisms have the same underlying
purpose:

 - Set up ssh to enable Ansible to have passwordless access to
   the target nodes

 - Make human ssh to the target nodes as painless as possible

But they have grown apart in implementation over the years. This
series is an attempt to build shared infrastructure that can be
triggered by the common "bringup" and "destroy" make targets.

Eventually IMO the SSH Kconfig options for terraform need to be
merged with the options from kconfigs/Kconfig.ssh. This series does
not attempt to do that.

In preparation for possibly moving the location of the
config_kdevops_* file, this series now treats the .ssh/config
Include directive a bit more carefully and consistently.

Changes since RFC:
- Split the RFC series into smaller patches for easier review
- The "Include" directive now follows CONFIG_KDEVOPS_SSH_CONFIG
- Various bug fixes and optimizations

Chuck Lever (12):
  guestfs: Rename the update_ssh_config_guestfs role
  update_ssh_config: Use {{ sshconfig }} instead of raw path
  update_ssh_config: Fix ansible-lint nits
  update_ssh_config: Run update_ssh_config during "make deps"
  update_ssh_config: Add tags to steps
  ssh.Makefile: Generate an sshdir variable
  update_ssh_config: make Include directive follow ssh config directory
  ssh.Makefile: Define a kdevops_ssh_config variable
  update_ssh_config: Add always-run ssh clean-up steps
  terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}
  terraform: "make mrproper" should remove terraform/*/.terraform
  terraform: Remove the terrraform update_ssh_config module

 Makefile                                      |  1 +
 playbooks/add_ssh_hosts_terraform.yml         |  5 ++
 .../add_ssh_hosts_terraform/defaults/main.yml |  2 +
 .../add_ssh_hosts_terraform/tasks/main.yml    | 33 +++++++
 .../templates/ssh_config.j2                   | 15 ++++
 .../roles/update_ssh_config/tasks/main.yml    | 88 +++++++++++++++++++
 .../update_ssh_config_guestfs/tasks/main.yml  | 71 ---------------
 playbooks/update_ssh_config.yml               |  4 +
 playbooks/update_ssh_config_guestfs.yml       |  4 -
 scripts/bringup.Makefile                      |  6 ++
 scripts/destroy_guestfs.sh                    |  6 --
 scripts/guestfs.Makefile                      |  6 --
 scripts/ssh.Makefile                          | 22 +++++
 scripts/terraform.Makefile                    |  6 +-
 terraform/aws/output.tf                       | 30 ++-----
 terraform/aws/update_ssh_config.tf            |  1 -
 terraform/azure/output.tf                     | 22 ++---
 terraform/azure/update_ssh_config.tf          |  1 -
 terraform/gce/output.tf                       | 31 ++-----
 terraform/gce/update_ssh_config.tf            |  1 -
 terraform/oci/output.tf                       |  9 ++
 terraform/oci/update_ssh_config.tf            |  1 -
 terraform/openstack/output.tf                 |  9 ++
 terraform/openstack/update_ssh_config.tf      |  1 -
 terraform/update_ssh_config.tf                | 17 ----
 25 files changed, 218 insertions(+), 174 deletions(-)
 create mode 100644 playbooks/add_ssh_hosts_terraform.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2
 create mode 100644 playbooks/roles/update_ssh_config/tasks/main.yml
 delete mode 100644 playbooks/roles/update_ssh_config_guestfs/tasks/main.yml
 create mode 100644 playbooks/update_ssh_config.yml
 delete mode 100644 playbooks/update_ssh_config_guestfs.yml
 delete mode 120000 terraform/aws/update_ssh_config.tf
 delete mode 120000 terraform/azure/update_ssh_config.tf
 delete mode 120000 terraform/gce/update_ssh_config.tf
 create mode 100644 terraform/oci/output.tf
 delete mode 120000 terraform/oci/update_ssh_config.tf
 delete mode 120000 terraform/openstack/update_ssh_config.tf
 delete mode 100644 terraform/update_ssh_config.tf

-- 
2.48.1

[PATCH v2 01/12] guestfs: Rename the update_ssh_config_guestfs role

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The plan is to use this same mechanism for terraform as well. Rename
the role with a generic name.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../tasks/main.yml                                            | 0
 playbooks/update_ssh_config.yml                               | 4 ++++
 playbooks/update_ssh_config_guestfs.yml                       | 4 ----
 scripts/guestfs.Makefile                                      | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
 rename playbooks/roles/{update_ssh_config_guestfs => update_ssh_config}/tasks/main.yml (100%)
 create mode 100644 playbooks/update_ssh_config.yml
 delete mode 100644 playbooks/update_ssh_config_guestfs.yml

diff --git a/playbooks/roles/update_ssh_config_guestfs/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
similarity index 100%
rename from playbooks/roles/update_ssh_config_guestfs/tasks/main.yml
rename to playbooks/roles/update_ssh_config/tasks/main.yml
diff --git a/playbooks/update_ssh_config.yml b/playbooks/update_ssh_config.yml
new file mode 100644
index 000000000000..4c0315c24725
--- /dev/null
+++ b/playbooks/update_ssh_config.yml
@@ -0,0 +1,4 @@
+---
+- hosts: localhost
+  roles:
+    - role: update_ssh_config
diff --git a/playbooks/update_ssh_config_guestfs.yml b/playbooks/update_ssh_config_guestfs.yml
deleted file mode 100644
index 346b90245637..000000000000
--- a/playbooks/update_ssh_config_guestfs.yml
+++ /dev/null
@@ -1,4 +0,0 @@
----
-- hosts: localhost
-  roles:
-    - role: update_ssh_config_guestfs
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 52397d54b89b..fb47bf21ab2f 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -64,7 +64,7 @@ $(KDEVOPS_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 			--inventory localhost, \
-			playbooks/update_ssh_config_guestfs.yml \
+			playbooks/update_ssh_config.yml \
 			--extra-vars=@./extra_vars.yaml \
 			-e 'ansible_python_interpreter=/usr/bin/python3' ;\
 		LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \
-- 
2.48.1

[PATCH v2 02/12] update_ssh_config: Use {{ sshconfig }} instead of raw path

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The sshconfig variable is set by Kconfig. The update_ssh_config role
needs to follow that setting, but was using a fixed "~/.ssh/config"
string instead.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../roles/update_ssh_config/tasks/main.yml    | 23 ++++++++-----------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
index 98c86f164612..01fac9fbb69e 100644
--- a/playbooks/roles/update_ssh_config/tasks/main.yml
+++ b/playbooks/roles/update_ssh_config/tasks/main.yml
@@ -1,12 +1,12 @@
-- name: Check if the ssh config file exists
+- name: Check that the controller's ssh config file exists
   stat:
-    path: "~/.ssh/config"
+    path: "{{ sshconfig }}"
   register: ssh_config
 
 # Check if the include directive is already presetn
 - name: Check if the kdevops include directive was used
   lineinfile:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     regexp: "Include ~/.ssh/config_kdevops_*"
     state: absent
   check_mode: yes
@@ -17,7 +17,7 @@
 # Check if the the kdevops_version was added in a comment
 - name: Check if the new include directive was used with a kdevops_version comment
   lineinfile:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops_version(.*)"
     state: absent
   check_mode: yes
@@ -34,34 +34,31 @@
     - kdevops_ssh_include.found
     - fixed_ssh_entry.found
 
-# If we're still running it means the correct include directive following a new
-# line was not found. So remove old stale include directives which may be
-# buggy.
-- name: Remove buggy stale include directive to ~/.ssh/config without a new line
+- name: Remove the stale Include directive
   lineinfile:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     line: "Include ~/.ssh/config_kdevops_*"
     state: absent
   when: ssh_config.stat.exists
 
 - name: Remove any stale kdevops comments
   lineinfile:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops(.*)"
     state: absent
   when: ssh_config.stat.exists
 
 - name: Remove any extra new lines
   replace:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     regexp: '(^\s*$)'
     replace: ''
   when: ssh_config.stat.exists
 
 # ssh include directives must follow a new line.
-- name: Add Include directive to ~/.ssh/config
+- name: Add a proper Include directive to {{ sshconfig }}
   blockinfile:
-    path: ~/.ssh/config
+    path: "{{ sshconfig }}"
     insertbefore: BOF
     marker: "{mark}"
     marker_begin: "# Automatically added by kdevops\n# kdevops_version: {{ kdevops_version }}"
-- 
2.48.1

[PATCH v2 03/12] update_ssh_config: Fix ansible-lint nits

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

ansible-lint called out a few nits:

 - Use FQDN for module names
 - Update truthy values to true/false
 - Add a "mode:" parameter to blockinfile

As an additional clean-up, remove comments that more-or-less repeat
what is already in the "- name:" of a step.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../roles/update_ssh_config/tasks/main.yml    | 32 +++++++++----------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
index 01fac9fbb69e..6a7b58d8a556 100644
--- a/playbooks/roles/update_ssh_config/tasks/main.yml
+++ b/playbooks/roles/update_ssh_config/tasks/main.yml
@@ -1,26 +1,25 @@
+---
 - name: Check that the controller's ssh config file exists
-  stat:
+  ansible.builtin.stat:
     path: "{{ sshconfig }}"
   register: ssh_config
 
-# Check if the include directive is already presetn
-- name: Check if the kdevops include directive was used
-  lineinfile:
+- name: Check that the kdevops Include directive is present
+  ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "Include ~/.ssh/config_kdevops_*"
     state: absent
-  check_mode: yes
+  check_mode: true
   changed_when: false
   register: kdevops_ssh_include
   when: ssh_config.stat.exists
 
-# Check if the the kdevops_version was added in a comment
-- name: Check if the new include directive was used with a kdevops_version comment
-  lineinfile:
+- name: Check that the Include directive has a kdevops_version comment
+  ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops_version(.*)"
     state: absent
-  check_mode: yes
+  check_mode: true
   changed_when: false
   register: fixed_ssh_entry
   when: ssh_config.stat.exists
@@ -28,28 +27,28 @@
 # If both the include directive was found and kdevops version comment was found
 # we bail right away to avoid updating the ssh config file always.
 - name: Check if the new fixed include directive was used
-  meta: end_play
+  ansible.builtin.meta: end_play
   when:
     - ssh_config.stat.exists
     - kdevops_ssh_include.found
     - fixed_ssh_entry.found
 
 - name: Remove the stale Include directive
-  lineinfile:
+  ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     line: "Include ~/.ssh/config_kdevops_*"
     state: absent
   when: ssh_config.stat.exists
 
-- name: Remove any stale kdevops comments
-  lineinfile:
+- name: Remove stale kdevops comments
+  ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops(.*)"
     state: absent
   when: ssh_config.stat.exists
 
-- name: Remove any extra new lines
-  replace:
+- name: Remove extraneous extra new lines
+  ansible.builtin.replace:
     path: "{{ sshconfig }}"
     regexp: '(^\s*$)'
     replace: ''
@@ -57,12 +56,13 @@
 
 # ssh include directives must follow a new line.
 - name: Add a proper Include directive to {{ sshconfig }}
-  blockinfile:
+  ansible.builtin.blockinfile:
     path: "{{ sshconfig }}"
     insertbefore: BOF
     marker: "{mark}"
     marker_begin: "# Automatically added by kdevops\n# kdevops_version: {{ kdevops_version }}"
     marker_end: ""
     create: true
+    mode: "u=rw,g=r,o=r"
     block: |
       Include ~/.ssh/config_kdevops_*
-- 
2.48.1

[PATCH v2 04/12] update_ssh_config: Run update_ssh_config during "make deps"

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

This situates the Include directive into the controller's
.ssh/config for all virtualization methods.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/guestfs.Makefile |  5 -----
 scripts/ssh.Makefile     | 12 ++++++++++++
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index fb47bf21ab2f..5a75889ec148 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -62,11 +62,6 @@ libvirt_pcie_passthrough_permissions:
 
 $(KDEVOPS_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
-		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-			--inventory localhost, \
-			playbooks/update_ssh_config.yml \
-			--extra-vars=@./extra_vars.yaml \
-			-e 'ansible_python_interpreter=/usr/bin/python3' ;\
 		LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \
 	fi
 	$(Q)ansible $(ANSIBLE_VERBOSE) -i hosts all -e 'ansible_python_interpreter=/usr/bin/python3' -m wait_for_connection
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index 3ee9437b1b4c..cf79a1dd6c27 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -21,3 +21,15 @@ $(KDEVOPS_SSH_PRIVKEY): .config
 	$(NQ) Generating new private key: $(KDEVOPS_SSH_PRIVKEY)
 	$(NQ) Generating new public key: $(KDEVOPS_SSH_PUBKEY)
 	$(Q)$(TOPDIR)/scripts/gen_ssh_key.sh
+
+PHONY += update-ssh-config
+update-ssh-config:
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+		--inventory localhost, \
+		playbooks/update_ssh_config.yml \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3'
+
+ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
+LOCALHOST_SETUP_WORK += update-ssh-config
+endif
-- 
2.48.1

[PATCH v2 05/12] update_ssh_config: Add tags to steps

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

I'm about to add new functions to the update_ssh_config role that
can be run by specifying tags on the ansible-playbook command line.

To run only the steps that update the controller's .ssh/config
Include directive, add a "deps" tag.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/update_ssh_config/tasks/main.yml | 8 ++++++++
 scripts/ssh.Makefile                             | 2 +-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
index 6a7b58d8a556..417781936f7f 100644
--- a/playbooks/roles/update_ssh_config/tasks/main.yml
+++ b/playbooks/roles/update_ssh_config/tasks/main.yml
@@ -1,10 +1,12 @@
 ---
 - name: Check that the controller's ssh config file exists
+  tags: deps
   ansible.builtin.stat:
     path: "{{ sshconfig }}"
   register: ssh_config
 
 - name: Check that the kdevops Include directive is present
+  tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "Include ~/.ssh/config_kdevops_*"
@@ -15,6 +17,7 @@
   when: ssh_config.stat.exists
 
 - name: Check that the Include directive has a kdevops_version comment
+  tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops_version(.*)"
@@ -27,6 +30,7 @@
 # If both the include directive was found and kdevops version comment was found
 # we bail right away to avoid updating the ssh config file always.
 - name: Check if the new fixed include directive was used
+  tags: deps
   ansible.builtin.meta: end_play
   when:
     - ssh_config.stat.exists
@@ -34,6 +38,7 @@
     - fixed_ssh_entry.found
 
 - name: Remove the stale Include directive
+  tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     line: "Include ~/.ssh/config_kdevops_*"
@@ -41,6 +46,7 @@
   when: ssh_config.stat.exists
 
 - name: Remove stale kdevops comments
+  tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
     regexp: "^#(.*)kdevops(.*)"
@@ -48,6 +54,7 @@
   when: ssh_config.stat.exists
 
 - name: Remove extraneous extra new lines
+  tags: deps
   ansible.builtin.replace:
     path: "{{ sshconfig }}"
     regexp: '(^\s*$)'
@@ -56,6 +63,7 @@
 
 # ssh include directives must follow a new line.
 - name: Add a proper Include directive to {{ sshconfig }}
+  tags: deps
   ansible.builtin.blockinfile:
     path: "{{ sshconfig }}"
     insertbefore: BOF
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index cf79a1dd6c27..caa4bc797c48 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -25,7 +25,7 @@ $(KDEVOPS_SSH_PRIVKEY): .config
 PHONY += update-ssh-config
 update-ssh-config:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-		--inventory localhost, \
+		--inventory localhost, --tags deps \
 		playbooks/update_ssh_config.yml \
 		--extra-vars=@./extra_vars.yaml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
-- 
2.48.1

[PATCH v2 06/12] ssh.Makefile: Generate an sshdir variable

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The update_ssh_config role needs to update some files under .ssh/
besides the ssh config file. Derive a pathname of that directory
for the Kconfig-specified sshconfig variable, and set that as a
global variable (it's needed in more than one place).

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/ssh.Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index caa4bc797c48..150001d8bbbd 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -9,6 +9,7 @@ endif
 ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(CONFIG_KDEVOPS_SSH_CONFIG)
+ANSIBLE_EXTRA_ARGS += sshdir=$(shell dirname $(SSH_CONFIG_FILE))
 endif
 
 PHONY += remove-ssh-key
-- 
2.48.1

[PATCH v2 07/12] update_ssh_config: make Include directive follow ssh config directory

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The file containing the kdevops target node entries needs to be
placed in the same directory as the controller's .ssh/config.
Usually that's ~/.ssh/config, but there is a Kconfig setting that
can change that.

XXX: Actually, to make this work 100% correctly, "make mrproper"
should remove the ssh Include directive. That way a user can modify
the ssh config via "make menuconfig", then do a "make" to get the
updated include path added into her .ssh/config.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../roles/update_ssh_config/tasks/main.yml     | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)

diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
index 417781936f7f..72c5f2e365f5 100644
--- a/playbooks/roles/update_ssh_config/tasks/main.yml
+++ b/playbooks/roles/update_ssh_config/tasks/main.yml
@@ -5,18 +5,7 @@
     path: "{{ sshconfig }}"
   register: ssh_config
 
-- name: Check that the kdevops Include directive is present
-  tags: deps
-  ansible.builtin.lineinfile:
-    path: "{{ sshconfig }}"
-    regexp: "Include ~/.ssh/config_kdevops_*"
-    state: absent
-  check_mode: true
-  changed_when: false
-  register: kdevops_ssh_include
-  when: ssh_config.stat.exists
-
-- name: Check that the Include directive has a kdevops_version comment
+- name: Check for a kdevops_version comment
   tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
@@ -34,10 +23,9 @@
   ansible.builtin.meta: end_play
   when:
     - ssh_config.stat.exists
-    - kdevops_ssh_include.found
     - fixed_ssh_entry.found
 
-- name: Remove the stale Include directive
+- name: Remove any stale Include directives
   tags: deps
   ansible.builtin.lineinfile:
     path: "{{ sshconfig }}"
@@ -73,4 +61,4 @@
     create: true
     mode: "u=rw,g=r,o=r"
     block: |
-      Include ~/.ssh/config_kdevops_*
+      Include {{ sshdir }}/config_kdevops_*
-- 
2.48.1

[PATCH v2 08/12] ssh.Makefile: Define a kdevops_ssh_config variable

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

Create a global variable that contains the pathname of the config
file that stores ephemeral ssh host entries. That way the scripts
don't have to keep creating this pathname.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/ssh.Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index 150001d8bbbd..58ffe1e05d68 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -7,10 +7,19 @@ ANSIBLE_EXTRA_ARGS += kexalgorithms=$(SSH_KEXALGORITHMS)
 endif
 
 ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
+
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(CONFIG_KDEVOPS_SSH_CONFIG)
 ANSIBLE_EXTRA_ARGS += sshdir=$(shell dirname $(SSH_CONFIG_FILE))
+
+ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
+SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
+else
+SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
 endif
+ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(CONFIG_KDEVOPS_SSH_CONFIG)_kdevops_$(SSH_CONFIG_SUFFIX)
+
+endif # CONFIG_KDEVOPS_SSH_CONFIG_UPDATE
 
 PHONY += remove-ssh-key
 remove-ssh-key:
-- 
2.48.1

[PATCH v2 09/12] update_ssh_config: Add always-run ssh clean-up steps

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

This common bit of logic is now run by the "make destroy" target in
order that it affects all virtualization methods.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../roles/update_ssh_config/tasks/main.yml    | 24 +++++++++++++++++++
 scripts/bringup.Makefile                      |  6 +++++
 scripts/destroy_guestfs.sh                    |  6 -----
 scripts/guestfs.Makefile                      |  1 -
 scripts/terraform.Makefile                    |  1 -
 5 files changed, 30 insertions(+), 8 deletions(-)

diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
index 72c5f2e365f5..781d70fbaf53 100644
--- a/playbooks/roles/update_ssh_config/tasks/main.yml
+++ b/playbooks/roles/update_ssh_config/tasks/main.yml
@@ -62,3 +62,27 @@
     mode: "u=rw,g=r,o=r"
     block: |
       Include {{ sshdir }}/config_kdevops_*
+
+- name: Remove saved keys for the target nodes
+  tags: destroy
+  throttle: 1
+  ansible.builtin.command:
+    argv:
+      - "ssh-keygen"
+      - "-q"
+      - "-f"
+      - "{{ sshdir }}/known_hosts"
+      - "-R"
+      - "{{ inventory_hostname }}"
+  register: result
+  failed_when: false
+  changed_when:
+    - result is success
+    - not "not found in" in result.stderr
+
+- name: Remove the ephemeral ssh config file on the control host
+  tags: destroy
+  run_once: true
+  ansible.builtin.file:
+    path: "{{ kdevops_ssh_config }}"
+    state: absent
diff --git a/scripts/bringup.Makefile b/scripts/bringup.Makefile
index 148547c0644a..a19cffa39cae 100644
--- a/scripts/bringup.Makefile
+++ b/scripts/bringup.Makefile
@@ -23,6 +23,12 @@ endif
 bringup: $(KDEVOPS_BRING_UP_DEPS)
 
 destroy: $(KDEVOPS_DESTROY_DEPS)
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+		--inventory localhost, --tags destroy \
+		playbooks/update_ssh_config.yml \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3' \
+	$(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
 bringup-help-menu:
 	@echo "Bringup targets:"
diff --git a/scripts/destroy_guestfs.sh b/scripts/destroy_guestfs.sh
index 58dca78d85cf..dd32e7398365 100755
--- a/scripts/destroy_guestfs.sh
+++ b/scripts/destroy_guestfs.sh
@@ -23,14 +23,8 @@ if [ -f "$GUESTFSDIR/kdevops_nodes.yaml" ]; then
 		fi
 		rm -rf "$GUESTFSDIR/$name"
 		rm -rf "$STORAGEDIR/$name"
-		ssh-keygen -q -f ~/.ssh/known_hosts -R $name 1> /dev/null 2>&1
 	done
 fi
 
-if [[ "$CONFIG_TOPDIR_PATH_HAS_SHA256SUM" == "y" ]]; then
-	rm -f ~/.ssh/config_kdevops_$CONFIG_TOPDIR_PATH_SHA256SUM
-else
-	rm -f ~/.ssh/config_kdevops_$CONFIG_KDEVOPS_HOSTS_PREFIX
-fi
 rm -f $GUESTFSDIR/.provisioned_once
 rm -f $GUESTFSDIR/kdevops_nodes.yaml
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 5a75889ec148..5a86c3def412 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -85,7 +85,6 @@ PHONY += bringup_guestfs
 
 destroy_guestfs:
 	$(Q)$(TOPDIR)/scripts/destroy_guestfs.sh
-	$(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
 PHONY += destroy_guestfs
 
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index 888d3af88e3e..b4543d2561fb 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -168,7 +168,6 @@ bringup_terraform:
 
 destroy_terraform:
 	$(Q)$(TOPDIR)/scripts/destroy_terraform.sh
-	$(Q)rm -f $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
 $(KDEVOPS_TFVARS): $(KDEVOPS_TFVARS_TEMPLATE) .config
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-- 
2.48.1

[PATCH v2 10/12] terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The fixed update_ssh_config module is still not removing ssh Host
configuration information with "make destroy".

Also, we want to have more control over how the control host's
ssh config is managed. Updating a separate terraform module is
getting awkward.

Let's replace the independent terraform module that handles ssh
configuration with a playbook that operates the same as guestfs:
the host config is stuffed into a common file under ~/.ssh that
is included in ~/.ssh/config, and is easily located and deleted
by "make destroy".

XXX: I'm not 100% sold on this organization: it might be better
to fold the new playbook into scripts/bringup_terraform.sh
somehow.

Suggested-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/add_ssh_hosts_terraform.yml         |  5 +++
 .../add_ssh_hosts_terraform/defaults/main.yml |  2 ++
 .../add_ssh_hosts_terraform/tasks/main.yml    | 33 +++++++++++++++++++
 .../templates/ssh_config.j2                   | 15 +++++++++
 scripts/terraform.Makefile                    |  5 +++
 terraform/aws/output.tf                       |  7 ++++
 terraform/azure/output.tf                     |  8 +++++
 terraform/gce/output.tf                       |  8 +++++
 terraform/oci/output.tf                       |  9 +++++
 terraform/openstack/output.tf                 |  7 ++++
 10 files changed, 99 insertions(+)
 create mode 100644 playbooks/add_ssh_hosts_terraform.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml
 create mode 100644 playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2
 create mode 100644 terraform/oci/output.tf

diff --git a/playbooks/add_ssh_hosts_terraform.yml b/playbooks/add_ssh_hosts_terraform.yml
new file mode 100644
index 000000000000..b5ef86d09ac9
--- /dev/null
+++ b/playbooks/add_ssh_hosts_terraform.yml
@@ -0,0 +1,5 @@
+---
+- hosts: all
+  gather_facts: false
+  roles:
+    - role: add_ssh_hosts_terraform
diff --git a/playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml b/playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml
new file mode 100644
index 000000000000..33bd00e6d1a4
--- /dev/null
+++ b/playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml
@@ -0,0 +1,2 @@
+---
+ssh_config_kexalgorithms: ""
diff --git a/playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml b/playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml
new file mode 100644
index 000000000000..d10db0175294
--- /dev/null
+++ b/playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml
@@ -0,0 +1,33 @@
+---
+- name: Retrieve the public_ip_map from terraform
+  run_once: true
+  ansible.builtin.command:
+    chdir: "{{ topdir_path }}/terraform/{{ kdevops_terraform_provider }}"
+    cmd: "terraform output -json public_ip_map"
+  register: terraform_output
+  changed_when: false
+
+- name: Convert the public_ip_map into a dictionary
+  run_once: true
+  ansible.builtin.set_fact:
+    public_ip_map: "{{ terraform_output.stdout | from_json }}"
+
+- name: Insert or update the controller's ssh Host entry for {{ inventory_hostname }}
+  vars:
+    hostname: "{{ inventory_hostname }}"
+    ipaddr: "{{ public_ip_map[inventory_hostname] }}"
+    port: "22"
+    user: "{{ kdevops_terraform_ssh_config_user }}"
+    sshkey: "{{ sshdir }}/{{ kdevops_terraform_ssh_config_pubkey_file|basename|replace('.pub', '') }}"
+    strict: "{{ kdevops_terraform_ssh_config_update_strict|bool }}"
+    kexalgorithms: "{{ ssh_config_kexalgorithms }}"
+  throttle: 1
+  ansible.builtin.blockinfile:
+    block: "{{ lookup('template', 'ssh_config.j2') }}"
+    create: true
+    dest: "{{ kdevops_ssh_config }}"
+    insertafter: "EOF"
+    marker: "# {mark} host configuration for {{ inventory_hostname }}"
+    marker_begin: "begin"
+    marker_end: "end"
+    mode: "u=rw,g=r,o=r"
diff --git a/playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2 b/playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2
new file mode 100644
index 000000000000..f212e6e48607
--- /dev/null
+++ b/playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2
@@ -0,0 +1,15 @@
+Host {{ hostname }} {{ ipaddr }}
+	HostName {{ ipaddr }}
+	User {{ user }}
+	Port {{ port }}
+	IdentityFile {{ sshkey }}
+{% if kexalgorithms %}
+	KexAlgorithms {{ kexalgorithms }}
+{% endif %}
+{% if strict %}
+	UserKnownHostsFile /dev/null
+	StrictHostKeyChecking no
+	PasswordAuthentication no
+	IdentitiesOnly yes
+	LogLevel FATAL
+{% endif %}
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index b4543d2561fb..1e86096717cb 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -165,6 +165,11 @@ ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS)
 
 bringup_terraform:
 	$(Q)$(TOPDIR)/scripts/bringup_terraform.sh
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+		--inventory hosts \
+		playbooks/add_ssh_hosts_terraform.yml \
+		--extra-vars=@./extra_vars.yaml \
+		-e 'ansible_python_interpreter=/usr/bin/python3'
 
 destroy_terraform:
 	$(Q)$(TOPDIR)/scripts/destroy_terraform.sh
diff --git a/terraform/aws/output.tf b/terraform/aws/output.tf
index 6ff195be2515..cb8cab4afcdd 100644
--- a/terraform/aws/output.tf
+++ b/terraform/aws/output.tf
@@ -25,3 +25,10 @@ output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
 
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], aws_eip.kdevops_eip[*].public_ip)}"
+}
diff --git a/terraform/azure/output.tf b/terraform/azure/output.tf
index 5a2654970011..a8e32b605a47 100644
--- a/terraform/azure/output.tf
+++ b/terraform/azure/output.tf
@@ -37,3 +37,11 @@ data "null_data_source" "group_hostnames_and_ips" {
 output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], azurerm_public_ip.kdevops_publicip[*].name)}"
+}
diff --git a/terraform/gce/output.tf b/terraform/gce/output.tf
index 7b96c829173b..b95667cc7efd 100644
--- a/terraform/gce/output.tf
+++ b/terraform/gce/output.tf
@@ -24,3 +24,11 @@ data "null_data_source" "group_hostnames_and_ips" {
 output "login_using" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], local.ipv4s[*])}"
+}
diff --git a/terraform/oci/output.tf b/terraform/oci/output.tf
new file mode 100644
index 000000000000..83a85a388055
--- /dev/null
+++ b/terraform/oci/output.tf
@@ -0,0 +1,9 @@
+# All generic output goes here
+
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], aws_eip.kdevops_eip[*].public_ip)}"
+}
diff --git a/terraform/openstack/output.tf b/terraform/openstack/output.tf
index 148343561ae5..aff44d1b45f9 100644
--- a/terraform/openstack/output.tf
+++ b/terraform/openstack/output.tf
@@ -16,3 +16,10 @@ output "kdevops_hosts_and_ipv4" {
   value = data.null_data_source.group_hostnames_and_ips.*.outputs
 }
 
+# Each provider's output.tf needs to define a public_ip_map. This
+# map is used to build the Ansible controller's ssh configuration.
+# Each map entry contains the node's hostname and public IP address.
+output "public_ip_map" {
+  description = "The public IP addresses assigned to each instance"
+  value = "${zipmap(var.kdevops_nodes[*], openstack_compute_instance_v2.kdevops_instances[*].access_ip_v4)}"
+}
-- 
2.48.1

[PATCH v2 11/12] terraform: "make mrproper" should remove terraform/*/.terraform

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

"make mrproper" should clear this directory out so that the next
kdevops run can retrieve up-to-date modules, as needed.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index a0441b2c7bb8..e0e5d8f962d2 100644
--- a/Makefile
+++ b/Makefile
@@ -252,6 +252,7 @@ mrproper:
 	$(Q)$(MAKE) -f scripts/build.Makefile $@
 	$(Q)rm -f $(KDEVOPS_DEPCHECK)
 	$(Q)rm -f terraform/*/terraform.tfvars
+	$(Q)rm -rf terraform/*/.terraform
 	$(Q)rm -f $(KDEVOPS_NODES)
 	$(Q)rm -f $(KDEVOPS_HOSTFILE) $(KDEVOPS_MRPROPER)
 	$(Q)rm -f .config .config.old extra_vars.yaml $(KCONFIG_YAMLCFG)
-- 
2.48.1

[PATCH v2 12/12] terraform: Remove the terrraform update_ssh_config module

[cel]

From: Chuck Lever <chuck.lever@oracle.com>

The add_ssh_hosts_terraform playbook has taken its place.

XXX: Need to add a public_ip_map output for all cloud providers
before removing the update_ssh_config terraform module.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/aws/output.tf                  | 25 ----------------------
 terraform/aws/update_ssh_config.tf       |  1 -
 terraform/azure/output.tf                | 18 ----------------
 terraform/azure/update_ssh_config.tf     |  1 -
 terraform/gce/output.tf                  | 27 +-----------------------
 terraform/gce/update_ssh_config.tf       |  1 -
 terraform/oci/update_ssh_config.tf       |  1 -
 terraform/openstack/output.tf            |  2 ++
 terraform/openstack/update_ssh_config.tf |  1 -
 terraform/update_ssh_config.tf           | 17 ---------------
 10 files changed, 3 insertions(+), 91 deletions(-)
 delete mode 120000 terraform/aws/update_ssh_config.tf
 delete mode 120000 terraform/azure/update_ssh_config.tf
 delete mode 120000 terraform/gce/update_ssh_config.tf
 delete mode 120000 terraform/oci/update_ssh_config.tf
 delete mode 120000 terraform/openstack/update_ssh_config.tf
 delete mode 100644 terraform/update_ssh_config.tf

diff --git a/terraform/aws/output.tf b/terraform/aws/output.tf
index cb8cab4afcdd..83a85a388055 100644
--- a/terraform/aws/output.tf
+++ b/terraform/aws/output.tf
@@ -1,30 +1,5 @@
 # All generic output goes here
 
-locals {
-  ssh_key_i = format(
-    " %s%s ",
-    var.ssh_config_pubkey_file != "" ? "-i " : "",
-    var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "",
-  )
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    value = format(
-      "%30s  :  ssh %s@%s %s ",
-      element(var.kdevops_nodes, count.index),
-      var.ssh_config_user,
-      element(aws_eip.kdevops_eip.*.public_ip, count.index),
-      local.ssh_key_i,
-    )
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
-
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
 # Each map entry contains the node's hostname and public IP address.
diff --git a/terraform/aws/update_ssh_config.tf b/terraform/aws/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/aws/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/azure/output.tf b/terraform/azure/output.tf
index a8e32b605a47..ce5ac79ff694 100644
--- a/terraform/azure/output.tf
+++ b/terraform/azure/output.tf
@@ -20,24 +20,6 @@ output "kdevops_public_ip_addresses" {
   value = data.azurerm_public_ip.public_ips.*.ip_address
 }
 
-locals {
-  ssh_key_i = format(" %s%s ", var.ssh_config_pubkey_file != "" ? "-i " : "", var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "")
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    # In theory using "${self.triggers["name"]}" and "${self.triggersp["ip"]}"
-    # would be nice but it is not supported in this context, only in the
-    # provisioner and connection contexts.
-    value = "${format("%30s  :  ssh %s@%s %s ", element(azurerm_linux_virtual_machine.kdevops_vm.*.name, count.index), var.ssh_config_user, element(azurerm_public_ip.kdevops_publicip.*.ip_address, count.index), local.ssh_key_i)}"
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
-
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
 # Each map entry contains the node's hostname and public IP address.
diff --git a/terraform/azure/update_ssh_config.tf b/terraform/azure/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/azure/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/gce/output.tf b/terraform/gce/output.tf
index b95667cc7efd..470617f700ee 100644
--- a/terraform/gce/output.tf
+++ b/terraform/gce/output.tf
@@ -1,29 +1,4 @@
-locals {
-  ssh_key_i          = format(" %s%s ", var.ssh_config_pubkey_file != "" ? "-i " : "", var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "")
-  network_interfaces = google_compute_instance.kdevops_instances.*.network_interface
-  access_configs = [
-    for net_interface in local.network_interfaces :
-    net_interface[0].access_config
-  ]
-  ipv4s = [
-    for access_config in local.access_configs :
-    access_config[0].nat_ip
-  ]
-}
-
-data "null_data_source" "group_hostnames_and_ips" {
-  count = local.kdevops_num_boxes
-  inputs = {
-    # In theory using "${self.triggers["name"]}" and "${self.triggersp["ip"]}"
-    # would be nice but it is not supported in this context, only in the
-    # provisioner and connection contexts.
-    value = "${format("%30s  :  ssh %s@%s %s ", element(google_compute_instance.kdevops_instances.*.name, count.index), var.ssh_config_user, element(local.ipv4s, count.index), local.ssh_key_i)}"
-  }
-}
-
-output "login_using" {
-  value = data.null_data_source.group_hostnames_and_ips.*.outputs
-}
+# All generic output goes here
 
 # Each provider's output.tf needs to define a public_ip_map. This
 # map is used to build the Ansible controller's ssh configuration.
diff --git a/terraform/gce/update_ssh_config.tf b/terraform/gce/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/gce/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/oci/update_ssh_config.tf b/terraform/oci/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/oci/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/openstack/output.tf b/terraform/openstack/output.tf
index aff44d1b45f9..2d60cc46c030 100644
--- a/terraform/openstack/output.tf
+++ b/terraform/openstack/output.tf
@@ -1,3 +1,5 @@
+# All generic output goes here
+
 data "null_data_source" "group_hostnames_and_ips" {
   count = local.kdevops_num_boxes
   inputs = {
diff --git a/terraform/openstack/update_ssh_config.tf b/terraform/openstack/update_ssh_config.tf
deleted file mode 120000
index 03cd77a65841..000000000000
--- a/terraform/openstack/update_ssh_config.tf
+++ /dev/null
@@ -1 +0,0 @@
-../update_ssh_config.tf
\ No newline at end of file
diff --git a/terraform/update_ssh_config.tf b/terraform/update_ssh_config.tf
deleted file mode 100644
index 03f0cbed424a..000000000000
--- a/terraform/update_ssh_config.tf
+++ /dev/null
@@ -1,17 +0,0 @@
-module "ssh_config_update_host_entries" {
-  source  = "linux-kdevops/add-host-ssh-config/kdevops"
-  version = "3.0.0"
-
-  ssh_config               = var.ssh_config
-  update_ssh_config_enable = var.ssh_config_update
-  cmd                      = "update"
-  shorthosts               = join(",", slice(local.shorthosts, 0, local.kdevops_num_boxes))
-  hostnames                = join(",", slice(local.ipv4s, 0, local.kdevops_num_boxes))
-  ports                    = "22"
-  user                     = var.ssh_config_user == "" ? "" : var.ssh_config_user
-  id                       = replace(var.ssh_config_pubkey_file, ".pub", "")
-  strict                   = var.ssh_config_use_strict_settings ? "true" : ""
-  use_backup               = !var.ssh_config_backup || var.ssh_config == "/dev/null" ? "" : "true"
-  backup_postfix           = "kdevops"
-  kexalgorithms            = var.ssh_config_kexalgorithms == "" ? "" : var.ssh_config_kexalgorithms
-}
-- 
2.48.1

Re: [PATCH v2 08/12] ssh.Makefile: Define a kdevops_ssh_config variable

[Luis Chamberlain]

On Wed, Feb 05, 2025 at 10:52:48AM -0500, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Create a global variable that contains the pathname of the config
> file that stores ephemeral ssh host entries. That way the scripts
> don't have to keep creating this pathname.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  scripts/ssh.Makefile | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
> index 150001d8bbbd..58ffe1e05d68 100644
> --- a/scripts/ssh.Makefile
> +++ b/scripts/ssh.Makefile
> @@ -7,10 +7,19 @@ ANSIBLE_EXTRA_ARGS += kexalgorithms=$(SSH_KEXALGORITHMS)
>  endif
>  
>  ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
> +
>  SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
>  ANSIBLE_EXTRA_ARGS += sshconfig=$(CONFIG_KDEVOPS_SSH_CONFIG)
>  ANSIBLE_EXTRA_ARGS += sshdir=$(shell dirname $(SSH_CONFIG_FILE))
> +
> +ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
> +SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
> +else
> +SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
>  endif
> +ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(CONFIG_KDEVOPS_SSH_CONFIG)_kdevops_$(SSH_CONFIG_SUFFIX)

Can we we just define this through Kconfig and use output yaml on the
entry for CONFIG_KDEVOPS_SSH_CONFIG ? Appending variables together
is stitched together on Kconfig via append-makefile-vars.sh.

This may require either cleaning up the usage / definition of
CONFIG_KDEVOPS_SSH_CONFIG and kdevops_ssh_config further so it is
consistent.

  Luis

Re: [PATCH v2 00/12] Replace terraform update_ssh_config module

[Luis Chamberlain]

On Wed, Feb 05, 2025 at 10:52:40AM -0500, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Luis suggested merging the way guestfs and terraform handle ssh
> host config. Luis, please let me know if I've grossly misunderstood
> your suggestion.

Looks good to me, specially if it works :) I just made a comment about
one small nit.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>

 Luis

Re: [PATCH v2 08/12] ssh.Makefile: Define a kdevops_ssh_config variable

[Chuck Lever]

On 2/5/25 12:42 PM, Luis Chamberlain wrote:
> On Wed, Feb 05, 2025 at 10:52:48AM -0500, cel@kernel.org wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Create a global variable that contains the pathname of the config
>> file that stores ephemeral ssh host entries. That way the scripts
>> don't have to keep creating this pathname.
>>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  scripts/ssh.Makefile | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
>> index 150001d8bbbd..58ffe1e05d68 100644
>> --- a/scripts/ssh.Makefile
>> +++ b/scripts/ssh.Makefile
>> @@ -7,10 +7,19 @@ ANSIBLE_EXTRA_ARGS += kexalgorithms=$(SSH_KEXALGORITHMS)
>>  endif
>>  
>>  ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
>> +
>>  SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
>>  ANSIBLE_EXTRA_ARGS += sshconfig=$(CONFIG_KDEVOPS_SSH_CONFIG)
>>  ANSIBLE_EXTRA_ARGS += sshdir=$(shell dirname $(SSH_CONFIG_FILE))
>> +
>> +ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
>> +SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
>> +else
>> +SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
>>  endif
>> +ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(CONFIG_KDEVOPS_SSH_CONFIG)_kdevops_$(SSH_CONFIG_SUFFIX)
> 
> Can we we just define this through Kconfig and use output yaml on the
> entry for CONFIG_KDEVOPS_SSH_CONFIG ? Appending variables together
> is stitched together on Kconfig via append-makefile-vars.sh.
> 
> This may require either cleaning up the usage / definition of
> CONFIG_KDEVOPS_SSH_CONFIG and kdevops_ssh_config further so it is
> consistent.

Yeah, this is not the prettiest scripting in the world. I will try to
fix this up as I get through testing the other terraform providers.


-- 
Chuck Lever

Re: [PATCH v2 07/12] update_ssh_config: make Include directive follow ssh config directory

[Chuck Lever]

On 2/5/25 10:52 AM, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> The file containing the kdevops target node entries needs to be
> placed in the same directory as the controller's .ssh/config.
> Usually that's ~/.ssh/config, but there is a Kconfig setting that
> can change that.
> 
> XXX: Actually, to make this work 100% correctly, "make mrproper"
> should remove the ssh Include directive. That way a user can modify
> the ssh config via "make menuconfig", then do a "make" to get the
> updated include path added into her .ssh/config.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  .../roles/update_ssh_config/tasks/main.yml     | 18 +++---------------
>  1 file changed, 3 insertions(+), 15 deletions(-)
> 
> diff --git a/playbooks/roles/update_ssh_config/tasks/main.yml b/playbooks/roles/update_ssh_config/tasks/main.yml
> index 417781936f7f..72c5f2e365f5 100644
> --- a/playbooks/roles/update_ssh_config/tasks/main.yml
> +++ b/playbooks/roles/update_ssh_config/tasks/main.yml
> @@ -5,18 +5,7 @@
>      path: "{{ sshconfig }}"
>    register: ssh_config
>  
> -- name: Check that the kdevops Include directive is present
> -  tags: deps
> -  ansible.builtin.lineinfile:
> -    path: "{{ sshconfig }}"
> -    regexp: "Include ~/.ssh/config_kdevops_*"
> -    state: absent
> -  check_mode: true
> -  changed_when: false
> -  register: kdevops_ssh_include
> -  when: ssh_config.stat.exists
> -
> -- name: Check that the Include directive has a kdevops_version comment
> +- name: Check for a kdevops_version comment

One minor misbehavior I noticed while working on this series is that if
I use "make defconfig-yada", it doesn't populate the kdevops_version
variable. Looks like the defconfig-* target does not build
include/config/project.release , so kdevops_version is an empty string
in that case.

"make defconfig-yada menuconfig" works fine, though.


>    tags: deps
>    ansible.builtin.lineinfile:
>      path: "{{ sshconfig }}"
> @@ -34,10 +23,9 @@
>    ansible.builtin.meta: end_play
>    when:
>      - ssh_config.stat.exists
> -    - kdevops_ssh_include.found
>      - fixed_ssh_entry.found
>  
> -- name: Remove the stale Include directive
> +- name: Remove any stale Include directives
>    tags: deps
>    ansible.builtin.lineinfile:
>      path: "{{ sshconfig }}"
> @@ -73,4 +61,4 @@
>      create: true
>      mode: "u=rw,g=r,o=r"
>      block: |
> -      Include ~/.ssh/config_kdevops_*
> +      Include {{ sshdir }}/config_kdevops_*


-- 
Chuck Lever