[PATCH v2 00/12] Convert bringup_guestfs to a single Ansible role

[PATCH v2 00/12] Convert bringup_guestfs to a single Ansible role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

This series demonstrates some ideas about how libvirt/guestfs bring-
up might be expressed using only Ansible (ie. with a minimal use of
shell scripting). Benefits include:

- Starting guests is now done in parallel
- "make bringup" is more idempotent
- Lets us create utilities that both guestfs and Terraform can use

The new parts that handle custom upstream images has not yet been
tested.

Changes since RFC:
- All of bringup_guestfs.sh has now been converted to Ansible
- Remove the old code as the last step in the series
- Reorganize the series to reduce the size of each patch
- Modernize steps that Ansible can now do with builtin modules

Chuck Lever (12):
  guestfs: Replace scripts/destroy_guestfs.sh with an Ansible playbook
  Move the guestfs install-deps to the guestfs playbook
  guestfs: Do not use the config-check tag
  guestfs: Add a "bringup" tag to the guestfs role
  guestfs: Copy "network" tag steps to guestfs role
  guestfs: Move the QEMU_GROUP check
  Add a base-image role
  guestfs: Convert scripts/bringup_guestfs.sh to Ansible
  guestfs: Move console-related steps to guestfs role
  bringup_guestfs: Remove the role
  scripts: Remove the bringup_guestfs.sh script
  scripts: Remove the destroy_guestfs.sh script

 .gitignore                                    |   1 +
 playbooks/base_image.yml                      |   7 +
 playbooks/bringup_guestfs.yml                 |   4 -
 playbooks/guestfs.yml                         |   7 +
 playbooks/roles/base_image/README.md          |  45 ++
 playbooks/roles/base_image/defaults/main.yml  |   7 +
 .../roles/base_image/tasks/base-image.yml     |  80 ++++
 .../roles/base_image/tasks/custom-image.yml   | 159 +++++++
 playbooks/roles/base_image/tasks/main.yml     |  19 +
 .../base_image/templates/custom-index.j2      |   2 +
 .../base_image/templates/custom-source.j2     |   3 +
 .../base_image/templates/virt-builder.j2      |  77 ++++
 .../roles/bringup_guestfs/defaults/main.yml   |  12 -
 .../tasks/install-deps/main.yml               |  17 -
 .../roles/bringup_guestfs/tasks/main.yml      | 243 -----------
 playbooks/roles/guestfs/defaults/main.yml     |   5 +
 .../tasks/bringup/console-permissions.yml     |  31 ++
 .../guestfs/tasks/bringup/extra-disks.yml     |  16 +
 .../roles/guestfs/tasks/bringup/largeio.yml   |  11 +
 .../roles/guestfs/tasks/bringup/main.yml      | 132 ++++++
 .../roles/guestfs/tasks/bringup/network.yml   |  83 ++++
 .../tasks/bringup/storage-pool-path.yml       |  92 ++++
 playbooks/roles/guestfs/tasks/destroy.yml     |  32 ++
 .../tasks/install-deps/debian/main.yml        |   3 +-
 .../roles/guestfs/tasks/install-deps/main.yml |  21 +
 .../tasks/install-deps/redhat/main.yml        |   5 +-
 .../tasks/install-deps/suse/main.yml          |   3 +-
 playbooks/roles/guestfs/tasks/main.yml        |  67 +++
 scripts/bringup_guestfs.sh                    | 399 ------------------
 scripts/destroy_guestfs.sh                    |  36 --
 scripts/guestfs.Makefile                      |  23 +-
 31 files changed, 911 insertions(+), 731 deletions(-)
 create mode 100644 playbooks/base_image.yml
 delete mode 100644 playbooks/bringup_guestfs.yml
 create mode 100644 playbooks/guestfs.yml
 create mode 100644 playbooks/roles/base_image/README.md
 create mode 100644 playbooks/roles/base_image/defaults/main.yml
 create mode 100644 playbooks/roles/base_image/tasks/base-image.yml
 create mode 100644 playbooks/roles/base_image/tasks/custom-image.yml
 create mode 100644 playbooks/roles/base_image/tasks/main.yml
 create mode 100644 playbooks/roles/base_image/templates/custom-index.j2
 create mode 100644 playbooks/roles/base_image/templates/custom-source.j2
 create mode 100644 playbooks/roles/base_image/templates/virt-builder.j2
 delete mode 100644 playbooks/roles/bringup_guestfs/defaults/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/main.yml
 create mode 100644 playbooks/roles/guestfs/defaults/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/console-permissions.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/extra-disks.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/largeio.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/network.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
 create mode 100644 playbooks/roles/guestfs/tasks/destroy.yml
 rename playbooks/roles/{bringup_guestfs => guestfs}/tasks/install-deps/debian/main.yml (78%)
 create mode 100644 playbooks/roles/guestfs/tasks/install-deps/main.yml
 rename playbooks/roles/{bringup_guestfs => guestfs}/tasks/install-deps/redhat/main.yml (62%)
 rename playbooks/roles/{bringup_guestfs => guestfs}/tasks/install-deps/suse/main.yml (71%)
 create mode 100644 playbooks/roles/guestfs/tasks/main.yml
 delete mode 100755 scripts/bringup_guestfs.sh
 delete mode 100755 scripts/destroy_guestfs.sh

-- 
2.49.0

[PATCH v2 01/12] guestfs: Replace scripts/destroy_guestfs.sh with an Ansible playbook

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Use Ansible for destroying libvirt-based target nodes. This replaces
an open-coded loop over the target node hosts, and makes the destroy
target more idempotent.

Also, this change is a prerequisite to making guestfs and terraform
manage their ssh key material in the same way, eventually.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .gitignore                                |  1 +
 playbooks/guestfs.yml                     |  7 +++++
 playbooks/roles/guestfs/tasks/destroy.yml | 32 +++++++++++++++++++++++
 playbooks/roles/guestfs/tasks/main.yml    |  6 +++++
 scripts/guestfs.Makefile                  |  6 +++--
 5 files changed, 50 insertions(+), 2 deletions(-)
 create mode 100644 playbooks/guestfs.yml
 create mode 100644 playbooks/roles/guestfs/tasks/destroy.yml
 create mode 100644 playbooks/roles/guestfs/tasks/main.yml

diff --git a/.gitignore b/.gitignore
index 45113a669390..706ef3fca950 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@
 .provisioned_once*
 
 guestfs/
+!playbooks/roles/guestfs/
 
 linux/
 !workflows/linux/
diff --git a/playbooks/guestfs.yml b/playbooks/guestfs.yml
new file mode 100644
index 000000000000..8bb496e0f05a
--- /dev/null
+++ b/playbooks/guestfs.yml
@@ -0,0 +1,7 @@
+---
+- name: Provision target nodes with libvirt/guestfs
+  gather_facts: false
+  connection: local
+  hosts: all
+  roles:
+    - role: guestfs
diff --git a/playbooks/roles/guestfs/tasks/destroy.yml b/playbooks/roles/guestfs/tasks/destroy.yml
new file mode 100644
index 000000000000..e26aacde4cff
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/destroy.yml
@@ -0,0 +1,32 @@
+---
+- name: Destroy each target node
+  community.libvirt.virt:
+    name: "{{ inventory_hostname }}"
+    command: "destroy"
+    uri: "{{ libvirt_uri }}"
+  failed_when: false # Do not fail if the target node is not currently running
+
+- name: Undefine each target node
+  community.libvirt.virt:
+    name: "{{ inventory_hostname }}"
+    command: "undefine"
+    uri: "{{ libvirt_uri }}"
+    force: true
+  failed_when: false # Do not fail if the target node is not currently defined
+
+- name: Remove per-node configuration files
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "{{ guestfs_path }}/{{ inventory_hostname }}"
+    - "{{ kdevops_storage_pool_path }}/guestfs/{{ inventory_hostname }}"
+
+- name: Remove global configuration files
+  run_once: true
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: absent
+  loop:
+    - "{{ kdevops_ssh_config }}"
+    - "{{ topdir_path }}/{{ kdevops_nodes }}"
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
new file mode 100644
index 000000000000..60d4ffd40a20
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -0,0 +1,6 @@
+---
+- name: Shut down and destroy each target node
+  tags:
+    - destroy
+  ansible.builtin.import_tasks:
+    file: "{{ role_path }}/tasks/destroy.yml"
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 8d4aac3e3669..5d355ec70f8c 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -93,9 +93,11 @@ status_guestfs:
 PHONY += status_guestfs
 
 destroy_guestfs:
-	$(Q)$(TOPDIR)/scripts/destroy_guestfs.sh
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		-i hosts playbooks/guestfs.yml \
+		--extra-vars=@./extra_vars.yaml \
+		--tags destroy
 	$(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG)
-
 PHONY += destroy_guestfs
 
 cleancache:
-- 
2.49.0

[PATCH v2 02/12] Move the guestfs install-deps to the guestfs playbook

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

As a clean up, handle "install-deps" in the new guestfs role,
just like is done for terraform.

In addition to copying the install-deps scripts, switch to using
import_tasks, which has more dependable behavior and avoids the
need to add a "tags:" keyword on each step.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../tasks/install-deps/debian/main.yml        | 10 +++++++++
 .../roles/guestfs/tasks/install-deps/main.yml | 21 +++++++++++++++++++
 .../tasks/install-deps/redhat/main.yml        | 12 +++++++++++
 .../guestfs/tasks/install-deps/suse/main.yml  |  9 ++++++++
 playbooks/roles/guestfs/tasks/main.yml        |  6 ++++++
 scripts/guestfs.Makefile                      |  4 ++--
 6 files changed, 60 insertions(+), 2 deletions(-)
 create mode 100644 playbooks/roles/guestfs/tasks/install-deps/debian/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/install-deps/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/install-deps/redhat/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/install-deps/suse/main.yml

diff --git a/playbooks/roles/guestfs/tasks/install-deps/debian/main.yml b/playbooks/roles/guestfs/tasks/install-deps/debian/main.yml
new file mode 100644
index 000000000000..5935f752a493
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/install-deps/debian/main.yml
@@ -0,0 +1,10 @@
+---
+- name: Install guestfs dependencies for Debian
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.package:
+    update_cache: true
+    name:
+      - libguestfs-tools
+      - isc-dhcp-client
+    state: present
diff --git a/playbooks/roles/guestfs/tasks/install-deps/main.yml b/playbooks/roles/guestfs/tasks/install-deps/main.yml
new file mode 100644
index 000000000000..5cbc55dcb188
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/install-deps/main.yml
@@ -0,0 +1,21 @@
+---
+- name: Gathering facts
+  ansible.builtin.gather_facts:
+
+- name: Debian-specific setup
+  ansible.builtin.import_tasks:
+    file: debian/main.yml
+  when:
+    - ansible_os_family == "Debian"
+
+- name: SuSE-specific setup
+  ansible.builtin.import_tasks:
+    file: suse/main.yml
+  when:
+    - ansible_os_family == "Suse"
+
+- name: Red Hat-specific setup
+  ansible.builtin.import_tasks:
+    file: redhat/main.yml
+  when:
+    - ansible_os_family == "Redhat"
diff --git a/playbooks/roles/guestfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/guestfs/tasks/install-deps/redhat/main.yml
new file mode 100644
index 000000000000..c28a16a07c7a
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/install-deps/redhat/main.yml
@@ -0,0 +1,12 @@
+---
+- name: Install guestfs dependencies for Red Hat Enterprise
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.package:
+    update_cache: true
+    name:
+      - libguestfs-tools
+      - dhcpcd
+    state: present
+  when:
+    - ansible_distribution != "Fedora"
diff --git a/playbooks/roles/guestfs/tasks/install-deps/suse/main.yml b/playbooks/roles/guestfs/tasks/install-deps/suse/main.yml
new file mode 100644
index 000000000000..c1bf24354612
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/install-deps/suse/main.yml
@@ -0,0 +1,9 @@
+---
+- name: Install guestfs dependencies for Suse
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.package:
+    name:
+      - libguestfs-tools
+      - dhcpcd
+    state: present
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
index 60d4ffd40a20..ba38a67c4baf 100644
--- a/playbooks/roles/guestfs/tasks/main.yml
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -1,4 +1,10 @@
 ---
+- name: Install guestfs dependencies on the Ansible controller
+  tags:
+    - install-deps
+  ansible.builtin.import_tasks:
+    file: "{{role_path }}/tasks/install-deps/main.yml"
+
 - name: Shut down and destroy each target node
   tags:
     - destroy
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 5d355ec70f8c..4ad347544cf6 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -68,9 +68,9 @@ $(KDEVOPS_PROVISIONED_SSH):
 	$(Q)touch $(KDEVOPS_PROVISIONED_SSH)
 
 install_libguestfs:
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
 		--inventory localhost, \
-		playbooks/bringup_guestfs.yml \
+		playbooks/guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
 		--tags install-deps
 
-- 
2.49.0

[PATCH v2 03/12] guestfs: Do not use the config-check tag

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

These days, "make bringup" fails immediately if "make" has not
already been run, because ansible.cfg does not yet exist.

The guestfs config-check steps are no longer necessary.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/guestfs.Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 4ad347544cf6..7f5fb6334846 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -79,7 +79,7 @@ bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 		--inventory localhost, \
 		playbooks/bringup_guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
-		--tags config-check,network,storage-pool-path
+		--tags network,storage-pool-path
 	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
-- 
2.49.0

[PATCH v2 04/12] guestfs: Add a "bringup" tag to the guestfs role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Create a new "bringup" tag that bookends the existing "destroy" tag.
All guestfs bringup steps will get this tag.

Start by populating this new tag with the steps that provision
the kdevops storage pool for guestfs.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/guestfs/defaults/main.yml     |  2 +
 .../tasks/bringup/storage-pool-path.yml       | 78 +++++++++++++++++++
 playbooks/roles/guestfs/tasks/main.yml        |  8 ++
 scripts/guestfs.Makefile                      |  6 +-
 4 files changed, 93 insertions(+), 1 deletion(-)
 create mode 100644 playbooks/roles/guestfs/defaults/main.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml

diff --git a/playbooks/roles/guestfs/defaults/main.yml b/playbooks/roles/guestfs/defaults/main.yml
new file mode 100644
index 000000000000..6112b384bb57
--- /dev/null
+++ b/playbooks/roles/guestfs/defaults/main.yml
@@ -0,0 +1,2 @@
+---
+libvirt_uri_system: false
diff --git a/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml b/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
new file mode 100644
index 000000000000..50f804cc304c
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
@@ -0,0 +1,78 @@
+---
+- name: Create storage pool path directory (libvirt session uri)
+  # noqa: risky-file-permissions
+  ansible.builtin.file:
+    path: "{{ libvirt_storage_pool_path }}"
+    state: directory
+  # mode: "u=rwx,g=rwx,o=rx"
+  when:
+    - not libvirt_uri_system|bool
+
+- name: Create storage pool path directory and set group (libvirt system uri)
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    path: "{{ libvirt_storage_pool_path }}"
+    state: directory
+    owner: root
+    group: "{{ libvirt_qemu_group }}"
+    mode: "u=rwx,g=rwx,o=rx"
+  when:
+    - libvirt_uri_system|bool
+
+- name: Create kdevops guestfs storage directory (libvirt session uri)
+  ansible.builtin.file:
+    path: "{{ guestfs_base_image_dir }}"
+    state: directory
+    mode: "u=rwx,g=rx,o=rx"
+  when:
+    - not libvirt_uri_system|bool
+
+- name: Create kdevops guestfs storage directory (libvirt system uri)
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    path: "{{ guestfs_base_image_dir }}"
+    state: directory
+    mode: "u=rwx,g=rwx,o=rx"
+    group: "{{ libvirt_qemu_group }}"
+  when:
+    - libvirt_uri_system|bool
+
+- name: Check if directory is owned by the correct group (libvirt system uri)
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    cmd: stat -c '%G' "{{ libvirt_storage_pool_path }}"
+  register: dir_group
+  changed_when: false
+  when:
+    - libvirt_uri_system|bool
+
+- name: Check if directory has group write permissions (libvirt system uri)
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    cmd: stat -c '%A' "{{ libvirt_storage_pool_path }}"
+  register: dir_perms
+  changed_when: false
+  when:
+    - libvirt_uri_system|bool
+
+- name: Verify storage pool path directory is group-writable (libvirt system uri)
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.fail:
+    msg: |
+      The permissions for {{ libvirt_storage_pool_path }} should be group
+      writeable by the group used by libvirt: {{ libvirt_qemu_group }}
+      Current group: {{ dir_group.stdout }}
+      Current permissions: {{ dir_perms.stdout }}
+  when:
+    - libvirt_uri_system|bool
+    - (dir_group.stdout != libvirt_qemu_group) or (dir_perms.stdout[5] != 'w')
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
index ba38a67c4baf..12064cb91d73 100644
--- a/playbooks/roles/guestfs/tasks/main.yml
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -5,6 +5,14 @@
   ansible.builtin.import_tasks:
     file: "{{role_path }}/tasks/install-deps/main.yml"
 
+- name: Ensure a storage pool for guestfs exists
+  delegate_to: localhost
+  run_once: true
+  tags:
+    - bringup
+  ansible.builtin.import_tasks:
+    file: "{{role_path }}/tasks/bringup/storage-pool-path.yml"
+
 - name: Shut down and destroy each target node
   tags:
     - destroy
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 7f5fb6334846..571662d187ae 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -75,11 +75,15 @@ install_libguestfs:
 		--tags install-deps
 
 bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		-i hosts playbooks/guestfs.yml \
+		--extra-vars=@./extra_vars.yaml \
+		--tags bringup
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
 		playbooks/bringup_guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
-		--tags network,storage-pool-path
+		--tags network
 	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
-- 
2.49.0

[PATCH v2 05/12] guestfs: Copy "network" tag steps to guestfs role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Copy the "network" tag steps into the guestfs role to remove a usage
of the bringup_guestfs role.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/guestfs/defaults/main.yml     |  2 +
 .../roles/guestfs/tasks/bringup/network.yml   | 83 +++++++++++++++++++
 playbooks/roles/guestfs/tasks/main.yml        |  8 ++
 scripts/guestfs.Makefile                      |  5 --
 4 files changed, 93 insertions(+), 5 deletions(-)
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/network.yml

diff --git a/playbooks/roles/guestfs/defaults/main.yml b/playbooks/roles/guestfs/defaults/main.yml
index 6112b384bb57..e250a40fd27a 100644
--- a/playbooks/roles/guestfs/defaults/main.yml
+++ b/playbooks/roles/guestfs/defaults/main.yml
@@ -1,2 +1,4 @@
 ---
+distro_debian_based: false
+
 libvirt_uri_system: false
diff --git a/playbooks/roles/guestfs/tasks/bringup/network.yml b/playbooks/roles/guestfs/tasks/bringup/network.yml
new file mode 100644
index 000000000000..8e7c9f1df7f3
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/network.yml
@@ -0,0 +1,83 @@
+---
+- name: Check for dnsmasq configuration files
+  ansible.builtin.stat:
+    path: "{{ item }}"
+  loop:
+    - /etc/dnsmasq.conf
+    - /etc/dnsmasq.d
+  register: dnsmasq_config_files
+  when:
+    - distro_debian_based|bool
+
+- name: Fail if dnsmasq configuration files exist
+  ansible.builtin.fail:
+    msg: |
+      dnsmasq configuration files or directories still exist.
+      Please remove the following to fully uninstall
+      dnsmasq:\n{{ dnsmasq_config_files | join('\n') }}
+  when:
+    - distro_debian_based|bool
+    - dnsmasq_config_files.results | selectattr('stat.exists') | list | length > 0
+
+- name: Check if dnsmasq service is enabled
+  # noqa: command-instead-of-module
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    cmd: "systemctl is-enabled dnsmasq"
+  register: dnsmasq_enabled
+  failed_when: false
+  changed_when: false
+  when:
+    - distro_debian_based|bool
+    - dnsmasq_config_files | length > 0
+
+- name: Check if dnsmasq service is active
+  # noqa: command-instead-of-module
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    cmd: "systemctl is-active dnsmasq"
+  register: dnsmasq_active
+  failed_when: false
+  changed_when: false
+  when:
+    - distro_debian_based|bool
+    - dnsmasq_config_files | length > 0
+
+- name: Fail if dnsmasq service is enabled or active
+  ansible.builtin.fail:
+    msg: |
+      dnsmasq service is
+      {{ 'enabled' if dnsmasq_enabled.rc == 0 else 'active' if dnsmasq_active.rc == 0 else 'present' }}.
+      Please ensure dnsmasq is fully uninstalled and disabled.
+      Run 'sudo systemctl disable dnsmasq' and 'sudo systemctl
+      stop dnsmasq' to disable and stop the service.
+  when:
+    - distro_debian_based|bool
+    - dnsmasq_config_files | length > 0
+    - (dnsmasq_enabled.rc == 0) or (dnsmasq_active.rc == 0)
+
+- name: Check if libvirt default network is running
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.shell: virsh net-list | grep -q default
+  register: libvirt_default_net
+  ignore_errors: true
+  changed_when: false
+  when:
+    - libvirt_uri_system|bool
+
+- name: Start the libvirt default network
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    cmd: "virsh net-start default"
+  changed_when: true
+  when:
+    - libvirt_uri_system|bool
+    - libvirt_default_net.rc != 0
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
index 12064cb91d73..6981d3c1a70b 100644
--- a/playbooks/roles/guestfs/tasks/main.yml
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -13,6 +13,14 @@
   ansible.builtin.import_tasks:
     file: "{{role_path }}/tasks/bringup/storage-pool-path.yml"
 
+- name: Ensure libvirt networking has started
+  delegate_to: localhost
+  run_once: true
+  tags:
+    - bringup
+  ansible.builtin.import_tasks:
+    file: "{{role_path }}/tasks/bringup/network.yml"
+
 - name: Shut down and destroy each target node
   tags:
     - destroy
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 571662d187ae..f2900a684960 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -79,11 +79,6 @@ bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 		-i hosts playbooks/guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
 		--tags bringup
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-		--inventory localhost, \
-		playbooks/bringup_guestfs.yml \
-		--extra-vars=@./extra_vars.yaml \
-		--tags network
 	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
-- 
2.49.0

[PATCH v2 06/12] guestfs: Move the QEMU_GROUP check

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The qemu group check is needed by both the stoage-pool-path tag
and the bringup_guestfs.sh, but is done only by the latter. The
makefile runs the former first, however.

Move the check earlier during bring-up.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../guestfs/tasks/bringup/storage-pool-path.yml    | 14 ++++++++++++++
 scripts/bringup_guestfs.sh                         |  8 --------
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml b/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
index 50f804cc304c..385b6cdfdbe7 100644
--- a/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
+++ b/playbooks/roles/guestfs/tasks/bringup/storage-pool-path.yml
@@ -1,4 +1,18 @@
 ---
+- name: Get the user who invoked Ansible
+  ansible.builtin.command:
+    cmd: "id -Gn"
+  register: id_group
+  changed_when: false
+
+- name: Group membership check failed
+  vars:
+    user_groups: "{{ id_group.stdout | split }}"
+  ansible.builtin.fail:
+    msg: "Ansible user is not part of the {{ libvirt_qemu_group }} group."
+  when:
+    - libvirt_qemu_group not in user_groups
+
 - name: Create storage pool path directory (libvirt session uri)
   # noqa: risky-file-permissions
   ansible.builtin.file:
diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
index 67f85a5fdb0a..55b13a0cb2cc 100755
--- a/scripts/bringup_guestfs.sh
+++ b/scripts/bringup_guestfs.sh
@@ -31,14 +31,6 @@ OS_VERSION=${CONFIG_VIRT_BUILDER_OS_VERSION}
 BASE_IMAGE_DIR="${STORAGEDIR}/base_images"
 BASE_IMAGE="${BASE_IMAGE_DIR}/${OS_VERSION}.raw"
 
-if id -nG "$(whoami)" | grep -qw "$QEMU_GROUP"; then
-    echo "User $USER is part of the $QEMU_GROUP group."
-else
-    echo "Error: User $USER is not part of the $QEMU_GROUP group. Exiting..."
-    echo "Fix this and come back and try again."
-    exit 1
-fi
-
 build_custom_source()
 {
 	SOURCE_TMP=$(mktemp)
-- 
2.49.0

[PATCH v2 07/12] Add a base-image role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Add a base-image role that ensures a base OS image exists for
libvirt to use when provisioning guests. Copy the steps from
scripts/bringup_guestfs.sh.

This procedure is maintained outside of the guestfs role in
order to reduce the complexity of guestfs and base-image.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/base_image.yml                      |   7 +
 playbooks/roles/base_image/README.md          |  45 +++++
 playbooks/roles/base_image/defaults/main.yml  |   7 +
 .../roles/base_image/tasks/base-image.yml     |  80 +++++++++
 .../roles/base_image/tasks/custom-image.yml   | 159 ++++++++++++++++++
 playbooks/roles/base_image/tasks/main.yml     |  19 +++
 .../base_image/templates/custom-index.j2      |   2 +
 .../base_image/templates/custom-source.j2     |   3 +
 .../base_image/templates/virt-builder.j2      |  77 +++++++++
 9 files changed, 399 insertions(+)
 create mode 100644 playbooks/base_image.yml
 create mode 100644 playbooks/roles/base_image/README.md
 create mode 100644 playbooks/roles/base_image/defaults/main.yml
 create mode 100644 playbooks/roles/base_image/tasks/base-image.yml
 create mode 100644 playbooks/roles/base_image/tasks/custom-image.yml
 create mode 100644 playbooks/roles/base_image/tasks/main.yml
 create mode 100644 playbooks/roles/base_image/templates/custom-index.j2
 create mode 100644 playbooks/roles/base_image/templates/custom-source.j2
 create mode 100644 playbooks/roles/base_image/templates/virt-builder.j2

diff --git a/playbooks/base_image.yml b/playbooks/base_image.yml
new file mode 100644
index 000000000000..41126844ab30
--- /dev/null
+++ b/playbooks/base_image.yml
@@ -0,0 +1,7 @@
+---
+- name: Create a libvirt base OS image
+  gather_facts: false
+  connection: local
+  hosts: localhost
+  roles:
+    - role: base_image
diff --git a/playbooks/roles/base_image/README.md b/playbooks/roles/base_image/README.md
new file mode 100644
index 000000000000..e279005e9c44
--- /dev/null
+++ b/playbooks/roles/base_image/README.md
@@ -0,0 +1,45 @@
+base_image
+==========
+
+The base_image role manages libvirt base OS images. These images
+contain an installed operating system and are used to quickly
+create new libvirt guests with virt-sysprep.
+
+Requirements
+------------
+
+Network access to the public libvirt image repositories. The
+virt-builder program must be installed.
+
+Role Variables
+--------------
+
+  * base_image_os_version: OS to install on the image
+  * base_image_pathname: pathname of local file to contain the image
+
+Dependencies
+------------
+
+None.
+
+Example Playbook
+----------------
+
+Below is an example playbook task:
+
+```
+- name: Create /test/nfs if needed
+  ansible.builtin.import_role:
+    name: base_image
+  vars:
+    base_image_os_version: "fedora-39"
+    base_image_pathname: "/var/lib/libvirt/images/kdevops/base-images/fedora-39.raw"
+```
+
+For further examples refer to one of this role's users, the
+[https://github.com/linux-kdevops/kdevops](kdevops) project.
+
+License
+-------
+
+copyleft-next-0.3.1
diff --git a/playbooks/roles/base_image/defaults/main.yml b/playbooks/roles/base_image/defaults/main.yml
new file mode 100644
index 000000000000..dc9e8f6617d3
--- /dev/null
+++ b/playbooks/roles/base_image/defaults/main.yml
@@ -0,0 +1,7 @@
+---
+libvirt_uri_system: false
+
+guestfs_copy_sources_from_host_to_guest: false
+guestfs_has_custom_raw_image: false
+kdevops_uid: ""
+update_grub_cmd: "/usr/sbin/update-grub2"
diff --git a/playbooks/roles/base_image/tasks/base-image.yml b/playbooks/roles/base_image/tasks/base-image.yml
new file mode 100644
index 000000000000..84971b611ec0
--- /dev/null
+++ b/playbooks/roles/base_image/tasks/base-image.yml
@@ -0,0 +1,80 @@
+---
+- name: Gather facts
+  ansible.builtin.gather_facts:
+
+- name: Get the UID of the kdevops user on the control host
+  ansible.builtin.command:
+    cmd: "id -u kdevops"
+  register: id_output
+  changed_when: false
+  failed_when: false
+
+- name: Set the kdevops UID in the base image
+  ansible.builtin.set_fact:
+    kdevops_uid: "-u {{ id_output.stdout }}"
+  when:
+    - id_output.rc == 0
+
+- name: Select the grub command for the base image (Red Hat)
+  ansible.builtin.set_fact:
+    update_grub_cmd: "/usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg"
+  when:
+    - base_image_os_version is match("^(rhel|fedora|centos)")
+
+- name: Create a temporary file for virt-builder commands
+  ansible.builtin.tempfile:
+    state: file
+  register: command_file
+
+- name: Construct the virt-builder command file
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/virt-builder.j2"
+    dest: "{{ command_file.path }}"
+    mode: "u=rw"
+
+- name: Generate a new base image for {{ base_image_os_version }}
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.command:
+    argv:
+      - "virt-builder"
+      - "{{ base_image_os_version }}"
+      - "--arch"
+      - "{{ ansible_machine }}"
+      - "-o"
+      - "{{ base_image_pathname }}"
+      - "--size"
+      - "20G"
+      - "--format"
+      - "raw"
+      - "--commands-from-file"
+      - "{{ command_file.path }}"
+    creates: "{{ base_image_pathname }}"
+  when:
+    - libvirt_uri_system|bool
+
+- name: Generate a new base image for {{ base_image_os_version }}
+  ansible.builtin.command:
+    argv:
+      - "virt-builder"
+      - "{{ base_image_os_version }}"
+      - "--arch"
+      - "{{ ansible_machine }}"
+      - "-o"
+      - "{{ base_image_pathname }}"
+      - "--size"
+      - "20G"
+      - "--format"
+      - "raw"
+      - "--commands-from-file"
+      - "{{ command_file.path }}"
+    creates: "{{ base_image_pathname }}"
+  when:
+    - not libvirt_uri_system|bool
+
+- name: Clean up the virt-builder command file
+  ansible.builtin.file:
+    path: "{{ command_file.path }}"
+    state: absent
+  when:
+    - command_file.path is defined
diff --git a/playbooks/roles/base_image/tasks/custom-image.yml b/playbooks/roles/base_image/tasks/custom-image.yml
new file mode 100644
index 000000000000..ede44bd7da4d
--- /dev/null
+++ b/playbooks/roles/base_image/tasks/custom-image.yml
@@ -0,0 +1,159 @@
+---
+- name: Set the pathname of the custom image directory
+  ansible.builtin.set_fact:
+    custom_image_dir: "{{ kdevops_storage_pool_path }}/guestfs/custom_images/{{ base_image_os_version }}"
+
+- name: Ensure the custom image directory exists
+  ansible.builtin.file:
+    path: "{{ custom_image_dir }}"
+    state: directory
+    mode: "u=rwx,g=rx,o=rx"
+
+- name: Set the pathname of the custom image
+  ansible.builtin.set_fact:
+    custom_image: "{{ custom_image_dir }}/{{ base_image_os_version }}.raw"
+
+- name: Set the pathname of the custom image sentinel
+  ansible.builtin.set_fact:
+    custom_image_ok: "{{ custom_image_dir }}.ok"
+
+- name: Set the pathname of the custom source configuration file
+  ansible.builtin.set_fact:
+    custom_source: "/etc/virt-builder/repos.d/kdevops-custom-images-{{ base_image_os_version }}.conf"
+
+- name: Set the pathname of the custom index file
+  ansible.builtin.set_fact:
+    custom_index: "{{ custom_image_dir | realpath }}/index"
+
+- name: Check if the custom image file already exists
+  ansible.builtin.stat:
+    path: "{{ custom_image }}"
+    get_attributes: false
+    get_checksum: false
+    get_mime: false
+  register: result
+
+- name: Fetch the custom image
+  ansible.builtin.get_url:
+    url: "{{ guestfs_custom_raw_image_url }}"
+    dest: "{{ custom_image_dir }}"
+    mode: "u=rw,g=r,o=r"
+  when:
+    - not result.stat.exists
+    - guestfs_has_custom_raw_image_url|bool
+
+- name: Check if the custom image sentinel file already exists
+  ansible.builtin.stat:
+    path: "{{ custom_image_ok }}"
+    get_attributes: false
+    get_checksum: false
+    get_mime: false
+  register: result
+
+- name: Check the custom image
+  when:
+    - not result.stat.exists
+    - guestfs_has_custom_raw_image_sha512sums|bool
+  block:
+    - name: Get the base name of the sha512sums file
+      ansible.builtin.set_fact:
+        sha512sums_file: "{{ guestfs_custom_raw_image_sha512sums_url | basename }}"
+
+    - name: Set the full pathname of sha512sums file
+      ansible.builtin.set_fact:
+        custom_image_sha512sum: "{{ custom_image_dir }}/{{ sha512sums_file }}"
+
+    - name: Check if the sha512sums file already exists
+      ansible.builtin.stat:
+        path: "{{ custom_image_sha512sum }}"
+        get_attributes: false
+        get_checksum: false
+        get_mime: false
+      register: result
+
+    - name: Fetch the sha512sums file
+      ansible.builtin.get_url:
+        url: "{{ guestfs_custom_raw_image_sha512sums_url }}"
+        dest: "{{ custom_image_dir }}"
+        mode: "u=rw,g=r,o=r"
+      when:
+        - not result.stat.exists
+
+    - name: Compute checksum of something
+      ansible.builtin.command:
+        cmd: "sha512sum --ignore-missing -c {{ sha512sums_file }}"
+        chdir: "{{ custom_image_dir }}"
+      changed_when: false
+
+    - name: Touch the custom image sentinel
+      ansible.builtin.file:
+        path: "{{ custom_image_ok }}"
+        state: touch
+        mode: "u=rw,g=r,o=r"
+
+- name: Check if the custom source exists
+  ansible.builtin.stat:
+    path: "{{ custom_source }}"
+    get_attributes: false
+    get_checksum: false
+    get_mime: false
+  register: result
+
+- name: Build the custom source
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/custom-source.j2"
+    dst: "{{ custom_source }}"
+    mode: "u=rw,g=r,o=r"
+  when:
+    - not result.stat.exists
+
+- name: Check if the custom index exists
+  ansible.builtin.stat:
+    path: "{{ custom_index }}"
+    get_attributes: false
+    get_checksum: false
+    get_mime: false
+  register: result
+
+- name: Build the custom index
+  ansible.builtin.template:
+    src: "{{ role_path }}/templates/custom-index.j2"
+    dst: "{{ custom_index }}"
+    mode: "u=rw,g=r,o=r"
+  when:
+    - not result.stat.exists
+
+- name: Show rolling distribution release warning
+  ansible.builtin.debug:
+    msg: |
+      ------------------------------------------------------------------
+      This is a rolling distribution release! To upgrade just do:
+
+      rm -rf {{ custom_image }}/*
+      rm -f  {{ custom_source }}
+      rm -f  {{ custom_index }}
+
+      Running guests always use their own copy. To rebuild your custom
+      base image from the custom image, also remove the base image:
+
+      rm -f  ${BASE_IMAGE}
+
+      This can always be done safely without affecting running guests.
+      ------------------------------------------------------------------
+  when:
+    - guestfs_has_custom_raw_image_rolling|bool
+
+- name: Show the custom virt-builder database
+  ansible.builtin.debug:
+    msg: |
+      Custom virt-builder source: {{ custom_source }}
+      Custom virt-builder index: {{ custom_index }}
+      Custom virt-builder image: {{ custom_image }}
+
+- name: Generating the index for {{ base_image_os_version }}
+  ansible.builtin.command:
+    argv:
+      - "virt-builder-repository"
+      - "--no-compression"
+      - "{{ custom_image_dir }}"
+  changed_when: true
diff --git a/playbooks/roles/base_image/tasks/main.yml b/playbooks/roles/base_image/tasks/main.yml
new file mode 100644
index 000000000000..a708fd8dff29
--- /dev/null
+++ b/playbooks/roles/base_image/tasks/main.yml
@@ -0,0 +1,19 @@
+---
+- name: Stat {{ base_image_pathname }}
+  ansible.builtin.stat:
+    path: "{{ base_image_pathname }}"
+    get_checksum: false
+    get_mime: false
+  register: result
+
+- name: Create custom upstream OS image
+  ansible.builtin.include_tasks:
+    file: "{{ role_path }}/tasks/custom_image.yml"
+  when:
+    - guestfs_has_custom_raw_image|bool
+
+- name: Create the base OS image
+  ansible.builtin.include_tasks:
+    file: "{{ role_path }}/tasks/base-image.yml"
+  when:
+    - not result.stat.exists
diff --git a/playbooks/roles/base_image/templates/custom-index.j2 b/playbooks/roles/base_image/templates/custom-index.j2
new file mode 100644
index 000000000000..32edd8f9e1e5
--- /dev/null
+++ b/playbooks/roles/base_image/templates/custom-index.j2
@@ -0,0 +1,2 @@
+[{{ base_image_os_version }}]
+file={{ base_image_os_version }}.raw
diff --git a/playbooks/roles/base_image/templates/custom-source.j2 b/playbooks/roles/base_image/templates/custom-source.j2
new file mode 100644
index 000000000000..cb4af83fb5f3
--- /dev/null
+++ b/playbooks/roles/base_image/templates/custom-source.j2
@@ -0,0 +1,3 @@
+[local]
+uri=file:///{{ custom_index }}
+proxy=off
diff --git a/playbooks/roles/base_image/templates/virt-builder.j2 b/playbooks/roles/base_image/templates/virt-builder.j2
new file mode 100644
index 000000000000..6abb7e6643af
--- /dev/null
+++ b/playbooks/roles/base_image/templates/virt-builder.j2
@@ -0,0 +1,77 @@
+{% if rhel_org_id is defined %}
+run-command subscription-manager register --org={{ rhel_org_id }} --activationkey={{ rhel_activation_key }}
+{% endif %}
+
+{% if kdevops_custom_yum_repofile is defined and kdevops_custom_yum_repofile != "" %}
+copy-in {{ kdevops_custom_yum_repofile }}:/etc/yum.repos.d
+{% endif %}
+
+{% if guestfs_copy_sources_from_host_to_guest %}
+mkdir {{ target_dir }}
+copy-in {{ guestfs_distro_source_and_dest_file }}:{{ target_dir }}
+{% endif %}
+
+install sudo,qemu-guest-agent,python3,bash
+run-command useradd {{ kdevops_uid }} -s /bin/bash -m kdevops
+append-line /etc/sudoers.d/kdevops:kdevops   ALL=(ALL)       NOPASSWD: ALL
+edit /etc/default/grub:s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0"/
+run-command {{ update_grub_cmd }}
+root-password password:kdevops
+
+{% if rhel_org_id is defined %}
+sm-unregister
+{% endif %}
+
+{% if distro_debian_based is defined and distro_debian_based %}
+{# Ugh, debian has to be told to bring up the network and regenerate ssh keys #}
+{# Hope we get that interface name right! #}
+install isc-dhcp-client,ifupdown
+mkdir /etc/network/interfaces.d/
+append-line /etc/network/interfaces.d/enp1s0:auto enp1s0
+append-line /etc/network/interfaces.d/enp1s0:allow-hotplug enp1s0
+append-line /etc/network/interfaces.d/enp1s0:iface enp1s0 inet dhcp
+firstboot-command systemctl disable systemd-networkd-wait-online.service
+firstboot-command systemctl stop ssh
+firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force openssh-server
+firstboot-command systemctl start ssh
+firstboot-command apt update && apt upgrade --yes
+uninstall unattended-upgrades
+
+{% if distro_debian_trixie is defined and distro_debian_trixie %}
+{# CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST will not work #}
+{# if /etc/nsswitch.conf has a line like this: #}
+{# #}
+{# hosts:          files myhostname resolve [!UNAVAIL=return] dns #}
+{# #}
+{# We need DNS to be used so virb0 will be used for a DNS request #}
+{# For the life of me I can't get the following line to work with #}
+{# the virt-builder command and so we do a full edit of the file for now #}
+{# edit /etc/nsswitch.conf:'s/\[!UNAVAIL=return\]//' #}
+write /etc/nsswitch.conf: # kdevops generated /etc/nsswitch.conf
+append-line /etc/nsswitch.conf:passwd:         files
+append-line /etc/nsswitch.conf:group:          files
+append-line /etc/nsswitch.conf:shadow:         files
+append-line /etc/nsswitch.conf:gshadow:        files
+append-line /etc/nsswitch.conf:hosts:          files myhostname resolve dns
+append-line /etc/nsswitch.conf:networks:       files
+append-line /etc/nsswitch.conf:protocols:      db files
+append-line /etc/nsswitch.conf:services:       db files
+append-line /etc/nsswitch.conf:ethers:         db files
+append-line /etc/nsswitch.conf:rpc:            db files
+append-line /etc/nsswitch.conf:netgroup:       nis
+uninstall cloud-init
+write /etc/default/locale:LANG=en_US.UTF-8
+append-line /etc/default/locale:LANGUAGE=en_US:en
+write /etc/locale.gen:en_US.UTF-8 UTF-8
+firstboot-command locale-gen en_US.UTF-8
+firstboot-command update-locale LANG=en_US.UTF-8
+firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force locales
+firstboot-command systemctl stop ssh
+firstboot-command systemctl start ssh
+
+{% if guestfs_copy_sources_from_host_to_guest %}
+delete /etc/apt/sources.list.d/debian.sources
+{% endif %}
+
+{% endif %}
+{% endif %}
-- 
2.49.0

[PATCH v2 08/12] guestfs: Convert scripts/bringup_guestfs.sh to Ansible

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The bringup_guestfs.sh is converted to Ansible to enable the
virt-builder part of bring to run in parallel. This also gives us
a fighting chance to integrate terraform and guestfs ssh key
management in the future.

In addition to those benefits, the new Ansible code takes a stab at
being more idempotent than the shell script was. Idempotency is
achieved by checking that each guest is defined, and then defining
it if not; then using community.libvirt.virt to boot the guest,
which does nothing if the guest is already running.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/guestfs.yml                         |   2 +-
 playbooks/roles/guestfs/defaults/main.yml     |   1 +
 .../guestfs/tasks/bringup/extra-disks.yml     |  16 +++
 .../roles/guestfs/tasks/bringup/largeio.yml   |  11 ++
 .../roles/guestfs/tasks/bringup/main.yml      | 132 ++++++++++++++++++
 playbooks/roles/guestfs/tasks/main.yml        |  29 ++++
 scripts/guestfs.Makefile                      |   1 -
 7 files changed, 190 insertions(+), 2 deletions(-)
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/extra-disks.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/largeio.yml
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/main.yml

diff --git a/playbooks/guestfs.yml b/playbooks/guestfs.yml
index 8bb496e0f05a..784775cdb818 100644
--- a/playbooks/guestfs.yml
+++ b/playbooks/guestfs.yml
@@ -1,6 +1,6 @@
 ---
 - name: Provision target nodes with libvirt/guestfs
-  gather_facts: false
+  gather_facts: true
   connection: local
   hosts: all
   roles:
diff --git a/playbooks/roles/guestfs/defaults/main.yml b/playbooks/roles/guestfs/defaults/main.yml
index e250a40fd27a..eec137bd730a 100644
--- a/playbooks/roles/guestfs/defaults/main.yml
+++ b/playbooks/roles/guestfs/defaults/main.yml
@@ -2,3 +2,4 @@
 distro_debian_based: false
 
 libvirt_uri_system: false
+libvirt_enable_largeio: false
diff --git a/playbooks/roles/guestfs/tasks/bringup/extra-disks.yml b/playbooks/roles/guestfs/tasks/bringup/extra-disks.yml
new file mode 100644
index 000000000000..438b65ed5d3d
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/extra-disks.yml
@@ -0,0 +1,16 @@
+---
+- name: Create the new drive image
+  ansible.builtin.command:
+    argv:
+      - "qemu-img"
+      - "create"
+      - "-f"
+      - "{{ libvirt_extra_drive_format }}"
+      - "{{ path }}"
+      - "100G"
+
+- name: Update the permission settings of the drive image file
+  ansible.builtin.file:
+    path: "{{ path }}"
+    group: "{{ libvirt_qemu_group }}"
+    mode: "g+rw,o-rw"
diff --git a/playbooks/roles/guestfs/tasks/bringup/largeio.yml b/playbooks/roles/guestfs/tasks/bringup/largeio.yml
new file mode 100644
index 000000000000..4246677d18d8
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/largeio.yml
@@ -0,0 +1,11 @@
+---
+- name: Compute the total number of devices to build
+  ansible.builtin.set_fact:
+    total_devices: "{{ libvirt_largeio_pow_limit * libvirt_largeio_drives_per_space }}"
+
+- name: Create largeio block devices
+  ansible.builtin.include_tasks:
+    file: "{{ role_path }}/tasks/extra_disks.yml"
+  vars:
+    path: "{{ storagedir }}/{{ inventory_hostname }}/extra{{ item }}.{{ libvirt_extra_drive_format }}"
+  loop: "{{ range(0, total_devices) | list }}"
diff --git a/playbooks/roles/guestfs/tasks/bringup/main.yml b/playbooks/roles/guestfs/tasks/bringup/main.yml
new file mode 100644
index 000000000000..007dabddb2b4
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/main.yml
@@ -0,0 +1,132 @@
+---
+- name: List defined libvirt guests
+  run_once: true
+  community.libvirt.virt:
+    command: list_vms
+    uri: "{{ libvirt_uri }}"
+  register: defined_vms
+
+- name: Provision each target node
+  when:
+    - 'inventory_hostname not in defined_vms.list_vms'
+  block:
+    - name: Set the pathname of the ssh directory for each target node
+      ansible.builtin.set_fact:
+        ssh_key_dir: "{{ guestfs_path }}/{{ inventory_hostname }}/ssh"
+
+    - name: Set the pathname of the ssh key for each target node
+      ansible.builtin.set_fact:
+        ssh_key: "{{ ssh_key_dir }}/id_ed25519"
+
+    - name: Generate ssh keys for each target node
+      block:
+        - name: Create the ssh key directory on the control host
+          ansible.builtin.file:
+            path: "{{ ssh_key_dir }}"
+            state: directory
+            mode: "u=rwx"
+
+        - name: Generate fresh keys for each target node
+          ansible.builtin.command:
+            cmd: 'ssh-keygen -q -t ed25519 -f {{ ssh_key }} -N ""'
+
+    - name: Set the pathname of the root disk image for each target node
+      ansible.builtin.set_fact:
+        root_image: "{{ storagedir }}/{{ inventory_hostname }}/root.raw"
+
+    - name: Create the storage pool directory for each target node
+      ansible.builtin.file:
+        path: "{{ storagedir }}/{{ inventory_hostname }}"
+        state: directory
+
+    - name: Duplicate the root disk image for each target node
+      ansible.builtin.command:
+        cmd: "cp --reflink=auto {{ base_image }} {{ root_image }}"
+
+    - name: Get the timezone of the control host
+      ansible.builtin.command:
+        cmd: "timedatectl show -p Timezone --value"
+      register: host_timezone
+
+    - name: Build the root image for each target node (as root)
+      become: true
+      become_method: ansible.builtin.sudo
+      ansible.builtin.command:
+        argv:
+          - "virt-sysprep"
+          - "-a"
+          - "{{ root_image }}"
+          - "--hostname"
+          - "{{ inventory_hostname }}"
+          - "--ssh-inject"
+          - "kdevops:file:{{ ssh_key }}.pub"
+          - "--timezone"
+          - "{{ host_timezone.stdout }}"
+      when:
+        - libvirt_uri_system|bool
+
+    - name: Build the root image for each target node (non-root)
+      ansible.builtin.command:
+        argv:
+          - "virt-sysprep"
+          - "-a"
+          - "{{ root_image }}"
+          - "--hostname"
+          - "{{ inventory_hostname }}"
+          - "--ssh-inject"
+          - "kdevops:file:{{ ssh_key }}.pub"
+          - "--timezone"
+          - "{{ host_timezone.stdout }}"
+      when:
+        - not libvirt_uri_system|bool
+
+    - name: Create largeio devices
+      ansible.builtin.include_tasks:
+        file: "{{ role_path }}/tasks/bringup/largeio.yml"
+      when:
+        - libvirt_enable_largeio|bool
+
+    - name: Create extra disks
+      vars:
+        path: "{{ storagedir }}/{{ inventory_hostname }}/extra{{ item }}.{{ libvirt_extra_drive_format }}"
+      ansible.builtin.include_tasks:
+        file: "{{ role_path }}/tasks/bringup/extra-disks.yml"
+      loop: "{{ range(0, 4) | list }}"
+      when:
+        - not libvirt_enable_largeio|bool
+
+    - name: Define each target node
+      vars:
+        xml_file: "{{ guestfs_path }}/{{ inventory_hostname }}/{{ inventory_hostname }}.xml"
+      community.libvirt.virt:
+        command: define
+        name: "{{ inventory_hostname }}"
+        xml: "{{ lookup('file', xml_file) }}"
+        uri: "{{ libvirt_uri }}"
+
+    - name: Find PCIe passthrough devices
+      ansible.builtin.find:
+        paths: "{{ guestfs_path }}/{{ inventory_hostname }}"
+        file_type: file
+        patterns: "pcie_passthrough_*.xml"
+      register: passthrough_devices
+
+    - name: Attach PCIe passthrough devices to each target node
+      ansible.builtin.command:
+        argv:
+          - "virsh"
+          - "attach-device"
+          - "{{ inventory_hostname }}"
+          - "{{ item }}"
+          - "--config"
+      loop: "{{ passthrough_devices.files }}"
+      loop_control:
+        label: "Doing PCI-E passthrough for device {{ item }}"
+      when:
+        - passthrough_devices.matched > 0
+
+- name: Boot each target node
+  community.libvirt.virt:
+    name: "{{ inventory_hostname }}"
+    uri: "{{ libvirt_uri }}"
+    state: running
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
index 6981d3c1a70b..8201c8a4dcb2 100644
--- a/playbooks/roles/guestfs/tasks/main.yml
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -21,6 +21,35 @@
   ansible.builtin.import_tasks:
     file: "{{role_path }}/tasks/bringup/network.yml"
 
+- name: Set the pathname of storage pool directory
+  tags:
+    - bringup
+  ansible.builtin.set_fact:
+    storagedir: "{{ kdevops_storage_pool_path }}/guestfs"
+
+- name: Set the pathname of the OS base image
+  tags:
+    - bringup
+  ansible.builtin.set_fact:
+    base_image: "{{ storagedir }}/base_images/{{ virtbuilder_os_version }}.raw"
+
+- name: Ensure the base OS image exists
+  delegate_to: localhost
+  run_once: true
+  tags:
+    - bringup
+  ansible.builtin.import_role:
+    name: base_image
+  vars:
+    base_image_os_version: "{{ virtbuilder_os_version }}"
+    base_image_pathname: "{{ base_image }}"
+
+- name: Bring up each target node
+  tags:
+    - bringup
+  ansible.builtin.import_tasks:
+    file: "{{role_path }}/tasks/bringup/main.yml"
+
 - name: Shut down and destroy each target node
   tags:
     - destroy
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index f2900a684960..0f0a8b858dba 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -79,7 +79,6 @@ bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 		-i hosts playbooks/guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
 		--tags bringup
-	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
 		playbooks/bringup_guestfs.yml \
-- 
2.49.0

[PATCH v2 09/12] guestfs: Move console-related steps to guestfs role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Copy these steps to the guestfs role because my plan is to remove
the bringup_guestfs role eventually.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../tasks/bringup/console-permissions.yml     | 31 +++++++++++++++++++
 playbooks/roles/guestfs/tasks/main.yml        | 10 ++++++
 scripts/guestfs.Makefile                      |  5 ---
 3 files changed, 41 insertions(+), 5 deletions(-)
 create mode 100644 playbooks/roles/guestfs/tasks/bringup/console-permissions.yml

diff --git a/playbooks/roles/guestfs/tasks/bringup/console-permissions.yml b/playbooks/roles/guestfs/tasks/bringup/console-permissions.yml
new file mode 100644
index 000000000000..ad169a4eab31
--- /dev/null
+++ b/playbooks/roles/guestfs/tasks/bringup/console-permissions.yml
@@ -0,0 +1,31 @@
+---
+- name: Get the user who invoked Ansible
+  ansible.builtin.command:
+    cmd: whoami
+  register: reg_user
+  changed_when: false
+
+- name: Look for console.log files in guestfs subdirectories to check for CI enablement
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.find:
+    paths: "{{ topdir_path }}/guestfs"
+    patterns: "console.log"
+    file_type: file
+    recurse: true
+  register: console_log_files
+
+- name: Ensure console.log files are owned by the main user for CI monitoring
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    path: "{{ item.path }}"
+    owner: "{{ reg_user.stdout }}"
+    group: "{{ reg_user.stdout }}"
+  loop: "{{ console_log_files.files }}"
+  loop_control:
+    label: "{{ item.path | regex_replace('^.*guestfs/', 'guestfs/') }}"
+  when:
+    - console_log_files.matched > 0
diff --git a/playbooks/roles/guestfs/tasks/main.yml b/playbooks/roles/guestfs/tasks/main.yml
index 8201c8a4dcb2..0cc9dc43cbe8 100644
--- a/playbooks/roles/guestfs/tasks/main.yml
+++ b/playbooks/roles/guestfs/tasks/main.yml
@@ -50,6 +50,16 @@
   ansible.builtin.import_tasks:
     file: "{{role_path }}/tasks/bringup/main.yml"
 
+- name: Set up target node console permissions
+  delegate_to: localhost
+  run_once: true
+  tags:
+    - bringup
+  ansible.builtin.import_tasks:
+    file: "{{ role_path }}/tasks/bringup/console-permissions.yml"
+  when:
+    - libvirt_uri_system|bool
+
 - name: Shut down and destroy each target node
   tags:
     - destroy
diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index 0f0a8b858dba..ebd7f53d1bde 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -79,11 +79,6 @@ bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 		-i hosts playbooks/guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
 		--tags bringup
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
-		--inventory localhost, \
-		playbooks/bringup_guestfs.yml \
-		--extra-vars=@./extra_vars.yaml \
-		--tags console-permissions
 PHONY += bringup_guestfs
 
 status_guestfs:
-- 
2.49.0

[PATCH v2 10/12] bringup_guestfs: Remove the role

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up: The tasks performed by the bringup_guestfs role have been
superceded by the new guestfs role.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/bringup_guestfs.yml                 |   4 -
 .../roles/bringup_guestfs/defaults/main.yml   |  12 -
 .../tasks/install-deps/debian/main.yml        |  11 -
 .../tasks/install-deps/main.yml               |  17 --
 .../tasks/install-deps/redhat/main.yml        |  11 -
 .../tasks/install-deps/suse/main.yml          |  10 -
 .../roles/bringup_guestfs/tasks/main.yml      | 243 ------------------
 7 files changed, 308 deletions(-)
 delete mode 100644 playbooks/bringup_guestfs.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/defaults/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml
 delete mode 100644 playbooks/roles/bringup_guestfs/tasks/main.yml

diff --git a/playbooks/bringup_guestfs.yml b/playbooks/bringup_guestfs.yml
deleted file mode 100644
index 80a1afcd0a69..000000000000
--- a/playbooks/bringup_guestfs.yml
+++ /dev/null
@@ -1,4 +0,0 @@
----
-- hosts: localhost
-  roles:
-    - role: bringup_guestfs
diff --git a/playbooks/roles/bringup_guestfs/defaults/main.yml b/playbooks/roles/bringup_guestfs/defaults/main.yml
deleted file mode 100644
index 65867cd1093a..000000000000
--- a/playbooks/roles/bringup_guestfs/defaults/main.yml
+++ /dev/null
@@ -1,12 +0,0 @@
-# SPDX-License-Identifier GPL-2.0+
----
-libvirt_uri_system: False
-distro_debian: False
-distro_debian_based: False
-distro_redhat: False
-distro_redhat_based: False
-distro_fedora: False
-distro_suse: False
-distro_suse_based: False
-distro_ubuntu: False
-dnsmasq_files_exist: False
diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml
deleted file mode 100644
index 6b502f59f3e4..000000000000
--- a/playbooks/roles/bringup_guestfs/tasks/install-deps/debian/main.yml
+++ /dev/null
@@ -1,11 +0,0 @@
----
-- name: Install libguestfs
-  become: true
-  become_method: ansible.builtin.sudo
-  ansible.builtin.package:
-    update_cache: true
-    name:
-      - libguestfs-tools
-      - isc-dhcp-client
-    state: present
-  tags: ['install-deps']
diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml
deleted file mode 100644
index af3be6dea3c3..000000000000
--- a/playbooks/roles/bringup_guestfs/tasks/install-deps/main.yml
+++ /dev/null
@@ -1,17 +0,0 @@
----
-- name: Debian-specific setup
-  ansible.builtin.include_tasks: debian/main.yml
-  when: ansible_facts['os_family']|lower == 'debian'
-  tags: ['install-deps']
-
-- name: SuSE-specific setup
-  ansible.builtin.include_tasks: suse/main.yml
-  when: ansible_facts['os_family']|lower == 'suse'
-  tags: ['install-deps']
-
-- name: Fedora/Red Hat Enterprise-specific setup
-  ansible.builtin.include_tasks: redhat/main.yml
-  when:
-    - ansible_facts['os_family']|lower == 'redhat'
-    - ansible_facts['distribution']|lower != "fedora"
-  tags: ['install-deps']
diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml
deleted file mode 100644
index 5e4bfa9c9aeb..000000000000
--- a/playbooks/roles/bringup_guestfs/tasks/install-deps/redhat/main.yml
+++ /dev/null
@@ -1,11 +0,0 @@
----
-- name: Install libguestfs on Fedora/RedHat
-  become: true
-  become_method: ansible.builtin.sudo
-  ansible.builtin.package:
-    update_cache: true
-    name:
-      - libguestfs-tools
-      - dhcpcd
-    state: present
-  tags: ['install-deps']
diff --git a/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml b/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml
deleted file mode 100644
index 730bc20b5a7b..000000000000
--- a/playbooks/roles/bringup_guestfs/tasks/install-deps/suse/main.yml
+++ /dev/null
@@ -1,10 +0,0 @@
----
-- name: Install libguestfs on SUSE
-  become: true
-  become_method: ansible.builtin.sudo
-  ansible.builtin.package:
-    name:
-      - libguestfs-tools
-      - dhcpcd
-    state: present
-  tags: ['install-deps']
diff --git a/playbooks/roles/bringup_guestfs/tasks/main.yml b/playbooks/roles/bringup_guestfs/tasks/main.yml
deleted file mode 100644
index 3c4e61a7a095..000000000000
--- a/playbooks/roles/bringup_guestfs/tasks/main.yml
+++ /dev/null
@@ -1,243 +0,0 @@
-- name: Install dependencies
-  ansible.builtin.include_tasks: install-deps/main.yml
-  tags: ['install-deps']
-
-- name: Verify we're configured {{ topdir_path }}/.config directory exists
-  stat:
-    path: "{{ topdir_path }}/.config"
-  register: config_file
-  tags: [ 'config-check' ]
-
-- name: Fail if {{ topdir_path }}/.config is not present
-  fail:
-    msg: "kdevops is not confgured. You must run 'make menuconfig'"
-  when: not config_file.stat.exists or not config_file.stat.isreg
-  tags: [ 'config-check' ]
-
-- name: Check if guestfs directory exists
-  stat:
-    path: "{{ topdir_path }}/guestfs"
-  register: guestfs_dir_stat
-  tags: [ 'config-check' ]
-
-- name: Fail if guestfs directory does not exist
-  fail:
-    msg: "The guestfs directory does not exist. You must run make first."
-  when: not guestfs_dir_stat.stat.exists
-  tags: [ 'config-check' ]
-
-- name: Check for directories in {{ topdir_path }}/guestfs
-  find:
-    paths: "{{ topdir_path }}/guestfs"
-    file_type: directory
-    recurse: no
-    depth: 1
-  register: guestfs_subdirectories
-  when: guestfs_dir_stat.stat.exists
-  tags: [ 'config-check' ]
-
-- name: Fail if no directories found in {{ topdir_path }}/guestfs
-  fail:
-    msg: "No directories found in guestfs. You must run make first."
-  when: guestfs_subdirectories.matched == 0
-  tags: [ 'config-check' ]
-
-- name: Create storage pool path directory if (libvirt session uri)
-  file:
-    path: "{{ libvirt_storage_pool_path }}"
-    state: directory
-  when: 'not libvirt_uri_system|bool'
-  tags: ['storage-pool-path']
-
-- name: Create storage pool path directory and set group if using (libvirt system uri)
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  file:
-    path: "{{ libvirt_storage_pool_path }}"
-    state: directory
-    owner: root
-    group: "{{ libvirt_qemu_group }}"
-    mode: "0775"
-  when: 'libvirt_uri_system|bool'
-  tags: ['storage-pool-path']
-
-- name: Create kdevops guestfs storage directory if missing (libvirt session uri)
-  file:
-    path: "{{ guestfs_base_image_dir }}"
-    state: directory
-    mode: '0755'
-  tags: ['storage-pool-path']
-  when:
-    - 'not libvirt_uri_system|bool'
-
-- name: Create kdevops guestfs storage directory if missing (libvirt system uri)
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  file:
-    path: "{{ guestfs_base_image_dir }}"
-    state: directory
-    mode: '0775'
-    group: "{{ libvirt_qemu_group }}"
-  tags: ['storage-pool-path']
-  when:
-    - 'libvirt_uri_system|bool'
-
-- name: Check if directory is owned by the correct group (libvirt system uri)
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  command: stat -c '%G' "{{ libvirt_storage_pool_path }}"
-  register: dir_group
-  changed_when: false
-  tags: ['storage-pool-path']
-  when:
-    - 'libvirt_uri_system|bool'
-
-- name: Check if directory has group write permissions (libvirt system uri)
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  command: stat -c '%A' "{{ libvirt_storage_pool_path }}"
-  register: dir_perms
-  changed_when: false
-  tags: ['storage-pool-path']
-  when:
-    - 'libvirt_uri_system|bool'
-
-- name: Verify storage pool path directory is group-writable (libvirt system uri)
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  fail:
-    msg: |
-      The permissions for {{ libvirt_storage_pool_path }} should be group
-      writeable by the group used by libvirt: {{ libvirt_qemu_group }}
-      Current group: {{ dir_group.stdout }}
-      Current permissions: {{ dir_perms.stdout }}
-  tags: ['storage-pool-path']
-  when:
-    - 'libvirt_uri_system|bool'
-    - (dir_group.stdout != libvirt_qemu_group) or (dir_perms.stdout[5] != 'w')
-
-- name: Check for dnsmasq configuration files
-  stat:
-    path: "{{ item }}"
-  loop:
-    - /etc/dnsmasq.conf
-    - /etc/dnsmasq.d
-  register: dnsmasq_config_files
-  when:
-    - distro_debian_based|bool
-  tags: [ 'network' ]
-
-- name: Fail if dnsmasq configuration files exist
-  fail:
-    msg: |
-      dnsmasq configuration files or directories still exist.
-      Please remove the following to fully uninstall
-      dnsmasq:\n{{ dnsmasq_config_files | join('\n') }}
-  when:
-    - distro_debian_based|bool
-    - dnsmasq_config_files.results | selectattr('stat.exists') | list | length > 0
-  tags: [ 'network' ]
-
-- name: Check dnsmasq service status
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  command: systemctl is-enabled dnsmasq
-  register: dnsmasq_enabled
-  failed_when: false
-  changed_when: false
-  when:
-    - distro_debian_based|bool
-    - dnsmasq_config_files | length > 0
-  tags: [ 'network' ]
-
-- name: Check if dnsmasq service is active
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  command: systemctl is-active dnsmasq
-  register: dnsmasq_active
-  failed_when: false
-  changed_when: false
-  when:
-    - distro_debian_based|bool
-    - dnsmasq_config_files | length > 0
-  tags: [ 'network' ]
-
-- name: Fail if dnsmasq service is enabled or active
-  fail:
-    msg: |
-      dnsmasq service is
-      {{ 'enabled' if dnsmasq_enabled.rc == 0 else 'active' if dnsmasq_active.rc == 0 else 'present' }}.
-      Please ensure dnsmasq is fully uninstalled and disabled.
-      Run 'sudo systemctl disable dnsmasq' and 'sudo systemctl
-      stop dnsmasq' to disable and stop the service.
-  when:
-    - distro_debian_based|bool
-    - dnsmasq_config_files | length > 0
-    - (dnsmasq_enabled.rc == 0) or (dnsmasq_active.rc == 0)
-  tags: [ 'network' ]
-
-- name: Check if libvirt default network is running
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  shell: virsh net-list | grep -q default
-  register: libvirt_default_net
-  ignore_errors: yes
-  when:
-    - 'libvirt_uri_system|bool'
-  tags: [ 'network' ]
-
-- name: Start the libvirt default network if not running
-  command: virsh net-start default
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  when:
-    - 'libvirt_uri_system|bool'
-    - libvirt_default_net.rc != 0
-  tags: [ 'network' ]
-
-- name: Get the actual user who invoked Ansible
-  command: whoami
-  register: reg_user
-  changed_when: false
-  when:
-    - 'libvirt_uri_system|bool'
-  tags: ['console-permissions']
-
-- name: Look for console.log files in guestfs subdirectories to check for CI enablement
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  find:
-    paths: "{{ topdir_path }}/guestfs"
-    patterns: "console.log"
-    file_type: file
-    recurse: yes
-  register: console_log_files
-  when:
-    - 'libvirt_uri_system|bool'
-  tags: ['console-permissions']
-
-- name: Ensure console.log files are owned by the main user for CI monitoring
-  become: yes
-  become_flags: 'su - -c'
-  become_method: sudo
-  file:
-    path: "{{ item.path }}"
-    owner: "{{ reg_user.stdout }}"
-    group: "{{ reg_user.stdout }}"
-  loop: "{{ console_log_files.files }}"
-  loop_control:
-    label: "{{ item.path | regex_replace('^.*guestfs/', 'guestfs/') }}"
-  when:
-    - 'libvirt_uri_system|bool'
-    - console_log_files.matched > 0
-  tags: ['console-permissions']
-- 
2.49.0

[PATCH v2 11/12] scripts: Remove the bringup_guestfs.sh script

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up: This script is no longer used.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/bringup_guestfs.sh | 391 -------------------------------------
 1 file changed, 391 deletions(-)
 delete mode 100755 scripts/bringup_guestfs.sh

diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
deleted file mode 100755
index 55b13a0cb2cc..000000000000
--- a/scripts/bringup_guestfs.sh
+++ /dev/null
@@ -1,391 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: copyleft-next-0.3.1
-
-[ -z "${TOPDIR}" ] && TOPDIR='.'
-source ${TOPDIR}/.config
-source ${TOPDIR}/scripts/lib.sh
-
-if [[ "${CONFIG_GUESTFS_BRINGUP_DEBUG_0+x}" && \
-	   "${CONFIG_GUESTFS_BRINGUP_DEBUG_0}" == "y" ]]; then
-	set -x
-fi
-
-if [[ "${CONFIG_GUESTFS_BRINGUP_DEBUG_1+x}" && \
-	  "${CONFIG_GUESTFS_BRINGUP_DEBUG_1}" == "y" ]]; then
-	set -euxo pipefail
-fi
-
-export LIBVIRT_DEFAULT_URI=$CONFIG_LIBVIRT_URI
-
-# We use the NVMe setting for virtio too (go figure), but IDE
-# requires qcow2
-#
-IMG_FMT="qcow2"
-if [ "${CONFIG_LIBVIRT_EXTRA_DRIVE_FORMAT_RAW}" = "y" ]; then
-	IMG_FMT="raw"
-fi
-STORAGEDIR="${CONFIG_LIBVIRT_STORAGE_POOL_PATH}/${CONFIG_KDEVOPS_STORAGE_POOL_USER}/guestfs"
-QEMU_GROUP=$CONFIG_LIBVIRT_QEMU_GROUP
-GUESTFSDIR="${TOPDIR}/guestfs"
-OS_VERSION=${CONFIG_VIRT_BUILDER_OS_VERSION}
-BASE_IMAGE_DIR="${STORAGEDIR}/base_images"
-BASE_IMAGE="${BASE_IMAGE_DIR}/${OS_VERSION}.raw"
-
-build_custom_source()
-{
-	SOURCE_TMP=$(mktemp)
-	cat <<_EOT >$SOURCE_TMP
-[local]
-uri=file:///${CUSTOM_INDEX}
-proxy=off
-_EOT
-	sudo mv $SOURCE_TMP $CUSTOM_SOURCE
-}
-
-build_custom_index()
-{
-	cat <<_EOT >$CUSTOM_INDEX
-[$OS_VERSION]
-file=${OS_VERSION}.raw
-_EOT
-}
-
-fetch_custom_image()
-{
-	wget --directory-prefix=$CUSTOM_IMAGE_DIR $CONFIG_GUESTFS_CUSTOM_RAW_IMAGE_URL
-	if [[ $? -ne 0 ]]; then
-		echo -e "Could not download:\n$CONFIG_GUESTFS_CUSTOM_RAW_IMAGE_URL"
-		exit 1
-	fi
-}
-
-check_custom_image()
-{
-	SHA512SUMS_FILE="$(basename $CONFIG_GUESTFS_CUSTOM_RAW_IMAGE_SHA512SUMS_URL)"
-	CUSTOM_IMAGE_SHA512SUM="$CUSTOM_IMAGE_DIR/$SHA512SUMS_FILE"
-	if [[ ! -f $CUSTOM_IMAGE_SHA512SUM ]]; then
-		wget --directory-prefix=$CUSTOM_IMAGE_DIR $CONFIG_GUESTFS_CUSTOM_RAW_IMAGE_SHA512SUMS_URL
-		if [[ $? -ne 0 ]]; then
-			echo "Could not get sha512sum file: $CONFIG_GUESTFS_CUSTOM_RAW_IMAGE_SHA512SUMS_URL"
-			exit 1
-		fi
-	fi
-	echo "Checking $CUSTOM_IMAGE_DIR/$SHA512SUMS_FILE"
-
-	# This subshell let's us keep below in the current directory.
-	# sha512sum files are relative to the local directory
-	(cd $CUSTOM_IMAGE_DIR && sha512sum --ignore-missing -c $SHA512SUMS_FILE)
-	if [[ $? -ne 0 ]]; then
-		echo "Invalid SHA512SUM checksum for $CUSTOM_IMAGE as per $SHA512SUMS_FILE"
-		exit 1
-	fi
-	touch $CUSTOM_IMAGE_OK
-}
-
-# Ensure folks are not surprised. If you're using rolling distros you know what
-# you are doing. This gives us the right later to change this at will.
-#
-# In the future we can make this smoother, as we used to have it with vagrant
-# update, but for now downloading *once* for a rolling distro seems ok to start.
-# We give enough information so you can update.
-build_warn_rolling_distro()
-{
-	echo "------------------------------------------------------------------"
-	echo "This is a rolling distribution release! To upgrade just do:"
-	echo
-	echo "rm -rf ${CUSTOM_IMAGE}/*"
-	echo "rm -f  ${CUSTOM_SOURCE}"
-	echo "rm -f  ${CUSTOM_INDEX}"
-	echo
-	echo "Running guests always use their own copy. To rebuild your custom"
-	echo "base image from the custom image, also remove the base image:"
-	echo
-	echo "rm -f  ${BASE_IMAGE}"
-	echo
-	echo "This can always be done safely without affecting running guests."
-	echo "------------------------------------------------------------------"
-}
-
-build_custom_image()
-{
-	CUSTOM_IMAGE_DIR="${STORAGEDIR}/custom_images/${OS_VERSION}"
-	CUSTOM_IMAGE="${CUSTOM_IMAGE_DIR}/${OS_VERSION}.raw"
-	CUSTOM_IMAGE_OK="${CUSTOM_IMAGE_DIR}.ok"
-	CUSTOM_SOURCE="/etc/virt-builder/repos.d/kdevops-custom-images-${OS_VERSION}.conf"
-	mkdir -p ${CUSTOM_IMAGE_DIR}
-	CUSTOM_INDEX="$(realpath ${CUSTOM_IMAGE_DIR}/index)"
-
-	if [[ ! -f $CUSTOM_IMAGE && "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE_URL" == "y" ]]; then
-		fetch_custom_image
-	fi
-
-	if [[ ! -f $CUSTOM_IMAGE_OK && "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE_SHA512SUMS" == "y" ]]; then
-		check_custom_image
-	fi
-
-	if [[ ! -f $CUSTOM_IMAGE ]]; then
-		echo "Custom image on path $CUSTOM_IMAGE not found"
-		exit 1
-	fi
-
-	if [[ ! -f $CUSTOM_SOURCE ]]; then
-		build_custom_source
-	fi
-
-	if [[ ! -f $CUSTOM_INDEX ]]; then
-		build_custom_index
-	fi
-
-	echo "Custom virt-builder source: $CUSTOM_SOURCE"
-	echo "Custom virt-builder index:  $CUSTOM_INDEX"
-	echo "Custom image source:        $CUSTOM_IMAGE"
-
-	if [[ "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE_ROLLING" == "y" ]]; then
-		build_warn_rolling_distro
-	fi
-
-	echo "Going to build index for $OS_VERSION ..."
-	virt-builder-repository --no-compression $CUSTOM_IMAGE_DIR
-	if [[ $? -ne 0 ]]; then
-		echo "Failed to build repository ..."
-		exit 1
-	fi
-
-	# Note, we don't build $BASE_IMAGE, virt-builder does that later. We
-	# just build $virt-builder, which is the pristine upstream image.
-}
-
-handle_rhel_activation()
-{
-	if [ -n "$CONFIG_RHEL_ORG_ID" -a -n "$CONFIG_RHEL_ACTIVATION_KEY" ]; then
-		DO_UNREG=1
-		cat <<_EOT >>$cmdfile
-run-command subscription-manager register --org=${CONFIG_RHEL_ORG_ID} --activationkey=${CONFIG_RHEL_ACTIVATION_KEY}
-_EOT
-	fi
-}
-
-handle_rhel_unreg()
-{
-	cat <<_EOT >>$cmdfile
-sm-unregister
-_EOT
-}
-
-copy_yum_repo()
-{
-	cat <<_EOT >>$cmdfile
-copy-in $CONFIG_KDEVOPS_CUSTOM_YUM_REPOFILE:/etc/yum.repos.d
-_EOT
-}
-
-copy_host_sources()
-{
-	TARGET_DIR="$(dirname $CONFIG_GUESTFS_DISTRO_SOURCE_AND_DEST_FILE)"
-	cat <<_EOT >>$cmdfile
-mkdir $TARGET_DIR
-copy-in $CONFIG_GUESTFS_DISTRO_SOURCE_AND_DEST_FILE:$TARGET_DIR
-_EOT
-}
-
-pre_install_customizations()
-{
-	KDEVOPS_UID=""
-	if getent passwd kdevops > /dev/null 2>&1; then
-		KDEVOPS_UID="-u `id -u kdevops`"
-	fi
-	if echo $OS_VERSION | grep -qE "^(rhel|fedora|centos)"; then
-		UPDATE_GRUB_CMD="/usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg"
-	else
-		UPDATE_GRUB_CMD="/usr/sbin/update-grub2"
-	fi
-	cat <<_EOT >>$cmdfile
-install sudo,qemu-guest-agent,python3,bash
-run-command useradd ${KDEVOPS_UID} -s /bin/bash -m kdevops
-append-line /etc/sudoers.d/kdevops:kdevops   ALL=(ALL)       NOPASSWD: ALL
-edit /etc/default/grub:s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0"/
-run-command $UPDATE_GRUB_CMD
-root-password password:kdevops
-_EOT
-}
-
-# Ugh, debian has to be told to bring up the network and regenerate ssh keys
-# Hope we get that interface name right!
-debian_pre_install_hacks()
-{
-	cat <<_EOT >>$cmdfile
-install isc-dhcp-client,ifupdown
-mkdir /etc/network/interfaces.d/
-append-line /etc/network/interfaces.d/enp1s0:auto enp1s0
-append-line /etc/network/interfaces.d/enp1s0:allow-hotplug enp1s0
-append-line /etc/network/interfaces.d/enp1s0:iface enp1s0 inet dhcp
-firstboot-command systemctl disable systemd-networkd-wait-online.service
-firstboot-command systemctl stop ssh
-firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force openssh-server
-firstboot-command systemctl start ssh
-firstboot-command apt update && apt upgrade --yes
-uninstall unattended-upgrades
-_EOT
-	# CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST will not work
-	# if /etc/nsswitch.conf has a line like this:
-	#
-	# hosts:          files myhostname resolve [!UNAVAIL=return] dns
-	#
-	# We need DNS to be used so virb0 will be used for a DNS request
-	# For the life of me I can't get the following line to work with
-	# the virt-builder command and so we do a full edit of the file for now
-	# edit /etc/nsswitch.conf:'s/\[!UNAVAIL=return\]//'
-	if [[ "${CONFIG_GUESTFS_DEBIAN_TRIXIE+x}" && \
-		  "$CONFIG_GUESTFS_DEBIAN_TRIXIE" == "y" ]]; then
-		cat <<_EOT >>$cmdfile
-write /etc/nsswitch.conf: # kdevops generated /etc/nsswitch.conf
-append-line /etc/nsswitch.conf:passwd:         files
-append-line /etc/nsswitch.conf:group:          files
-append-line /etc/nsswitch.conf:shadow:         files
-append-line /etc/nsswitch.conf:gshadow:        files
-append-line /etc/nsswitch.conf:hosts:          files myhostname resolve dns
-append-line /etc/nsswitch.conf:networks:       files
-append-line /etc/nsswitch.conf:protocols:      db files
-append-line /etc/nsswitch.conf:services:       db files
-append-line /etc/nsswitch.conf:ethers:         db files
-append-line /etc/nsswitch.conf:rpc:            db files
-append-line /etc/nsswitch.conf:netgroup:       nis
-uninstall cloud-init
-write /etc/default/locale:LANG=en_US.UTF-8
-append-line /etc/default/locale:LANGUAGE=en_US:en
-write /etc/locale.gen:en_US.UTF-8 UTF-8
-firstboot-command locale-gen en_US.UTF-8
-firstboot-command update-locale LANG=en_US.UTF-8
-firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -p low --force locales
-firstboot-command systemctl stop ssh
-firstboot-command systemctl start ssh
-_EOT
-		if [[ "${CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST+x}" && \
-			  "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
-		cat <<_EOT >>$cmdfile
-delete /etc/apt/sources.list.d/debian.sources
-_EOT
-		fi
-	fi
-}
-
-USE_SUDO=""
-if [[ "$CONFIG_LIBVIRT_URI_SYSTEM" == "y" ]]; then
-	USE_SUDO="sudo "
-fi
-
-cmdfile=$(mktemp)
-
-if [ ! -f $BASE_IMAGE ]; then
-	if [[ "${CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE+x}" && \
-		  "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE" == "y" ]]; then
-		build_custom_image
-	fi
-
-	DO_UNREG=0
-	if echo $OS_VERSION | grep -q '^rhel'; then
-		handle_rhel_activation
-	fi
-
-	if [ -n "$CONFIG_KDEVOPS_CUSTOM_YUM_REPOFILE" ]; then
-		copy_yum_repo
-	fi
-
-	if [[ "${CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST+x}" && \
-		  "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
-		copy_host_sources
-	fi
-
-	pre_install_customizations
-
-	if [ $DO_UNREG -ne 0 ]; then
-		handle_rhel_unreg
-	fi
-
-	if echo $OS_VERSION | grep -q '^debian'; then
-		debian_pre_install_hacks
-	fi
-
-	echo "Generating new base image for ${OS_VERSION}"
-	$USE_SUDO virt-builder ${OS_VERSION} --arch `uname -m` -o $BASE_IMAGE --size 20G --format raw --commands-from-file $cmdfile
-	if [[ $? -ne 0 ]]; then
-		echo "Failed to build custom image $BASE_IMAGE"
-		exit 1
-	fi
-fi
-
-# FIXME: is there a yaml equivalent of jq?
-grep -e '^  - name: ' ${TOPDIR}/guestfs/kdevops_nodes.yaml | sed 's/^  - name: //' | while read name
-do
-	#
-	# If the guest is already defined, then just stop what we're doing
-	# and plead to the developer to clean things up.
-	#
-	if virsh list --all | grep --quiet --word-regexp "$name"; then
-		output_domstate=$(virsh domstate $name 2>/dev/null)
-		echo "Domain $name is already defined. (state: $output_domstate)"
-		if [ "$output_domstate" != "running" ]; then
-			virsh start $name
-		fi
-		exit 0
-	fi
-
-	SSH_KEY_DIR="${GUESTFSDIR}/$name/ssh"
-	SSH_KEY="${SSH_KEY_DIR}/id_ed25519"
-
-	# Generate a new ssh key
-	mkdir -p "$SSH_KEY_DIR"
-	chmod 0700 "$SSH_KEY_DIR"
-	rm -f $SSH_KEY $SSH_KEY.pub
-	ssh-keygen -q -t ed25519 -f $SSH_KEY -N ""
-
-	mkdir -p "$STORAGEDIR/$name"
-
-	# Copy the base image and prep it
-	ROOTIMG="$STORAGEDIR/$name/root.raw"
-	cp --reflink=auto $BASE_IMAGE $ROOTIMG
-	TZ="$(timedatectl show -p Timezone --value)"
-	$USE_SUDO virt-sysprep -a $ROOTIMG --hostname $name --ssh-inject "kdevops:file:$SSH_KEY.pub" --timezone $TZ
-
-	if [[ "${CONFIG_LIBVIRT_ENABLE_LARGEIO+x}" && \
-		  "$CONFIG_LIBVIRT_ENABLE_LARGEIO" == "y" ]]; then
-		lbs_idx=0
-		for i in $(seq 1 $(($CONFIG_QEMU_LARGEIO_MAX_POW_LIMIT+1))); do
-			for x in $(seq 0 $CONFIG_QEMU_EXTRA_DRIVE_LARGEIO_NUM_DRIVES_PER_SPACE); do
-				diskimg="$STORAGEDIR/$name/extra${lbs_idx}.${IMG_FMT}"
-				rm -f $diskimg
-				qemu-img create -f $IMG_FMT "$diskimg" 100G
-				if [[ "$CONFIG_LIBVIRT_URI_SYSTEM" == "y" ]]; then
-					chmod g+rw $diskimg
-					chgrp $QEMU_GROUP $diskimg
-				fi
-				let lbs_idx=$lbs_idx+1
-			done
-		done
-	else
-		# build some extra disks
-		for i in $(seq 0 3); do
-			diskimg="$STORAGEDIR/$name/extra${i}.${IMG_FMT}"
-			rm -f $diskimg
-			qemu-img create -f $IMG_FMT "$STORAGEDIR/$name/extra${i}.$IMG_FMT" 100G
-			if [[ "$CONFIG_LIBVIRT_URI_SYSTEM" == "y" ]]; then
-				chmod g+rw $STORAGEDIR/$name/extra${i}.$IMG_FMT
-				chgrp $QEMU_GROUP $STORAGEDIR/$name/extra${i}.$IMG_FMT
-			fi
-		done
-	fi
-
-	virsh define $GUESTFSDIR/$name/$name.xml
-	XML_DEVICES_COUNT=$(find $GUESTFSDIR/$name/ -name pcie_passthrough_*.xml | wc -l)
-	if [[ $XML_DEVICES_COUNT -gt 0 ]]; then
-		for xml in $GUESTFSDIR/$name/pcie_passthrough_*.xml; do
-			echo "Doing PCI-E passthrough for device $xml"
-			virsh attach-device $name $xml --config
-		done
-	fi
-	virsh start $name
-	if [[ $? -ne 0 ]]; then
-		echo "Failed to start $name"
-		exit 1
-	fi
-done
-- 
2.49.0

[PATCH v2 12/12] scripts: Remove the destroy_guestfs.sh script

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up: This script is no longer used.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/destroy_guestfs.sh | 36 ------------------------------------
 1 file changed, 36 deletions(-)
 delete mode 100755 scripts/destroy_guestfs.sh

diff --git a/scripts/destroy_guestfs.sh b/scripts/destroy_guestfs.sh
deleted file mode 100755
index dfbb4f15f4ca..000000000000
--- a/scripts/destroy_guestfs.sh
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: copyleft-next-0.3.1
-
-[ -z "${TOPDIR}" ] && TOPDIR='.'
-source ${TOPDIR}/.config
-source ${TOPDIR}/scripts/lib.sh
-
-export LIBVIRT_DEFAULT_URI=$CONFIG_LIBVIRT_URI
-
-STORAGEDIR="${CONFIG_LIBVIRT_STORAGE_POOL_PATH}/${CONFIG_KDEVOPS_STORAGE_POOL_USER}/guestfs"
-GUESTFSDIR="${TOPDIR}/guestfs"
-
-if [ -f "$GUESTFSDIR/kdevops_nodes.yaml" ]; then
-	# FIXME: is there a yaml equivalent to jq ?
-	grep -e '^  - name: ' "${GUESTFSDIR}/kdevops_nodes.yaml"  | sed 's/^  - name: //' | while read name
-	do
-		domstate=$(virsh domstate $name 2>/dev/null)
-		if [ $? -eq 0 ]; then
-			if [ "$domstate" = 'running' ]; then
-				virsh destroy $name
-			fi
-			virsh undefine --nvram $name
-		fi
-		rm -rf "$GUESTFSDIR/$name"
-		rm -rf "$STORAGEDIR/$name"
-		ssh-keygen -q -f ~/.ssh/known_hosts -R $name 1> /dev/null 2>&1
-	done
-fi
-
-if [[ "$CONFIG_TOPDIR_PATH_HAS_SHA256SUM" == "y" ]]; then
-	rm -f ~/.ssh/config_kdevops_$CONFIG_TOPDIR_PATH_SHA256SUM
-else
-	rm -f ~/.ssh/config_kdevops_$CONFIG_KDEVOPS_HOSTS_PREFIX
-fi
-rm -f $GUESTFSDIR/.provisioned_once
-rm -f $GUESTFSDIR/kdevops_nodes.yaml
-- 
2.49.0

Re: [PATCH v2 00/12] Convert bringup_guestfs to a single Ansible role

[Luis Chamberlain]

On Fri, May 30, 2025 at 01:52:17PM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> This series demonstrates some ideas about how libvirt/guestfs bring-
> up might be expressed using only Ansible (ie. with a minimal use of
> shell scripting). Benefits include:
> 
> - Starting guests is now done in parallel
> - "make bringup" is more idempotent
> - Lets us create utilities that both guestfs and Terraform can use

All looks good but I would have removed parts of the script as its
replacement is added on ansible piece-meal wise. But at this point
I'd rather have this merged than ask for a respin.

> The new parts that handle custom upstream images has not yet been
> tested.

What is missing exactly here to test, is this just to test to ensure
that debian testing custom images work?

  Luis

Re: [PATCH v2 00/12] Convert bringup_guestfs to a single Ansible role

[Chuck Lever]

On 6/3/25 3:29 PM, Luis Chamberlain wrote:
> On Fri, May 30, 2025 at 01:52:17PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> This series demonstrates some ideas about how libvirt/guestfs bring-
>> up might be expressed using only Ansible (ie. with a minimal use of
>> shell scripting). Benefits include:
>>
>> - Starting guests is now done in parallel
>> - "make bringup" is more idempotent
>> - Lets us create utilities that both guestfs and Terraform can use
> 
> All looks good but I would have removed parts of the script as its
> replacement is added on ansible piece-meal wise.

I tried that way at first, actually. It made each patch more messy and
the series difficult to re-order.


> But at this point
> I'd rather have this merged than ask for a respin.
> >> The new parts that handle custom upstream images has not yet been
>> tested.
> 
> What is missing exactly here to test, is this just to test to ensure
> that debian testing custom images work?

Well I've never had need to use the custom image mechanism, so I'm not
real sure what are the use cases that should be tested. My usual
wheelhouse is Fedora, not Debian, fwiw.

The first 6 patches are probably well enough tested to push now. Let
me know if you have ideas about how to test custom images, or we can
push everything now and just fix the bugs as they are found.

-- 
Chuck Lever

Re: [PATCH v2 00/12] Convert bringup_guestfs to a single Ansible role

[Luis Chamberlain]

On Wed, Jun 04, 2025 at 10:29:23AM -0400, Chuck Lever wrote:
> On 6/3/25 3:29 PM, Luis Chamberlain wrote:
> > On Fri, May 30, 2025 at 01:52:17PM -0400, Chuck Lever wrote:
> >> From: Chuck Lever <chuck.lever@oracle.com>
> >>
> >> This series demonstrates some ideas about how libvirt/guestfs bring-
> >> up might be expressed using only Ansible (ie. with a minimal use of
> >> shell scripting). Benefits include:
> >>
> >> - Starting guests is now done in parallel
> >> - "make bringup" is more idempotent
> >> - Lets us create utilities that both guestfs and Terraform can use
> > 
> > All looks good but I would have removed parts of the script as its
> > replacement is added on ansible piece-meal wise.
> 
> I tried that way at first, actually. It made each patch more messy and
> the series difficult to re-order.

Oh wow! Hopefully LLMs will pick up the delta between different commits.
I menion this as I expect each kdevops commit later to be useful for
further automation / conversions.

> > But at this point
> > I'd rather have this merged than ask for a respin.
> > >> The new parts that handle custom upstream images has not yet been
> >> tested.
> > 
> > What is missing exactly here to test, is this just to test to ensure
> > that debian testing custom images work?
> 
> Well I've never had need to use the custom image mechanism, so I'm not
> real sure what are the use cases that should be tested. My usual
> wheelhouse is Fedora, not Debian, fwiw.
> 
> Let
> me know if you have ideas about how to test custom images, or we can
> push everything now and just fix the bugs as they are found.

Let's just merge it all!

  Luis