[PATCH v2 00/37] Various fixes

[PATCH v2 00/37] Various fixes

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

I've finally gotten most of my test runners moved to cloud providers
so I'm able to rebase my version of kdevops on

  linux-kdevops/kdevops : HEAD

In the process I've found and fixed myriad small issues. One or two
of these might need some attention and additional work. Please have
a close look -- they are all short subjects.

Changes since v1:
- Add R-b tags
- Completed the patch that adds new GCE machine sizes
- Added 11 new fixes on the end

Daniel, this series includes the patches that change the Ansible
config and inventory pathname behavior. It would be most convenient
for me if these were merged now and then you can do your sandbox
work on that, but if you would like me to hold off, I can drop them
from this series.

Chuck Lever (37):
  scripts: Update gen_ssh_key.sh
  terraform/azure: Fix count of managed disks
  terraform/azure: Remove tags from terraform configuration
  terraform/azure: Enable network acceleration
  terraform/azure: Make the resource group name a variable
  terraform/azure: Remove the provider version constraint
  terraform/azure: Update the RHEL publishers
  terraform/oci: Work around oracle/oci provider bug
  terraform/oci: Add Kconfig choices for disk performance provisioning
  terraform/oci: Enable preemptible compute instances
  terraform/aws: Fix zone selection for us_west1
  Revert "terraform/aws: Add Oracle Linux AMI selections"
  terraform/aws: Remove unused terraform tags
  terraform/GCE: Fix the default disk throughput values
  terraform/gce: Remove unused tags
  terraform/gce: Remove support for Fedora
  terraform/gce: Update existing machine types and introduce new ones
  terraform: Create the user's .ssh/config if it does not already exist
  pynfs: Don't install xdrlib3 on older RHEL
  guestfs: Update the guestfs playbook description
  gen_hosts: Add 'localhost' to the [all] group
  gen_hosts: Add [service] group to the builder host template
  guestfs: Don't run the 'destroy' tags on the controller
  base_image: Skip "locales-all" package on Fedora
  update_etc_hosts: ipaddr
  Add an Ansible requirements.yml file
  terraform/oci: Clean up white space in route table resource
  guestfs: Correct the spelling of "drives"
  guestfs: Configure how many extra drives to provision
  ltp: Replace /opt with a symlink to data partition
  terraform: Squelch Ansible complaints about missing inventory file
  terraform: Skip localhost when waiting for targets to reboot
  devconfig: include the "service" group in the playbooks hosts list
  Makefile: Remove CONFIG_TOPDIR_PATH_HAS_SHA256SUM
  Makefile: Set TOPDIR_PATH and generate its sha256sum at run time
  ansible.cfg: generate an ansible.cfg file in TOPDIR
  inventory: generate the inventory pathname dynamically

 Kconfig                                       |  15 --
 Makefile                                      |  18 +++
 Makefile.kdevops                              |   1 +
 kconfigs/Kconfig.ansible_cfg                  |  24 ++++
 kconfigs/Kconfig.libvirt                      |  45 ++++++
 playbooks/devconfig.yml                       |   2 +-
 playbooks/gen_tfvars.yml                      |   1 +
 playbooks/guestfs.yml                         |   2 +-
 playbooks/install_terraform.yml               |   1 +
 .../base_image/templates/virt-builder.j2      |   2 +-
 .../roles/gen_hosts/templates/blktests.j2     |   3 +
 .../roles/gen_hosts/templates/builder.j2      |   5 +
 playbooks/roles/gen_hosts/templates/cxl.j2    |   3 +
 playbooks/roles/gen_hosts/templates/ltp.j2    |   4 +
 .../roles/gen_hosts/templates/mmtests.j2      |   3 +
 .../roles/gen_hosts/templates/selftests.j2    |   3 +
 .../roles/gen_hosts/templates/sysbench.j2     |   3 +
 playbooks/roles/gen_nodes/templates/drives.j2 |   8 +-
 .../roles/gen_nodes/templates/gen_drives.j2   |   8 +-
 .../templates/azure/terraform.tfvars.j2       |   1 +
 .../templates/oci/terraform.tfvars.j2         |   1 +
 .../roles/guestfs/tasks/bringup/main.yml      |   2 +-
 playbooks/roles/ltp/tasks/main.yml            |  45 +++++-
 .../pynfs/tasks/install-deps/redhat/main.yml  |   2 +
 playbooks/roles/terraform/tasks/main.yml      |   1 +
 .../roles/update_etc_hosts/tasks/main.yml     |   4 +-
 requirements.yml                              |   8 ++
 scripts/ansible.Makefile                      |   5 +
 scripts/devconfig.Makefile                    |   9 +-
 scripts/firstconfig.Makefile                  |   2 +-
 scripts/gen-hosts.Makefile                    |   1 +
 scripts/gen_ssh_key.sh                        |   2 +-
 scripts/guestfs.Makefile                      |   1 +
 scripts/provision.Makefile                    |   2 +-
 scripts/ssh.Makefile                          |   7 +-
 scripts/terraform.Makefile                    |   5 +-
 scripts/update_ssh_config_guestfs.py          |  11 +-
 terraform/aws/kconfigs/Kconfig.compute        |   4 -
 terraform/aws/kconfigs/Kconfig.location       |  21 ++-
 terraform/aws/kconfigs/distros/Kconfig.oracle |  48 -------
 terraform/aws/main.tf                         |  13 --
 terraform/azure/kconfigs/Kconfig.location     |  16 +++
 terraform/azure/kconfigs/Kconfig.storage      |   1 +
 .../azure/kconfigs/publishers/Kconfig.rhel    |  10 +-
 terraform/azure/main.tf                       |  35 +----
 terraform/azure/provider.tf                   |   2 +-
 terraform/azure/vars.tf                       |   5 +
 terraform/gce/kconfigs/Kconfig.compute        |  12 +-
 terraform/gce/kconfigs/Kconfig.storage        |   6 +-
 terraform/gce/kconfigs/images/Kconfig.fedora  |  95 -------------
 .../gce/kconfigs/machines/Kconfig.a2-standard |  56 ++++++++
 .../gce/kconfigs/machines/Kconfig.c3-standard |  41 ++++++
 .../kconfigs/machines/Kconfig.c3d-standard    |  46 ++++++
 .../gce/kconfigs/machines/Kconfig.c4-standard |  23 +--
 .../kconfigs/machines/Kconfig.c4a-standard    |  57 ++++++++
 .../kconfigs/machines/Kconfig.c4d-standard    |  51 +++++++
 .../gce/kconfigs/machines/Kconfig.e2-highcpu  |  58 ++++++++
 .../gce/kconfigs/machines/Kconfig.e2-standard |  58 ++++++++
 .../gce/kconfigs/machines/Kconfig.n1-highcpu  |  55 ++++++++
 .../gce/kconfigs/machines/Kconfig.n1-standard |  26 ++--
 .../gce/kconfigs/machines/Kconfig.n2-highcpu  |  16 +--
 .../gce/kconfigs/machines/Kconfig.n2-standard |  16 +--
 .../gce/kconfigs/machines/Kconfig.n2d-highcpu |  15 +-
 .../kconfigs/machines/Kconfig.n2d-standard    |  17 ++-
 .../gce/kconfigs/machines/Kconfig.n4-highcpu  |  16 +--
 .../gce/kconfigs/machines/Kconfig.n4-standard |  16 +--
 terraform/gce/main.tf                         |   2 -
 terraform/oci/kconfigs/Kconfig.storage        | 132 ++++++++++++++++++
 terraform/oci/main.tf                         |  10 ++
 terraform/oci/provider.tf                     |   4 +-
 terraform/oci/vars.tf                         |   5 +
 terraform/oci/volumes/main.tf                 |  12 +-
 terraform/oci/volumes/vars.tf                 |   5 +
 73 files changed, 928 insertions(+), 337 deletions(-)
 create mode 100644 requirements.yml
 delete mode 100644 terraform/aws/kconfigs/distros/Kconfig.oracle
 delete mode 100644 terraform/gce/kconfigs/images/Kconfig.fedora
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.a2-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c3-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c3d-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c4a-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c4d-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.e2-highcpu
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.e2-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.n1-highcpu

-- 
2.51.0

[PATCH v2 01/37] scripts: Update gen_ssh_key.sh

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Drop the -b option. We're stuck with RSA because some cloud providers
require it, but ssh-keygen's default bit length for RSA keys is 3072,
which is better than the value specified here.

Leaving off -b means when ssh-keygen increases its default, kdevops
will get that change without human intervention.

I'm also adding a -C option here because I got really confused when
I looked at the terraform state imported back from the provider. The
public ssh key comment was "cel@ ... ". I thought that this was my
personal public key.

It's is not my public key, thankfully. Rather the comment chosen by
ssh-keygen happens to be the same as the one in my personal public
key. So let's pick a more distinct eye-catcher to avoid future
myocardial infarction.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/gen_ssh_key.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/gen_ssh_key.sh b/scripts/gen_ssh_key.sh
index 72bdb22694aa..4f656fe35b7c 100755
--- a/scripts/gen_ssh_key.sh
+++ b/scripts/gen_ssh_key.sh
@@ -5,4 +5,4 @@ source ${TOPDIR}/.config
 source ${TOPDIR}/scripts/lib.sh
 
 echo "Creating $KDEVOPS_SSH_PRIVKEY"
-ssh-keygen -b 2048 -t rsa -f $KDEVOPS_SSH_PRIVKEY -q -N ""
+ssh-keygen -t rsa -C generated-by-kdevops -f $KDEVOPS_SSH_PRIVKEY -q -N ""
-- 
2.51.0

[PATCH v2 02/37] terraform/azure: Fix count of managed disks

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The Kconfig choice menu allows the selection of a single disk, but
there was no default setting for that, causing the gen_tfvars
playbook to fail with:

ansible.errors.AnsibleUndefinedVariable:
  'terraform_azure_managed_disks_per_instance' is undefined.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/azure/kconfigs/Kconfig.storage | 1 +
 1 file changed, 1 insertion(+)

diff --git a/terraform/azure/kconfigs/Kconfig.storage b/terraform/azure/kconfigs/Kconfig.storage
index 5bfe1441c09c..d4885163a3b6 100644
--- a/terraform/azure/kconfigs/Kconfig.storage
+++ b/terraform/azure/kconfigs/Kconfig.storage
@@ -64,6 +64,7 @@ endchoice
 config TERRAFORM_AZURE_MANAGED_DISKS_PER_INSTANCE
 	int
 	output yaml
+	default 1 if TERRAFORM_AZURE_MANAGED_DISKS_PER_INSTANCE_1
 	default 2 if TERRAFORM_AZURE_MANAGED_DISKS_PER_INSTANCE_2
 	default 3 if TERRAFORM_AZURE_MANAGED_DISKS_PER_INSTANCE_3
 	default 4 if TERRAFORM_AZURE_MANAGED_DISKS_PER_INSTANCE_4
-- 
2.51.0

[PATCH v2 03/37] terraform/azure: Remove tags from terraform configuration

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up: as far as I can tell these are unused anywhere.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/azure/main.tf | 24 ------------------------
 1 file changed, 24 deletions(-)

diff --git a/terraform/azure/main.tf b/terraform/azure/main.tf
index 2df0dc609381..e1d99cecf003 100644
--- a/terraform/azure/main.tf
+++ b/terraform/azure/main.tf
@@ -3,10 +3,6 @@
 resource "azurerm_resource_group" "kdevops_group" {
   name     = "kdevops_resource_group"
   location = var.azure_location
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 locals {
@@ -18,10 +14,6 @@ resource "azurerm_virtual_network" "kdevops_network" {
   address_space       = [local.kdevops_private_net]
   location            = var.azure_location
   resource_group_name = azurerm_resource_group.kdevops_group.name
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 resource "azurerm_subnet" "kdevops_subnet" {
@@ -37,10 +29,6 @@ resource "azurerm_public_ip" "kdevops_publicip" {
   location            = var.azure_location
   resource_group_name = azurerm_resource_group.kdevops_group.name
   allocation_method   = "Static"
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 resource "azurerm_network_security_group" "kdevops_sg" {
@@ -59,10 +47,6 @@ resource "azurerm_network_security_group" "kdevops_sg" {
     source_address_prefix      = "*"
     destination_address_prefix = "*"
   }
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 resource "azurerm_network_interface_security_group_association" "kdevops_sg_assoc" {
@@ -83,10 +67,6 @@ resource "azurerm_network_interface" "kdevops_nic" {
     private_ip_address_allocation = "Dynamic"
     public_ip_address_id          = element(azurerm_public_ip.kdevops_publicip.*.id, count.index)
   }
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 resource "azurerm_linux_virtual_machine" "kdevops_vm" {
@@ -132,10 +112,6 @@ resource "azurerm_linux_virtual_machine" "kdevops_vm" {
     username   = var.ssh_config_user
     public_key = var.ssh_config_pubkey_file != "" ? file(var.ssh_config_pubkey_file) : ""
   }
-
-  tags = {
-    environment = "kdevops tests"
-  }
 }
 
 module "kdevops_managed_disks" {
-- 
2.51.0

[PATCH v2 04/37] terraform/azure: Enable network acceleration

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

This option appears to be on by default when creating VM machines
via the Azure console.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/azure/main.tf | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/terraform/azure/main.tf b/terraform/azure/main.tf
index e1d99cecf003..c178c34d0180 100644
--- a/terraform/azure/main.tf
+++ b/terraform/azure/main.tf
@@ -56,10 +56,11 @@ resource "azurerm_network_interface_security_group_association" "kdevops_sg_asso
 }
 
 resource "azurerm_network_interface" "kdevops_nic" {
-  count               = local.kdevops_num_boxes
-  name                = format("kdevops_nic_%02d", count.index + 1)
-  location            = var.azure_location
-  resource_group_name = azurerm_resource_group.kdevops_group.name
+  count                          = local.kdevops_num_boxes
+  accelerated_networking_enabled = true
+  name                           = format("kdevops_nic_%02d", count.index + 1)
+  location                       = var.azure_location
+  resource_group_name            = azurerm_resource_group.kdevops_group.name
 
   ip_configuration {
     name                          = "kdevops_nic_configuration"
-- 
2.51.0

[PATCH v2 05/37] terraform/azure: Make the resource group name a variable

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Resource groups appear to be global. To run more than one instance
of kdevops in Azure at the same time, each instance needs a unique
resource group. Hoist the resource group name into Kconfig so that
it can be set to a unique value.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../templates/azure/terraform.tfvars.j2          |  1 +
 terraform/azure/kconfigs/Kconfig.location        | 16 ++++++++++++++++
 terraform/azure/main.tf                          |  2 +-
 terraform/azure/vars.tf                          |  5 +++++
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/playbooks/roles/gen_tfvars/templates/azure/terraform.tfvars.j2 b/playbooks/roles/gen_tfvars/templates/azure/terraform.tfvars.j2
index 8be48e800036..7ce0f6170e22 100644
--- a/playbooks/roles/gen_tfvars/templates/azure/terraform.tfvars.j2
+++ b/playbooks/roles/gen_tfvars/templates/azure/terraform.tfvars.j2
@@ -1,4 +1,5 @@
 azure_subscription_id = "{{ terraform_azure_subscription_id }}"
+azure_resource_group_name = "{{ terraform_azure_resource_group_name }}"
 azure_location = "{{ terraform_azure_location }}"
 azure_vmsize = "{{ terraform_azure_vm_size }}"
 azure_image_publisher = "{{ terraform_azure_image_publisher }}"
diff --git a/terraform/azure/kconfigs/Kconfig.location b/terraform/azure/kconfigs/Kconfig.location
index a7e8bae37507..2cc0e419d54d 100644
--- a/terraform/azure/kconfigs/Kconfig.location
+++ b/terraform/azure/kconfigs/Kconfig.location
@@ -134,3 +134,19 @@ config TERRAFORM_AZURE_LOCATION
 	default "westus2" if TERRAFORM_AZURE_REGION_WESTUS2
 	default "westus3" if TERRAFORM_AZURE_REGION_WESTUS3
 	default "westcentralus" if TERRAFORM_AZURE_REGION_WESTCENTRALUS
+
+config TERRAFORM_AZURE_RESOURCE_GROUP_NAME
+	string "Azure resource group name"
+	output yaml
+	default "kdevops_resource_group"
+	help
+	  An Azure resource group is a container that holds related
+	  resources so they can be managed as a single unit. These
+	  resources share the same life cycle and are deployed,
+	  updated, and deleted together. Resource groups are global
+	  to your subscription.
+
+	  To run concurrent kdevops jobs in Azure, each run must
+	  have a unique resouce group name.
+
+	  https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/overview#resource-groups
diff --git a/terraform/azure/main.tf b/terraform/azure/main.tf
index c178c34d0180..8dcead78b5fd 100644
--- a/terraform/azure/main.tf
+++ b/terraform/azure/main.tf
@@ -1,7 +1,7 @@
 # Azure terraform provider main
 
 resource "azurerm_resource_group" "kdevops_group" {
-  name     = "kdevops_resource_group"
+  name     = var.azure_resource_group_name
   location = var.azure_location
 }
 
diff --git a/terraform/azure/vars.tf b/terraform/azure/vars.tf
index 2d06857905e2..dd3c20ed05c7 100644
--- a/terraform/azure/vars.tf
+++ b/terraform/azure/vars.tf
@@ -33,6 +33,11 @@ variable "azure_managed_disks_tier" {
   type        = string
 }
 
+variable "azure_resource_group_name" {
+  description = "Name of the set of resources for this run"
+  type        = string
+}
+
 variable "azure_subscription_id" {
   description = "Your Azure subscription ID"
   type        = string
-- 
2.51.0

[PATCH v2 06/37] terraform/azure: Remove the provider version constraint

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The most recent version 3 release of hashicorp/azurerm was 3.117.1,
released on February 28, 2025. Version 3 is no longer getting fixes.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/azure/provider.tf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/terraform/azure/provider.tf b/terraform/azure/provider.tf
index 97baad4245d7..4b6380bbc68c 100644
--- a/terraform/azure/provider.tf
+++ b/terraform/azure/provider.tf
@@ -2,7 +2,7 @@ terraform {
   required_providers {
     azurerm = {
       source  = "hashicorp/azurerm"
-      version = "~>3"
+      version = "> 4"
     }
   }
 }
-- 
2.51.0

[PATCH v2 07/37] terraform/azure: Update the RHEL publishers

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Make the most recent RHEL 9 update (update 6, announced two months
ago) the default selection.

Since update 10 has been available for more than a year, add a RHEL
8.10 menu choice.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/azure/kconfigs/publishers/Kconfig.rhel | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/terraform/azure/kconfigs/publishers/Kconfig.rhel b/terraform/azure/kconfigs/publishers/Kconfig.rhel
index 16ec92903976..3a6a7d9a1797 100644
--- a/terraform/azure/kconfigs/publishers/Kconfig.rhel
+++ b/terraform/azure/kconfigs/publishers/Kconfig.rhel
@@ -4,7 +4,7 @@ if TARGET_ARCH_X86_64
 
 choice
 	prompt "Red Hat Enterprise Linux release"
-	default TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_5
+	default TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_6
 	help
 	  This option specifies which of a publisher's offers to use
 	  when creating kdevops compute instances.
@@ -21,6 +21,12 @@ config TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_9
 	  This option sets the OS image to Red Hat Enterprise Linux
 	  release 8 update 9.
 
+config TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_10
+	bool "RHEL 8.10 x64"
+	help
+	  This option sets the OS image to Red Hat Enterprise Linux
+	  release 8 update 10.
+
 config TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_5
 	bool "RHEL 9.5 x64"
 	help
@@ -40,6 +46,7 @@ config TERRAFORM_AZURE_IMAGE_OFFER
 	output yaml
 	default "RHEL" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_7_9
 	default "RHEL" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_9
+	default "RHEL" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_10
 	default "RHEL" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_5
 	default "RHEL" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_6
 
@@ -48,6 +55,7 @@ config TERRAFORM_AZURE_IMAGE_SKU
 	output yaml
 	default "7_9" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_7_9
 	default "8_9" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_9
+	default "8_10" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_8_10
 	default "9_5" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_5
 	default "9_6" if TERRAFORM_AZURE_IMAGE_LINUX_RHEL_9_6
 
-- 
2.51.0

[PATCH v2 08/37] terraform/oci: Work around oracle/oci provider bug

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

First, I was getting

module.volumes[0].oci_core_volume.kdevops_volume_extra[0]: Creating...
╷
│ Error: 400-CannotParseRequest, Incorrectly formatted request. Please refer to our documentation for help.

No idea why, though maybe this is the first time I've tried using
us-east1 when my home region is us-central. Nothing I tried changed
this result, and there's no obvious problem with the way that the
oci_core_volume resource is specified, which hasn't changed in many
months.

All documentation I consulted stated that because the root module
explicitly specifies exactly one provider, child modules are
supposed to inherit that provider. But that isn't happening here:

$ terraform providers

Providers required by configuration:
.
├── provider[registry.terraform.io/oracle/oci] ~> 6.0
└── module.volumes
    └── provider[registry.terraform.io/hashicorp/oci]

And looks like it's a bug that was never fixed:
https://github.com/oracle/terraform-provider-oci/issues/1608

The only workaround I understood was to switch the root module
back to using the hashicorp/oci provider so that the root and
"volumes" module use the same provider.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/oci/provider.tf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/terraform/oci/provider.tf b/terraform/oci/provider.tf
index 08d0d2c526c6..610d2059577a 100644
--- a/terraform/oci/provider.tf
+++ b/terraform/oci/provider.tf
@@ -2,8 +2,8 @@ terraform {
   required_version = ">= 0.12.6"
   required_providers {
     oci = {
-      source = "oracle/oci"
-      version = "~> 6"
+      source = "hashicorp/oci"
+      version = "> 7"
     }
   }
 }
-- 
2.51.0

[PATCH v2 09/37] terraform/oci: Add Kconfig choices for disk performance provisioning

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Add the vpus_per_gb argument when provisioning disk drives to
control how much performance is available. This is similar to
the performance settings available for other providers.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .../templates/oci/terraform.tfvars.j2         |   1 +
 terraform/oci/kconfigs/Kconfig.storage        | 132 ++++++++++++++++++
 terraform/oci/main.tf                         |   1 +
 terraform/oci/vars.tf                         |   5 +
 terraform/oci/volumes/main.tf                 |  12 +-
 terraform/oci/volumes/vars.tf                 |   5 +
 6 files changed, 151 insertions(+), 5 deletions(-)

diff --git a/playbooks/roles/gen_tfvars/templates/oci/terraform.tfvars.j2 b/playbooks/roles/gen_tfvars/templates/oci/terraform.tfvars.j2
index 9c32ea5884d4..0839bfacfb24 100644
--- a/playbooks/roles/gen_tfvars/templates/oci/terraform.tfvars.j2
+++ b/playbooks/roles/gen_tfvars/templates/oci/terraform.tfvars.j2
@@ -18,6 +18,7 @@ oci_subnet_ocid = "{{ terraform_oci_subnet_ocid }}"
 {% endif %}
 oci_volumes_per_instance = {{ terraform_oci_volumes_per_instance }}
 oci_volumes_size = {{ terraform_oci_volumes_size }}
+oci_vpus_per_gb = {{ terraform_oci_vpus_per_gb }}
 oci_data_volume_device_file_name = "{{ terraform_oci_data_volume_device_file_name }}"
 oci_sparse_volume_device_file_name = "{{ terraform_oci_sparse_volume_device_file_name }}"
 
diff --git a/terraform/oci/kconfigs/Kconfig.storage b/terraform/oci/kconfigs/Kconfig.storage
index ed872a26834e..15a2a05bb26a 100644
--- a/terraform/oci/kconfigs/Kconfig.storage
+++ b/terraform/oci/kconfigs/Kconfig.storage
@@ -122,6 +122,138 @@ config TERRAFORM_OCI_VOLUMES_SIZE
 	default 4096 if TERRAFORM_OCI_VOLUMES_SIZE_4096G
 	default 8192 if TERRAFORM_OCI_VOLUMES_SIZE_8192G
 
+choice
+	prompt "Volume Performance Units for each additional volume"
+	default TERRAFORM_OCI_VPUS_PER_GB_10
+	help
+	  Higher VPUs per GB increases maximum IOPS and throughput,
+	  but costs more. For more information, see:
+
+	  https://docs.oracle.com/en-us/iaas/Content/Block/Concepts/blockvolumeperformance.htm#vpus
+
+config TERRAFORM_OCI_VPUS_PER_GB_0
+	bool "0 VPUs"
+	help
+	  Lowest cost.
+
+	  2 IOPS/GB, 3,000 IOPS maximum/volume.
+	  480 MBPS maximum/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_10
+	bool "10 VPUs"
+	help
+	  Balanced performance.
+
+	  60 IOPS/GB, 25,000 IOPS maximum/volume.
+	  480 MBPS maximum/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_20
+	bool "20 VPUs"
+	help
+	  Higher performance.
+
+	  75 IOPS/GB, 50,000 IOPS maximum/volume.
+	  680 MBPS maximum/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_30
+	bool "30 VPUs"
+	help
+	  Ultra high performance.
+
+	  90 IOPS/GB, 75,000 IOPS maximum/volume.
+	  880 MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_40
+	bool "40 VPUs"
+	help
+	  Ultra high performance.
+
+	  105 IOPS/GB, 100,000 IOPS maximum/volume.
+	  1,080 MBPS maximum/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_50
+	bool "50 VPUs"
+	help
+	  Ultra high performance.
+
+	  120 IOPS/GB, 125,000 IOPS maximum/volume.
+	  1,280 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_60
+	bool "60 VPUs"
+	help
+	  Ultra high performance.
+
+	  135 IOPS/GB, 150,000 IOPS maximum/volume.
+	  1,480 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_70
+	bool "70 VPUs"
+	help
+	  Ultra high performance.
+
+	  150 IOPS/GB, 175,000 IOPS maximum/volume.
+	  1,680 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_80
+	bool "80 VPUs"
+	help
+	  Ultra high performance.
+
+	  165 IOPS/GB, 200,000 IOPS maximum/volume.
+	  1,880 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_90
+	bool "90 VPUs"
+	help
+	  Ultra high performance.
+
+	  180 IOPS/GB, 225,000 IOPS maximum/volume.
+	  2,080 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_100
+	bool "100 VPUs"
+	help
+	  Ultra high performance.
+
+	  195 IOPS/GB, 250,000 IOPS maximum/volume.
+	  2,280 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_110
+	bool "110 VPUs"
+	help
+	  Ultra high performance.
+
+	  210 IOPS/GB, 275,000 IOPS maximum/volume.
+	  2,480 max MBPS/volume.
+
+config TERRAFORM_OCI_VPUS_PER_GB_120
+	bool "120 VPUs"
+	help
+	  Ultra high performance.
+
+	  225 IOPS/GB, 300,000 IOPS maximum/volume.
+	  2,680 max MBPS/volume.
+
+endchoice
+
+config TERRAFORM_OCI_VPUS_PER_GB
+	int
+	output yaml
+	default 0 if TERRAFORM_OCI_VPUS_PER_GB_0
+	default 10 if TERRAFORM_OCI_VPUS_PER_GB_10
+	default 20 if TERRAFORM_OCI_VPUS_PER_GB_20
+	default 30 if TERRAFORM_OCI_VPUS_PER_GB_30
+	default 40 if TERRAFORM_OCI_VPUS_PER_GB_40
+	default 50 if TERRAFORM_OCI_VPUS_PER_GB_50
+	default 60 if TERRAFORM_OCI_VPUS_PER_GB_60
+	default 70 if TERRAFORM_OCI_VPUS_PER_GB_70
+	default 80 if TERRAFORM_OCI_VPUS_PER_GB_80
+	default 90 if TERRAFORM_OCI_VPUS_PER_GB_90
+	default 100 if TERRAFORM_OCI_VPUS_PER_GB_100
+	default 110 if TERRAFORM_OCI_VPUS_PER_GB_110
+	default 120 if TERRAFORM_OCI_VPUS_PER_GB_120
+
 config TERRAFORM_OCI_DATA_VOLUME_DEVICE_FILE_NAME
 	string "Data volume's device file name"
 	output yaml
diff --git a/terraform/oci/main.tf b/terraform/oci/main.tf
index d231f2225e5d..1d2d0f82e00f 100644
--- a/terraform/oci/main.tf
+++ b/terraform/oci/main.tf
@@ -50,6 +50,7 @@ module "volumes" {
   vol_instance_name       = element(var.kdevops_nodes, count.index)
   vol_volume_count        = var.oci_volumes_per_instance
   vol_volume_size         = var.oci_volumes_size
+  vol_vpus_per_gb         = var.oci_vpus_per_gb
 }
 
 resource "oci_core_vcn" "kdevops_vcn" {
diff --git a/terraform/oci/vars.tf b/terraform/oci/vars.tf
index c116829fbc72..401341feb34b 100644
--- a/terraform/oci/vars.tf
+++ b/terraform/oci/vars.tf
@@ -98,3 +98,8 @@ variable "oci_volumes_size" {
   description = "The size of additional block volumes, in gibibytes"
   type        = number
 }
+
+variable "oci_vpus_per_gb" {
+  description = "Volume performance units"
+  type        = number
+}
diff --git a/terraform/oci/volumes/main.tf b/terraform/oci/volumes/main.tf
index 56b9a476015a..7483ff72836e 100644
--- a/terraform/oci/volumes/main.tf
+++ b/terraform/oci/volumes/main.tf
@@ -1,9 +1,11 @@
 resource "oci_core_volume" "kdevops_volume_extra" {
-  count               = var.vol_volume_count
-  availability_domain = var.vol_availability_domain
-  display_name        = format("kdevops_%s_volume%02d", var.vol_instance_name, count.index + 1)
-  compartment_id      = var.vol_compartment_ocid
-  size_in_gbs         = var.vol_volume_size
+  count                = var.vol_volume_count
+  availability_domain  = var.vol_availability_domain
+  compartment_id       = var.vol_compartment_ocid
+  display_name         = format("kdevops_%s_vol%02d", var.vol_instance_name, count.index + 1)
+  is_auto_tune_enabled = false
+  size_in_gbs          = var.vol_volume_size
+  vpus_per_gb          = var.vol_vpus_per_gb
 }
 
 locals {
diff --git a/terraform/oci/volumes/vars.tf b/terraform/oci/volumes/vars.tf
index 57c5e2bf646b..ee702c2d713f 100644
--- a/terraform/oci/volumes/vars.tf
+++ b/terraform/oci/volumes/vars.tf
@@ -27,3 +27,8 @@ variable "vol_volume_size" {
   description = "Size of each volume, in gibibytes"
   type        = number
 }
+
+variable "vol_vpus_per_gb" {
+  description = "Volume performance units"
+  type        = number
+}
-- 
2.51.0

[PATCH v2 10/37] terraform/oci: Enable preemptible compute instances

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Enabling instance preemption allows the provider to destroy the
instance if it needs the resources. The benefit is a 50%
reduction in price.

Typically, there's nothing on these instances that can't be
re-cloned or reproduced.

This should really be a Kconfig setting.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/oci/main.tf | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/terraform/oci/main.tf b/terraform/oci/main.tf
index 1d2d0f82e00f..45a4b4bc0c63 100644
--- a/terraform/oci/main.tf
+++ b/terraform/oci/main.tf
@@ -37,6 +37,13 @@ resource "oci_core_instance" "kdevops_instance" {
     ssh_authorized_keys = file(var.ssh_config_pubkey_file)
   }
 
+  preemptible_instance_config {
+    preemption_action {
+      type                 = "TERMINATE"
+      preserve_boot_volume = false
+    }
+  }
+
   preserve_boot_volume = false
 }
 
-- 
2.51.0

[PATCH v2 11/37] terraform/aws: Fix zone selection for us_west1

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The us_west1 region was missing its zone selection menu.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/aws/kconfigs/Kconfig.location | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/terraform/aws/kconfigs/Kconfig.location b/terraform/aws/kconfigs/Kconfig.location
index b457166e0552..e5dacb9dd630 100644
--- a/terraform/aws/kconfigs/Kconfig.location
+++ b/terraform/aws/kconfigs/Kconfig.location
@@ -573,11 +573,11 @@ endchoice
 
 endif # TERRAFORM_AWS_REGION_US_EAST_2
 
-if TERRAFORM_AWS_REGION_US_WEST_2
+if TERRAFORM_AWS_REGION_US_WEST_1
 
 choice
 	prompt "AWS availability zone"
-	default TERRAFORM_AWS_AV_ZONE_US_WEST_2B
+	default TERRAFORM_AWS_AV_ZONE_US_WEST_1B
 	help
 	  This option sets the AWS availablity zone to the specified value.
 	  If you wish to expand on this list send a patch after reading this
@@ -591,6 +591,23 @@ config TERRAFORM_AWS_AV_ZONE_US_WEST_1B
 	help
 	  This option selects the us-west-1b availability zone.
 
+endchoice
+
+endif # TERRAFORM_AWS_REGION_US_WEST_1
+
+if TERRAFORM_AWS_REGION_US_WEST_2
+
+choice
+	prompt "AWS availability zone"
+	default TERRAFORM_AWS_AV_ZONE_US_WEST_2B
+	help
+	  This option sets the AWS availablity zone to the specified value.
+	  If you wish to expand on this list send a patch after reading this
+	  list:
+
+	  https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html
+	  https://gist.github.com/neilstuartcraig/0ccefcf0887f29b7f240
+
 config TERRAFORM_AWS_AV_ZONE_US_WEST_2B
 	bool "us-west-2b"
 	help
-- 
2.51.0

[PATCH v2 12/37] Revert "terraform/aws: Add Oracle Linux AMI selections"

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The Oracle Linux AMIs are only for-pay marketplace options, not
created by Amazon itself.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/aws/kconfigs/Kconfig.compute        |  4 --
 terraform/aws/kconfigs/distros/Kconfig.oracle | 48 -------------------
 2 files changed, 52 deletions(-)
 delete mode 100644 terraform/aws/kconfigs/distros/Kconfig.oracle

diff --git a/terraform/aws/kconfigs/Kconfig.compute b/terraform/aws/kconfigs/Kconfig.compute
index bae0ea1c9d7b..4b9c6efb6372 100644
--- a/terraform/aws/kconfigs/Kconfig.compute
+++ b/terraform/aws/kconfigs/Kconfig.compute
@@ -99,9 +99,6 @@ config TERRAFORM_AWS_DISTRO_DEBIAN
 config TERRAFORM_AWS_DISTRO_FEDORA
 	bool "Fedora Core"
 
-config TERRAFORM_AWS_DISTRO_OL
-	bool "Oracle Linux"
-
 config TERRAFORM_AWS_DISTRO_RHEL
 	bool "Red Hat Enterprise Linux"
 
@@ -116,7 +113,6 @@ endchoice
 source "terraform/aws/kconfigs/distros/Kconfig.amazon"
 source "terraform/aws/kconfigs/distros/Kconfig.debian"
 source "terraform/aws/kconfigs/distros/Kconfig.fedora"
-source "terraform/aws/kconfigs/distros/Kconfig.oracle"
 source "terraform/aws/kconfigs/distros/Kconfig.rhel"
 source "terraform/aws/kconfigs/distros/Kconfig.sles"
 source "terraform/aws/kconfigs/distros/Kconfig.custom"
diff --git a/terraform/aws/kconfigs/distros/Kconfig.oracle b/terraform/aws/kconfigs/distros/Kconfig.oracle
deleted file mode 100644
index fd4166c457c5..000000000000
--- a/terraform/aws/kconfigs/distros/Kconfig.oracle
+++ /dev/null
@@ -1,48 +0,0 @@
-if TERRAFORM_AWS_DISTRO_OL
-
-config TERRAFORM_AWS_AMI_OWNER
-	string
-	output yaml
-	default "amazon"
-
-if TARGET_ARCH_X86_64
-
-choice
-	prompt "Oracle Linux release"
-	default TERRAFORM_AWS_OL9_X86_64
-
-config TERRAFORM_AWS_OL8_X86_64
-	bool "Oracle Linux 8 (x86)"
-
-config TERRAFORM_AWS_OL9_X86_64
-	bool "Oracle Linux 9 (x86)"
-
-endchoice
-
-config TERRAFORM_AWS_NS
-	string
-	output yaml
-	default "Oracle-Linux-8.*x86_64-*" if TERRAFORM_AWS_OL8_X86_64
-	default "Oracle-Linux-9.*x86_64-*" if TERRAFORM_AWS_OL9_X86_64
-
-endif # TARGET_ARCH_X86_64
-
-if TARGET_ARCH_ARM64
-
-choice
-	prompt "Oracle Linux release"
-	default TERRAFORM_AWS_OL9_ARM64
-
-config TERRAFORM_AWS_OL9_ARM64
-	bool "Oracle Linux 9 (arm64)"
-
-endchoice
-
-config TERRAFORM_AWS_NS
-	string
-	output yaml
-	default "Oracle-Linux-9.*arm64-*" if TERRAFORM_AWS_OL9_ARM64
-
-endif # TARGET_ARCH_ARM64
-
-endif # TERRAFORM_AWS_DISTRO_OL
-- 
2.51.0

[PATCH v2 13/37] terraform/aws: Remove unused terraform tags

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/aws/main.tf | 13 -------------
 1 file changed, 13 deletions(-)

diff --git a/terraform/aws/main.tf b/terraform/aws/main.tf
index 8b343484f1ab..82a936c77f01 100644
--- a/terraform/aws/main.tf
+++ b/terraform/aws/main.tf
@@ -15,9 +15,6 @@ resource "aws_vpc" "kdevops_vpc" {
   cidr_block           = "10.0.0.0/16"
   enable_dns_hostnames = true
   enable_dns_support   = true
-  tags = {
-    Name = "kdevops"
-  }
 }
 
 resource "aws_subnet" "kdevops_subnet" {
@@ -132,10 +129,6 @@ resource "aws_instance" "kdevops_instance" {
     data.template_cloudinit_config.kdevops_config.*.rendered,
     count.index,
   )
-
-  tags = {
-    Name = element(var.kdevops_nodes, count.index),
-  }
 }
 
 module "kdevops_ebs_volumes" {
@@ -158,9 +151,6 @@ resource "aws_eip" "kdevops_eip" {
 
 resource "aws_internet_gateway" "kdevops_gw" {
   vpc_id = aws_vpc.kdevops_vpc.id
-  tags = {
-    Name = "kdevops-gw"
-  }
 }
 
 resource "aws_route_table" "kdevops_rt" {
@@ -169,9 +159,6 @@ resource "aws_route_table" "kdevops_rt" {
     cidr_block = "0.0.0.0/0"
     gateway_id = aws_internet_gateway.kdevops_gw.id
   }
-  tags = {
-    Name = "kdevops_rt"
-  }
 }
 
 resource "aws_route_table_association" "kdevops_rt_assoc" {
-- 
2.51.0

[PATCH v2 14/37] terraform/GCE: Fix the default disk throughput values

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/gce/kconfigs/Kconfig.storage | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/terraform/gce/kconfigs/Kconfig.storage b/terraform/gce/kconfigs/Kconfig.storage
index a71a5d62753b..717f50dfe32d 100644
--- a/terraform/gce/kconfigs/Kconfig.storage
+++ b/terraform/gce/kconfigs/Kconfig.storage
@@ -195,14 +195,14 @@ config TERRAFORM_GCE_DISK_IOPS
 	int "Provisioned IOPS for each attached disk"
 	output yaml
 	range 125 120000
-	default 240
+	default 3000
 	depends on TERRAFORM_GCE_DISK_NEEDS_IOPS
 
 config TERRAFORM_GCE_DISK_THROUGHPUT
 	int "Provisioned throughput for each attached disk"
 	output yaml
-	range 125 600
-	default 240
+	range 125 2400
+	default 300
 	depends on TERRAFORM_GCE_DISK_NEEDS_THROUGHPUT
 
 config TERRAFORM_GCE_DATA_VOLUME_DEVICE_FILE_NAME
-- 
2.51.0

[PATCH v2 15/37] terraform/gce: Remove unused tags

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/gce/main.tf | 2 --
 1 file changed, 2 deletions(-)

diff --git a/terraform/gce/main.tf b/terraform/gce/main.tf
index 0e8120686bfb..816f43098e88 100644
--- a/terraform/gce/main.tf
+++ b/terraform/gce/main.tf
@@ -9,8 +9,6 @@ resource "google_compute_instance" "kdevops_instance" {
   machine_type = var.gce_machine_type
   zone         = var.gce_zone
 
-  tags = ["kdevops"]
-
   boot_disk {
     initialize_params {
       image = data.google_compute_image.kdevops_image.self_link
-- 
2.51.0

[PATCH v2 16/37] terraform/gce: Remove support for Fedora

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Unlike the other Linux distributions, Fedora requires some
cloud-init magic to set up the log-in user. I don't feel like
digging into that right now, and no one is demanding that kdevops
support Fedora on GCE at the moment.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/gce/kconfigs/Kconfig.compute       |  4 -
 terraform/gce/kconfigs/images/Kconfig.fedora | 95 --------------------
 2 files changed, 99 deletions(-)
 delete mode 100644 terraform/gce/kconfigs/images/Kconfig.fedora

diff --git a/terraform/gce/kconfigs/Kconfig.compute b/terraform/gce/kconfigs/Kconfig.compute
index e647c9bb2ba7..0d3ae98e53d2 100644
--- a/terraform/gce/kconfigs/Kconfig.compute
+++ b/terraform/gce/kconfigs/Kconfig.compute
@@ -77,9 +77,6 @@ config TERRAFORM_GCE_DISTRO_CENTOS
 config TERRAFORM_GCE_DISTRO_DEBIAN
 	bool "Debian"
 
-config TERRAFORM_GCE_DISTRO_FEDORA
-	bool "Fedora"
-
 config TERRAFORM_GCE_DISTRO_OL
 	bool "Oracle Linux"
 
@@ -99,7 +96,6 @@ endchoice
 
 source "terraform/gce/kconfigs/images/Kconfig.centos"
 source "terraform/gce/kconfigs/images/Kconfig.debian"
-source "terraform/gce/kconfigs/images/Kconfig.fedora"
 source "terraform/gce/kconfigs/images/Kconfig.oracle"
 source "terraform/gce/kconfigs/images/Kconfig.opensuse"
 source "terraform/gce/kconfigs/images/Kconfig.rhel"
diff --git a/terraform/gce/kconfigs/images/Kconfig.fedora b/terraform/gce/kconfigs/images/Kconfig.fedora
deleted file mode 100644
index 90999617f9ce..000000000000
--- a/terraform/gce/kconfigs/images/Kconfig.fedora
+++ /dev/null
@@ -1,95 +0,0 @@
-if TERRAFORM_GCE_DISTRO_FEDORA
-
-config TERRAFORM_GCE_IMAGE_PROJECT
-	string
-	output yaml
-	default "fedora-coreos-cloud"
-
-config TERRAFORM_GCE_IMAGE_SIZE
-	int
-	output yaml
-	default 10
-
-if TARGET_ARCH_X86_64
-
-choice
-	prompt "Fedora stream to use"
-	default TERRAFORM_GCE_IMAGE_FEDORA_STABLE_X86_64
-
-config TERRAFORM_GCE_IMAGE_FEDORA_STABLE_X86_64
-	bool "stable (x86)"
-	help
-	  The stable stream is the most reliable stream offered with
-	  changes only reaching that stream after spending a period
-	  of time in the testing stream.
-
-config TERRAFORM_GCE_IMAGE_FEDORA_TESTING_X86_64
-	bool "testing (x86)"
-	help
-	  The testing stream represents what is coming in the next
-	  stable release. Content in this stream is updated
-	  regularly and offers our community an opportunity to catch
-	  breaking changes before they hit the stable stream.
-
-config TERRAFORM_GCE_IMAGE_FEDORA_NEXT_X86_64
-	bool "next (x86)"
-	help
-	  The next stream represents the future. It will often be
-	  used to experiment with new features and also test out
-	  rebases of our platform on top of the next major version
-	  of Fedora. The content in the next stream will also
-	  eventually filter down into testing and on to stable.
-
-endchoice
-
-config TERRAFORM_GCE_IMAGE_FAMILY
-	string
-	output yaml
-	default "fedora-coreos-stable" if TERRAFORM_GCE_IMAGE_FEDORA_STABLE_X86_64
-	default "fedora-coreos-testing" if TERRAFORM_GCE_IMAGE_FEDORA_TESTING_X86_64
-	default "fedora-coreos-next" if TERRAFORM_GCE_IMAGE_FEDORA_NEXT_X86_64
-
-endif # TARGET_ARCH_X86_64
-
-if TARGET_ARCH_ARM64
-
-choice
-	prompt "Fedora stream to use"
-	default TERRAFORM_GCE_IMAGE_FEDORA_STABLE_ARM64
-
-config TERRAFORM_GCE_IMAGE_FEDORA_STABLE_ARM64
-	bool "stable (arm64)"
-	help
-	  The stable stream is the most reliable stream offered with
-	  changes only reaching that stream after spending a period
-	  of time in the testing stream.
-
-config TERRAFORM_GCE_IMAGE_FEDORA_TESTING_ARM64
-	bool "testing (arm64)"
-	help
-	  The testing stream represents what is coming in the next
-	  stable release. Content in this stream is updated
-	  regularly and offers our community an opportunity to catch
-	  breaking changes before they hit the stable stream.
-
-config TERRAFORM_GCE_IMAGE_FEDORA_NEXT_ARM64
-	bool "next (arm64)"
-	help
-	  The next stream represents the future. It will often be
-	  used to experiment with new features and also test out
-	  rebases of our platform on top of the next major version
-	  of Fedora. The content in the next stream will also
-	  eventually filter down into testing and on to stable.
-
-endchoice
-
-config TERRAFORM_GCE_IMAGE_FAMILY
-	string
-	output yaml
-	default "fedora-coreos-stable-arm64" if TERRAFORM_GCE_IMAGE_FEDORA_STABLE_ARM64
-	default "fedora-coreos-testing-arm64" if TERRAFORM_GCE_IMAGE_FEDORA_TESTING_ARM64
-	default "fedora-coreos-next-arm64" if TERRAFORM_GCE_IMAGE_FEDORA_NEXT_ARM64
-
-endif # TARGET_ARCH_ARM64
-
-endif # TERRAFORM_GCE_DISTRO_FEDORA
-- 
2.51.0

[PATCH v2 17/37] terraform/gce: Update existing machine types and introduce new ones

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Add a spectrum of cost and performance choices, plus a new GPU
platform.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/gce/kconfigs/Kconfig.compute        |  8 +++
 .../gce/kconfigs/machines/Kconfig.a2-standard | 56 ++++++++++++++++++
 .../gce/kconfigs/machines/Kconfig.c3-standard | 41 +++++++++++++
 .../kconfigs/machines/Kconfig.c3d-standard    | 46 +++++++++++++++
 .../gce/kconfigs/machines/Kconfig.c4-standard | 23 +++++---
 .../kconfigs/machines/Kconfig.c4a-standard    | 57 ++++++++++++++++++
 .../kconfigs/machines/Kconfig.c4d-standard    | 51 ++++++++++++++++
 .../gce/kconfigs/machines/Kconfig.e2-highcpu  | 58 +++++++++++++++++++
 .../gce/kconfigs/machines/Kconfig.e2-standard | 58 +++++++++++++++++++
 .../gce/kconfigs/machines/Kconfig.n1-highcpu  | 55 ++++++++++++++++++
 .../gce/kconfigs/machines/Kconfig.n1-standard | 26 +++++----
 .../gce/kconfigs/machines/Kconfig.n2-highcpu  | 16 +++--
 .../gce/kconfigs/machines/Kconfig.n2-standard | 16 +++--
 .../gce/kconfigs/machines/Kconfig.n2d-highcpu | 15 +++--
 .../kconfigs/machines/Kconfig.n2d-standard    | 17 +++---
 .../gce/kconfigs/machines/Kconfig.n4-highcpu  | 16 +++--
 .../gce/kconfigs/machines/Kconfig.n4-standard | 16 +++--
 17 files changed, 503 insertions(+), 72 deletions(-)
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.a2-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c3-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c3d-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c4a-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.c4d-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.e2-highcpu
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.e2-standard
 create mode 100644 terraform/gce/kconfigs/machines/Kconfig.n1-highcpu

diff --git a/terraform/gce/kconfigs/Kconfig.compute b/terraform/gce/kconfigs/Kconfig.compute
index 0d3ae98e53d2..b0c999293cfd 100644
--- a/terraform/gce/kconfigs/Kconfig.compute
+++ b/terraform/gce/kconfigs/Kconfig.compute
@@ -55,14 +55,22 @@ config TERRAFORM_GCE_MACHINE_FAMILY_C4_STANDARD
 
 endchoice
 
+source "terraform/gce/kconfigs/machines/Kconfig.e2-highcpu"
+source "terraform/gce/kconfigs/machines/Kconfig.e2-standard"
 source "terraform/gce/kconfigs/machines/Kconfig.n1-standard"
+source "terraform/gce/kconfigs/machines/Kconfig.n1-highcpu"
 source "terraform/gce/kconfigs/machines/Kconfig.n2-standard"
 source "terraform/gce/kconfigs/machines/Kconfig.n2-highcpu"
 source "terraform/gce/kconfigs/machines/Kconfig.n2d-standard"
 source "terraform/gce/kconfigs/machines/Kconfig.n2d-highcpu"
 source "terraform/gce/kconfigs/machines/Kconfig.n4-standard"
 source "terraform/gce/kconfigs/machines/Kconfig.n4-highcpu"
+source "terraform/gce/kconfigs/machines/Kconfig.c3-standard"
+source "terraform/gce/kconfigs/machines/Kconfig.c3d-standard"
 source "terraform/gce/kconfigs/machines/Kconfig.c4-standard"
+source "terraform/gce/kconfigs/machines/Kconfig.c4a-standard"
+source "terraform/gce/kconfigs/machines/Kconfig.c4d-standard"
+source "terraform/gce/kconfigs/machines/Kconfig.a2-standard"
 
 choice
 	prompt "OS Distributor"
diff --git a/terraform/gce/kconfigs/machines/Kconfig.a2-standard b/terraform/gce/kconfigs/machines/Kconfig.a2-standard
new file mode 100644
index 000000000000..5e54db36b3ce
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.a2-standard
@@ -0,0 +1,56 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_A2_HIGHGPU
+
+choice
+	prompt "GCE Machine Type"
+	default TERRAFORM_GCE_MACHINE_A2_HIGHGPU_1G
+	help
+	  A2 High GPU machine types have a fixed number of A100
+	  40GB GPUs.
+
+config TERRAFORM_GCE_MACHINE_A2_HIGHGPU_1G
+	bool "a2-highgpu-1g"
+	help
+	  12 vCPUs, 85GB of memory, and 1 GPU with 40GB of GPU
+	  memory.
+
+config TERRAFORM_GCE_MACHINE_A2_HIGHGPU_2G
+	bool "a2-highgpu-2g"
+	help
+	  24 vCPUs, 170GB of memory, and 2 GPUs with 80GB of GPU
+	  memory.
+
+config TERRAFORM_GCE_MACHINE_A2_HIGHGPU_4G
+	bool "a2-highgpu-4g"
+	help
+	  48 vCPUs, 340GB of memory, and 4 GPUs with 160GB of GPU
+	  memory.
+
+config TERRAFORM_GCE_MACHINE_A2_HIGHGPU_8G
+	bool "a2-highgpu-8g"
+	help
+	  96 vCPUs, 680GB of memory, and 8 GPUs with 320GB of GPU
+	  memory.
+
+config TERRAFORM_GCE_MACHINE_A2_HIGHGPU_16G
+	bool "a2-highgpu-16g"
+	help
+	  96 vCPUs, 1360GB of memory, and 16 GPUs with 640GB of GPU
+	  memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "a2-highgpu-1g" if TERRAFORM_GCE_MACHINE_A2_HIGHGPU_1G
+	default "a2-highgpu-2g" if TERRAFORM_GCE_MACHINE_A2_HIGHGPU_2G
+	default "a2-highgpu-4g" if TERRAFORM_GCE_MACHINE_A2_HIGHGPU_4G
+	default "a2-highgpu-8g" if TERRAFORM_GCE_MACHINE_A2_HIGHGPU_8G
+	default "a2-highgpu-16g" if TERRAFORM_GCE_MACHINE_A2_HIGHGPU_16G
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "pd-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_A2_HIGHGPU
diff --git a/terraform/gce/kconfigs/machines/Kconfig.c3-standard b/terraform/gce/kconfigs/machines/Kconfig.c3-standard
new file mode 100644
index 000000000000..183ac315cabf
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.c3-standard
@@ -0,0 +1,41 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_C3_STANDARD
+
+choice
+	prompt "GCE Machine Type"
+	default TERRAFORM_GCE_MACHINE_C3_STANDARD_4
+	help
+	  C3 VMs are powered by the 4th generation Intel Xeon
+	  Scalable processors (code-named Sapphire Rapids), DDR5
+	  memory, and Titanium, enabling higher levels of networking
+	  performance, isolation and security.
+
+config TERRAFORM_GCE_MACHINE_C3_STANDARD_4
+	bool "c3-standard-4"
+	help
+	  4 vCPUs (2 cores) and 16GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C3_STANDARD_8
+	bool "c3-standard-8"
+	help
+	  8 vCPUs (4 cores) and 32GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C3_STANDARD_22
+	bool "c3-standard-22"
+	help
+	  22 vCPUs (11 cores) and 88GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "c3-standard-4" if TERRAFORM_GCE_MACHINE_C3_STANDARD_4
+	default "c3-standard-8" if TERRAFORM_GCE_MACHINE_C3_STANDARD_8
+	default "c3-standard-22" if TERRAFORM_GCE_MACHINE_C3_STANDARD_22
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "hyperdisk-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_C3_STANDARD
diff --git a/terraform/gce/kconfigs/machines/Kconfig.c3d-standard b/terraform/gce/kconfigs/machines/Kconfig.c3d-standard
new file mode 100644
index 000000000000..4af821e5a14e
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.c3d-standard
@@ -0,0 +1,46 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_C3D_STANDARD
+
+choice
+	prompt "GCE Machine Type"
+	default TERRAFORM_GCE_MACHINE_C3D_STANDARD_4
+	help
+	  C3D VMs are powered by the 4th-gen AMD EPYC (Genoa)
+	  processor. C3D uses Titanium, enabling higher levels
+	  of networking performance, isolation and security.
+
+config TERRAFORM_GCE_MACHINE_C3D_STANDARD_4
+	bool "c3d-standard-4"
+	help
+	  4 vCPUs (2 cores) and 16GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C3D_STANDARD_8
+	bool "c3d-standard-8"
+	help
+	  8 vCPUs (4 cores) and 32GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C3D_STANDARD_16
+	bool "c3d-standard-16"
+	help
+	  16 vCPUs (8 cores) and 64GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C3D_STANDARD_30
+	bool "c3d-standard-30"
+	help
+	  30 vCPUs (15 cores) and 120GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "c3d-standard-4" if TERRAFORM_GCE_MACHINE_C3D_STANDARD_4
+	default "c3d-standard-8" if TERRAFORM_GCE_MACHINE_C3D_STANDARD_8
+	default "c3d-standard-16" if TERRAFORM_GCE_MACHINE_C3D_STANDARD_16
+	default "c3d-standard-30" if TERRAFORM_GCE_MACHINE_C3D_STANDARD_30
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "hyperdisk-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_C3D_STANDARD
diff --git a/terraform/gce/kconfigs/machines/Kconfig.c4-standard b/terraform/gce/kconfigs/machines/Kconfig.c4-standard
index 75c2eed2d111..3ec7f4ec1620 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.c4-standard
+++ b/terraform/gce/kconfigs/machines/Kconfig.c4-standard
@@ -4,35 +4,39 @@ choice
 	prompt "GCE Machine Type"
 	default TERRAFORM_GCE_MACHINE_C4_STANDARD_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  C4 VMs are powered by 6th generation (code-named Granite
+	  Rapids) or 5th generation (code-named Emerald Rapids)
+	  Intel Xeon Scalable processors and Titanium.
 
 config TERRAFORM_GCE_MACHINE_C4_STANDARD_2
 	bool "c4-standard-2"
 	help
-	  2 vCPU (1 core) and 7GB of memory.
+	  2 vCPUs (1 core) and 7GB of memory.
 
 config TERRAFORM_GCE_MACHINE_C4_STANDARD_4
 	bool "c4-standard-4"
 	help
-	  4 vCPU (2 core) and 15GB of memory.
+	  4 vCPUs (2 core) and 15GB of memory.
 
 config TERRAFORM_GCE_MACHINE_C4_STANDARD_8
 	bool "c4-standard-8"
 	help
-	  8 vCPU (4 cores) and 30GB of memory.
+	  8 vCPUs (4 cores) and 30GB of memory.
 
 config TERRAFORM_GCE_MACHINE_C4_STANDARD_16
 	bool "c4-standard-16"
 	help
-	  16 vCPU (8 cores) and 60GB of memory.
+	  16 vCPUs (8 cores) and 60GB of memory.
 
 config TERRAFORM_GCE_MACHINE_C4_STANDARD_24
 	bool "c4-standard-24"
 	help
-	  24 vCPU (12 cores) and 90GB of memory.
+	  24 vCPUs (12 cores) and 90GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4_STANDARD_32
+	bool "c4-standard-32"
+	help
+	  32 vCPUs (16 cores) and 120 of memory.
 
 endchoice
 
@@ -44,6 +48,7 @@ config TERRAFORM_GCE_MACHINE_TYPE
 	default "c4-standard-8" if TERRAFORM_GCE_MACHINE_C4_STANDARD_8
 	default "c4-standard-16" if TERRAFORM_GCE_MACHINE_C4_STANDARD_16
 	default "c4-standard-24" if TERRAFORM_GCE_MACHINE_C4_STANDARD_24
+	default "c4-standard-32" if TERRAFORM_GCE_MACHINE_C4_STANDARD_32
 
 config TERRAFORM_GCE_IMAGE_TYPE
 	string
diff --git a/terraform/gce/kconfigs/machines/Kconfig.c4a-standard b/terraform/gce/kconfigs/machines/Kconfig.c4a-standard
new file mode 100644
index 000000000000..3a486fe4a6c3
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.c4a-standard
@@ -0,0 +1,57 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_C4A_STANDARD
+
+choice
+	prompt "GCE Machine Type"
+	default TERRAFORM_GCE_MACHINE_C4A_STANDARD_4
+	help
+	  C4A VMs are powered by Google's first Arm-based Axion™
+	  processor, DDR5 memory, and local Titanium SSDs.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_1
+	bool "c4a-standard-1"
+	help
+	  1 vCPU and 4GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_2
+	bool "c4a-standard-2"
+	help
+	  2 vCPUs and 8GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_4
+	bool "c4a-standard-4"
+	help
+	  4 vCPUs and 16GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_8
+	bool "c4a-standard-8"
+	help
+	  8 vCPUs and 32GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_16
+	bool "c4a-standard-16"
+	help
+	  16 vCPUs and 64GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4A_STANDARD_32
+	bool "c4a-standard-32"
+	help
+	  32 vCPUs and 128GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "c4a-standard-1" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_1
+	default "c4a-standard-2" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_2
+	default "c4a-standard-4" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_4
+	default "c4a-standard-8" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_8
+	default "c4a-standard-16" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_16
+	default "c4a-standard-32" if TERRAFORM_GCE_MACHINE_C4A_STANDARD_32
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "hyperdisk-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_C4A_STANDARD
diff --git a/terraform/gce/kconfigs/machines/Kconfig.c4d-standard b/terraform/gce/kconfigs/machines/Kconfig.c4d-standard
new file mode 100644
index 000000000000..6d7b4bb96c8b
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.c4d-standard
@@ -0,0 +1,51 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_C4D_STANDARD
+
+choice
+	prompt "GCE Machine Type"
+	default TERRAFORM_GCE_MACHINE_C4D_STANDARD_4
+	help
+	  C4D VMs are powered by the fifth generation AMD EPYC Turin
+	  processor and Titanium.
+
+config TERRAFORM_GCE_MACHINE_C4D_STANDARD_2
+	bool "c4d-standard-2"
+	help
+	  2 vCPUs (1 core) and 7GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4D_STANDARD_4
+	bool "c4d-standard-4"
+	help
+	  4 vCPUs (2 core) and 15GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4D_STANDARD_8
+	bool "c4d-standard-8"
+	help
+	  8 vCPUs (4 cores) and 31GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4D_STANDARD_16
+	bool "c4d-standard-16"
+	help
+	  16 vCPUs (8 cores) and 62GB of memory.
+
+config TERRAFORM_GCE_MACHINE_C4D_STANDARD_24
+	bool "c4d-standard-24"
+	help
+	  32 vCPUs (16 cores) and 124GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "c4d-standard-2" if TERRAFORM_GCE_MACHINE_C4D_STANDARD_2
+	default "c4d-standard-4" if TERRAFORM_GCE_MACHINE_C4D_STANDARD_4
+	default "c4d-standard-8" if TERRAFORM_GCE_MACHINE_C4D_STANDARD_8
+	default "c4d-standard-16" if TERRAFORM_GCE_MACHINE_C4D_STANDARD_16
+	default "c4d-standard-32" if TERRAFORM_GCE_MACHINE_C4D_STANDARD_32
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "hyperdisk-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_C4D_STANDARD
diff --git a/terraform/gce/kconfigs/machines/Kconfig.e2-highcpu b/terraform/gce/kconfigs/machines/Kconfig.e2-highcpu
new file mode 100644
index 000000000000..eb401451b3c8
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.e2-highcpu
@@ -0,0 +1,58 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_E2_HIGHCPU
+
+choice
+	prompt "GCE Machine type"
+	default TERRAFORM_GCE_MACHINE_E2_HIGHCPU_8
+	help
+	  The E2 machine series offers both Intel and AMD EPYC
+	  processors, selected for you at the time of VM creation.
+	  Machine types in this series are available in all regions
+	  and zones and support a virtio memory balloon device.
+
+config TERRAFORM_GCE_MACHINE_E2_HIGHCPU_2
+	bool "e2-highcpu-2"
+	depends on TARGET_ARCH_X86_64
+	help
+	  2 vCPUs (1 core) and 2GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_HIGHCPU_4
+	bool "e2-highcpu-4"
+	depends on TARGET_ARCH_X86_64
+	help
+	  4 vCPUs (2 cores) and 4GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_HIGHCPU_8
+	bool "e2-highcpu-8"
+	depends on TARGET_ARCH_X86_64
+	help
+	  8 vCPUs (4 cores) and 8GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_HIGHCPU_16
+	bool "e2-highcpu-16"
+	depends on TARGET_ARCH_X86_64
+	help
+	  16 vCPUs (8 cores) and 16GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_HIGHCPU_32
+	bool "e2-highcpu-32"
+	depends on TARGET_ARCH_X86_64
+	help
+	  32 vCPUs (16 cores) and 32GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "e2-highcpu-2" if TERRAFORM_GCE_MACHINE_E2_HIGHCPU_2
+	default "e2-highcpu-4" if TERRAFORM_GCE_MACHINE_E2_HIGHCPU_4
+	default "e2-highcpu-8" if TERRAFORM_GCE_MACHINE_E2_HIGHCPU_8
+	default "e2-highcpu-16" if TERRAFORM_GCE_MACHINE_E2_HIGHCPU_16
+	default "e2-highcpu-32" if TERRAFORM_GCE_MACHINE_E2_HIGHCPU_32
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "pd-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_E2_HIGHCPU
diff --git a/terraform/gce/kconfigs/machines/Kconfig.e2-standard b/terraform/gce/kconfigs/machines/Kconfig.e2-standard
new file mode 100644
index 000000000000..26180f841d5a
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.e2-standard
@@ -0,0 +1,58 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_E2_STANDARD
+
+choice
+	prompt "GCE Machine type"
+	default TERRAFORM_GCE_MACHINE_E2_STANDARD_8
+	help
+	  The E2 machine series offers both Intel and AMD EPYC
+	  processors, selected for you at the time of VM creation.
+	  Machine types in this series are available in all regions
+	  and zones and support a virtio memory balloon device.
+
+config TERRAFORM_GCE_MACHINE_E2_STANDARD_2
+	bool "e2-standard-2"
+	depends on TARGET_ARCH_X86_64
+	help
+	  2 vCPUs (1 core) and 8GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_STANDARD_4
+	bool "e2-standard-4"
+	depends on TARGET_ARCH_X86_64
+	help
+	  4 vCPUs (2 cores) and 16GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_STANDARD_8
+	bool "e2-standard-8"
+	depends on TARGET_ARCH_X86_64
+	help
+	  8 vCPUs (4 cores) and 32GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_STANDARD_16
+	bool "e2-standard-16"
+	depends on TARGET_ARCH_X86_64
+	help
+	  16 vCPUs (8 cores) and 64GB of memory.
+
+config TERRAFORM_GCE_MACHINE_E2_STANDARD_32
+	bool "e2-standard-32"
+	depends on TARGET_ARCH_X86_64
+	help
+	  32 vCPUs (16 cores) and 128GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "e2-standard-2" if TERRAFORM_GCE_MACHINE_E2_STANDARD_2
+	default "e2-standard-4" if TERRAFORM_GCE_MACHINE_E2_STANDARD_4
+	default "e2-standard-8" if TERRAFORM_GCE_MACHINE_E2_STANDARD_8
+	default "e2-standard-16" if TERRAFORM_GCE_MACHINE_E2_STANDARD_16
+	default "e2-standard-32" if TERRAFORM_GCE_MACHINE_E2_STANDARD_32
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "pd-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_E2_STANDARD
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n1-highcpu b/terraform/gce/kconfigs/machines/Kconfig.n1-highcpu
new file mode 100644
index 000000000000..7684b2961962
--- /dev/null
+++ b/terraform/gce/kconfigs/machines/Kconfig.n1-highcpu
@@ -0,0 +1,55 @@
+if TERRAFORM_GCE_MACHINE_FAMILY_N1_HIGHCPU
+
+choice
+	prompt "GCE Machine type"
+	default TERRAFORM_GCE_MACHINE_N1_HIGHCPU_8
+	help
+	  The N1 machine series is Compute Engine's first generation
+	  general-purpose machine series available on Intel Skylake,
+	  Broadwell, Haswell, Sandy Bridge, and Ivy Bridge CPU
+	  platforms.
+
+config TERRAFORM_GCE_MACHINE_N1_HIGHCPU_2
+	bool "n1-highcpu-2"
+	help
+	  2 vCPUs (1 core) and 1.8GB of memory.
+
+config TERRAFORM_GCE_MACHINE_N1_HIGHCPU_4
+	bool "n1-highcpu-4"
+	help
+	  4 vCPUs (2 cores) and 3.6GB of memory.
+
+config TERRAFORM_GCE_MACHINE_N1_HIGHCPU_8
+	bool "n1-highcpu-8"
+	help
+	  8 vCPUs (4 cores) and 7.2GB of memory.
+
+config TERRAFORM_GCE_MACHINE_N1_HIGHCPU_16
+	bool "n1-highcpu-16"
+	depends on TARGET_ARCH_X86_64
+	help
+	  16 vCPUs (8 cores) and 14.4GB of memory.
+
+config TERRAFORM_GCE_MACHINE_N1_HIGHCPU_32
+	bool "n1-highcpu-32"
+	depends on TARGET_ARCH_X86_64
+	help
+	  32 vCPUs (16 cores) and 28.8GB of memory.
+
+endchoice
+
+config TERRAFORM_GCE_MACHINE_TYPE
+	string
+	output yaml
+	default "n1-highcpu-2" if TERRAFORM_GCE_MACHINE_N1_HIGHCPU_2
+	default "n1-highcpu-4" if TERRAFORM_GCE_MACHINE_N1_HIGHCPU_4
+	default "n1-highcpu-8" if TERRAFORM_GCE_MACHINE_N1_HIGHCPU_8
+	default "n1-highcpu-16" if TERRAFORM_GCE_MACHINE_N1_HIGHCPU_16
+	default "n1-highcpu-32" if TERRAFORM_GCE_MACHINE_N1_HIGHCPU_32
+
+config TERRAFORM_GCE_IMAGE_TYPE
+	string
+	output yaml
+	default "pd-balanced"
+
+endif # TERRAFORM_GCE_MACHINE_FAMILY_N1_HIGHCPU
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n1-standard b/terraform/gce/kconfigs/machines/Kconfig.n1-standard
index 65dbd672893d..7201231c74a0 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n1-standard
+++ b/terraform/gce/kconfigs/machines/Kconfig.n1-standard
@@ -4,12 +4,10 @@ choice
 	prompt "GCE Machine type"
 	default TERRAFORM_GCE_MACHINE_N1_STANDARD_8
 	help
-	  This option sets the GCE machine type. For other machine types,
-	  refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
-
-	  Documentation: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance#machine_type
+	  The N1 machine series is Compute Engine's first generation
+	  general-purpose machine series available on Intel Skylake,
+	  Broadwell, Haswell, Sandy Bridge, and Ivy Bridge CPU
+	  platforms.
 
 config TERRAFORM_GCE_MACHINE_N1_STANDARD_1
 	bool "n1-standard-1"
@@ -19,23 +17,30 @@ config TERRAFORM_GCE_MACHINE_N1_STANDARD_1
 config TERRAFORM_GCE_MACHINE_N1_STANDARD_2
 	bool "n1-standard-2"
 	help
-	  2 vCPU (1 core) and 7.5GB of memory.
+	  2 vCPUs (1 core) and 7.5GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N1_STANDARD_4
 	bool "n1-standard-4"
 	help
-	  4 vCPU (2 cores) and 15GB of memory.
+	  4 vCPUs (2 cores) and 15GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N1_STANDARD_8
 	bool "n1-standard-8"
 	help
-	  8 vCPU (4 cores) and 30GB of memory.
+	  8 vCPUs (4 cores) and 30GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N1_STANDARD_16
 	bool "n1-standard-16"
 	depends on TARGET_ARCH_X86_64
 	help
-	  16 vCPU (8 cores) and 60GB of memory.
+	  16 vCPUs (8 cores) and 60GB of memory.
+
+config TERRAFORM_GCE_MACHINE_N1_STANDARD_32
+	bool "n1-standard-32"
+	depends on TARGET_ARCH_X86_64
+	help
+	  32 vCPUs (16 cores) and 120GB of memory.
+
 endchoice
 
 config TERRAFORM_GCE_MACHINE_TYPE
@@ -46,6 +51,7 @@ config TERRAFORM_GCE_MACHINE_TYPE
 	default "n1-standard-4" if TERRAFORM_GCE_MACHINE_N1_STANDARD_4
 	default "n1-standard-8" if TERRAFORM_GCE_MACHINE_N1_STANDARD_8
 	default "n1-standard-16" if TERRAFORM_GCE_MACHINE_N1_STANDARD_16
+	default "n1-standard-32" if TERRAFORM_GCE_MACHINE_N1_STANDARD_32
 
 config TERRAFORM_GCE_IMAGE_TYPE
 	string
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n2-highcpu b/terraform/gce/kconfigs/machines/Kconfig.n2-highcpu
index 8aa6893c7252..83823753dc93 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n2-highcpu
+++ b/terraform/gce/kconfigs/machines/Kconfig.n2-highcpu
@@ -4,40 +4,38 @@ choice
 	prompt "GCE Machine type"
 	default TERRAFORM_GCE_MACHINE_N2_HIGHCPU_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  The N2 machine series can run on either Intel Cascade Lake
+	  (on smaller configurations) or Intel Ice Lake.
 
 config TERRAFORM_GCE_MACHINE_N2_HIGHCPU_2
 	bool "n2-highcpu-2"
 	depends on TARGET_ARCH_X86_64
 	help
-	  2 vCPU (1 core) and 2GB of memory.
+	  2 vCPUs (1 core) and 2GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_HIGHCPU_4
 	bool "n2-highcpu-4"
 	depends on TARGET_ARCH_X86_64
 	help
-	  4 vCPU (2 core) and 4GB of memory.
+	  4 vCPUs (2 cores) and 4GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_HIGHCPU_8
 	bool "n2-highcpu-8"
 	depends on TARGET_ARCH_X86_64
 	help
-	  8 vCPU (4 cores) and 8GB of memory.
+	  8 vCPUs (4 cores) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_HIGHCPU_16
 	bool "n2-highcpu-16"
 	depends on TARGET_ARCH_X86_64
 	help
-	  16 vCPU (8 cores) and 16GB of memory.
+	  16 vCPUs (8 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_HIGHCPU_32
 	bool "n2-highcpu-32"
 	depends on TARGET_ARCH_X86_64
 	help
-	  32 vCPU (16 cores) and 32GB of memory.
+	  32 vCPUs (16 cores) and 32GB of memory.
 
 endchoice
 
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n2-standard b/terraform/gce/kconfigs/machines/Kconfig.n2-standard
index b7b2d68fa522..a485a754ab44 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n2-standard
+++ b/terraform/gce/kconfigs/machines/Kconfig.n2-standard
@@ -4,40 +4,38 @@ choice
 	prompt "GCE Machine type"
 	default TERRAFORM_GCE_MACHINE_N2_STANDARD_8
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  The N2 machine series can run on either Intel Cascade Lake
+	  (on smaller configurations) or Intel Ice Lake.
 
 config TERRAFORM_GCE_MACHINE_N2_STANDARD_2
 	bool "n2-standard-2"
 	depends on TARGET_ARCH_X86_64
 	help
-	  2 vCPU (1 core) and 8GB of memory.
+	  2 vCPUs (1 core) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_STANDARD_4
 	bool "n2-standard-4"
 	depends on TARGET_ARCH_X86_64
 	help
-	  4 vCPU (2 core) and 16GB of memory.
+	  4 vCPUs (2 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_STANDARD_8
 	bool "n2-standard-8"
 	depends on TARGET_ARCH_X86_64
 	help
-	  8 vCPU (4 cores) and 32GB of memory.
+	  8 vCPUs (4 cores) and 32GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_STANDARD_16
 	bool "n2-standard-16"
 	depends on TARGET_ARCH_X86_64
 	help
-	  16 vCPU (8 cores) and 64GB of memory.
+	  16 vCPUs (8 cores) and 64GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2_STANDARD_32
 	bool "n2-standard-32"
 	depends on TARGET_ARCH_X86_64
 	help
-	  32 vCPU (16 cores) and 128GB of memory.
+	  32 vCPUs (16 cores) and 128GB of memory.
 
 endchoice
 
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n2d-highcpu b/terraform/gce/kconfigs/machines/Kconfig.n2d-highcpu
index 3050349416fd..e3cf9c6e5c41 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n2d-highcpu
+++ b/terraform/gce/kconfigs/machines/Kconfig.n2d-highcpu
@@ -4,10 +4,9 @@ choice
 	prompt "GCE Machine type"
 	default TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  The N2D machine series can run on either AMD EPYC Milan or
+	  AMD EPYC Rome processors. The third generation AMD EPYC
+	  Milan processor is available only in specific regions and zones.
 
 config TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_2
 	bool "n2d-highcpu-2"
@@ -19,25 +18,25 @@ config TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_4
 	bool "n2d-highcpu-4"
 	depends on TARGET_ARCH_X86_64
 	help
-	  4 vCPU (2 core) and 4GB of memory.
+	  4 vCPUs (2 cores) and 4GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_8
 	bool "n2d-highcpu-8"
 	depends on TARGET_ARCH_X86_64
 	help
-	  8 vCPU (4 cores) and 8GB of memory.
+	  8 vCPUs (4 cores) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_16
 	bool "n2d-highcpu-16"
 	depends on TARGET_ARCH_X86_64
 	help
-	  16 vCPU (8 cores) and 16GB of memory.
+	  16 vCPUs (8 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_HIGHCPU_32
 	bool "n2d-highcpu-32"
 	depends on TARGET_ARCH_X86_64
 	help
-	  32 vCPU (16 cores) and 32GB of memory.
+	  32 vCPUs (16 cores) and 32GB of memory.
 
 endchoice
 
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n2d-standard b/terraform/gce/kconfigs/machines/Kconfig.n2d-standard
index 08744d396595..4f86e0b7d3cf 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n2d-standard
+++ b/terraform/gce/kconfigs/machines/Kconfig.n2d-standard
@@ -4,40 +4,39 @@ choice
 	prompt "GCE Machine type"
 	default TERRAFORM_GCE_MACHINE_N2D_STANDARD_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  The N2D machine series can run on either AMD EPYC Milan or
+	  AMD EPYC Rome processors. The third generation AMD EPYC
+	  Milan processor is available only in specific regions and zones.
 
 config TERRAFORM_GCE_MACHINE_N2D_STANDARD_2
 	bool "n2d-standard-2"
 	depends on TARGET_ARCH_X86_64
 	help
-	  2 vCPU (1 core) and 8GB of memory.
+	  2 vCPUs (1 core) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_STANDARD_4
 	bool "n2d-standard-4"
 	depends on TARGET_ARCH_X86_64
 	help
-	  4 vCPU (2 core) and 16GB of memory.
+	  4 vCPUs (2 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_STANDARD_8
 	bool "n2d-standard-8"
 	depends on TARGET_ARCH_X86_64
 	help
-	  8 vCPU (4 cores) and 32GB of memory.
+	  8 vCPUs (4 cores) and 32GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_STANDARD_16
 	bool "n2d-standard-16"
 	depends on TARGET_ARCH_X86_64
 	help
-	  16 vCPU (8 cores) and 64GB of memory.
+	  16 vCPUs (8 cores) and 64GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N2D_STANDARD_32
 	bool "n2d-standard-32"
 	depends on TARGET_ARCH_X86_64
 	help
-	  32 vCPU (16 cores) and 128GB of memory.
+	  32 vCPUs (16 cores) and 128GB of memory.
 
 endchoice
 
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n4-highcpu b/terraform/gce/kconfigs/machines/Kconfig.n4-highcpu
index 9f871940a73b..880a386ea01d 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n4-highcpu
+++ b/terraform/gce/kconfigs/machines/Kconfig.n4-highcpu
@@ -4,35 +4,33 @@ choice
 	prompt "GCE Machine Type"
 	default TERRAFORM_GCE_MACHINE_N4_HIGHCPU_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  N4 VMs are powered by the 5th generation Intel Xeon Scalable
+	  processors (code-named Emerald Rapids) and Titanium.
 
 config TERRAFORM_GCE_MACHINE_N4_HIGHCPU_2
 	bool "n4-highcpu-2"
 	help
-	  2 vCPU (1 core) and 4GB of memory.
+	  2 vCPUs (1 core) and 4GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_HIGHCPU_4
 	bool "n4-highcpu-4"
 	help
-	  4 vCPU (2 core) and 8GB of memory.
+	  4 vCPUs (2 cores) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_HIGHCPU_8
 	bool "n4-highcpu-8"
 	help
-	  8 vCPU (4 cores) and 16GB of memory.
+	  8 vCPUs (4 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_HIGHCPU_16
 	bool "n4-highcpu-16"
 	help
-	  16 vCPU (8 cores) and 32GB of memory.
+	  16 vCPUs (8 cores) and 32GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_HIGHCPU_32
 	bool "n4-highcpu-32"
 	help
-	  32 vCPU (16 cores) and 64GB of memory.
+	  32 vCPUs (16 cores) and 64GB of memory.
 
 endchoice
 
diff --git a/terraform/gce/kconfigs/machines/Kconfig.n4-standard b/terraform/gce/kconfigs/machines/Kconfig.n4-standard
index d1d1202cbb7b..807f8638e865 100644
--- a/terraform/gce/kconfigs/machines/Kconfig.n4-standard
+++ b/terraform/gce/kconfigs/machines/Kconfig.n4-standard
@@ -4,35 +4,33 @@ choice
 	prompt "GCE Machine Type"
 	default TERRAFORM_GCE_MACHINE_N4_STANDARD_4
 	help
-	  This option sets the GCE machine type. For other machine
-	  types, refer to this page:
-
-	  https://cloud.google.com/compute/docs/machine-types
+	  N4 VMs are powered by the 5th generation Intel Xeon Scalable
+	  processors (code-named Emerald Rapids) and Titanium.
 
 config TERRAFORM_GCE_MACHINE_N4_STANDARD_2
 	bool "n4-standard-2"
 	help
-	  2 vCPU (1 core) and 8GB of memory.
+	  2 vCPUs (1 core) and 8GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_STANDARD_4
 	bool "n4-standard-4"
 	help
-	  4 vCPU (2 core) and 16GB of memory.
+	  4 vCPUs (2 cores) and 16GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_STANDARD_8
 	bool "n4-standard-8"
 	help
-	  8 vCPU (4 cores) and 32GB of memory.
+	  8 vCPUs (4 cores) and 32GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_STANDARD_16
 	bool "n4-standard-16"
 	help
-	  16 vCPU (8 cores) and 64GB of memory.
+	  16 vCPUs (8 cores) and 64GB of memory.
 
 config TERRAFORM_GCE_MACHINE_N4_STANDARD_32
 	bool "n4-standard-32"
 	help
-	  32 vCPU (16 cores) and 128GB of memory.
+	  32 vCPUs (16 cores) and 128GB of memory.
 
 endchoice
 
-- 
2.51.0

[PATCH v2 18/37] terraform: Create the user's .ssh/config if it does not already exist

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

While setting up some new test runners, I encountered this failure:

TASK [terraform : Ensure the Include directive is present on the controller] ***
task path: /usr/local/home/queue-5-4/worker/queue-5-4-kernel/build/playbooks/roles/terraform/tasks/main.yml:31
fatal: [localhost]: FAILED! => {
    "changed": false,
    "rc": 257
}
MSG:
Path /usr/local/home/queue-5-4/.ssh/config does not exist !

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/terraform/tasks/main.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/playbooks/roles/terraform/tasks/main.yml b/playbooks/roles/terraform/tasks/main.yml
index 91c424c4ab6b..dabc9384e470 100644
--- a/playbooks/roles/terraform/tasks/main.yml
+++ b/playbooks/roles/terraform/tasks/main.yml
@@ -118,6 +118,7 @@
     path: "{{ sshconfig }}"
     insertbefore: BOF
     append_newline: true
+    create: true
     marker: "# {mark} Managed by kdevops"
     mode: "u=rw,g=r,o=r"
     block: "Include {{ kdevops_ssh_config_prefix }}*"
-- 
2.51.0

[PATCH v2 19/37] pynfs: Don't install xdrlib3 on older RHEL

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

When running the pynfs workflow on RHEL 8:

> :stderr:   Could not find a version that satisfies the requirement
> xdrlib3 (from versions: )
> No matching distribution found for xdrlib3

RHEL 8 sports a version of Python that neither requires or supports
the use of "pip install xdrlib3".

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
index 5139825ea872..6046c84ac4c4 100644
--- a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
@@ -27,3 +27,5 @@
 - name: Install xdrlib3
   ansible.builtin.pip:
     name: xdrlib3
+  when:
+    - ansible_distribution == "Fedora" or ansible_distribution_major_version | int > 8
-- 
2.51.0

[PATCH v2 20/37] guestfs: Update the guestfs playbook description

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Match the description for the terraform playbook

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/guestfs.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/playbooks/guestfs.yml b/playbooks/guestfs.yml
index 784775cdb818..62334463f05e 100644
--- a/playbooks/guestfs.yml
+++ b/playbooks/guestfs.yml
@@ -1,5 +1,5 @@
 ---
-- name: Provision target nodes with libvirt/guestfs
+- name: Manage infrastructure lifecycle and SSH access with libvirt/guestfs
   gather_facts: true
   connection: local
   hosts: all
-- 
2.51.0

[PATCH v2 21/37] gen_hosts: Add 'localhost' to the [all] group

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Commit 58a0745e8fa2 ("Add a guest/instance for building the test
kernel") was applied just after commit 1cf0800c9ffc ("gen_hosts:
templates: include localhost in the all group"). The former should
have been updated to include localhost in its [all] group.

Without 'localhost', trying to provision the kernel builder node
fails.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/gen_hosts/templates/builder.j2 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/playbooks/roles/gen_hosts/templates/builder.j2 b/playbooks/roles/gen_hosts/templates/builder.j2
index 0c9ba1e8e01a..eb8c736409b9 100644
--- a/playbooks/roles/gen_hosts/templates/builder.j2
+++ b/playbooks/roles/gen_hosts/templates/builder.j2
@@ -1,4 +1,5 @@
 [all]
+localhost ansible_connection=local
 {{ kdevops_hosts_prefix }}-builder
 [all:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-- 
2.51.0

[PATCH v2 22/37] gen_hosts: Add [service] group to the builder host template

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Ensure that the [service] group is defined in the builder
node's host template, to quash some noisy warnings during
bringup.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/gen_hosts/templates/builder.j2 | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/playbooks/roles/gen_hosts/templates/builder.j2 b/playbooks/roles/gen_hosts/templates/builder.j2
index eb8c736409b9..164201dc5ca4 100644
--- a/playbooks/roles/gen_hosts/templates/builder.j2
+++ b/playbooks/roles/gen_hosts/templates/builder.j2
@@ -12,3 +12,7 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 [dev]
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
-- 
2.51.0

[PATCH v2 23/37] guestfs: Don't run the 'destroy' tags on the controller

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

"make destroy" was skipping some important file removals, and
removing other files that do not exist.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/guestfs.Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
index f6c350a43f27..953c61493a88 100644
--- a/scripts/guestfs.Makefile
+++ b/scripts/guestfs.Makefile
@@ -96,6 +96,7 @@ PHONY += status_guestfs
 
 destroy_guestfs:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		--limit 'baseline:dev:service' \
 		playbooks/guestfs.yml \
 		--extra-vars=@./extra_vars.yaml \
 		--tags destroy
-- 
2.51.0

[PATCH v2 24/37] base_image: Skip "locales-all" package on Fedora

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/base_image/templates/virt-builder.j2 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/playbooks/roles/base_image/templates/virt-builder.j2 b/playbooks/roles/base_image/templates/virt-builder.j2
index 7df4229405e7..138d06eede6b 100644
--- a/playbooks/roles/base_image/templates/virt-builder.j2
+++ b/playbooks/roles/base_image/templates/virt-builder.j2
@@ -11,7 +11,7 @@ mkdir {{ target_dir }}
 copy-in {{ guestfs_distro_source_and_dest_file }}:{{ target_dir }}
 {% endif %}
 
-install sudo,qemu-guest-agent,python3,bash,locales-all
+install sudo,qemu-guest-agent,python3,bash
 run-command useradd {{ kdevops_uid }} -s /bin/bash -m kdevops
 append-line /etc/sudoers.d/kdevops:kdevops   ALL=(ALL)       NOPASSWD: ALL
 edit /etc/default/grub:s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0"/
-- 
2.51.0

[PATCH v2 25/37] update_etc_hosts: ipaddr

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

ansible.netcommon.ipaddr is deprecated. See:

 https://docs.ansible.com/ansible/latest/collections/ansible/netcommon/ipaddr_filter.html

Ensure the update_etc_hosts playbook uses the ansible.utils.ipaddr
plugin instead.

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/update_etc_hosts/tasks/main.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/playbooks/roles/update_etc_hosts/tasks/main.yml b/playbooks/roles/update_etc_hosts/tasks/main.yml
index dc40ededd539..17c85dc539b5 100644
--- a/playbooks/roles/update_etc_hosts/tasks/main.yml
+++ b/playbooks/roles/update_etc_hosts/tasks/main.yml
@@ -51,8 +51,8 @@
   become_method: sudo
   ansible.builtin.lineinfile:
     dest: /etc/hosts
-    regexp: ".*{{ item }}$"
-    line: "{{ hostvars[item].ansible_all_ipv4_addresses | ipaddr(private_network) | first }} {{ item }}"
+    regexp: '.*{{ item }}$'
+    line: "{{ hostvars[item].ansible_all_ipv4_addresses | ansible.utils.ipaddr(private_network) | first }} {{ item }}"
     state: present
   with_items: "{{ ueh_hosts }}"
   when:
-- 
2.51.0

[PATCH v2 26/37] Add an Ansible requirements.yml file

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
module") introduced the use of the cloud.terraform module, and
commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
with an Ansible playbook") introduced the use of the
community.libvirt module. It would be friendly if kdevops could
pull in the Ansible modules it needs transparently.

The requirements.yml file is a manifest of Ansible collections that
the project needs to run. Installation of these collections is made
automatic by adding:

  ansible-galaxy install -r requirements.yml

to the "make ansible_cfg" step. This mechanism can keep cached
versions of collections up to date, and can also constrain a
cached collection to a specific version, if that's needed.

The initial file contains requirements I could find easily, and
should be updated over time as new collection dependencies are
introduced.

See also:

https://docs.ansible.com/ansible/latest/user_guide/collections_using.html

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 requirements.yml         | 8 ++++++++
 scripts/ansible.Makefile | 5 +++++
 2 files changed, 13 insertions(+)
 create mode 100644 requirements.yml

diff --git a/requirements.yml b/requirements.yml
new file mode 100644
index 000000000000..a22691d3f86b
--- /dev/null
+++ b/requirements.yml
@@ -0,0 +1,8 @@
+---
+collections:
+  - name: ansible.posix
+  - name: ansible.utils
+  - name: cloud.terraform
+  - name: community.docker
+  - name: community.general
+  - name: community.libvirt
diff --git a/scripts/ansible.Makefile b/scripts/ansible.Makefile
index b0d2a8c12ee7..4efcf3e93b0e 100644
--- a/scripts/ansible.Makefile
+++ b/scripts/ansible.Makefile
@@ -2,3 +2,8 @@
 
 AV ?= 0
 export ANSIBLE_VERBOSE := $(shell scripts/validate_av.py --av "$(AV)")
+
+ansible-requirements:
+	@$(Q)ansible-galaxy install -r requirements.yml
+PHONY += ansible-requirements
+DEFAULT_DEPS += ansible-requirements
-- 
2.51.0

[PATCH v2 27/37] terraform/oci: Clean up white space in route table resource

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 terraform/oci/main.tf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/terraform/oci/main.tf b/terraform/oci/main.tf
index 45a4b4bc0c63..15660aa02614 100644
--- a/terraform/oci/main.tf
+++ b/terraform/oci/main.tf
@@ -103,10 +103,12 @@ resource "oci_core_route_table" "kdevops_route_table" {
   compartment_id = data.oci_identity_compartments.kdevops_compartment.compartments[0].id
   display_name   = "kdevops route table"
   vcn_id         = one(oci_core_vcn.kdevops_vcn[*].id)
+
   route_rules {
     destination       = "0.0.0.0/0"
     destination_type  = "CIDR_BLOCK"
     network_entity_id = one(oci_core_internet_gateway.kdevops_internet_gateway[*].id)
+    route_type        = "STATIC"
   }
 }
 
-- 
2.51.0

[PATCH v2 28/37] guestfs: Correct the spelling of "drives"

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/gen_nodes/templates/drives.j2 | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/playbooks/roles/gen_nodes/templates/drives.j2 b/playbooks/roles/gen_nodes/templates/drives.j2
index 1241b88a9379..472b4b27067f 100644
--- a/playbooks/roles/gen_nodes/templates/drives.j2
+++ b/playbooks/roles/gen_nodes/templates/drives.j2
@@ -8,7 +8,7 @@ the drives can vary by type, so we have one macro by type of drive.
 			libvirt_extra_drive_format,
 			libvirt_extra_storage_aio_mode,
 			libvirt_extra_storage_aio_cache_mode) -%}
-<!-- We generated {{ num_drives }} ide dives -->
+<!-- We generated {{ num_drives }} ide drives -->
 {% for n in range(0,num_drives) %}
     <!-- This is IDE drive # {{ n + 1 }} -->
     <qemu:arg value='-drive'/>
@@ -26,7 +26,7 @@ the drives can vary by type, so we have one macro by type of drive.
 			   libvirt_extra_storage_aio_cache_mode,
 			   libvirt_extra_storage_virtio_logical_block_size,
 			   libvirt_extra_storage_virtio_physical_block_size) -%}
-<!-- We generated {{ num_drives }} virtio dives -->
+<!-- We generated {{ num_drives }} virtio drives -->
 {% for n in range(0,num_drives) %}
     <!-- This is virtio drive # {{ n + 1 }} -->
     <qemu:arg value='-device'/>
@@ -46,7 +46,7 @@ the drives can vary by type, so we have one macro by type of drive.
                          libvirt_extra_drive_format,
                          libvirt_extra_storage_aio_mode,
                          libvirt_extra_storage_aio_cache_mode) -%}
-<!-- We generated {{ num_drives }} scsi dives -->
+<!-- We generated {{ num_drives }} scsi drives -->
 {% for n in range(0,num_drives) %}
     <!-- This is scsi drive # {{ n + 1 }} -->
     <qemu:arg value='-device'/>
@@ -103,7 +103,7 @@ the drives can vary by type, so we have one macro by type of drive.
 			 libvirt_extra_storage_aio_mode,
 			 libvirt_extra_storage_aio_cache_mode,
 			 libvirt_extra_storage_nvme_logical_block_size) -%}
-<!-- We generated {{ num_drives }} NVMe dives -->
+<!-- We generated {{ num_drives }} NVMe drives -->
 {% for n in range(0,num_drives) %}
     <!-- This is NVMe drive # {{ n + 1 }} -->
     <qemu:arg value='-device'/>
-- 
2.51.0

[PATCH v2 29/37] guestfs: Configure how many extra drives to provision

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Some workflows create several guests but use only one or two extra
storage drives per guest. Enable configurations that reduce the
amount of storage allocated per guest to conserve local persistent
storage space on the host.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 kconfigs/Kconfig.libvirt                      | 45 +++++++++++++++++++
 .../roles/gen_nodes/templates/gen_drives.j2   |  8 ++--
 .../roles/guestfs/tasks/bringup/main.yml      |  2 +-
 3 files changed, 50 insertions(+), 5 deletions(-)

diff --git a/kconfigs/Kconfig.libvirt b/kconfigs/Kconfig.libvirt
index 7fe231668229..961276ccc1b1 100644
--- a/kconfigs/Kconfig.libvirt
+++ b/kconfigs/Kconfig.libvirt
@@ -556,6 +556,51 @@ config LIBVIRT_HOST_PASSTHROUGH
 	      --pre 'make -s mrproper defconfig' \
 	    \-- make -s -j$(nproc) bzImage
 
+choice
+	prompt "Libvirt extra storage drive count"
+	default LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
+	help
+	  This option selects the number of extra storage drives to
+	  provision for each libvirt guest. Reduce this number to
+	  conserve local host storage capacity, if your workflows do
+	  not use all the extra drives.
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
+	bool "0"
+	help
+	  Provision no extra storage drives per guest.
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
+	bool "1"
+	help
+	  Provision one extra storage drive per guest.
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
+	bool "2"
+	help
+	  Provision two extra storage drives per guest.
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
+	bool "3"
+	help
+	  Provision three extra storage drives per guest.
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
+	bool "4"
+	help
+	  Provision four extra storages drive per guest.
+
+endchoice
+
+config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT
+	int
+	output yaml
+	default 0 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
+	default 1 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
+	default 2 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
+	default 3 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
+	default 4 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
+
 choice
 	prompt "Libvirt extra storage driver to use"
 	default LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO
diff --git a/playbooks/roles/gen_nodes/templates/gen_drives.j2 b/playbooks/roles/gen_nodes/templates/gen_drives.j2
index 2de13da4ab8e..1040c0e19d84 100644
--- a/playbooks/roles/gen_nodes/templates/gen_drives.j2
+++ b/playbooks/roles/gen_nodes/templates/gen_drives.j2
@@ -1,6 +1,6 @@
 {% import './templates/drives.j2' as drives %}
 {% if libvirt_extra_storage_drive_ide %}
-{{ drives.gen_drive_ide(4,
+{{ drives.gen_drive_ide(libvirt_extra_storage_drive_count,
 			kdevops_storage_pool_path,
 			hostname,
 			libvirt_extra_drive_format,
@@ -18,7 +18,7 @@
 				    libvirt_extra_storage_aio_cache_mode,
 				    kdevops_storage_pool_path) }}
 {% else %}
-{{ drives.gen_drive_virtio(4,
+{{ drives.gen_drive_virtio(libvirt_extra_storage_drive_count,
                            kdevops_storage_pool_path,
                            hostname,
                            libvirt_extra_drive_format,
@@ -28,7 +28,7 @@
                            libvirt_extra_storage_virtio_physical_block_size) }}
 {% endif %}
 {% elif libvirt_extra_storage_drive_scsi %}
-{{ drives.gen_drive_scsi(4,
+{{ drives.gen_drive_scsi(libvirt_extra_storage_drive_count,
                          kdevops_storage_pool_path,
                          hostname,
                          libvirt_extra_drive_format,
@@ -46,7 +46,7 @@
 				  libvirt_extra_storage_aio_cache_mode,
 				  kdevops_storage_pool_path) }}
 {% else %}
-{{ drives.gen_drive_nvme(4,
+{{ drives.gen_drive_nvme(libvirt_extra_storage_drive_count,
 			 kdevops_storage_pool_path,
 			 hostname,
 			 libvirt_extra_drive_format,
diff --git a/playbooks/roles/guestfs/tasks/bringup/main.yml b/playbooks/roles/guestfs/tasks/bringup/main.yml
index bd9f52603a3b..3fd677ff4659 100644
--- a/playbooks/roles/guestfs/tasks/bringup/main.yml
+++ b/playbooks/roles/guestfs/tasks/bringup/main.yml
@@ -102,7 +102,7 @@
         path: "{{ storagedir }}/{{ inventory_hostname }}/extra{{ item }}.{{ libvirt_extra_drive_format }}"
       ansible.builtin.include_tasks:
         file: "{{ role_path }}/tasks/bringup/extra-disks.yml"
-      loop: "{{ range(0, 4) | list }}"
+      loop: "{{ range(0, libvirt_extra_storage_drive_count) | list }}"
       when:
         - not libvirt_enable_largeio|bool
 
-- 
2.51.0

[PATCH v2 30/37] ltp: Replace /opt with a symlink to data partition

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

The size of some cloud OS images is too small for the ltp
installation (eg, the RHEL 8 OS image on Azure is about 10GB). This
causes the ltp workflow to fail 100% of the time during the test
build step.

A simple solution is to replace /opt with a symlink to /data/opt, as
kdevops can make /data large enough to hold the install.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/roles/ltp/tasks/main.yml | 45 +++++++++++++++++++++++++++---
 1 file changed, 41 insertions(+), 4 deletions(-)

diff --git a/playbooks/roles/ltp/tasks/main.yml b/playbooks/roles/ltp/tasks/main.yml
index 830ec30515f0..1b966771f757 100644
--- a/playbooks/roles/ltp/tasks/main.yml
+++ b/playbooks/roles/ltp/tasks/main.yml
@@ -162,15 +162,52 @@
     target: all
     jobs: "{{ ansible_processor_nproc }}"
 
-- name: Ensure /opt has correct permissions
-  tags: ["ltp"]
+- name: Set the pathname of the install directory
+  tags: ["build", "ltp"]
+  ansible.builtin.set_fact:
+    ltp_install_dir: "{{ data_path }}/opt"
+
+- name: Remove existing ltp install directory
+  tags: ["build", "ltp"]
   become: true
   become_flags: "su - -c"
   become_method: ansible.builtin.sudo
   ansible.builtin.file:
-    state: directory
-    path: "/opt"
+    path: "{{ ltp_install_dir }}"
+    state: absent
+
+- name: Create a fresh ltp install directory
+  tags: ["build", "ltp"]
+  become: true
+  become_flags: "su - -c"
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    path: "{{ ltp_install_dir }}"
+    owner: "root"
+    group: "root"
     mode: "u=rwx,g=rwx,o=rwxt"
+    state: directory
+
+- name: Remove /opt
+  tags: ["build", "ltp"]
+  become: true
+  become_flags: "su - -c"
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    path: "/opt"
+    state: absent
+
+- name: Replace /opt with a symlink
+  tags: ["build", "ltp"]
+  become: true
+  become_flags: "su - -c"
+  become_method: ansible.builtin.sudo
+  ansible.builtin.file:
+    src: "{{ ltp_install_dir }}"
+    dest: "/opt"
+    owner: "root"
+    group: "root"
+    state: link
 
 - name: Install ltp on the test systems
   tags: ["ltp"]
-- 
2.51.0

[PATCH v2 31/37] terraform: Squelch Ansible complaints about missing inventory file

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

During "make", I see this warning twice when preparing a terraform-
based kdevops run:

[WARNING]: provided hosts list is empty, only localhost is available.
   Note that the implicit localhost does not match 'all'

The install_terraform and gen_tfvars playbooks both run before the
inventory file is created. AIUI they have to be invoked with an
explicit inventory on the command line.

Fixes: 0987c30034c9 ("Makefile: use inventory from ansible.cfg")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile.kdevops                | 1 +
 playbooks/gen_tfvars.yml        | 1 +
 playbooks/install_terraform.yml | 1 +
 scripts/terraform.Makefile      | 1 +
 4 files changed, 4 insertions(+)

diff --git a/Makefile.kdevops b/Makefile.kdevops
index 9c6109f3a114..fd03b392f1d9 100644
--- a/Makefile.kdevops
+++ b/Makefile.kdevops
@@ -9,6 +9,7 @@ PHONY := kdevops_all
 
 kdevops_terraform_deps:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		--inventory localhost, \
 		$(KDEVOPS_PLAYBOOKS_DIR)/install_terraform.yml
 PHONY += kdevops_terraform_deps
 
diff --git a/playbooks/gen_tfvars.yml b/playbooks/gen_tfvars.yml
index 9d75c86810b0..11437fa252d8 100644
--- a/playbooks/gen_tfvars.yml
+++ b/playbooks/gen_tfvars.yml
@@ -1,5 +1,6 @@
 ---
 - name: Generate Terraform variables file (terraform/terraform.tfvars)
+  connection: local
   hosts: localhost
   roles:
     - role: gen_tfvars
diff --git a/playbooks/install_terraform.yml b/playbooks/install_terraform.yml
index 5a5fa724a212..3342d32a6269 100644
--- a/playbooks/install_terraform.yml
+++ b/playbooks/install_terraform.yml
@@ -1,5 +1,6 @@
 ---
 - name: Install Terraform
+  connection: local
   hosts: localhost
   roles:
     - role: install_terraform
diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index d1411a1868f1..305b287a5e6c 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -232,5 +232,6 @@ destroy_terraform_base:
 
 $(KDEVOPS_TFVARS): $(KDEVOPS_TFVARS_TEMPLATE) .config
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+		--inventory localhost, \
 		$(KDEVOPS_PLAYBOOKS_DIR)/gen_tfvars.yml \
 		--extra-vars=@./extra_vars.yaml
-- 
2.51.0

[PATCH v2 32/37] terraform: Skip localhost when waiting for targets to reboot

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Eliminate this warning:

  [WARNING]: Reset is not implemented for this connection

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 scripts/terraform.Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile
index 305b287a5e6c..4d738c82120b 100644
--- a/scripts/terraform.Makefile
+++ b/scripts/terraform.Makefile
@@ -212,7 +212,9 @@ $(KDEVOPS_PROVISIONED_SSH):
 		--inventory localhost, \
 		playbooks/terraform.yml --tags ssh \
 		--extra-vars=@./extra_vars.yaml
-	$(Q)ansible $(ANSIBLE_VERBOSE) -i hosts all -m wait_for_connection
+	$(Q)ansible $(ANSIBLE_VERBOSE) \
+		-i hosts baseline:dev:service \
+		-m wait_for_connection
 	$(Q)touch $(KDEVOPS_PROVISIONED_SSH)
 
 status_terraform:
-- 
2.51.0

[PATCH v2 33/37] devconfig: include the "service" group in the playbooks hosts list

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Since commit 1cf0800c9ffc ("gen_hosts: templates: include localhost
in the all group"), the devconfig playbook is no longer updating the
nfsd service host.

1. Actually all the service hosts need to be updated by devconfig,
   so ensure devconfig runs against the service group.

2. Since devconfig can run against most workflows, add blank
   [service] groups to any inventory template that doesn't already
   have it.

Fixes: 1cf0800c9ffc ("gen_hosts: templates: include localhost in the all group")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 playbooks/devconfig.yml                          | 2 +-
 playbooks/roles/gen_hosts/templates/blktests.j2  | 3 +++
 playbooks/roles/gen_hosts/templates/cxl.j2       | 3 +++
 playbooks/roles/gen_hosts/templates/ltp.j2       | 4 ++++
 playbooks/roles/gen_hosts/templates/mmtests.j2   | 3 +++
 playbooks/roles/gen_hosts/templates/selftests.j2 | 3 +++
 playbooks/roles/gen_hosts/templates/sysbench.j2  | 3 +++
 scripts/devconfig.Makefile                       | 9 ++++-----
 scripts/firstconfig.Makefile                     | 2 +-
 scripts/provision.Makefile                       | 2 +-
 10 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/playbooks/devconfig.yml b/playbooks/devconfig.yml
index cebaa87a7228..f0ca7d767efe 100644
--- a/playbooks/devconfig.yml
+++ b/playbooks/devconfig.yml
@@ -1,6 +1,6 @@
 ---
 - name: Configure developer environment on target systems
-  hosts: baseline:dev
+  hosts: baseline:dev:service
   gather_facts: false
   roles:
     - role: devconfig
diff --git a/playbooks/roles/gen_hosts/templates/blktests.j2 b/playbooks/roles/gen_hosts/templates/blktests.j2
index 5bf50acbddac..6aa434c3383c 100644
--- a/playbooks/roles/gen_hosts/templates/blktests.j2
+++ b/playbooks/roles/gen_hosts/templates/blktests.j2
@@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endif %}
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/cxl.j2 b/playbooks/roles/gen_hosts/templates/cxl.j2
index 64bec4561b6c..cab7da246b11 100644
--- a/playbooks/roles/gen_hosts/templates/cxl.j2
+++ b/playbooks/roles/gen_hosts/templates/cxl.j2
@@ -16,3 +16,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endif %}
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/ltp.j2 b/playbooks/roles/gen_hosts/templates/ltp.j2
index 828fd6362718..6946b3f3593b 100644
--- a/playbooks/roles/gen_hosts/templates/ltp.j2
+++ b/playbooks/roles/gen_hosts/templates/ltp.j2
@@ -24,3 +24,7 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endif %}
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/mmtests.j2 b/playbooks/roles/gen_hosts/templates/mmtests.j2
index 1252fe876e6c..8df4a3c31213 100644
--- a/playbooks/roles/gen_hosts/templates/mmtests.j2
+++ b/playbooks/roles/gen_hosts/templates/mmtests.j2
@@ -31,3 +31,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endfor %}
 [mmtests:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/selftests.j2 b/playbooks/roles/gen_hosts/templates/selftests.j2
index e505bbf194b4..af0d6b341048 100644
--- a/playbooks/roles/gen_hosts/templates/selftests.j2
+++ b/playbooks/roles/gen_hosts/templates/selftests.j2
@@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endif %}
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/playbooks/roles/gen_hosts/templates/sysbench.j2 b/playbooks/roles/gen_hosts/templates/sysbench.j2
index 107eebc5fbce..e30773c1b84a 100644
--- a/playbooks/roles/gen_hosts/templates/sysbench.j2
+++ b/playbooks/roles/gen_hosts/templates/sysbench.j2
@@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
 {% endif %}
 [dev:vars]
 ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
+[service]
+[service:vars]
+ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
diff --git a/scripts/devconfig.Makefile b/scripts/devconfig.Makefile
index 146c70f78897..fedf1f2ca21f 100644
--- a/scripts/devconfig.Makefile
+++ b/scripts/devconfig.Makefile
@@ -57,11 +57,10 @@ extend-extra-args-devconfig:
 PHONY += devconfig
 devconfig: $(KDEVOPS_NODES)
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
-		--limit 'baseline:dev:nfsd' \
+		--limit 'baseline:dev:service' \
 		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
 		--extra-vars="$(BOOTLINUX_ARGS)" \
-		--extra-vars '{ kdevops_cli_install: True }' \
-		$(LIMIT_HOSTS)
+		--extra-vars '{ kdevops_cli_install: True }'
 
 devconfig-generic-help-menu:
 	@echo "devconfig          - Ensures generic system setup and is up to date"
@@ -72,9 +71,9 @@ ifeq (y,$(CONFIG_SYSCTL_TUNING))
 PHONY += sysctl-tunings
 sysctl-tunings: $(KDEVOPS_NODES)
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
-		--limit 'baseline:dev:nfsd' \
+		--limit 'baseline:dev:service' \
 		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
-		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS) --tags vars,sysctl
+		--extra-vars="$(BOOTLINUX_ARGS)" --tags vars,sysctl
 
 devconfig-help-menu:
 	@echo "Target node configuration options"
diff --git a/scripts/firstconfig.Makefile b/scripts/firstconfig.Makefile
index 9f8a63a8ff84..5651a07d0b66 100644
--- a/scripts/firstconfig.Makefile
+++ b/scripts/firstconfig.Makefile
@@ -2,7 +2,7 @@
 
 firstconfig:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
-		--limit 'baseline:dev' \
+		--limit 'baseline:dev:service' \
 		--extra-vars '{ kdevops_cli_install: True }' \
 		--tags vars_simple,firstconfig \
 		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml
diff --git a/scripts/provision.Makefile b/scripts/provision.Makefile
index f04264f6b9ea..62a5692904c9 100644
--- a/scripts/provision.Makefile
+++ b/scripts/provision.Makefile
@@ -70,7 +70,7 @@ KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG)
 $(KDEVOPS_PROVISIONED_DEVCONFIG):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) \
-			--limit 'baseline:dev' \
+			--limit 'baseline:dev:service' \
 			$(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
 	fi
 	$(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG)
-- 
2.51.0

[PATCH v2 34/37] Makefile: Remove CONFIG_TOPDIR_PATH_HAS_SHA256SUM

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Clean up: Sufficient time has passed that this compatibility setting
is no longer necessary.

Suggested-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Kconfig                              |  5 -----
 scripts/ssh.Makefile                 |  7 +------
 scripts/update_ssh_config_guestfs.py | 11 +++--------
 3 files changed, 4 insertions(+), 19 deletions(-)

diff --git a/Kconfig b/Kconfig
index 3f2bc8cf019a..54921617b8b2 100644
--- a/Kconfig
+++ b/Kconfig
@@ -10,11 +10,6 @@ config TOPDIR_PATH
 	output yaml
 	default $(shell, echo $TOPDIR_PATH)
 
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
 config TOPDIR_PATH_SHA256SUM
 	string
 	output yaml
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index d18a03e094ee..68c6dcdd4dcd 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -10,12 +10,7 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
 
-ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
-SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
-else
-SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
-endif
-ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(shell scripts/append-makefile-vars.sh $(CONFIG_KDEVOPS_SSH_CONFIG_PREFIX) $(SSH_CONFIG_SUFFIX))
+ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(shell scripts/append-makefile-vars.sh $(CONFIG_KDEVOPS_SSH_CONFIG_PREFIX) $(CONFIG_TOPDIR_PATH_SHA256SUM))
 
 endif # CONFIG_KDEVOPS_SSH_CONFIG_UPDATE
 
diff --git a/scripts/update_ssh_config_guestfs.py b/scripts/update_ssh_config_guestfs.py
index 8b212a9c15a2..143ff4fc24c4 100755
--- a/scripts/update_ssh_config_guestfs.py
+++ b/scripts/update_ssh_config_guestfs.py
@@ -85,14 +85,9 @@ def main():
     with open(f'{topdir}/{extra_vars["kdevops_nodes"]}') as stream:
         nodes = yaml.safe_load(stream)
 
-    if extra_vars.get("topdir_path_has_sha256sum", False):
-        ssh_config = (
-            f'{Path.home()}/.ssh/config_kdevops_{extra_vars["topdir_path_sha256sum"]}'
-        )
-    else:
-        ssh_config = (
-            f'{Path.home()}/.ssh/config_kdevops_{extra_vars["kdevops_host_prefix"]}'
-        )
+    ssh_config = (
+        f'{Path.home()}/.ssh/config_kdevops_{extra_vars["topdir_path_sha256sum"]}'
+    )
 
     # make a stanza for each node
     sshconf = open(ssh_config, "w")
-- 
2.51.0

[PATCH v2 35/37] Makefile: Set TOPDIR_PATH and generate its sha256sum at run time

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a
uniquifier to the name of the ssh config to avoid multiple kdevops
runs clobbering each other's ssh config entries. It uses the TOPDIR
pathname set in the current .config to benefit from the nice
convenience of generating an Ansible extra_var using "output yaml".

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") broke the
ability to share a .config with other users because
CONFIG_TOPDIR_PATH contains a fixed string pointing to the home
directory where the .config was created.

The TOPDIR and its checksum need to be generated not by "make
menuconfig" but rather by "make" in order that .config files (as
saved in defconfigs/) can be truly portable.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Kconfig                    | 10 ----------
 Makefile                   |  2 ++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  2 +-
 4 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/Kconfig b/Kconfig
index 54921617b8b2..948befbce4de 100644
--- a/Kconfig
+++ b/Kconfig
@@ -5,16 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
diff --git a/Makefile b/Makefile
index 83c67340d0ce..a9cba02c0829 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,7 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_SHA256SUM = $(shell ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
@@ -82,6 +83,7 @@ INCLUDES = -I include/
 CFLAGS += $(INCLUDES)
 
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
+ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
diff --git a/scripts/gen-hosts.Makefile b/scripts/gen-hosts.Makefile
index 0d1e0cd77c2e..c809b1da18e5 100644
--- a/scripts/gen-hosts.Makefile
+++ b/scripts/gen-hosts.Makefile
@@ -3,6 +3,7 @@
 KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR)
 KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates
 
+GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'
 GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)'
 
 # Relative path so that ansible can work with it
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index 68c6dcdd4dcd..82e9a8dd33b5 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -10,7 +10,7 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
 
-ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(shell scripts/append-makefile-vars.sh $(CONFIG_KDEVOPS_SSH_CONFIG_PREFIX) $(CONFIG_TOPDIR_PATH_SHA256SUM))
+ANSIBLE_EXTRA_ARGS += kdevops_ssh_config=$(shell scripts/append-makefile-vars.sh $(CONFIG_KDEVOPS_SSH_CONFIG_PREFIX) $(TOPDIR_PATH_SHA256SUM))
 
 endif # CONFIG_KDEVOPS_SSH_CONFIG_UPDATE
 
-- 
2.51.0

[PATCH v2 36/37] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

I need an ansible.cfg that is generated by the Kconfig menu, but
whose pathname is selected dynamically based on where kdevops is
being run rather than having that path baked into the .config.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/Makefile b/Makefile
index a9cba02c0829..f13d1f5c91ce 100644
--- a/Makefile
+++ b/Makefile
@@ -26,10 +26,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_CUSTOM))
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+else
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -84,6 +89,9 @@ CFLAGS += $(INCLUDES)
 
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifneq (y,$(CONFIG_ANSIBLE_CFG_FILE_CUSTOM))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
index dbd189f5d19e..8b32d4986a76 100644
--- a/kconfigs/Kconfig.ansible_cfg
+++ b/kconfigs/Kconfig.ansible_cfg
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 
+config ANSIBLE_CFG_FILE_CUSTOM
+	bool "Set a custom location for the ansible.cfg file"
+	help
+	  When this option is set to N, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu. If it is set to Y, you can set the
+	  path and filename of the Ansible configuration file.
+
+if ANSIBLE_CFG_FILE_CUSTOM
+
 config ANSIBLE_CFG_FILE
 	string "Ansible configuration file"
 	output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
 
+endif # ANSIBLE_CFG_FILE_CUSTOM
+
 menu "Ansible Callback Plugin Configuration"
 choice
 	prompt "Ansible Callback Plugin"
-- 
2.51.0

[PATCH v2 37/37] inventory: generate the inventory pathname dynamically

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

My CI workflows need the inventory location to be determined at run
time, not set by the kdevops .config file, so the .configs can be
portable amongst test runners and test branches.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/Makefile b/Makefile
index f13d1f5c91ce..7d8b91f4c163 100644
--- a/Makefile
+++ b/Makefile
@@ -35,7 +35,12 @@ else
 ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+
+ifeq (y,$(CONFIG_ANSIBLE_CFG_INVENTORY_CUSTOM))
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
+else
+ANSIBLE_INVENTORY_FILE := $(TOPDIR_PATH)/hosts
+endif
 
 KDEVOPS_INSTALL_TARGETS :=
 
@@ -92,6 +97,9 @@ ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
 ifneq (y,$(CONFIG_ANSIBLE_CFG_FILE_CUSTOM))
 ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
 endif
+ifneq (y,$(CONFIG_ANSIBLE_CFG_INVENTORY_CUSTOM))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_inventory='$(ANSIBLE_INVENTORY_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
index 8b32d4986a76..a5812995c3fa 100644
--- a/kconfigs/Kconfig.ansible_cfg
+++ b/kconfigs/Kconfig.ansible_cfg
@@ -263,6 +263,16 @@ config ANSIBLE_CFG_TASK_DEBUGGER
 
 	  https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_debugger.html#id10
 
+config ANSIBLE_CFG_INVENTORY_CUSTOM
+	bool "Set a custom location for the Ansible inventory"
+	help
+	  When this option is set to N, kdevops creates its
+	  inventory file in a sensible default location. If it is
+	  set to Y, you can set the path and filename of the Ansible
+	  inventory.
+
+if ANSIBLE_CFG_INVENTORY_CUSTOM
+
 config ANSIBLE_CFG_INVENTORY
 	string "Ansible inventory sources"
 	output yaml
@@ -282,6 +292,8 @@ config ANSIBLE_CFG_INVENTORY
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-host-list
 
+endif # ANSIBLE_CFG_INVENTORY_CUSTOM
+
 if DISTRO_OPENSUSE
 
 config ANSIBLE_CFG_RECONNECTION_RETRIES
-- 
2.51.0

Re: [PATCH v2 00/37] Various fixes

[Luis Chamberlain]

On Tue, Sep 02, 2025 at 09:53:47AM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> I've finally gotten most of my test runners moved to cloud providers
> so I'm able to rebase my version of kdevops on
> 
>   linux-kdevops/kdevops : HEAD
> 
> In the process I've found and fixed myriad small issues. One or two
> of these might need some attention and additional work. Please have
> a close look -- they are all short subjects.

For anything missing:

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>

  Luis

Re: [PATCH v2 20/37] guestfs: Update the guestfs playbook description

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Match the description for the terraform playbook
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

Re: [PATCH v2 21/37] gen_hosts: Add 'localhost' to the [all] group

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Commit 58a0745e8fa2 ("Add a guest/instance for building the test
> kernel") was applied just after commit 1cf0800c9ffc ("gen_hosts:
> templates: include localhost in the all group"). The former should
> have been updated to include localhost in its [all] group.
> 
> Without 'localhost', trying to provision the kernel builder node
> fails.
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

Re: [PATCH v2 22/37] gen_hosts: Add [service] group to the builder host template

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Ensure that the [service] group is defined in the builder
> node's host template, to quash some noisy warnings during
> bringup.
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

I wonder if we should apply this to other templates too?

We have service group declared for fio-tests, fstests, gitr, hosts, nfstest
and pynfs. I'll try to remember when running other workflows!

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

Re: [PATCH v2 22/37] gen_hosts: Add [service] group to the builder host template

[Chuck Lever]

On 9/2/25 3:43 PM, Daniel Gomez wrote:
> On 02/09/2025 15.54, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Ensure that the [service] group is defined in the builder
>> node's host template, to quash some noisy warnings during
>> bringup.
>>
>> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> 
> I wonder if we should apply this to other templates too?
> 
> We have service group declared for fio-tests, fstests, gitr, hosts, nfstest
> and pynfs. I'll try to remember when running other workflows!
> 
> Reviewed-by: Daniel Gomez <da.gomez@samsung.com>
> 

I didn't want to just scatter-shot this change, was waiting to see
actual warnings come up. But I think it's likely that the builder
template is not the only one that might need [service].


-- 
Chuck Lever

Re: [PATCH v2 23/37] guestfs: Don't run the 'destroy' tags on the controller

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> "make destroy" was skipping some important file removals, and
> removing other files that do not exist.
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  scripts/guestfs.Makefile | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
> index f6c350a43f27..953c61493a88 100644
> --- a/scripts/guestfs.Makefile
> +++ b/scripts/guestfs.Makefile
> @@ -96,6 +96,7 @@ PHONY += status_guestfs
>  
>  destroy_guestfs:
>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
> +		--limit 'baseline:dev:service' \

Do we need to be explicit here?

The guestfs.yml playbook already defines hosts: all. Not sure if I understand
the reason why things were skipped. Can you elaborate?

>		playbooks/guestfs.yml \
>  		--extra-vars=@./extra_vars.yaml \
>  		--tags destroy

Re: [PATCH v2 24/37] base_image: Skip "locales-all" package on Fedora

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  playbooks/roles/base_image/templates/virt-builder.j2 | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/playbooks/roles/base_image/templates/virt-builder.j2 b/playbooks/roles/base_image/templates/virt-builder.j2
> index 7df4229405e7..138d06eede6b 100644
> --- a/playbooks/roles/base_image/templates/virt-builder.j2
> +++ b/playbooks/roles/base_image/templates/virt-builder.j2
> @@ -11,7 +11,7 @@ mkdir {{ target_dir }}
>  copy-in {{ guestfs_distro_source_and_dest_file }}:{{ target_dir }}
>  {% endif %}
>  
> -install sudo,qemu-guest-agent,python3,bash,locales-all
> +install sudo,qemu-guest-agent,python3,bash

I think this also applies to Debian, not Fedora only.

The reason was added was to fix a local warning when ssh into a Debian guest.

Let me check if this is still needed.

>  run-command useradd {{ kdevops_uid }} -s /bin/bash -m kdevops
>  append-line /etc/sudoers.d/kdevops:kdevops   ALL=(ALL)       NOPASSWD: ALL
>  edit /etc/default/grub:s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0"/

Re: [PATCH v2 25/37] update_etc_hosts: ipaddr

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> ansible.netcommon.ipaddr is deprecated. See:
> 
>  https://docs.ansible.com/ansible/latest/collections/ansible/netcommon/ipaddr_filter.html
> 
> Ensure the update_etc_hosts playbook uses the ansible.utils.ipaddr
> plugin instead.
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

> ---
>  playbooks/roles/update_etc_hosts/tasks/main.yml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/playbooks/roles/update_etc_hosts/tasks/main.yml b/playbooks/roles/update_etc_hosts/tasks/main.yml
> index dc40ededd539..17c85dc539b5 100644
> --- a/playbooks/roles/update_etc_hosts/tasks/main.yml
> +++ b/playbooks/roles/update_etc_hosts/tasks/main.yml
> @@ -51,8 +51,8 @@
>    become_method: sudo
>    ansible.builtin.lineinfile:
>      dest: /etc/hosts
> -    regexp: ".*{{ item }}$"
> -    line: "{{ hostvars[item].ansible_all_ipv4_addresses | ipaddr(private_network) | first }} {{ item }}"
> +    regexp: '.*{{ item }}$'
> +    line: "{{ hostvars[item].ansible_all_ipv4_addresses | ansible.utils.ipaddr(private_network) | first }} {{ item }}"
>      state: present
>    with_items: "{{ ueh_hosts }}"
>    when:

Re: [PATCH v2 23/37] guestfs: Don't run the 'destroy' tags on the controller

[Chuck Lever]

On 9/2/25 3:54 PM, Daniel Gomez wrote:
> On 02/09/2025 15.54, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> "make destroy" was skipping some important file removals, and
>> removing other files that do not exist.
>>
>> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  scripts/guestfs.Makefile | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
>> index f6c350a43f27..953c61493a88 100644
>> --- a/scripts/guestfs.Makefile
>> +++ b/scripts/guestfs.Makefile
>> @@ -96,6 +96,7 @@ PHONY += status_guestfs
>>  
>>  destroy_guestfs:
>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>> +		--limit 'baseline:dev:service' \
> 
> Do we need to be explicit here?
> 
> The guestfs.yml playbook already defines hosts: all. Not sure if I understand
> the reason why things were skipped. Can you elaborate?

Before, the "destroy" inventory would have generated these tasks:

host1 -> localhost
host2 -> localhost
host3 -> localhost

But [all] now includes localhost. So we get:

host1 -> localhost
host2 -> localhost
host3 -> localhost
localhost -> localhost

It's that last item that is unwanted.


>> 		playbooks/guestfs.yml \
>>  		--extra-vars=@./extra_vars.yaml \
>>  		--tags destroy


-- 
Chuck Lever

Re: [PATCH v2 26/37] Add an Ansible requirements.yml file

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
> module") introduced the use of the cloud.terraform module, and
> commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
> with an Ansible playbook") introduced the use of the
> community.libvirt module. It would be friendly if kdevops could
> pull in the Ansible modules it needs transparently.
> 
> The requirements.yml file is a manifest of Ansible collections that
> the project needs to run. Installation of these collections is made
> automatic by adding:
> 
>   ansible-galaxy install -r requirements.yml
> 
> to the "make ansible_cfg" step. This mechanism can keep cached
> versions of collections up to date, and can also constrain a
> cached collection to a specific version, if that's needed.

I don't see the link between the galaxy step and the ansible_cfg target in the
changes below. But in case we have it, I think this should be distro specific.
Last time I checked, all these galaxy dependencies are provided in Debian by the
ansible package. So it's not needed there.

> 
> The initial file contains requirements I could find easily, and
> should be updated over time as new collection dependencies are
> introduced.
> 
> See also:
> 
> https://docs.ansible.com/ansible/latest/user_guide/collections_using.html
> 
> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  requirements.yml         | 8 ++++++++
>  scripts/ansible.Makefile | 5 +++++
>  2 files changed, 13 insertions(+)
>  create mode 100644 requirements.yml
> 
> diff --git a/requirements.yml b/requirements.yml
> new file mode 100644
> index 000000000000..a22691d3f86b
> --- /dev/null
> +++ b/requirements.yml
> @@ -0,0 +1,8 @@
> +---
> +collections:
> +  - name: ansible.posix
> +  - name: ansible.utils
> +  - name: cloud.terraform
> +  - name: community.docker
> +  - name: community.general
> +  - name: community.libvirt
> diff --git a/scripts/ansible.Makefile b/scripts/ansible.Makefile
> index b0d2a8c12ee7..4efcf3e93b0e 100644
> --- a/scripts/ansible.Makefile
> +++ b/scripts/ansible.Makefile
> @@ -2,3 +2,8 @@
>  
>  AV ?= 0
>  export ANSIBLE_VERBOSE := $(shell scripts/validate_av.py --av "$(AV)")
> +
> +ansible-requirements:
> +	@$(Q)ansible-galaxy install -r requirements.yml

Can you remove the "@"? That is controlled by the $(Q) var.

> +PHONY += ansible-requirements
> +DEFAULT_DEPS += ansible-requirements

Re: [PATCH v2 27/37] terraform/oci: Clean up white space in route table resource

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  terraform/oci/main.tf | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/terraform/oci/main.tf b/terraform/oci/main.tf
> index 45a4b4bc0c63..15660aa02614 100644
> --- a/terraform/oci/main.tf
> +++ b/terraform/oci/main.tf
> @@ -103,10 +103,12 @@ resource "oci_core_route_table" "kdevops_route_table" {
>    compartment_id = data.oci_identity_compartments.kdevops_compartment.compartments[0].id
>    display_name   = "kdevops route table"
>    vcn_id         = one(oci_core_vcn.kdevops_vcn[*].id)
> +
>    route_rules {
>      destination       = "0.0.0.0/0"
>      destination_type  = "CIDR_BLOCK"
>      network_entity_id = one(oci_core_internet_gateway.kdevops_internet_gateway[*].id)
> +    route_type        = "STATIC"

I think the patch title is misleading. It's doing something else.

>    }
>  }
>  

Re: [PATCH v2 26/37] Add an Ansible requirements.yml file

[Chuck Lever]

On 9/2/25 4:06 PM, Daniel Gomez wrote:
> On 02/09/2025 15.54, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
>> module") introduced the use of the cloud.terraform module, and
>> commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
>> with an Ansible playbook") introduced the use of the
>> community.libvirt module. It would be friendly if kdevops could
>> pull in the Ansible modules it needs transparently.
>>
>> The requirements.yml file is a manifest of Ansible collections that
>> the project needs to run. Installation of these collections is made
>> automatic by adding:
>>
>>   ansible-galaxy install -r requirements.yml
>>
>> to the "make ansible_cfg" step. This mechanism can keep cached
>> versions of collections up to date, and can also constrain a
>> cached collection to a specific version, if that's needed.
> 
> I don't see the link between the galaxy step and the ansible_cfg target in the
> changes below.

AIUI, just the requirements.yml file itself is not enough. The
"ansible-galaxy install" step is needed to actually do the install.

It's arguable whether the ansible_cfg target is the best place to
handle it, but that target is done quite early. Suggestions are
welcome!


> But in case we have it, I think this should be distro specific.
> Last time I checked, all these galaxy dependencies are provided in Debian by the
> ansible package. So it's not needed there.

On my systems, some of the packaged versions of the collections are
already installed anyway. But on some they are not.

It's better to not have to keep track of whether the distros provide
these or whether they are installed. We document the requirements in
requirements.yml, and the rest is handled automatically (and
declaratively).


>> The initial file contains requirements I could find easily, and
>> should be updated over time as new collection dependencies are
>> introduced.
>>
>> See also:
>>
>> https://docs.ansible.com/ansible/latest/user_guide/collections_using.html
>>
>> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  requirements.yml         | 8 ++++++++
>>  scripts/ansible.Makefile | 5 +++++
>>  2 files changed, 13 insertions(+)
>>  create mode 100644 requirements.yml
>>
>> diff --git a/requirements.yml b/requirements.yml
>> new file mode 100644
>> index 000000000000..a22691d3f86b
>> --- /dev/null
>> +++ b/requirements.yml
>> @@ -0,0 +1,8 @@
>> +---
>> +collections:
>> +  - name: ansible.posix
>> +  - name: ansible.utils
>> +  - name: cloud.terraform
>> +  - name: community.docker
>> +  - name: community.general
>> +  - name: community.libvirt
>> diff --git a/scripts/ansible.Makefile b/scripts/ansible.Makefile
>> index b0d2a8c12ee7..4efcf3e93b0e 100644
>> --- a/scripts/ansible.Makefile
>> +++ b/scripts/ansible.Makefile
>> @@ -2,3 +2,8 @@
>>  
>>  AV ?= 0
>>  export ANSIBLE_VERBOSE := $(shell scripts/validate_av.py --av "$(AV)")
>> +
>> +ansible-requirements:
>> +	@$(Q)ansible-galaxy install -r requirements.yml
> 
> Can you remove the "@"? That is controlled by the $(Q) var.

I copied that from elsewhere. It can be removed.


>> +PHONY += ansible-requirements
>> +DEFAULT_DEPS += ansible-requirements


-- 
Chuck Lever

Re: [PATCH v2 28/37] guestfs: Correct the spelling of "drives"

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

Re: [PATCH v2 29/37] guestfs: Configure how many extra drives to provision

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Some workflows create several guests but use only one or two extra
> storage drives per guest. Enable configurations that reduce the
> amount of storage allocated per guest to conserve local persistent
> storage space on the host.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

This should allow workflows to specify the nr of drives needed, right?

Looks great optimization, thanks!

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

> ---
>  kconfigs/Kconfig.libvirt                      | 45 +++++++++++++++++++
>  .../roles/gen_nodes/templates/gen_drives.j2   |  8 ++--
>  .../roles/guestfs/tasks/bringup/main.yml      |  2 +-
>  3 files changed, 50 insertions(+), 5 deletions(-)
> 
> diff --git a/kconfigs/Kconfig.libvirt b/kconfigs/Kconfig.libvirt
> index 7fe231668229..961276ccc1b1 100644
> --- a/kconfigs/Kconfig.libvirt
> +++ b/kconfigs/Kconfig.libvirt
> @@ -556,6 +556,51 @@ config LIBVIRT_HOST_PASSTHROUGH
>  	      --pre 'make -s mrproper defconfig' \
>  	    \-- make -s -j$(nproc) bzImage
>  
> +choice
> +	prompt "Libvirt extra storage drive count"
> +	default LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4

I see we are defaulting to what we had. So it's up to the user or defconfig to
leverage this.

> +	help
> +	  This option selects the number of extra storage drives to
> +	  provision for each libvirt guest. Reduce this number to
> +	  conserve local host storage capacity, if your workflows do
> +	  not use all the extra drives.
> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
> +	bool "0"
> +	help
> +	  Provision no extra storage drives per guest.

Curious, any reason to have a menu/choice instead of int?


> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
> +	bool "1"
> +	help
> +	  Provision one extra storage drive per guest.
> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
> +	bool "2"
> +	help
> +	  Provision two extra storage drives per guest.
> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
> +	bool "3"
> +	help
> +	  Provision three extra storage drives per guest.
> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
> +	bool "4"
> +	help
> +	  Provision four extra storages drive per guest.
> +
> +endchoice
> +
> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT
> +	int
> +	output yaml
> +	default 0 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
> +	default 1 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
> +	default 2 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
> +	default 3 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
> +	default 4 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
> +
>  choice
>  	prompt "Libvirt extra storage driver to use"
>  	default LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO
> diff --git a/playbooks/roles/gen_nodes/templates/gen_drives.j2 b/playbooks/roles/gen_nodes/templates/gen_drives.j2
> index 2de13da4ab8e..1040c0e19d84 100644
> --- a/playbooks/roles/gen_nodes/templates/gen_drives.j2
> +++ b/playbooks/roles/gen_nodes/templates/gen_drives.j2
> @@ -1,6 +1,6 @@
>  {% import './templates/drives.j2' as drives %}
>  {% if libvirt_extra_storage_drive_ide %}
> -{{ drives.gen_drive_ide(4,
> +{{ drives.gen_drive_ide(libvirt_extra_storage_drive_count,
>  			kdevops_storage_pool_path,
>  			hostname,
>  			libvirt_extra_drive_format,
> @@ -18,7 +18,7 @@
>  				    libvirt_extra_storage_aio_cache_mode,
>  				    kdevops_storage_pool_path) }}
>  {% else %}
> -{{ drives.gen_drive_virtio(4,
> +{{ drives.gen_drive_virtio(libvirt_extra_storage_drive_count,
>                             kdevops_storage_pool_path,
>                             hostname,
>                             libvirt_extra_drive_format,
> @@ -28,7 +28,7 @@
>                             libvirt_extra_storage_virtio_physical_block_size) }}
>  {% endif %}
>  {% elif libvirt_extra_storage_drive_scsi %}
> -{{ drives.gen_drive_scsi(4,
> +{{ drives.gen_drive_scsi(libvirt_extra_storage_drive_count,
>                           kdevops_storage_pool_path,
>                           hostname,
>                           libvirt_extra_drive_format,
> @@ -46,7 +46,7 @@
>  				  libvirt_extra_storage_aio_cache_mode,
>  				  kdevops_storage_pool_path) }}
>  {% else %}
> -{{ drives.gen_drive_nvme(4,
> +{{ drives.gen_drive_nvme(libvirt_extra_storage_drive_count,
>  			 kdevops_storage_pool_path,
>  			 hostname,
>  			 libvirt_extra_drive_format,
> diff --git a/playbooks/roles/guestfs/tasks/bringup/main.yml b/playbooks/roles/guestfs/tasks/bringup/main.yml
> index bd9f52603a3b..3fd677ff4659 100644
> --- a/playbooks/roles/guestfs/tasks/bringup/main.yml
> +++ b/playbooks/roles/guestfs/tasks/bringup/main.yml
> @@ -102,7 +102,7 @@
>          path: "{{ storagedir }}/{{ inventory_hostname }}/extra{{ item }}.{{ libvirt_extra_drive_format }}"
>        ansible.builtin.include_tasks:
>          file: "{{ role_path }}/tasks/bringup/extra-disks.yml"
> -      loop: "{{ range(0, 4) | list }}"
> +      loop: "{{ range(0, libvirt_extra_storage_drive_count) | list }}"
>        when:
>          - not libvirt_enable_largeio|bool
>  

Re: [PATCH v2 29/37] guestfs: Configure how many extra drives to provision

[Chuck Lever]

On 9/2/25 4:19 PM, Daniel Gomez wrote:
> On 02/09/2025 15.54, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Some workflows create several guests but use only one or two extra
>> storage drives per guest. Enable configurations that reduce the
>> amount of storage allocated per guest to conserve local persistent
>> storage space on the host.
>>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> 
> This should allow workflows to specify the nr of drives needed, right?
> 
> Looks great optimization, thanks!
> 
> Reviewed-by: Daniel Gomez <da.gomez@samsung.com>
> 
>> ---
>>  kconfigs/Kconfig.libvirt                      | 45 +++++++++++++++++++
>>  .../roles/gen_nodes/templates/gen_drives.j2   |  8 ++--
>>  .../roles/guestfs/tasks/bringup/main.yml      |  2 +-
>>  3 files changed, 50 insertions(+), 5 deletions(-)
>>
>> diff --git a/kconfigs/Kconfig.libvirt b/kconfigs/Kconfig.libvirt
>> index 7fe231668229..961276ccc1b1 100644
>> --- a/kconfigs/Kconfig.libvirt
>> +++ b/kconfigs/Kconfig.libvirt
>> @@ -556,6 +556,51 @@ config LIBVIRT_HOST_PASSTHROUGH
>>  	      --pre 'make -s mrproper defconfig' \
>>  	    \-- make -s -j$(nproc) bzImage
>>  
>> +choice
>> +	prompt "Libvirt extra storage drive count"
>> +	default LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
> 
> I see we are defaulting to what we had. So it's up to the user or defconfig to
> leverage this.
> 
>> +	help
>> +	  This option selects the number of extra storage drives to
>> +	  provision for each libvirt guest. Reduce this number to
>> +	  conserve local host storage capacity, if your workflows do
>> +	  not use all the extra drives.
>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
>> +	bool "0"
>> +	help
>> +	  Provision no extra storage drives per guest.
> 
> Curious, any reason to have a menu/choice instead of int?

This is the way drive count is handled in the terraform provider menus.
So, the reason is "consistency with existing input methods", no other
reason.


>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
>> +	bool "1"
>> +	help
>> +	  Provision one extra storage drive per guest.
>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
>> +	bool "2"
>> +	help
>> +	  Provision two extra storage drives per guest.
>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
>> +	bool "3"
>> +	help
>> +	  Provision three extra storage drives per guest.
>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
>> +	bool "4"
>> +	help
>> +	  Provision four extra storages drive per guest.
>> +
>> +endchoice
>> +
>> +config LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT
>> +	int
>> +	output yaml
>> +	default 0 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_0
>> +	default 1 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_1
>> +	default 2 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_2
>> +	default 3 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_3
>> +	default 4 if LIBVIRT_EXTRA_STORAGE_DRIVE_COUNT_4
>> +
>>  choice
>>  	prompt "Libvirt extra storage driver to use"
>>  	default LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO
>> diff --git a/playbooks/roles/gen_nodes/templates/gen_drives.j2 b/playbooks/roles/gen_nodes/templates/gen_drives.j2
>> index 2de13da4ab8e..1040c0e19d84 100644
>> --- a/playbooks/roles/gen_nodes/templates/gen_drives.j2
>> +++ b/playbooks/roles/gen_nodes/templates/gen_drives.j2
>> @@ -1,6 +1,6 @@
>>  {% import './templates/drives.j2' as drives %}
>>  {% if libvirt_extra_storage_drive_ide %}
>> -{{ drives.gen_drive_ide(4,
>> +{{ drives.gen_drive_ide(libvirt_extra_storage_drive_count,
>>  			kdevops_storage_pool_path,
>>  			hostname,
>>  			libvirt_extra_drive_format,
>> @@ -18,7 +18,7 @@
>>  				    libvirt_extra_storage_aio_cache_mode,
>>  				    kdevops_storage_pool_path) }}
>>  {% else %}
>> -{{ drives.gen_drive_virtio(4,
>> +{{ drives.gen_drive_virtio(libvirt_extra_storage_drive_count,
>>                             kdevops_storage_pool_path,
>>                             hostname,
>>                             libvirt_extra_drive_format,
>> @@ -28,7 +28,7 @@
>>                             libvirt_extra_storage_virtio_physical_block_size) }}
>>  {% endif %}
>>  {% elif libvirt_extra_storage_drive_scsi %}
>> -{{ drives.gen_drive_scsi(4,
>> +{{ drives.gen_drive_scsi(libvirt_extra_storage_drive_count,
>>                           kdevops_storage_pool_path,
>>                           hostname,
>>                           libvirt_extra_drive_format,
>> @@ -46,7 +46,7 @@
>>  				  libvirt_extra_storage_aio_cache_mode,
>>  				  kdevops_storage_pool_path) }}
>>  {% else %}
>> -{{ drives.gen_drive_nvme(4,
>> +{{ drives.gen_drive_nvme(libvirt_extra_storage_drive_count,
>>  			 kdevops_storage_pool_path,
>>  			 hostname,
>>  			 libvirt_extra_drive_format,
>> diff --git a/playbooks/roles/guestfs/tasks/bringup/main.yml b/playbooks/roles/guestfs/tasks/bringup/main.yml
>> index bd9f52603a3b..3fd677ff4659 100644
>> --- a/playbooks/roles/guestfs/tasks/bringup/main.yml
>> +++ b/playbooks/roles/guestfs/tasks/bringup/main.yml
>> @@ -102,7 +102,7 @@
>>          path: "{{ storagedir }}/{{ inventory_hostname }}/extra{{ item }}.{{ libvirt_extra_drive_format }}"
>>        ansible.builtin.include_tasks:
>>          file: "{{ role_path }}/tasks/bringup/extra-disks.yml"
>> -      loop: "{{ range(0, 4) | list }}"
>> +      loop: "{{ range(0, libvirt_extra_storage_drive_count) | list }}"
>>        when:
>>          - not libvirt_enable_largeio|bool
>>  


-- 
Chuck Lever

Re: [PATCH v2 33/37] devconfig: include the "service" group in the playbooks hosts list

[Daniel Gomez]

On 02/09/2025 15.54, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Since commit 1cf0800c9ffc ("gen_hosts: templates: include localhost
> in the all group"), the devconfig playbook is no longer updating the
> nfsd service host.
> 
> 1. Actually all the service hosts need to be updated by devconfig,
>    so ensure devconfig runs against the service group.
> 
> 2. Since devconfig can run against most workflows, add blank
>    [service] groups to any inventory template that doesn't already
>    have it.
> 
> Fixes: 1cf0800c9ffc ("gen_hosts: templates: include localhost in the all group")
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  playbooks/devconfig.yml                          | 2 +-
>  playbooks/roles/gen_hosts/templates/blktests.j2  | 3 +++
>  playbooks/roles/gen_hosts/templates/cxl.j2       | 3 +++
>  playbooks/roles/gen_hosts/templates/ltp.j2       | 4 ++++
>  playbooks/roles/gen_hosts/templates/mmtests.j2   | 3 +++
>  playbooks/roles/gen_hosts/templates/selftests.j2 | 3 +++
>  playbooks/roles/gen_hosts/templates/sysbench.j2  | 3 +++
>  scripts/devconfig.Makefile                       | 9 ++++-----
>  scripts/firstconfig.Makefile                     | 2 +-
>  scripts/provision.Makefile                       | 2 +-
>  10 files changed, 26 insertions(+), 8 deletions(-)
> 
> diff --git a/playbooks/devconfig.yml b/playbooks/devconfig.yml
> index cebaa87a7228..f0ca7d767efe 100644
> --- a/playbooks/devconfig.yml
> +++ b/playbooks/devconfig.yml
> @@ -1,6 +1,6 @@
>  ---
>  - name: Configure developer environment on target systems
> -  hosts: baseline:dev
> +  hosts: baseline:dev:service
>    gather_facts: false
>    roles:
>      - role: devconfig
> diff --git a/playbooks/roles/gen_hosts/templates/blktests.j2 b/playbooks/roles/gen_hosts/templates/blktests.j2
> index 5bf50acbddac..6aa434c3383c 100644
> --- a/playbooks/roles/gen_hosts/templates/blktests.j2
> +++ b/playbooks/roles/gen_hosts/templates/blktests.j2
> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endif %}
>  [dev:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/playbooks/roles/gen_hosts/templates/cxl.j2 b/playbooks/roles/gen_hosts/templates/cxl.j2
> index 64bec4561b6c..cab7da246b11 100644
> --- a/playbooks/roles/gen_hosts/templates/cxl.j2
> +++ b/playbooks/roles/gen_hosts/templates/cxl.j2
> @@ -16,3 +16,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endif %}
>  [dev:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/playbooks/roles/gen_hosts/templates/ltp.j2 b/playbooks/roles/gen_hosts/templates/ltp.j2
> index 828fd6362718..6946b3f3593b 100644
> --- a/playbooks/roles/gen_hosts/templates/ltp.j2
> +++ b/playbooks/roles/gen_hosts/templates/ltp.j2
> @@ -24,3 +24,7 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endif %}
>  [dev:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/playbooks/roles/gen_hosts/templates/mmtests.j2 b/playbooks/roles/gen_hosts/templates/mmtests.j2
> index 1252fe876e6c..8df4a3c31213 100644
> --- a/playbooks/roles/gen_hosts/templates/mmtests.j2
> +++ b/playbooks/roles/gen_hosts/templates/mmtests.j2
> @@ -31,3 +31,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endfor %}
>  [mmtests:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/playbooks/roles/gen_hosts/templates/selftests.j2 b/playbooks/roles/gen_hosts/templates/selftests.j2
> index e505bbf194b4..af0d6b341048 100644
> --- a/playbooks/roles/gen_hosts/templates/selftests.j2
> +++ b/playbooks/roles/gen_hosts/templates/selftests.j2
> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endif %}
>  [dev:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/playbooks/roles/gen_hosts/templates/sysbench.j2 b/playbooks/roles/gen_hosts/templates/sysbench.j2
> index 107eebc5fbce..e30773c1b84a 100644
> --- a/playbooks/roles/gen_hosts/templates/sysbench.j2
> +++ b/playbooks/roles/gen_hosts/templates/sysbench.j2
> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>  {% endif %}
>  [dev:vars]
>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> +[service]
> +[service:vars]
> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
> diff --git a/scripts/devconfig.Makefile b/scripts/devconfig.Makefile
> index 146c70f78897..fedf1f2ca21f 100644
> --- a/scripts/devconfig.Makefile
> +++ b/scripts/devconfig.Makefile
> @@ -57,11 +57,10 @@ extend-extra-args-devconfig:
>  PHONY += devconfig
>  devconfig: $(KDEVOPS_NODES)
>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
> -		--limit 'baseline:dev:nfsd' \
> +		--limit 'baseline:dev:service' \

You don't need --limit at all (I missed this cleanup!). The playbook already has
hosts: baseline:dev:service.

>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
>  		--extra-vars="$(BOOTLINUX_ARGS)" \
> -		--extra-vars '{ kdevops_cli_install: True }' \
> -		$(LIMIT_HOSTS)
> +		--extra-vars '{ kdevops_cli_install: True }'

Any reason to remove LIMIT_HOSTS support from here? Maybe not useful for these
targets?

>  
>  devconfig-generic-help-menu:
>  	@echo "devconfig          - Ensures generic system setup and is up to date"
> @@ -72,9 +71,9 @@ ifeq (y,$(CONFIG_SYSCTL_TUNING))
>  PHONY += sysctl-tunings
>  sysctl-tunings: $(KDEVOPS_NODES)
>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
> -		--limit 'baseline:dev:nfsd' \
> +		--limit 'baseline:dev:service' \

Same here. Not needed.

>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
> -		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS) --tags vars,sysctl
> +		--extra-vars="$(BOOTLINUX_ARGS)" --tags vars,sysctl
>  
>  devconfig-help-menu:
>  	@echo "Target node configuration options"
> diff --git a/scripts/firstconfig.Makefile b/scripts/firstconfig.Makefile
> index 9f8a63a8ff84..5651a07d0b66 100644
> --- a/scripts/firstconfig.Makefile
> +++ b/scripts/firstconfig.Makefile
> @@ -2,7 +2,7 @@
>  
>  firstconfig:
>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
> -		--limit 'baseline:dev' \
> +		--limit 'baseline:dev:service' \

Same here...

>  		--extra-vars '{ kdevops_cli_install: True }' \
>  		--tags vars_simple,firstconfig \
>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml
> diff --git a/scripts/provision.Makefile b/scripts/provision.Makefile
> index f04264f6b9ea..62a5692904c9 100644
> --- a/scripts/provision.Makefile
> +++ b/scripts/provision.Makefile
> @@ -70,7 +70,7 @@ KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG)
>  $(KDEVOPS_PROVISIONED_DEVCONFIG):
>  	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
>  		ansible-playbook $(ANSIBLE_VERBOSE) \
> -			--limit 'baseline:dev' \
> +			--limit 'baseline:dev:service' \

And here (playbook is devconfig.yml)

>  			$(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
>  	fi
>  	$(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG)

Re: [PATCH v2 33/37] devconfig: include the "service" group in the playbooks hosts list

[Chuck Lever]

On 9/2/25 4:27 PM, Daniel Gomez wrote:
> On 02/09/2025 15.54, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Since commit 1cf0800c9ffc ("gen_hosts: templates: include localhost
>> in the all group"), the devconfig playbook is no longer updating the
>> nfsd service host.
>>
>> 1. Actually all the service hosts need to be updated by devconfig,
>>    so ensure devconfig runs against the service group.
>>
>> 2. Since devconfig can run against most workflows, add blank
>>    [service] groups to any inventory template that doesn't already
>>    have it.
>>
>> Fixes: 1cf0800c9ffc ("gen_hosts: templates: include localhost in the all group")
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  playbooks/devconfig.yml                          | 2 +-
>>  playbooks/roles/gen_hosts/templates/blktests.j2  | 3 +++
>>  playbooks/roles/gen_hosts/templates/cxl.j2       | 3 +++
>>  playbooks/roles/gen_hosts/templates/ltp.j2       | 4 ++++
>>  playbooks/roles/gen_hosts/templates/mmtests.j2   | 3 +++
>>  playbooks/roles/gen_hosts/templates/selftests.j2 | 3 +++
>>  playbooks/roles/gen_hosts/templates/sysbench.j2  | 3 +++
>>  scripts/devconfig.Makefile                       | 9 ++++-----
>>  scripts/firstconfig.Makefile                     | 2 +-
>>  scripts/provision.Makefile                       | 2 +-
>>  10 files changed, 26 insertions(+), 8 deletions(-)
>>
>> diff --git a/playbooks/devconfig.yml b/playbooks/devconfig.yml
>> index cebaa87a7228..f0ca7d767efe 100644
>> --- a/playbooks/devconfig.yml
>> +++ b/playbooks/devconfig.yml
>> @@ -1,6 +1,6 @@
>>  ---
>>  - name: Configure developer environment on target systems
>> -  hosts: baseline:dev
>> +  hosts: baseline:dev:service
>>    gather_facts: false
>>    roles:
>>      - role: devconfig
>> diff --git a/playbooks/roles/gen_hosts/templates/blktests.j2 b/playbooks/roles/gen_hosts/templates/blktests.j2
>> index 5bf50acbddac..6aa434c3383c 100644
>> --- a/playbooks/roles/gen_hosts/templates/blktests.j2
>> +++ b/playbooks/roles/gen_hosts/templates/blktests.j2
>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endif %}
>>  [dev:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/playbooks/roles/gen_hosts/templates/cxl.j2 b/playbooks/roles/gen_hosts/templates/cxl.j2
>> index 64bec4561b6c..cab7da246b11 100644
>> --- a/playbooks/roles/gen_hosts/templates/cxl.j2
>> +++ b/playbooks/roles/gen_hosts/templates/cxl.j2
>> @@ -16,3 +16,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endif %}
>>  [dev:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/playbooks/roles/gen_hosts/templates/ltp.j2 b/playbooks/roles/gen_hosts/templates/ltp.j2
>> index 828fd6362718..6946b3f3593b 100644
>> --- a/playbooks/roles/gen_hosts/templates/ltp.j2
>> +++ b/playbooks/roles/gen_hosts/templates/ltp.j2
>> @@ -24,3 +24,7 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endif %}
>>  [dev:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/playbooks/roles/gen_hosts/templates/mmtests.j2 b/playbooks/roles/gen_hosts/templates/mmtests.j2
>> index 1252fe876e6c..8df4a3c31213 100644
>> --- a/playbooks/roles/gen_hosts/templates/mmtests.j2
>> +++ b/playbooks/roles/gen_hosts/templates/mmtests.j2
>> @@ -31,3 +31,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endfor %}
>>  [mmtests:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/playbooks/roles/gen_hosts/templates/selftests.j2 b/playbooks/roles/gen_hosts/templates/selftests.j2
>> index e505bbf194b4..af0d6b341048 100644
>> --- a/playbooks/roles/gen_hosts/templates/selftests.j2
>> +++ b/playbooks/roles/gen_hosts/templates/selftests.j2
>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endif %}
>>  [dev:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/playbooks/roles/gen_hosts/templates/sysbench.j2 b/playbooks/roles/gen_hosts/templates/sysbench.j2
>> index 107eebc5fbce..e30773c1b84a 100644
>> --- a/playbooks/roles/gen_hosts/templates/sysbench.j2
>> +++ b/playbooks/roles/gen_hosts/templates/sysbench.j2
>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>  {% endif %}
>>  [dev:vars]
>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> +[service]
>> +[service:vars]
>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>> diff --git a/scripts/devconfig.Makefile b/scripts/devconfig.Makefile
>> index 146c70f78897..fedf1f2ca21f 100644
>> --- a/scripts/devconfig.Makefile
>> +++ b/scripts/devconfig.Makefile
>> @@ -57,11 +57,10 @@ extend-extra-args-devconfig:
>>  PHONY += devconfig
>>  devconfig: $(KDEVOPS_NODES)
>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>> -		--limit 'baseline:dev:nfsd' \
>> +		--limit 'baseline:dev:service' \
> 
> You don't need --limit at all (I missed this cleanup!). The playbook already has
> hosts: baseline:dev:service.
> 
>>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
>>  		--extra-vars="$(BOOTLINUX_ARGS)" \
>> -		--extra-vars '{ kdevops_cli_install: True }' \
>> -		$(LIMIT_HOSTS)
>> +		--extra-vars '{ kdevops_cli_install: True }'
> 
> Any reason to remove LIMIT_HOSTS support from here? Maybe not useful for these
> targets?

Agreed, if we are simply removing the "--limit" option then I suppose
the $LIMIT_HOSTS is no longer redundant and should stay.


>>  devconfig-generic-help-menu:
>>  	@echo "devconfig          - Ensures generic system setup and is up to date"
>> @@ -72,9 +71,9 @@ ifeq (y,$(CONFIG_SYSCTL_TUNING))
>>  PHONY += sysctl-tunings
>>  sysctl-tunings: $(KDEVOPS_NODES)
>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>> -		--limit 'baseline:dev:nfsd' \
>> +		--limit 'baseline:dev:service' \
> 
> Same here. Not needed.
> 
>>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
>> -		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS) --tags vars,sysctl
>> +		--extra-vars="$(BOOTLINUX_ARGS)" --tags vars,sysctl
>>  
>>  devconfig-help-menu:
>>  	@echo "Target node configuration options"
>> diff --git a/scripts/firstconfig.Makefile b/scripts/firstconfig.Makefile
>> index 9f8a63a8ff84..5651a07d0b66 100644
>> --- a/scripts/firstconfig.Makefile
>> +++ b/scripts/firstconfig.Makefile
>> @@ -2,7 +2,7 @@
>>  
>>  firstconfig:
>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>> -		--limit 'baseline:dev' \
>> +		--limit 'baseline:dev:service' \
> 
> Same here...
> 
>>  		--extra-vars '{ kdevops_cli_install: True }' \
>>  		--tags vars_simple,firstconfig \
>>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml
>> diff --git a/scripts/provision.Makefile b/scripts/provision.Makefile
>> index f04264f6b9ea..62a5692904c9 100644
>> --- a/scripts/provision.Makefile
>> +++ b/scripts/provision.Makefile
>> @@ -70,7 +70,7 @@ KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG)
>>  $(KDEVOPS_PROVISIONED_DEVCONFIG):
>>  	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
>>  		ansible-playbook $(ANSIBLE_VERBOSE) \
>> -			--limit 'baseline:dev' \
>> +			--limit 'baseline:dev:service' \
> 
> And here (playbook is devconfig.yml)
> 
>>  			$(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
>>  	fi
>>  	$(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG)


-- 
Chuck Lever

Re: [PATCH v2 00/37] Various fixes

[Daniel Gomez]

On 02/09/2025 15.53, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> I've finally gotten most of my test runners moved to cloud providers
> so I'm able to rebase my version of kdevops on
> 
>   linux-kdevops/kdevops : HEAD
> 
> In the process I've found and fixed myriad small issues. One or two
> of these might need some attention and additional work. Please have
> a close look -- they are all short subjects.

Thanks for all the fixes!

> 
> Changes since v1:
> - Add R-b tags
> - Completed the patch that adds new GCE machine sizes
> - Added 11 new fixes on the end
> 
> Daniel, this series includes the patches that change the Ansible
> config and inventory pathname behavior. It would be most convenient

Let me double check if we have a conflict with the sandbox concept. But I don't
think so. The part missing for sandboxing is just decoupling TOPDIR for kdevops
sources and for kdevops artifacts (.config, extra_vars.yaml, ansible.cfg, etc).

> for me if these were merged now and then you can do your sandbox
> work on that, but if you would like me to hold off, I can drop them
> from this series.

That's okay with me. I'll double check tomorrow these patches to see if there's
anything.

Re: [PATCH v2 22/37] gen_hosts: Add [service] group to the builder host template

[Daniel Gomez]

On 02/09/2025 21.47, Chuck Lever wrote:
> On 9/2/25 3:43 PM, Daniel Gomez wrote:
>> On 02/09/2025 15.54, Chuck Lever wrote:
>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>
>>> Ensure that the [service] group is defined in the builder
>>> node's host template, to quash some noisy warnings during
>>> bringup.
>>>
>>> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
>>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>>
>> I wonder if we should apply this to other templates too?
>>
>> We have service group declared for fio-tests, fstests, gitr, hosts, nfstest
>> and pynfs. I'll try to remember when running other workflows!
>>
>> Reviewed-by: Daniel Gomez <da.gomez@samsung.com>
>>
> 
> I didn't want to just scatter-shot this change, was waiting to see
> actual warnings come up. But I think it's likely that the builder
> template is not the only one that might need [service].

Patch 33 actually adds the service group to the remaining templates. I just saw
 it later :)

Nit: I think both can be squashed.

> 
> 

Re: [PATCH v2 23/37] guestfs: Don't run the 'destroy' tags on the controller

[Daniel Gomez]

On 02/09/2025 21.59, Chuck Lever wrote:
> On 9/2/25 3:54 PM, Daniel Gomez wrote:
>> On 02/09/2025 15.54, Chuck Lever wrote:
>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>
>>> "make destroy" was skipping some important file removals, and
>>> removing other files that do not exist.
>>>
>>> Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
>>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>>> ---
>>>  scripts/guestfs.Makefile | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile
>>> index f6c350a43f27..953c61493a88 100644
>>> --- a/scripts/guestfs.Makefile
>>> +++ b/scripts/guestfs.Makefile
>>> @@ -96,6 +96,7 @@ PHONY += status_guestfs
>>>  
>>>  destroy_guestfs:
>>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>>> +		--limit 'baseline:dev:service' \
>>
>> Do we need to be explicit here?
>>
>> The guestfs.yml playbook already defines hosts: all. Not sure if I understand
>> the reason why things were skipped. Can you elaborate?
> 
> Before, the "destroy" inventory would have generated these tasks:
> 
> host1 -> localhost
> host2 -> localhost
> host3 -> localhost
> 
> But [all] now includes localhost. So we get:
> 
> host1 -> localhost
> host2 -> localhost
> host3 -> localhost
> localhost -> localhost
> 
> It's that last item that is unwanted.

Got it. I think I was confused by the commit message but makes sense now.

Reviewed-by: Daniel Gomez <da.gomez@samsung.com>

> 
> 
>>> 		playbooks/guestfs.yml \
>>>  		--extra-vars=@./extra_vars.yaml \
>>>  		--tags destroy
> 
> 

Re: [PATCH v2 33/37] devconfig: include the "service" group in the playbooks hosts list

[Daniel Gomez]

On 02/09/2025 22.30, Chuck Lever wrote:
> On 9/2/25 4:27 PM, Daniel Gomez wrote:
>> On 02/09/2025 15.54, Chuck Lever wrote:
>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>
>>> Since commit 1cf0800c9ffc ("gen_hosts: templates: include localhost
>>> in the all group"), the devconfig playbook is no longer updating the
>>> nfsd service host.
>>>
>>> 1. Actually all the service hosts need to be updated by devconfig,
>>>    so ensure devconfig runs against the service group.
>>>
>>> 2. Since devconfig can run against most workflows, add blank
>>>    [service] groups to any inventory template that doesn't already
>>>    have it.
>>>
>>> Fixes: 1cf0800c9ffc ("gen_hosts: templates: include localhost in the all group")
>>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>>> ---
>>>  playbooks/devconfig.yml                          | 2 +-
>>>  playbooks/roles/gen_hosts/templates/blktests.j2  | 3 +++
>>>  playbooks/roles/gen_hosts/templates/cxl.j2       | 3 +++
>>>  playbooks/roles/gen_hosts/templates/ltp.j2       | 4 ++++
>>>  playbooks/roles/gen_hosts/templates/mmtests.j2   | 3 +++
>>>  playbooks/roles/gen_hosts/templates/selftests.j2 | 3 +++
>>>  playbooks/roles/gen_hosts/templates/sysbench.j2  | 3 +++
>>>  scripts/devconfig.Makefile                       | 9 ++++-----
>>>  scripts/firstconfig.Makefile                     | 2 +-
>>>  scripts/provision.Makefile                       | 2 +-
>>>  10 files changed, 26 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/playbooks/devconfig.yml b/playbooks/devconfig.yml
>>> index cebaa87a7228..f0ca7d767efe 100644
>>> --- a/playbooks/devconfig.yml
>>> +++ b/playbooks/devconfig.yml
>>> @@ -1,6 +1,6 @@
>>>  ---
>>>  - name: Configure developer environment on target systems
>>> -  hosts: baseline:dev
>>> +  hosts: baseline:dev:service
>>>    gather_facts: false
>>>    roles:
>>>      - role: devconfig
>>> diff --git a/playbooks/roles/gen_hosts/templates/blktests.j2 b/playbooks/roles/gen_hosts/templates/blktests.j2
>>> index 5bf50acbddac..6aa434c3383c 100644
>>> --- a/playbooks/roles/gen_hosts/templates/blktests.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/blktests.j2
>>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endif %}
>>>  [dev:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/playbooks/roles/gen_hosts/templates/cxl.j2 b/playbooks/roles/gen_hosts/templates/cxl.j2
>>> index 64bec4561b6c..cab7da246b11 100644
>>> --- a/playbooks/roles/gen_hosts/templates/cxl.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/cxl.j2
>>> @@ -16,3 +16,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endif %}
>>>  [dev:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/playbooks/roles/gen_hosts/templates/ltp.j2 b/playbooks/roles/gen_hosts/templates/ltp.j2
>>> index 828fd6362718..6946b3f3593b 100644
>>> --- a/playbooks/roles/gen_hosts/templates/ltp.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/ltp.j2
>>> @@ -24,3 +24,7 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endif %}
>>>  [dev:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/playbooks/roles/gen_hosts/templates/mmtests.j2 b/playbooks/roles/gen_hosts/templates/mmtests.j2
>>> index 1252fe876e6c..8df4a3c31213 100644
>>> --- a/playbooks/roles/gen_hosts/templates/mmtests.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/mmtests.j2
>>> @@ -31,3 +31,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endfor %}
>>>  [mmtests:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/playbooks/roles/gen_hosts/templates/selftests.j2 b/playbooks/roles/gen_hosts/templates/selftests.j2
>>> index e505bbf194b4..af0d6b341048 100644
>>> --- a/playbooks/roles/gen_hosts/templates/selftests.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/selftests.j2
>>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endif %}
>>>  [dev:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/playbooks/roles/gen_hosts/templates/sysbench.j2 b/playbooks/roles/gen_hosts/templates/sysbench.j2
>>> index 107eebc5fbce..e30773c1b84a 100644
>>> --- a/playbooks/roles/gen_hosts/templates/sysbench.j2
>>> +++ b/playbooks/roles/gen_hosts/templates/sysbench.j2
>>> @@ -22,3 +22,6 @@ ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>>  {% endif %}
>>>  [dev:vars]
>>>  ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> +[service]
>>> +[service:vars]
>>> +ansible_python_interpreter =  "{{ kdevops_python_interpreter }}"
>>> diff --git a/scripts/devconfig.Makefile b/scripts/devconfig.Makefile
>>> index 146c70f78897..fedf1f2ca21f 100644
>>> --- a/scripts/devconfig.Makefile
>>> +++ b/scripts/devconfig.Makefile
>>> @@ -57,11 +57,10 @@ extend-extra-args-devconfig:
>>>  PHONY += devconfig
>>>  devconfig: $(KDEVOPS_NODES)
>>>  	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
>>> -		--limit 'baseline:dev:nfsd' \
>>> +		--limit 'baseline:dev:service' \
>>
>> You don't need --limit at all (I missed this cleanup!). The playbook already has
>> hosts: baseline:dev:service.
>>
>>>  		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml \
>>>  		--extra-vars="$(BOOTLINUX_ARGS)" \
>>> -		--extra-vars '{ kdevops_cli_install: True }' \
>>> -		$(LIMIT_HOSTS)
>>> +		--extra-vars '{ kdevops_cli_install: True }'
>>
>> Any reason to remove LIMIT_HOSTS support from here? Maybe not useful for these
>> targets?
> 
> Agreed, if we are simply removing the "--limit" option then I suppose
> the $LIMIT_HOSTS is no longer redundant and should stay.

Even if we keep the first --limit argument, it can be combined with LIMIT_HOSTS.
Ansible will take the last argument which may be more restrictive and user
controlled.