[RFC PATCH v1] Add an Ansible requirements.yml file

[RFC PATCH v1] Add an Ansible requirements.yml file

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
module") introduced the use of the cloud.terraform module, and
commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
with an Ansible playbook") introduced the use of the
community.libvirt module. It would be friendly if kdevops could
pull in the Ansible modules it needs transparently.

The requirements.yml file is a manifest of Ansible collections that
the project needs to run. Installation of these collections is made
automatic by adding:

  ansible-galaxy install -r requirements.yml

to the "make ansible_cfg" step. This mechanism can keep cached
versions of collections up to date, and can also constrain a
cached collection to a specific version, if that's needed.

The initial file contains requirements I could find easily, and
should be updated over time as new collection dependencies are
introduced.

See also:

https://docs.ansible.com/ansible/latest/user_guide/collections_using.html

Question: Can we assume that the Ansible controller has internet
access all the time?

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 requirements.yml         | 8 ++++++++
 scripts/ansible.Makefile | 5 +++++
 2 files changed, 13 insertions(+)
 create mode 100644 requirements.yml

diff --git a/requirements.yml b/requirements.yml
new file mode 100644
index 000000000000..a22691d3f86b
--- /dev/null
+++ b/requirements.yml
@@ -0,0 +1,8 @@
+---
+collections:
+  - name: ansible.posix
+  - name: ansible.utils
+  - name: cloud.terraform
+  - name: community.docker
+  - name: community.general
+  - name: community.libvirt
diff --git a/scripts/ansible.Makefile b/scripts/ansible.Makefile
index b0d2a8c12ee7..4efcf3e93b0e 100644
--- a/scripts/ansible.Makefile
+++ b/scripts/ansible.Makefile
@@ -2,3 +2,8 @@
 
 AV ?= 0
 export ANSIBLE_VERBOSE := $(shell scripts/validate_av.py --av "$(AV)")
+
+ansible-requirements:
+	@$(Q)ansible-galaxy install -r requirements.yml
+PHONY += ansible-requirements
+DEFAULT_DEPS += ansible-requirements
-- 
2.50.0

Re: [RFC PATCH v1] Add an Ansible requirements.yml file

[Luis Chamberlain]

On Sun, Aug 24, 2025 at 12:23:05PM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
> module") introduced the use of the cloud.terraform module, and
> commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
> with an Ansible playbook") introduced the use of the
> community.libvirt module. It would be friendly if kdevops could
> pull in the Ansible modules it needs transparently.
> 
> The requirements.yml file is a manifest of Ansible collections that
> the project needs to run. Installation of these collections is made
> automatic by adding:
> 
>   ansible-galaxy install -r requirements.yml
> 
> to the "make ansible_cfg" step. This mechanism can keep cached
> versions of collections up to date, and can also constrain a
> cached collection to a specific version, if that's needed.
> 
> The initial file contains requirements I could find easily, and
> should be updated over time as new collection dependencies are
> introduced.
> 
> See also:
> 
> https://docs.ansible.com/ansible/latest/user_guide/collections_using.html
> 
> Question: Can we assume that the Ansible controller has internet
> access all the time?

That's not really an issue at all, but what I learned using galaxy stuff
was if we required *our* own stuff upstream on galaxy it was not worth
it.

Given you're only using external stuff, then that seems sensible to me.
But now given our ongoing dialog over how to ease the pain to make it
easy to use kdevops for new folks, how would this be handled? Are we
OK with the request to phone home out?

  Luis

Re: [RFC PATCH v1] Add an Ansible requirements.yml file

[Daniel Gomez]

On 25/08/2025 10.38, Luis Chamberlain wrote:
> On Sun, Aug 24, 2025 at 12:23:05PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
>> module") introduced the use of the cloud.terraform module, and
>> commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
>> with an Ansible playbook") introduced the use of the
>> community.libvirt module. It would be friendly if kdevops could
>> pull in the Ansible modules it needs transparently.
>>
>> The requirements.yml file is a manifest of Ansible collections that
>> the project needs to run. Installation of these collections is made
>> automatic by adding:
>>
>>   ansible-galaxy install -r requirements.yml
>>
>> to the "make ansible_cfg" step. This mechanism can keep cached
>> versions of collections up to date, and can also constrain a
>> cached collection to a specific version, if that's needed.
>>
>> The initial file contains requirements I could find easily, and
>> should be updated over time as new collection dependencies are
>> introduced.
>>
>> See also:
>>
>> https://docs.ansible.com/ansible/latest/user_guide/collections_using.html
>>
>> Question: Can we assume that the Ansible controller has internet
>> access all the time?
> 
> That's not really an issue at all, but what I learned using galaxy stuff
> was if we required *our* own stuff upstream on galaxy it was not worth
> it.
> 
> Given you're only using external stuff, then that seems sensible to me.
> But now given our ongoing dialog over how to ease the pain to make it
> easy to use kdevops for new folks, how would this be handled? Are we
> OK with the request to phone home out?

I wonder if this is a non-Debian issue?

I see I have all these installed by the Debian ansible package:

Here the ansible package sources:
https://packages.debian.org/forky/all/ansible/filelist

ansible-galaxy collection list

# /usr/lib/python3/dist-packages/ansible/_internal/ansible_collections
Collection                               Version
---------------------------------------- -------
ansible._protomatter                     2.19.0

# /usr/lib/python3/dist-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               10.1.0
ansible.netcommon                        8.0.1
ansible.posix                            2.1.0
ansible.utils                            6.0.0
ansible.windows                          3.2.0
arista.eos                               11.0.1
awx.awx                                  24.6.1
azure.azcollection                       3.7.0
check_point.mgmt                         6.4.1
chocolatey.chocolatey                    1.5.3
cisco.aci                                2.12.0
cisco.dnac                               6.37.0
cisco.intersight                         2.2.0
cisco.ios                                10.1.1
cisco.iosxr                              11.1.0
cisco.meraki                             2.21.4
cisco.mso                                2.11.0
cisco.nxos                               10.2.0
cisco.ucs                                1.16.0
cloud.common                             5.0.0
cloudscale_ch.cloud                      2.5.2
community.aws                            10.0.0
community.ciscosmb                       1.0.11
community.crypto                         3.0.3
community.digitalocean                   1.27.0
community.dns                            3.2.7
community.docker                         4.7.0
community.general                        11.1.2
community.grafana                        2.3.0
community.hashi_vault                    7.0.0
community.hrobot                         2.5.0
community.library_inventory_filtering_v1 1.1.1
community.libvirt                        2.0.0
community.mongodb                        1.7.10
community.mysql                          3.15.0
community.okd                            5.0.0
community.postgresql                     4.1.0
community.proxmox                        1.2.0
community.proxysql                       1.6.0
community.rabbitmq                       1.6.0
community.routeros                       3.8.1
community.sap_libs                       1.4.2
community.sops                           2.2.1
community.vmware                         5.7.1
community.windows                        3.0.1
community.zabbix                         4.1.0
containers.podman                        1.17.0
cyberark.conjur                          1.3.6
cyberark.pas                             1.0.35
dellemc.enterprise_sonic                 3.0.0
dellemc.openmanage                       9.12.3
dellemc.powerflex                        2.6.1
dellemc.unity                            2.0.0
f5networks.f5_modules                    1.37.1
fortinet.fortimanager                    2.10.0
fortinet.fortios                         2.4.0
google.cloud                             1.7.0
grafana.grafana                          6.0.3
hetzner.hcloud                           5.1.0
hitachivantara.vspone_block              4.0.1
ibm.qradar                               4.0.0
ibm.storage_virtualize                   2.7.4
ieisystem.inmanage                       3.0.0
infinidat.infinibox                      1.6.3
infoblox.nios_modules                    1.8.0
inspur.ispim                             2.2.3
junipernetworks.junos                    10.0.0
kaytus.ksmanage                          2.0.0
kubernetes.core                          6.0.0
kubevirt.core                            2.2.3
lowlydba.sqlserver                       2.6.1
microsoft.ad                             1.9.2
microsoft.iis                            1.0.3
netapp.cloudmanager                      21.24.0
netapp.ontap                             23.1.0
netapp.storagegrid                       21.15.0
netapp_eseries.santricity                1.4.1
netbox.netbox                            3.21.0
ngine_io.cloudstack                      2.5.0
openstack.cloud                          2.4.1
ovirt.ovirt                              3.2.1
purestorage.flasharray                   1.36.0
purestorage.flashblade                   1.20.0
splunk.es                                4.0.0
telekom_mms.icinga_director              2.4.0
theforeman.foreman                       5.4.0
vmware.vmware                            2.2.0
vmware.vmware_rest                       4.8.1
vultr.cloud                              1.13.0
vyos.vyos                                6.0.0
wti.remote                               1.0.10

dpkg -l ansible
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version          Architecture Description
+++-==============-================-============-===============================================================
ii  ansible        12.0.0~b3+dfsg-1 all          Configuration management, deployment, and task execution system

dpkg -S /usr/lib/python3/dist-packages/ansible_collections/community
ansible: /usr/lib/python3/dist-packages/ansible_collections/community

Re: [RFC PATCH v1] Add an Ansible requirements.yml file

[Chuck Lever]

On 8/25/25 4:38 AM, Luis Chamberlain wrote:
> On Sun, Aug 24, 2025 at 12:23:05PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Commit b90d89d27659 ("Switch to the cloud.terraform.terraform
>> module") introduced the use of the cloud.terraform module, and
>> commit 7ccb64834eeb ("guestfs: Replace scripts/destroy_guestfs.sh
>> with an Ansible playbook") introduced the use of the
>> community.libvirt module. It would be friendly if kdevops could
>> pull in the Ansible modules it needs transparently.
>>
>> The requirements.yml file is a manifest of Ansible collections that
>> the project needs to run. Installation of these collections is made
>> automatic by adding:
>>
>>   ansible-galaxy install -r requirements.yml
>>
>> to the "make ansible_cfg" step. This mechanism can keep cached
>> versions of collections up to date, and can also constrain a
>> cached collection to a specific version, if that's needed.
>>
>> The initial file contains requirements I could find easily, and
>> should be updated over time as new collection dependencies are
>> introduced.
>>
>> See also:
>>
>> https://docs.ansible.com/ansible/latest/user_guide/collections_using.html
>>
>> Question: Can we assume that the Ansible controller has internet
>> access all the time?
> 
> That's not really an issue at all, but what I learned using galaxy stuff
> was if we required *our* own stuff upstream on galaxy it was not worth
> it.
> 
> Given you're only using external stuff, then that seems sensible to me.

Yes, only collections that are already in the Ansible galaxy.

Any missing collections are installed into the Ansible user's home
directory, so "sudo" is not needed.

Also, this mechanism seems smart enough to deal with collections that
are already installed in shared locations. It skips re-installing.

> But now given our ongoing dialog over how to ease the pain to make it
> easy to use kdevops for new folks, how would this be handled? Are we
> OK with the request to phone home out?

Exactly; phoning home is OK for my own use cases, but I can't answer
that for anyone else's.


-- 
Chuck Lever