[RFC PATCH 0/2] Set TOPDIR at run time

[RFC PATCH 0/2] Set TOPDIR at run time

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

A few recent changes have been difficult to integrate into my
current CI infrastructure. Here are a couple of ideas on how to
stretch in a couple of areas to help my particular arrangement.

Note that I would need to handle the inventory file location in a
similar way: add a boolean Kconfig option to use a sensible default
location rather than baking a fixed pathname into the kdevops
.config. So, work in progress here.

Thoughts and alternatives are welcome. Even the Internet standard
response ("you're doing it wrong") is OK.

Chuck Lever (2):
  Set TOPDIR_PATH and generate its sha256sum at run time
  ansible.cfg: generate an ansible.cfg file in TOPDIR

 Kconfig                      | 15 ---------------
 Makefile                     | 12 ++++++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 scripts/gen-hosts.Makefile   |  1 +
 scripts/ssh.Makefile         |  4 ++--
 5 files changed, 27 insertions(+), 17 deletions(-)

-- 
2.51.0

[RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a
uniquifier to the name of the ssh config to avoid multiple kdevops
runs clobbering each other's ssh config entries. It uses the TOPDIR
pathname set in the current .config to benefit from the nice
convenience of generating an Ansible extra_var using "output yaml".

However, if the same user generates all her .configs (that will be
run by other users under automation) while in the same directory
(say, if they are different branches in the same git repository),
the TOPDIR pathname and thus the sha256sum of that pathname will be
exactly the same in all of those .config files, allowing the path
names of the ephemeral ssh configs to collide.

The TOPDIR and its checksum need to be generated not by "make
menuconfig" but rather by "make" in order that .config files (as
saved in defconfigs/) can be truly portable.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Kconfig                    | 15 ---------------
 Makefile                   |  4 ++++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  4 ++--
 4 files changed, 7 insertions(+), 17 deletions(-)

diff --git a/Kconfig b/Kconfig
index 3f2bc8cf019a..948befbce4de 100644
--- a/Kconfig
+++ b/Kconfig
@@ -5,21 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
diff --git a/Makefile b/Makefile
index 8755577e4d85..b6fdbe67fc07 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_HAS_SHA256SUM=y
+export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
@@ -82,6 +84,8 @@ INCLUDES = -I include/
 CFLAGS += $(INCLUDES)
 
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
+ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
+ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
diff --git a/scripts/gen-hosts.Makefile b/scripts/gen-hosts.Makefile
index 0d1e0cd77c2e..c809b1da18e5 100644
--- a/scripts/gen-hosts.Makefile
+++ b/scripts/gen-hosts.Makefile
@@ -3,6 +3,7 @@
 KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR)
 KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates
 
+GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'
 GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)'
 
 # Relative path so that ansible can work with it
diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
index d18a03e094ee..d2a583f45da2 100644
--- a/scripts/ssh.Makefile
+++ b/scripts/ssh.Makefile
@@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
 
-ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
-SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
+ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM))
+SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM)
 else
 SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
 endif
-- 
2.51.0

[RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Chuck Lever]

From: Chuck Lever <chuck.lever@oracle.com>

I need an ansible.cfg that is generated by the Kconfig menu, but
whose pathname is selected dynamically based on where kdevops is
being run rather than having that path baked into the .config.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/Makefile b/Makefile
index b6fdbe67fc07..b014faa5632e 100644
--- a/Makefile
+++ b/Makefile
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
index dbd189f5d19e..a44f760912cf 100644
--- a/kconfigs/Kconfig.ansible_cfg
+++ b/kconfigs/Kconfig.ansible_cfg
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
+	default N
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.
+
+if !ANSIBLE_CFG_FILE_DEFAULT
+
 config ANSIBLE_CFG_FILE
 	string "Ansible configuration file"
 	output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
 
+endif # ANSIBLE_CFG_FILE_DEFAULT
+
 menu "Ansible Callback Plugin Configuration"
 choice
 	prompt "Ansible Callback Plugin"
-- 
2.51.0

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

[Luis Chamberlain]

On Thu, Aug 28, 2025 at 04:28:26PM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a
> uniquifier to the name of the ssh config to avoid multiple kdevops
> runs clobbering each other's ssh config entries. It uses the TOPDIR
> pathname set in the current .config to benefit from the nice
> convenience of generating an Ansible extra_var using "output yaml".
> 
> However, if the same user generates all her .configs (that will be
> run by other users under automation) while in the same directory
> (say, if they are different branches in the same git repository),
> the TOPDIR pathname and thus the sha256sum of that pathname will be
> exactly the same in all of those .config files, allowing the path
> names of the ephemeral ssh configs to collide.

Both paragraphs above took me quite a bit of time to grasp, I think
it was the parenthesis, as they make me try to switch contexts. So
I think the explanation of the issue can be simplified. I *think*
that the issue your are describing is essentially that we currently
cannot support sharing .config to other users because TOPDIR_PATH
expects the same path to be used. Would that be a summary of the
issue?

> The TOPDIR and its checksum need to be generated not by "make
> menuconfig" but rather by "make" in order that .config files (as
> saved in defconfigs/) can be truly portable.

This is nice.

> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  Kconfig                    | 15 ---------------
>  Makefile                   |  4 ++++
>  scripts/gen-hosts.Makefile |  1 +
>  scripts/ssh.Makefile       |  4 ++--
>  4 files changed, 7 insertions(+), 17 deletions(-)
> 
> diff --git a/Kconfig b/Kconfig
> index 3f2bc8cf019a..948befbce4de 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -5,21 +5,6 @@
>  
>  mainmenu "$(PROJECT) $(PROJECTRELEASE)"
>  
> -config TOPDIR_PATH
> -	string
> -	output yaml
> -	default $(shell, echo $TOPDIR_PATH)
> -
> -config TOPDIR_PATH_HAS_SHA256SUM
> -	bool
> -	output yaml
> -	default y
> -
> -config TOPDIR_PATH_SHA256SUM
> -	string
> -	output yaml
> -	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
> -
>  config HAVE_KDEVOPS_CUSTOM_DEFAULTS
>  	bool
>  	default n
> diff --git a/Makefile b/Makefile
> index 8755577e4d85..b6fdbe67fc07 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
>  export PYTHONUNBUFFERED=1
>  export TOPDIR=./
>  export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
> +export TOPDIR_PATH_HAS_SHA256SUM=y

We added this a long time ago, to remain compatible with users who
may git pull to the latest kdevops which didn't yet have the symbol
TOPDIR_PATH_HAS_SHA256SUM.

So I think a prep patch to just remove TOPDIR_PATH_HAS_SHA256SUM now
make sense.

Otherwise this is looking nice.

  Luis

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Luis Chamberlain]

On Thu, Aug 28, 2025 at 04:28:27PM -0400, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> I need an ansible.cfg that is generated by the Kconfig menu, but
> whose pathname is selected dynamically based on where kdevops is
> being run rather than having that path baked into the .config.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  Makefile                     |  8 ++++++++
>  kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index b6fdbe67fc07..b014faa5632e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
>  export KDEVOPS_NODES_TEMPLATE :=
>  export KDEVOPS_MRPROPER :=
>  
> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
> +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
> +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
> +else
>  ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
>  ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
>  export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>  endif
> +endif
>  ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
>  
>  KDEVOPS_INSTALL_TARGETS :=
> @@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
>  ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
>  ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
>  ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
> +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
> +endif
>  
>  export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
>  
> diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
> index dbd189f5d19e..a44f760912cf 100644
> --- a/kconfigs/Kconfig.ansible_cfg
> +++ b/kconfigs/Kconfig.ansible_cfg
> @@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
>  	bool
>  	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
>  
> +config ANSIBLE_CFG_FILE_DEFAULT
> +	bool "Use a sensible default location for the ansible.cfg file"

> +	default N

defaults are always n, so this line can be removed.

> +	help
> +	  When this option is set to Y, kdevops creates an ansible.cfg
> +	  in a sensible default location, using the settings defined
> +	  in this Kconfig menu.

Why would we not want this by default?

 Luis

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

[Chuck Lever]

On 8/29/25 7:19 AM, Luis Chamberlain wrote:
> On Thu, Aug 28, 2025 at 04:28:26PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a
>> uniquifier to the name of the ssh config to avoid multiple kdevops
>> runs clobbering each other's ssh config entries. It uses the TOPDIR
>> pathname set in the current .config to benefit from the nice
>> convenience of generating an Ansible extra_var using "output yaml".
>>
>> However, if the same user generates all her .configs (that will be
>> run by other users under automation) while in the same directory
>> (say, if they are different branches in the same git repository),
>> the TOPDIR pathname and thus the sha256sum of that pathname will be
>> exactly the same in all of those .config files, allowing the path
>> names of the ephemeral ssh configs to collide.
> 
> Both paragraphs above took me quite a bit of time to grasp, I think
> it was the parenthesis, as they make me try to switch contexts. So
> I think the explanation of the issue can be simplified. I *think*
> that the issue your are describing is essentially that we currently
> cannot support sharing .config to other users because TOPDIR_PATH
> expects the same path to be used. Would that be a summary of the
> issue?

I struggled with writing it. I wanted to capture my use case as the
rationale for the change. I'll try to integrate your suggestion.


>> The TOPDIR and its checksum need to be generated not by "make
>> menuconfig" but rather by "make" in order that .config files (as
>> saved in defconfigs/) can be truly portable.
> 
> This is nice.
> 
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  Kconfig                    | 15 ---------------
>>  Makefile                   |  4 ++++
>>  scripts/gen-hosts.Makefile |  1 +
>>  scripts/ssh.Makefile       |  4 ++--
>>  4 files changed, 7 insertions(+), 17 deletions(-)
>>
>> diff --git a/Kconfig b/Kconfig
>> index 3f2bc8cf019a..948befbce4de 100644
>> --- a/Kconfig
>> +++ b/Kconfig
>> @@ -5,21 +5,6 @@
>>  
>>  mainmenu "$(PROJECT) $(PROJECTRELEASE)"
>>  
>> -config TOPDIR_PATH
>> -	string
>> -	output yaml
>> -	default $(shell, echo $TOPDIR_PATH)
>> -
>> -config TOPDIR_PATH_HAS_SHA256SUM
>> -	bool
>> -	output yaml
>> -	default y
>> -
>> -config TOPDIR_PATH_SHA256SUM
>> -	string
>> -	output yaml
>> -	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
>> -
>>  config HAVE_KDEVOPS_CUSTOM_DEFAULTS
>>  	bool
>>  	default n
>> diff --git a/Makefile b/Makefile
>> index 8755577e4d85..b6fdbe67fc07 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
>>  export PYTHONUNBUFFERED=1
>>  export TOPDIR=./
>>  export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
>> +export TOPDIR_PATH_HAS_SHA256SUM=y
> 
> We added this a long time ago, to remain compatible with users who
> may git pull to the latest kdevops which didn't yet have the symbol
> TOPDIR_PATH_HAS_SHA256SUM.
> 
> So I think a prep patch to just remove TOPDIR_PATH_HAS_SHA256SUM now
> make sense.

Ha! I had a sentence in the patch description suggesting removal of
HAS_SHA256SUM, but decided to keep things simple and took it out.


> Otherwise this is looking nice.
> 
>   Luis


-- 
Chuck Lever

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Chuck Lever]

On 8/29/25 7:23 AM, Luis Chamberlain wrote:
> On Thu, Aug 28, 2025 at 04:28:27PM -0400, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> I need an ansible.cfg that is generated by the Kconfig menu, but
>> whose pathname is selected dynamically based on where kdevops is
>> being run rather than having that path baked into the .config.
>>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  Makefile                     |  8 ++++++++
>>  kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
>>  2 files changed, 20 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index b6fdbe67fc07..b014faa5632e 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
>>  export KDEVOPS_NODES_TEMPLATE :=
>>  export KDEVOPS_MRPROPER :=
>>  
>> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
>> +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
>> +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>> +else
>>  ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
>>  ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
>>  export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>>  endif
>> +endif
>>  ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
>>  
>>  KDEVOPS_INSTALL_TARGETS :=
>> @@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
>>  ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
>>  ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
>>  ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
>> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
>> +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
>> +endif
>>  
>>  export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
>>  
>> diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
>> index dbd189f5d19e..a44f760912cf 100644
>> --- a/kconfigs/Kconfig.ansible_cfg
>> +++ b/kconfigs/Kconfig.ansible_cfg
>> @@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
>>  	bool
>>  	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
>>  
>> +config ANSIBLE_CFG_FILE_DEFAULT
>> +	bool "Use a sensible default location for the ansible.cfg file"
> 
>> +	default N
> 
> defaults are always n, so this line can be removed.
> 
>> +	help
>> +	  When this option is set to Y, kdevops creates an ansible.cfg
>> +	  in a sensible default location, using the settings defined
>> +	  in this Kconfig menu.
> 
> Why would we not want this by default?

Because N matches the current behavior.

But I would prefer Y, actually.


-- 
Chuck Lever

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Luis Chamberlain]

On Fri, Aug 29, 2025 at 09:39:41AM -0400, Chuck Lever wrote:
> On 8/29/25 7:23 AM, Luis Chamberlain wrote:
> > On Thu, Aug 28, 2025 at 04:28:27PM -0400, Chuck Lever wrote:
> >> From: Chuck Lever <chuck.lever@oracle.com>
> >> +config ANSIBLE_CFG_FILE_DEFAULT
> >> +	bool "Use a sensible default location for the ansible.cfg file"
> > 
> >> +	default N
> > 
> > defaults are always n, so this line can be removed.
> > 
> >> +	help
> >> +	  When this option is set to Y, kdevops creates an ansible.cfg
> >> +	  in a sensible default location, using the settings defined
> >> +	  in this Kconfig menu.
> > 
> > Why would we not want this by default?
> 
> Because N matches the current behavior.
> 
> But I would prefer Y, actually.

I think its safe to use Y.

 LUis

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

[Daniel Gomez]

On 28/08/2025 22.28, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a
> uniquifier to the name of the ssh config to avoid multiple kdevops
> runs clobbering each other's ssh config entries. It uses the TOPDIR
> pathname set in the current .config to benefit from the nice
> convenience of generating an Ansible extra_var using "output yaml".
> 
> However, if the same user generates all her .configs (that will be
> run by other users under automation) while in the same directory
> (say, if they are different branches in the same git repository),
> the TOPDIR pathname and thus the sha256sum of that pathname will be
> exactly the same in all of those .config files, allowing the path
> names of the ephemeral ssh configs to collide.
> 

IIUC, this aligns very well with the sandboxing effort. Definitely, TOPDIR needs
to point to a sandbox directory.

Thanks for this series!

> The TOPDIR and its checksum need to be generated not by "make
> menuconfig" but rather by "make" in order that .config files (as
> saved in defconfigs/) can be truly portable.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  Kconfig                    | 15 ---------------
>  Makefile                   |  4 ++++
>  scripts/gen-hosts.Makefile |  1 +
>  scripts/ssh.Makefile       |  4 ++--
>  4 files changed, 7 insertions(+), 17 deletions(-)
> 
> diff --git a/Kconfig b/Kconfig
> index 3f2bc8cf019a..948befbce4de 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -5,21 +5,6 @@
>  
>  mainmenu "$(PROJECT) $(PROJECTRELEASE)"
>  
> -config TOPDIR_PATH
> -	string
> -	output yaml
> -	default $(shell, echo $TOPDIR_PATH)
> -
> -config TOPDIR_PATH_HAS_SHA256SUM
> -	bool
> -	output yaml
> -	default y
> -
> -config TOPDIR_PATH_SHA256SUM
> -	string
> -	output yaml
> -	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
> -
>  config HAVE_KDEVOPS_CUSTOM_DEFAULTS
>  	bool
>  	default n
> diff --git a/Makefile b/Makefile
> index 8755577e4d85..b6fdbe67fc07 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
>  export PYTHONUNBUFFERED=1
>  export TOPDIR=./
>  export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
> +export TOPDIR_PATH_HAS_SHA256SUM=y
> +export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
>  include scripts/refs.Makefile
>  
>  KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
> @@ -82,6 +84,8 @@ INCLUDES = -I include/
>  CFLAGS += $(INCLUDES)
>  
>  ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
> +ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
> +ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
>  
>  export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
>  
> diff --git a/scripts/gen-hosts.Makefile b/scripts/gen-hosts.Makefile
> index 0d1e0cd77c2e..c809b1da18e5 100644
> --- a/scripts/gen-hosts.Makefile
> +++ b/scripts/gen-hosts.Makefile
> @@ -3,6 +3,7 @@
>  KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR)
>  KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates
>  
> +GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'

We need to feed these vars as they are variables generated based on the
directory the user is currently running kdevops on, is that correct?

>  GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)'
>  
>  # Relative path so that ansible can work with it
> diff --git a/scripts/ssh.Makefile b/scripts/ssh.Makefile
> index d18a03e094ee..d2a583f45da2 100644
> --- a/scripts/ssh.Makefile
> +++ b/scripts/ssh.Makefile
> @@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
>  SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
>  ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
>  
> -ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
> -SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
> +ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM))
> +SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM)
>  else
>  SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
>  endif

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Daniel Gomez]

On 28/08/2025 22.28, Chuck Lever wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> I need an ansible.cfg that is generated by the Kconfig menu, but
> whose pathname is selected dynamically based on where kdevops is
> being run rather than having that path baked into the .config.

I'm trying to review the previous conversation before these patches as well as
this series. Please, let me know if I'm understanding something wrong:

I think what we need is not to have individual configuration paths (as we
have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox
folder, our TOPDIR, and have our kdevops sandbox directory there. All files
generated and consumed by kdevops should then use that path.

Would this work? See below:

kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0
# Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc
kdevops-tree $ make menuconfig O=path/to/sandbox0
kdevops-tree $ make O=path/to/sandbox0
kdevops-tree $ make bringup O=path/to/sandbox0
...
kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
...
path/to/sandbox0 $ make linux
path/to/sandbox0 $ make selftests
path/to/sandbox0 $ make selftests-xarray
...
path/to/sandbox1 $ make
path/to/sandbox1 $ make bringup
path/to/sandbox1 $ make linux
path/to/sandbox1 $ make fstests
path/to/sandbox1 $ make fstests-baseline

To clarify: instead of letting users override individual file paths, we allow
customization of the topdir directory (with O=) where all files are created.
If O= is not passed, then PWD is assumed. I think it would still be useful to
support customizing individual files, but only relative to TOPDIR. That would
keep the structure cleaner, for example, using subdirectories for Ansible files,
kconfigs, SSH, virsh, and so on.

> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  Makefile                     |  8 ++++++++
>  kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index b6fdbe67fc07..b014faa5632e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
>  export KDEVOPS_NODES_TEMPLATE :=
>  export KDEVOPS_MRPROPER :=
>  
> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
> +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
> +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
> +else

What I'm suggesting above is to always use this new option. And perhaps
keep ANSIBLE_CFG_FILE configurable but inside TOPDIR. For example:
ANSIBLE_CFG_FILE=ansible/ansible.cfg.

>  ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
>  ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
>  export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>  endif
> +endif
>  ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
>  
>  KDEVOPS_INSTALL_TARGETS :=
> @@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
>  ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
>  ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
>  ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
> +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
> +endif
>  
>  export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
>  
> diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
> index dbd189f5d19e..a44f760912cf 100644
> --- a/kconfigs/Kconfig.ansible_cfg
> +++ b/kconfigs/Kconfig.ansible_cfg
> @@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
>  	bool
>  	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
>  
> +config ANSIBLE_CFG_FILE_DEFAULT
> +	bool "Use a sensible default location for the ansible.cfg file"
> +	default N
> +	help
> +	  When this option is set to Y, kdevops creates an ansible.cfg
> +	  in a sensible default location, using the settings defined
> +	  in this Kconfig menu.
> +
> +if !ANSIBLE_CFG_FILE_DEFAULT
> +

I think we don't need this if you agree with the approach above.

>  config ANSIBLE_CFG_FILE
>  	string "Ansible configuration file"
>  	output yaml
> @@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
>  	  For more details, refer to the Ansible documentation:
>  	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
>  
> +endif # ANSIBLE_CFG_FILE_DEFAULT
> +
>  menu "Ansible Callback Plugin Configuration"
>  choice
>  	prompt "Ansible Callback Plugin"

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Chuck Lever]

On 9/1/25 8:00 AM, Daniel Gomez wrote:
> On 28/08/2025 22.28, Chuck Lever wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> I need an ansible.cfg that is generated by the Kconfig menu, but
>> whose pathname is selected dynamically based on where kdevops is
>> being run rather than having that path baked into the .config.
> 
> I'm trying to review the previous conversation before these patches as well as
> this series. Please, let me know if I'm understanding something wrong:
> 
> I think what we need is not to have individual configuration paths (as we
> have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox
> folder, our TOPDIR, and have our kdevops sandbox directory there. All files
> generated and consumed by kdevops should then use that path.
> 
> Would this work?

Well I think that would work for my particular usage, however: If I'm
reading between the lines of the Ansible pathname help texts, it appears
that these settings can specify search paths, not a single file.

In particular, the ANSIBLE_CFG_INVENTORY_FILE string is a comma-
separated list. And, if ANSIBLE_CFG_FILE is an empty string, then there
is a default set of locations /outside of/ the CWD that Ansible searches
for a configuration.

I'm not against your suggestion of specifying a sandbox pathname at all;
that would be a good UX simplification. There are these details, though,
and I don't have any suggestions about that at the moment.


> See below:
> 
> kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0
> # Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc
> kdevops-tree $ make menuconfig O=path/to/sandbox0
> kdevops-tree $ make O=path/to/sandbox0
> kdevops-tree $ make bringup O=path/to/sandbox0
> ...
> kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
> ...
> path/to/sandbox0 $ make linux
> path/to/sandbox0 $ make selftests
> path/to/sandbox0 $ make selftests-xarray
> ...
> path/to/sandbox1 $ make
> path/to/sandbox1 $ make bringup
> path/to/sandbox1 $ make linux
> path/to/sandbox1 $ make fstests
> path/to/sandbox1 $ make fstests-baseline
> 
> To clarify: instead of letting users override individual file paths, we allow
> customization of the topdir directory (with O=) where all files are created.
> If O= is not passed, then PWD is assumed. I think it would still be useful to
> support customizing individual files, but only relative to TOPDIR. That would
> keep the structure cleaner, for example, using subdirectories for Ansible files,
> kconfigs, SSH, virsh, and so on.
> 
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  Makefile                     |  8 ++++++++
>>  kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
>>  2 files changed, 20 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index b6fdbe67fc07..b014faa5632e 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
>>  export KDEVOPS_NODES_TEMPLATE :=
>>  export KDEVOPS_MRPROPER :=
>>  
>> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
>> +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
>> +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>> +else
> 
> What I'm suggesting above is to always use this new option. And perhaps
> keep ANSIBLE_CFG_FILE configurable but inside TOPDIR. For example:
> ANSIBLE_CFG_FILE=ansible/ansible.cfg.
> 
>>  ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
>>  ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
>>  export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
>>  endif
>> +endif
>>  ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
>>  
>>  KDEVOPS_INSTALL_TARGETS :=
>> @@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
>>  ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
>>  ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
>>  ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
>> +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
>> +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
>> +endif
>>  
>>  export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
>>  
>> diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
>> index dbd189f5d19e..a44f760912cf 100644
>> --- a/kconfigs/Kconfig.ansible_cfg
>> +++ b/kconfigs/Kconfig.ansible_cfg
>> @@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
>>  	bool
>>  	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
>>  
>> +config ANSIBLE_CFG_FILE_DEFAULT
>> +	bool "Use a sensible default location for the ansible.cfg file"
>> +	default N
>> +	help
>> +	  When this option is set to Y, kdevops creates an ansible.cfg
>> +	  in a sensible default location, using the settings defined
>> +	  in this Kconfig menu.
>> +
>> +if !ANSIBLE_CFG_FILE_DEFAULT
>> +
> 
> I think we don't need this if you agree with the approach above.
> 
>>  config ANSIBLE_CFG_FILE
>>  	string "Ansible configuration file"
>>  	output yaml
>> @@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
>>  	  For more details, refer to the Ansible documentation:
>>  	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
>>  
>> +endif # ANSIBLE_CFG_FILE_DEFAULT
>> +
>>  menu "Ansible Callback Plugin Configuration"
>>  choice
>>  	prompt "Ansible Callback Plugin"


-- 
Chuck Lever

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

[Daniel Gomez]

On 01/09/2025 17.03, Chuck Lever wrote:
> On 9/1/25 8:00 AM, Daniel Gomez wrote:
>> On 28/08/2025 22.28, Chuck Lever wrote:
>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>
>>> I need an ansible.cfg that is generated by the Kconfig menu, but
>>> whose pathname is selected dynamically based on where kdevops is
>>> being run rather than having that path baked into the .config.
>>
>> I'm trying to review the previous conversation before these patches as well as
>> this series. Please, let me know if I'm understanding something wrong:
>>
>> I think what we need is not to have individual configuration paths (as we
>> have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox
>> folder, our TOPDIR, and have our kdevops sandbox directory there. All files
>> generated and consumed by kdevops should then use that path.
>>
>> Would this work?
> 
> Well I think that would work for my particular usage, however: If I'm
> reading between the lines of the Ansible pathname help texts, it appears
> that these settings can specify search paths, not a single file.
> 
> In particular, the ANSIBLE_CFG_INVENTORY_FILE string is a comma-
> separated list. And, if ANSIBLE_CFG_FILE is an empty string, then there
> is a default set of locations /outside of/ the CWD that Ansible searches
> for a configuration.

That case will never happen, as we will ensure our sandbox var is always
defined (either with O=, or with PWD) and therefore the location of the kdevops
artifacts (.config, ansible.cfg, inventory, etc). That should preserve the
current functionality while also allowing others to generate kdevops artifacts
into separate directories.

> 
> I'm not against your suggestion of specifying a sandbox pathname at all;
> that would be a good UX simplification. There are these details, though,
> and I don't have any suggestions about that at the moment.

Great! And agreed. Even if my assessment above is wrong, I don't see any major
blockers.