[PATCH v2 0/4] guestfs: bringup: add debug mode

[PATCH v2 0/4] guestfs: bringup: add debug mode

[Daniel Gomez]

This has helped me during debug sessions of bringup script. "Resend"
with minor changes.

Adding debug option in bash bringup_guestfs.sh allows to check for
commands executions (virsh, chmod, cp, virt-sysprep, etc) and variables.

Add 3 levels of debug:
- Disabled: This is the current behaviour and is the default.
- set -x (debug mode 0):  Allows to print the executed commands but if
they fail, the script continues.
- set -euxo pipefail (debug mode 1): mode 0 + variables are printed +
script fails if a command does not succeed.

While debugging, I noticed about small bash errors that we ignore, such
as unbound variables. So, this also fixes the issues found while debug
mode 1 was enabled.

---
Changes in v2:
* Rebase on top of latest main branch
* Split long conditional lines into 2 
- Link to v1: https://lore.kernel.org/all/20241015-bringup-guestfs-debug-v1-0-bd74c0c31412@samsung.com/

---
Daniel Gomez (4):
      guestfs: bringup: check if domain exists
      guestfs: bringup: fix unbound variable when debug
      guestfs: bringup: fix user check when debug
      guestfs: bringup: add debug mode

 kconfigs/Kconfig.guestfs   | 21 +++++++++++++++++++++
 scripts/bringup_guestfs.sh | 38 +++++++++++++++++++++++++++-----------
 2 files changed, 48 insertions(+), 11 deletions(-)
---
base-commit: ef549598f2261ac89643d3e7a8ea0f050fc27c5f
change-id: 20241015-bringup-guestfs-debug-775f4dbd98c1

Best regards,
-- 
Daniel Gomez <da.gomez@samsung.com>

[PATCH v2 1/4] guestfs: bringup: check if domain exists

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

virsh domstate fails (exit 1) if domain checked does not exist. This
conflicts when debugging the script with set -euxo pipefile. Replace the
command with virsh list --all and search for the domain in the list.
Also, capture the state to avoid trying to start a domain that is
already running.

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 scripts/bringup_guestfs.sh | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
index 7cafa7ffafb058b31cca4baa7228b7fd4c0b9119..1d377faf387888a4523a838fd292654e5deddd82 100755
--- a/scripts/bringup_guestfs.sh
+++ b/scripts/bringup_guestfs.sh
@@ -316,10 +316,12 @@ do
 	# If the guest is already defined, then just stop what we're doing
 	# and plead to the developer to clean things up.
 	#
-	virsh domstate $name 1>/dev/null 2>&1
-	if [ $? -eq 0 ]; then
-		echo "Domain $name is already defined."
-		virsh start $name
+	if virsh list --all | grep --quiet --word-regexp "$name"; then
+		output_domstate=$(virsh domstate $name 2>/dev/null)
+		echo "Domain $name is already defined. (state: $output_domstate)"
+		if [ "$output_domstate" != "running" ]; then
+			virsh start $name
+		fi
 		exit 0
 	fi
 

-- 
2.49.0

[PATCH v2 2/4] guestfs: bringup: fix unbound variable when debug

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

Fix unbound variable errors. This is a silent error only found while
debugging. A variable may not be set, so check if set before comparing
the value to avoid 'unbound variable' error.

++ mktemp
+ cmdfile=/tmp/tmp.cn0hNYMZbt
+ '[' '!' -f
/var/lib/libvirt/images/kdevops/guestfs/base_images/debian-12.raw ']'
.//scripts/bringup_guestfs.sh: line 289:
CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE: unbound variable
make: *** [scripts/guestfs.Makefile:77: bringup_guestfs] Error 1

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 scripts/bringup_guestfs.sh | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
index 1d377faf387888a4523a838fd292654e5deddd82..3681423eff2849bc63519641ad09d974d421589a 100755
--- a/scripts/bringup_guestfs.sh
+++ b/scripts/bringup_guestfs.sh
@@ -234,7 +234,8 @@ _EOT
 	# For the life of me I can't get the following line to work with
 	# the virt-builder command and so we do a full edit of the file for now
 	# edit /etc/nsswitch.conf:'s/\[!UNAVAIL=return\]//'
-	if [[ "$CONFIG_GUESTFS_DEBIAN_TRIXIE" == "y" ]]; then
+	if [[ "${CONFIG_GUESTFS_DEBIAN_TRIXIE+x}" && \
+		  "$CONFIG_GUESTFS_DEBIAN_TRIXIE" == "y" ]]; then
 		cat <<_EOT >>$cmdfile
 write /etc/nsswitch.conf: # kdevops generated /etc/nsswitch.conf
 append-line /etc/nsswitch.conf:passwd:         files
@@ -258,7 +259,8 @@ firstboot-command DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=tru
 firstboot-command systemctl stop ssh
 firstboot-command systemctl start ssh
 _EOT
-		if [[ "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
+		if [[ "${CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST+x}" && \
+			  "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
 		cat <<_EOT >>$cmdfile
 delete /etc/apt/sources.list.d/debian.sources
 _EOT
@@ -274,7 +276,8 @@ fi
 cmdfile=$(mktemp)
 
 if [ ! -f $BASE_IMAGE ]; then
-	if [[ "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE" == "y" ]]; then
+	if [[ "${CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE+x}" && \
+		  "$CONFIG_GUESTFS_HAS_CUSTOM_RAW_IMAGE" == "y" ]]; then
 		build_custom_image
 	fi
 
@@ -287,7 +290,8 @@ if [ ! -f $BASE_IMAGE ]; then
 		copy_yum_repo
 	fi
 
-	if [[ "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
+	if [[ "${CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST+x}" && \
+		  "$CONFIG_GUESTFS_COPY_SOURCES_FROM_HOST_TO_GUEST" == "y" ]]; then
 		copy_host_sources
 	fi
 
@@ -342,7 +346,8 @@ do
 	TZ="$(timedatectl show -p Timezone --value)"
 	$USE_SUDO virt-sysprep -a $ROOTIMG --hostname $name --ssh-inject "kdevops:file:$SSH_KEY.pub" --timezone $TZ
 
-	if [[ "$CONFIG_LIBVIRT_ENABLE_LARGEIO" == "y" ]]; then
+	if [[ "${CONFIG_LIBVIRT_ENABLE_LARGEIO+x}" && \
+		  "$CONFIG_LIBVIRT_ENABLE_LARGEIO" == "y" ]]; then
 		lbs_idx=0
 		for i in $(seq 1 $(($CONFIG_QEMU_LARGEIO_MAX_POW_LIMIT+1))); do
 			for x in $(seq 0 $CONFIG_QEMU_EXTRA_DRIVE_LARGEIO_NUM_DRIVES_PER_SPACE); do

-- 
2.49.0

[PATCH v2 3/4] guestfs: bringup: fix user check when debug

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

Replace id -u with getent to check if the user exists.

Avoids id: 'kdevops': no such user.

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 scripts/bringup_guestfs.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
index 3681423eff2849bc63519641ad09d974d421589a..3fd48331b73f51e23818c553f8973d17d38dffca 100755
--- a/scripts/bringup_guestfs.sh
+++ b/scripts/bringup_guestfs.sh
@@ -189,8 +189,7 @@ _EOT
 pre_install_customizations()
 {
 	KDEVOPS_UID=""
-	id -u kdevops > /dev/null 2>&1
-	if [ $? -eq 0 ]; then
+	if getent passwd kdevops > /dev/null 2>&1; then
 		KDEVOPS_UID="-u `id -u kdevops`"
 	fi
 	if echo $OS_VERSION | grep -qE "^(rhel|fedora|centos)"; then

-- 
2.49.0

[PATCH v2 4/4] guestfs: bringup: add debug mode

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

To be able to run bringup_guestfs.sh script in bash debug mode:
1. disabled
2. set -x
3. set -euxo pipefile

From set manual:
-x  Print commands and their arguments as they are executed.
-e  Exit immediately if a command exits with a non-zero status.
-u  Treat unset variables as an error when substituting.
-o option-name
          Set the variable corresponding to option-name:
              pipefail     the return value of a pipeline is the status of
                           the last command to exit with a non-zero status,
                           or zero if no command exited with a non-zero status

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 kconfigs/Kconfig.guestfs   | 21 +++++++++++++++++++++
 scripts/bringup_guestfs.sh | 10 ++++++++++
 2 files changed, 31 insertions(+)

diff --git a/kconfigs/Kconfig.guestfs b/kconfigs/Kconfig.guestfs
index a2fddd691be9dbfa025c916708d8c14a07eb4514..ed08d293b3f6085350d09b68457aed6f8c09a468 100644
--- a/kconfigs/Kconfig.guestfs
+++ b/kconfigs/Kconfig.guestfs
@@ -214,4 +214,25 @@ config VIRT_BUILDER_OS_VERSION
          to get a list of operating systems and versions supported
          by guestfs.
 
+choice
+	prompt "Guestfs bringup debug mode"
+	default GUESTFS_BRINGUP_DEBUG_DISABLED
+
+config GUESTFS_BRINGUP_DEBUG_DISABLED
+	bool "Disabled"
+	help
+	  Disables bringup debug.
+
+config GUESTFS_BRINGUP_DEBUG_0
+	bool "set -x"
+	help
+	  Enables 'set -x' when running scripts/guestfs_bringup.sh.
+
+config GUESTFS_BRINGUP_DEBUG_1
+	bool "set -euxo pipefile"
+	help
+	  Enables 'set -euxo pipefile' when running scripts/guestfs_bringup.sh.
+
+endchoice
+
 endif # GUESTFS
diff --git a/scripts/bringup_guestfs.sh b/scripts/bringup_guestfs.sh
index 3fd48331b73f51e23818c553f8973d17d38dffca..67f85a5fdb0a41a12de01f6e9911c206d7ab9a5a 100755
--- a/scripts/bringup_guestfs.sh
+++ b/scripts/bringup_guestfs.sh
@@ -5,6 +5,16 @@
 source ${TOPDIR}/.config
 source ${TOPDIR}/scripts/lib.sh
 
+if [[ "${CONFIG_GUESTFS_BRINGUP_DEBUG_0+x}" && \
+	   "${CONFIG_GUESTFS_BRINGUP_DEBUG_0}" == "y" ]]; then
+	set -x
+fi
+
+if [[ "${CONFIG_GUESTFS_BRINGUP_DEBUG_1+x}" && \
+	  "${CONFIG_GUESTFS_BRINGUP_DEBUG_1}" == "y" ]]; then
+	set -euxo pipefail
+fi
+
 export LIBVIRT_DEFAULT_URI=$CONFIG_LIBVIRT_URI
 
 # We use the NVMe setting for virtio too (go figure), but IDE

-- 
2.49.0

Re: [PATCH v2 0/4] guestfs: bringup: add debug mode

[Chuck Lever]

On 5/12/25 9:36 AM, Daniel Gomez wrote:
> This has helped me during debug sessions of bringup script. "Resend"
> with minor changes.
> 
> Adding debug option in bash bringup_guestfs.sh allows to check for
> commands executions (virsh, chmod, cp, virt-sysprep, etc) and variables.
> 
> Add 3 levels of debug:
> - Disabled: This is the current behaviour and is the default.
> - set -x (debug mode 0):  Allows to print the executed commands but if
> they fail, the script continues.
> - set -euxo pipefail (debug mode 1): mode 0 + variables are printed +
> script fails if a command does not succeed.
> 
> While debugging, I noticed about small bash errors that we ignore, such
> as unbound variables. So, this also fixes the issues found while debug
> mode 1 was enabled.

Reviewed-by: Chuck Lever <chuck.lever@oracle.com>

In the long run, I think bringup_guestfs.sh should be converted to
Ansible (IaC), similar to playbooks/roles/terraform/.

Since it is a big honking shell script, the process will be less risky
to do parts of it at a time. The heavy lift will be replacing the shell
code that builds base OS images. Once that is done, what is left is
fairly straightforward, and I have a few old patches that do that part.

Unfortunately I'm running an NFS plug-fest this week, so I can't post
much at the moment.


> ---
> Changes in v2:
> * Rebase on top of latest main branch
> * Split long conditional lines into 2 
> - Link to v1: https://lore.kernel.org/all/20241015-bringup-guestfs-debug-v1-0-bd74c0c31412@samsung.com/
> 
> ---
> Daniel Gomez (4):
>       guestfs: bringup: check if domain exists
>       guestfs: bringup: fix unbound variable when debug
>       guestfs: bringup: fix user check when debug
>       guestfs: bringup: add debug mode
> 
>  kconfigs/Kconfig.guestfs   | 21 +++++++++++++++++++++
>  scripts/bringup_guestfs.sh | 38 +++++++++++++++++++++++++++-----------
>  2 files changed, 48 insertions(+), 11 deletions(-)
> ---
> base-commit: ef549598f2261ac89643d3e7a8ea0f050fc27c5f
> change-id: 20241015-bringup-guestfs-debug-775f4dbd98c1
> 
> Best regards,


-- 
Chuck Lever

Re: [PATCH v2 0/4] guestfs: bringup: add debug mode

[Daniel Gomez]

> Reviewed-by: Chuck Lever <chuck.lever@oracle.com>

Thanks!

> In the long run, I think bringup_guestfs.sh should be converted to
> Ansible (IaC), similar to playbooks/roles/terraform/.
> 
> Since it is a big honking shell script, the process will be less risky
> to do parts of it at a time. The heavy lift will be replacing the shell
> code that builds base OS images. Once that is done, what is left is
> fairly straightforward, and I have a few old patches that do that part.

I agree with the approach.

> 
> Unfortunately I'm running an NFS plug-fest this week, so I can't post
> much at the moment.

Happy to review that when you have the time to send it.

Re: [PATCH v2 0/4] guestfs: bringup: add debug mode

[Luis Chamberlain]

On Mon, May 12, 2025 at 03:36:01PM +0200, Daniel Gomez wrote:
> This has helped me during debug sessions of bringup script. "Resend"
> with minor changes.
> 
> Adding debug option in bash bringup_guestfs.sh allows to check for
> commands executions (virsh, chmod, cp, virt-sysprep, etc) and variables.
> 
> Add 3 levels of debug:
> - Disabled: This is the current behaviour and is the default.
> - set -x (debug mode 0):  Allows to print the executed commands but if
> they fail, the script continues.
> - set -euxo pipefail (debug mode 1): mode 0 + variables are printed +
> script fails if a command does not succeed.
> 
> While debugging, I noticed about small bash errors that we ignore, such
> as unbound variables. So, this also fixes the issues found while debug
> mode 1 was enabled.
> 
> ---

Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>

 Luis