[PATCH 0/2] Locales fixes

[PATCH 0/2] Locales fixes

[Daniel Gomez]

This drops locales-all from devconfig playbook and make en-US.UTF tasks
distro agnostic. While at it, also fix guest image permissions when the
base image is not copied and ensure idempotency with permissions.

Note, the locales-all was in the wrong location (before removing
unattended-install Debian package) provoking the apt concurrency issue
(logs below) and not needed for en_US-UTF locale configuration added in
commit 2514a694 ("guestfs: fix locale warnings in VM deployment").

Error: : Task failed: Module failed: '/usr/bin/apt-get -y -o
"Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"
-o DPkg::Lock::Timeout=60       install 'locales-all=2.41-12''
failed: E: Unable to acquire the dpkg frontend lock
(/var/lib/dpkg/lock-frontend), is another process using it?

Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
Daniel Gomez (2):
      devconfig: drop locales-all leftovers
      base_image: ensure base image permissions are set

 playbooks/roles/base_image/tasks/custom-image.yml |  1 -
 playbooks/roles/devconfig/tasks/main.yml          | 50 ++++++++---------------
 2 files changed, 17 insertions(+), 34 deletions(-)
---
base-commit: 0f7daa6bf64052343ff1dc42b3da3a35bc89b234
change-id: 20250929-locales-all-removal-e67b23f6724e

Best regards,
--  
Daniel Gomez <da.gomez@samsung.com>

[PATCH 1/2] devconfig: drop locales-all leftovers

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

When locales-all support was added in commit 2514a694 ("guestfs: fix
locale warnings in VM deployment"), it was added for all distros in the
virt-builder.j2 template as well as Ansible tasks to ensure idempotency
on previous base images.

But we just need to ensure en_US.UTF-8 is available on the guests
image, so keep just this config in both template and tasks but remove
locales-all.

While we are at it, also move the locale configuration after all deps
are installed and to the distro agnostic task section. We do it anyways
for all distros in virt-builder.j2, this ensures idempotency in all
of them.

Fixes: 6bb3bfb2 ("base_image: Skip "locales-all" package installation")
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 playbooks/roles/devconfig/tasks/main.yml | 50 +++++++++++---------------------
 1 file changed, 17 insertions(+), 33 deletions(-)

diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index ae16a698..1acca6b4 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -54,17 +54,25 @@
     dest: /etc/hostname
   tags: hostname
 
-- name: Ensure locales-all package is installed
-  become: true
-  become_flags: "su - -c"
-  become_method: sudo
-  ansible.builtin.package:
-    name: locales-all
-    state: present
+# Distro specific
+
+# Check and fix APT mirrors for Debian testing before installing dependencies
+- name: Check and fix APT mirrors for Debian testing
+  ansible.builtin.include_tasks: check-apt-mirrors.yml
   when:
-    - ansible_facts['os_family']|lower == 'debian'
-  tags: firstconfig
+    - devconfig_debian_testing is defined
+    - devconfig_debian_testing | bool
+
+- name: Install dependencies
+  ansible.builtin.include_tasks: install-deps/main.yml
+  tags: ["vars", "vars_simple"]
 
+- name: Configure custom repositories and install packages
+  ansible.builtin.import_tasks: config-custom-repos-and-packages/main.yml
+  when:
+    - ansible_facts['os_family']|lower == 'redhat'
+
+# Distro agnostic stuff goes below
 - name: Configure en_US.UTF-8 locale files
   become: true
   become_flags: "su - -c"
@@ -80,8 +88,6 @@
     - { path: "/etc/default/locale", line: "LANG=en_US.UTF-8" }
     - { path: "/etc/default/locale", line: "LANGUAGE=en_US:en" }
     - { path: "/etc/locale.gen", line: "en_US.UTF-8 UTF-8" }
-  when:
-    - ansible_facts['os_family']|lower == 'debian'
   tags: firstconfig
 
 - name: Generate and update locales
@@ -92,30 +98,8 @@
     locale-gen en_US.UTF-8
     update-locale LANG=en_US.UTF-8
   changed_when: true
-  when:
-    - ansible_facts['os_family']|lower == 'debian'
   tags: firstconfig
 
-# Distro specific
-
-# Check and fix APT mirrors for Debian testing before installing dependencies
-- name: Check and fix APT mirrors for Debian testing
-  ansible.builtin.include_tasks: check-apt-mirrors.yml
-  when:
-    - devconfig_debian_testing is defined
-    - devconfig_debian_testing | bool
-
-- name: Install dependencies
-  ansible.builtin.include_tasks: install-deps/main.yml
-  tags: ["vars", "vars_simple"]
-
-- name: Configure custom repositories and install packages
-  ansible.builtin.import_tasks: config-custom-repos-and-packages/main.yml
-  when:
-    - ansible_facts['os_family']|lower == 'redhat'
-
-# Distro agnostic stuff goes below
-
 - name: Check if /mirror directory exists on localhost
   delegate_to: localhost
   ansible.builtin.stat:

-- 
2.50.1

Re: [PATCH 1/2] devconfig: drop locales-all leftovers

[Daniel Gomez]

On 29/09/2025 21.04, Daniel Gomez wrote:
> From: Daniel Gomez <da.gomez@samsung.com>
> 
> When locales-all support was added in commit 2514a694 ("guestfs: fix
> locale warnings in VM deployment"), it was added for all distros in the
> virt-builder.j2 template as well as Ansible tasks to ensure idempotency
> on previous base images.
> 
> But we just need to ensure en_US.UTF-8 is available on the guests
> image, so keep just this config in both template and tasks but remove
> locales-all.
> 
> While we are at it, also move the locale configuration after all deps
> are installed and to the distro agnostic task section. We do it anyways
> for all distros in virt-builder.j2, this ensures idempotency in all
> of them.
> 
> Fixes: 6bb3bfb2 ("base_image: Skip "locales-all" package installation")
> Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
> ---
>  playbooks/roles/devconfig/tasks/main.yml | 50 +++++++++++---------------------
>  1 file changed, 17 insertions(+), 33 deletions(-)
> 
> diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
> index ae16a698..1acca6b4 100644
> --- a/playbooks/roles/devconfig/tasks/main.yml
> +++ b/playbooks/roles/devconfig/tasks/main.yml
...
> @@ -80,8 +88,6 @@
>      - { path: "/etc/default/locale", line: "LANG=en_US.UTF-8" }
>      - { path: "/etc/default/locale", line: "LANGUAGE=en_US:en" }
>      - { path: "/etc/locale.gen", line: "en_US.UTF-8 UTF-8" }
> -  when:
> -    - ansible_facts['os_family']|lower == 'debian'

FYI, I plan to merge this tomorrow as this patchset solves the current
linux-ci issue [1]. Please note that this change affects all distributions.
I believe this is the correct approach and will proceed with it to restore CI
functionality. Please, let me know if you find it breaks something on non-Debian
distros.

Link: https://github.com/linux-kdevops/kdevops/actions/runs/18347971703/job/52259752236 [1]

[PATCH 2/2] base_image: ensure base image permissions are set

[Daniel Gomez]

From: Daniel Gomez <da.gomez@samsung.com>

Permissions should be set regardless of if we executed the copy task
or not.

Workflow:

TASK: Show the custom virt-builder database [localhost]
Custom virt-builder source:
/etc/virt-builder/repos.d/
kdevops-custom-images-debian-13-generic-arm64-daily.conf
Custom virt-builder index:
/var/lib/libvirt/images/kdevops/guestfs/custom_images/
debian-13-generic-arm64-daily/index
Custom virt-builder image:
/var/lib/libvirt/images/kdevops/guestfs/custom_images/
debian-13-generic-arm64-daily/debian-13-generic-arm64-daily.raw
TASK: Generating the index for debian-13-generic-arm64-daily [localhost]
⠀⠀✓ [localhost]
TASK: Copy custom image to base image location (with automatic reflink
optimization) [localhost]
⠀⠀⊘ [localhost]
TASK: Set proper permissions on base image [localhost]
⠀⠀⊘ [localhost]
TASK: Create the base OS image [localhost]
⠀⠀⊘ [localhost]

...

Fixes error:
TASK: Duplicate the root disk image for each target node
[dgc-testing-block,dgc-testing-nvme]
⠀⠀FAILED: [dgc-testing-block] => non-zero
return code STDERR: cp: cannot open
'/var/lib/libvirt/images/kdevops/guestfs/custom_images/
debian-13-generic-arm64-daily/debian-13-generic-arm64-daily.raw' for
reading: Permission denied
[ERROR]: Task failed: Module failed: non-zero return code
Origin:
/home/dagomez.linux/src/kdevops/playbooks/roles/guestfs/tasks/bringup/
main.yml:53:7

51       delegate_to: localhost
52
53     - name: Duplicate the root disk image for each target node
         ^ column 7

Fixes: f9ef79cb ("base_image: optimize VM image copying with reflinks
and fix same-file handling")
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 playbooks/roles/base_image/tasks/custom-image.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/playbooks/roles/base_image/tasks/custom-image.yml b/playbooks/roles/base_image/tasks/custom-image.yml
index bac834e8..121fa112 100644
--- a/playbooks/roles/base_image/tasks/custom-image.yml
+++ b/playbooks/roles/base_image/tasks/custom-image.yml
@@ -355,4 +355,3 @@
     mode: "u=rw,g=r,o=r"
   when:
     - custom_image_stat.stat.exists or custom_image_download is changed
-    - custom_image != base_image_pathname

-- 
2.50.1

Re: [PATCH 0/2] Locales fixes

[Daniel Gomez]

On Mon, 29 Sep 2025 21:04:11 +0200, Daniel Gomez wrote:
> This drops locales-all from devconfig playbook and make en-US.UTF tasks
> distro agnostic. While at it, also fix guest image permissions when the
> base image is not copied and ensure idempotency with permissions.
> 
> Note, the locales-all was in the wrong location (before removing
> unattended-install Debian package) provoking the apt concurrency issue
> (logs below) and not needed for en_US-UTF locale configuration added in
> commit 2514a694 ("guestfs: fix locale warnings in VM deployment").
> 
> [...]

Applied, thanks!

[1/2] devconfig: drop locales-all leftovers
      (no commit info)
[2/2] base_image: ensure base image permissions are set
      (no commit info)

Best regards,
-- 
Daniel Gomez <da.gomez@samsung.com>

Re: [PATCH 0/2] Locales fixes

[Chuck Lever]

On 10/14/25 2:02 PM, Daniel Gomez wrote:
> 
> On Mon, 29 Sep 2025 21:04:11 +0200, Daniel Gomez wrote:
>> This drops locales-all from devconfig playbook and make en-US.UTF tasks
>> distro agnostic. While at it, also fix guest image permissions when the
>> base image is not copied and ensure idempotency with permissions.
>>
>> Note, the locales-all was in the wrong location (before removing
>> unattended-install Debian package) provoking the apt concurrency issue
>> (logs below) and not needed for en_US-UTF locale configuration added in
>> commit 2514a694 ("guestfs: fix locale warnings in VM deployment").
>>
>> [...]
> 
> Applied, thanks!
> 
> [1/2] devconfig: drop locales-all leftovers
>       (no commit info)
> [2/2] base_image: ensure base image permissions are set
>       (no commit info)
> 
> Best regards,


Hi Daniel!

I'm seeing this on Fedora 41 with guestfs:

TASK [devconfig : Configure en_US.UTF-8 locale files]
******************************************************************************************
changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
'LANG=en_US.UTF-8'})
changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
'LANGUAGE=en_US:en'})
changed: [kdevops-nfsd] => (item={'path': '/etc/locale.gen', 'line':
'en_US.UTF-8 UTF-8'})
 [started TASK: devconfig : Generate and update locales on kdevops-nfsd]

TASK [devconfig : Generate and update locales]
*************************************************************************************************
task path:
/home/cel/src/kdevops/buildbot-configs/playbooks/roles/devconfig/tasks/main.yml:93
fatal: [kdevops-nfsd]: FAILED! => {
    "changed": true,
    "cmd": "locale-gen en_US.UTF-8\nupdate-locale LANG=en_US.UTF-8\n",
    "delta": "0:00:00.003794",
    "end": "2025-10-14 17:30:14.766094",
    "rc": 127,
    "start": "2025-10-14 17:30:14.762300"
}

STDERR:

/bin/sh: line 1: locale-gen: command not found
/bin/sh: line 2: update-locale: command not found


MSG:

non-zero return code


Neither of those commands appears in the Fedora 41 package inventory.


-- 
Chuck Lever

Re: [PATCH 0/2] Locales fixes

[Daniel Gomez]

On 14/10/2025 23.35, Chuck Lever wrote:
> On 10/14/25 2:02 PM, Daniel Gomez wrote:
>>
>> On Mon, 29 Sep 2025 21:04:11 +0200, Daniel Gomez wrote:
>>> This drops locales-all from devconfig playbook and make en-US.UTF tasks
>>> distro agnostic. While at it, also fix guest image permissions when the
>>> base image is not copied and ensure idempotency with permissions.
>>>
>>> Note, the locales-all was in the wrong location (before removing
>>> unattended-install Debian package) provoking the apt concurrency issue
>>> (logs below) and not needed for en_US-UTF locale configuration added in
>>> commit 2514a694 ("guestfs: fix locale warnings in VM deployment").
>>>
>>> [...]
>>
>> Applied, thanks!
>>
>> [1/2] devconfig: drop locales-all leftovers
>>       (no commit info)
>> [2/2] base_image: ensure base image permissions are set
>>       (no commit info)
>>
>> Best regards,
> 
> 
> Hi Daniel!
> 
> I'm seeing this on Fedora 41 with guestfs:
> 
> TASK [devconfig : Configure en_US.UTF-8 locale files]
> ******************************************************************************************
> changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
> 'LANG=en_US.UTF-8'})
> changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
> 'LANGUAGE=en_US:en'})
> changed: [kdevops-nfsd] => (item={'path': '/etc/locale.gen', 'line':
> 'en_US.UTF-8 UTF-8'})
>  [started TASK: devconfig : Generate and update locales on kdevops-nfsd]
> 
> TASK [devconfig : Generate and update locales]
> *************************************************************************************************
> task path:
> /home/cel/src/kdevops/buildbot-configs/playbooks/roles/devconfig/tasks/main.yml:93
> fatal: [kdevops-nfsd]: FAILED! => {
>     "changed": true,
>     "cmd": "locale-gen en_US.UTF-8\nupdate-locale LANG=en_US.UTF-8\n",
>     "delta": "0:00:00.003794",
>     "end": "2025-10-14 17:30:14.766094",
>     "rc": 127,
>     "start": "2025-10-14 17:30:14.762300"
> }
> 
> STDERR:
> 
> /bin/sh: line 1: locale-gen: command not found
> /bin/sh: line 2: update-locale: command not found
> 
> 
> MSG:
> 
> non-zero return code
> 
> 
> Neither of those commands appears in the Fedora 41 package inventory.
> 
> 

Sorry for the trouble.

I guess restoring this hunk fixes the issue, can you give it a try and confirm?

diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index 39d883b7..ba55928e 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -88,6 +88,8 @@
     - { path: "/etc/default/locale", line: "LANG=en_US.UTF-8" }
     - { path: "/etc/default/locale", line: "LANGUAGE=en_US:en" }
     - { path: "/etc/locale.gen", line: "en_US.UTF-8 UTF-8" }
+  when:
+    - ansible_facts['os_family']|lower == 'debian'
   tags: firstconfig

 - name: Generate and update locales
@@ -98,6 +100,8 @@
     locale-gen en_US.UTF-8
     update-locale LANG=en_US.UTF-8
   changed_when: true
+  when:
+    - ansible_facts['os_family']|lower == 'debian'
   tags: firstconfig

 - name: Check if /mirror directory exists on localhost

Note, I can't fully test this or the patch, as it appears to be broken (at least
for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:

virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
[   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
###################################################################################################################################################################################################### 100.0%
[ 140.6] Planning how to build this image
[ 140.6] Uncompressing
[ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
[ 171.5] Opening the new disk
[ 181.7] Setting a random seed
[ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
[ 194.7] Uninstalling packages: dracut-config-rescue
[ 195.0] Deleting: /boot/*rescue*
[ 195.0] Deleting: /boot/loader/entries/*rescue*
[ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
[ 195.1] Appending line to /etc/sudoers.d/kdevops
[ 195.1] Editing: /etc/default/grub
[ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
[ 195.9] Installing packages: isc-dhcp-client ifupdown
Updating and loading repositories:
 Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
 Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
 Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
 Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
 Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
 Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
Repositories loaded.
Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
...
Complete!
Generating grub configuration file ...
Adding boot menu entry for UEFI Firmware Settings ...
done
Updating and loading repositories:
Repositories loaded.
Failed to resolve the transaction:
No match for argument: isc-dhcp-client
No match for argument: ifupdown
You can try to add to command line:
  --skip-unavailable to skip unavailable packages
virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
exited with an error

If reporting bugs, run virt-builder with debugging enabled and include the
complete output:

  virt-builder -v -x [...]

Re: [PATCH 0/2] Locales fixes

[Chuck Lever]

On 10/15/25 2:21 PM, Daniel Gomez wrote:
> 
> 
> On 14/10/2025 23.35, Chuck Lever wrote:
>> On 10/14/25 2:02 PM, Daniel Gomez wrote:
>>>
>>> On Mon, 29 Sep 2025 21:04:11 +0200, Daniel Gomez wrote:
>>>> This drops locales-all from devconfig playbook and make en-US.UTF tasks
>>>> distro agnostic. While at it, also fix guest image permissions when the
>>>> base image is not copied and ensure idempotency with permissions.
>>>>
>>>> Note, the locales-all was in the wrong location (before removing
>>>> unattended-install Debian package) provoking the apt concurrency issue
>>>> (logs below) and not needed for en_US-UTF locale configuration added in
>>>> commit 2514a694 ("guestfs: fix locale warnings in VM deployment").
>>>>
>>>> [...]
>>>
>>> Applied, thanks!
>>>
>>> [1/2] devconfig: drop locales-all leftovers
>>>       (no commit info)
>>> [2/2] base_image: ensure base image permissions are set
>>>       (no commit info)
>>>
>>> Best regards,
>>
>>
>> Hi Daniel!
>>
>> I'm seeing this on Fedora 41 with guestfs:
>>
>> TASK [devconfig : Configure en_US.UTF-8 locale files]
>> ******************************************************************************************
>> changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
>> 'LANG=en_US.UTF-8'})
>> changed: [kdevops-nfsd] => (item={'path': '/etc/default/locale', 'line':
>> 'LANGUAGE=en_US:en'})
>> changed: [kdevops-nfsd] => (item={'path': '/etc/locale.gen', 'line':
>> 'en_US.UTF-8 UTF-8'})
>>  [started TASK: devconfig : Generate and update locales on kdevops-nfsd]
>>
>> TASK [devconfig : Generate and update locales]
>> *************************************************************************************************
>> task path:
>> /home/cel/src/kdevops/buildbot-configs/playbooks/roles/devconfig/tasks/main.yml:93
>> fatal: [kdevops-nfsd]: FAILED! => {
>>     "changed": true,
>>     "cmd": "locale-gen en_US.UTF-8\nupdate-locale LANG=en_US.UTF-8\n",
>>     "delta": "0:00:00.003794",
>>     "end": "2025-10-14 17:30:14.766094",
>>     "rc": 127,
>>     "start": "2025-10-14 17:30:14.762300"
>> }
>>
>> STDERR:
>>
>> /bin/sh: line 1: locale-gen: command not found
>> /bin/sh: line 2: update-locale: command not found
>>
>>
>> MSG:
>>
>> non-zero return code
>>
>>
>> Neither of those commands appears in the Fedora 41 package inventory.
>>
>>
> 
> Sorry for the trouble.
> 
> I guess restoring this hunk fixes the issue, can you give it a try and confirm?
> 
> diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
> index 39d883b7..ba55928e 100644
> --- a/playbooks/roles/devconfig/tasks/main.yml
> +++ b/playbooks/roles/devconfig/tasks/main.yml
> @@ -88,6 +88,8 @@
>      - { path: "/etc/default/locale", line: "LANG=en_US.UTF-8" }
>      - { path: "/etc/default/locale", line: "LANGUAGE=en_US:en" }
>      - { path: "/etc/locale.gen", line: "en_US.UTF-8 UTF-8" }
> +  when:
> +    - ansible_facts['os_family']|lower == 'debian'
>    tags: firstconfig
> 
>  - name: Generate and update locales
> @@ -98,6 +100,8 @@
>      locale-gen en_US.UTF-8
>      update-locale LANG=en_US.UTF-8
>    changed_when: true
> +  when:
> +    - ansible_facts['os_family']|lower == 'debian'
>    tags: firstconfig
> 
>  - name: Check if /mirror directory exists on localhost

Tested-by: Chuck Lever <chuck.lever@oracle.com>


> Note, I can't fully test this or the patch, as it appears to be broken (at least
> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
> 
> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
> ###################################################################################################################################################################################################### 100.0%
> [ 140.6] Planning how to build this image
> [ 140.6] Uncompressing
> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
> [ 171.5] Opening the new disk
> [ 181.7] Setting a random seed
> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
> [ 194.7] Uninstalling packages: dracut-config-rescue
> [ 195.0] Deleting: /boot/*rescue*
> [ 195.0] Deleting: /boot/loader/entries/*rescue*
> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
> [ 195.1] Appending line to /etc/sudoers.d/kdevops
> [ 195.1] Editing: /etc/default/grub
> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
> [ 195.9] Installing packages: isc-dhcp-client ifupdown
> Updating and loading repositories:
>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
> Repositories loaded.
> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
> ...
> Complete!
> Generating grub configuration file ...
> Adding boot menu entry for UEFI Firmware Settings ...
> done
> Updating and loading repositories:
> Repositories loaded.
> Failed to resolve the transaction:
> No match for argument: isc-dhcp-client
> No match for argument: ifupdown
> You can try to add to command line:
>   --skip-unavailable to skip unavailable packages
> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
> exited with an error
> 
> If reporting bugs, run virt-builder with debugging enabled and include the
> complete output:
> 
>   virt-builder -v -x [...]

This appears to come from the new base_image virt_builder template,
where we have:

{% if distro_debian_based is defined and distro_debian_based %}
{# Ugh, debian has to be told to bring up the network and regenerate ssh
keys #}
{# Hope we get that interface name right! #}
install isc-dhcp-client,ifupdown
mkdir /etc/network/interfaces.d/

Which I would expect to apply only when the /guest/ is Debian, not
the host. It could be that the distro detection logic works only when
both the host and guest are the same?


-- 
Chuck Lever

Re: [PATCH 0/2] Locales fixes

[Chuck Lever]

On 10/15/25 5:17 PM, Chuck Lever wrote:
>> Note, I can't fully test this or the patch, as it appears to be broken (at least
>> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
>>
>> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
>> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
>> ###################################################################################################################################################################################################### 100.0%
>> [ 140.6] Planning how to build this image
>> [ 140.6] Uncompressing
>> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
>> [ 171.5] Opening the new disk
>> [ 181.7] Setting a random seed
>> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
>> [ 194.7] Uninstalling packages: dracut-config-rescue
>> [ 195.0] Deleting: /boot/*rescue*
>> [ 195.0] Deleting: /boot/loader/entries/*rescue*
>> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
>> [ 195.1] Appending line to /etc/sudoers.d/kdevops
>> [ 195.1] Editing: /etc/default/grub
>> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
>> [ 195.9] Installing packages: isc-dhcp-client ifupdown
>> Updating and loading repositories:
>>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
>> Repositories loaded.
>> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
>> ...
>> Complete!
>> Generating grub configuration file ...
>> Adding boot menu entry for UEFI Firmware Settings ...
>> done
>> Updating and loading repositories:
>> Repositories loaded.
>> Failed to resolve the transaction:
>> No match for argument: isc-dhcp-client
>> No match for argument: ifupdown
>> You can try to add to command line:
>>   --skip-unavailable to skip unavailable packages
>> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
>> exited with an error
>>
>> If reporting bugs, run virt-builder with debugging enabled and include the
>> complete output:
>>
>>   virt-builder -v -x [...]
> This appears to come from the new base_image virt_builder template,
> where we have:
> 
> {% if distro_debian_based is defined and distro_debian_based %}
> {# Ugh, debian has to be told to bring up the network and regenerate ssh
> keys #}
> {# Hope we get that interface name right! #}
> install isc-dhcp-client,ifupdown
> mkdir /etc/network/interfaces.d/
> 
> Which I would expect to apply only when the /guest/ is Debian, not
> the host. It could be that the distro detection logic works only when
> both the host and guest are the same?

I think the problem arises because the new base_image role uses these
distro checks instead of what bringup_guestfs.sh did, which was select
these commands based on {{ virtbuilder_os_version }}

virt-builder.j2 needs to use virtbuilder_os_version for these checks.


-- 
Chuck Lever

Re: [PATCH 0/2] Locales fixes

[Chuck Lever]

On 10/16/25 2:56 PM, Chuck Lever wrote:
> On 10/15/25 5:17 PM, Chuck Lever wrote:
>>> Note, I can't fully test this or the patch, as it appears to be broken (at least
>>> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
>>>
>>> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
>>> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
>>> ###################################################################################################################################################################################################### 100.0%
>>> [ 140.6] Planning how to build this image
>>> [ 140.6] Uncompressing
>>> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
>>> [ 171.5] Opening the new disk
>>> [ 181.7] Setting a random seed
>>> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
>>> [ 194.7] Uninstalling packages: dracut-config-rescue
>>> [ 195.0] Deleting: /boot/*rescue*
>>> [ 195.0] Deleting: /boot/loader/entries/*rescue*
>>> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
>>> [ 195.1] Appending line to /etc/sudoers.d/kdevops
>>> [ 195.1] Editing: /etc/default/grub
>>> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
>>> [ 195.9] Installing packages: isc-dhcp-client ifupdown
>>> Updating and loading repositories:
>>>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>>>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>>>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>>>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
>>> Repositories loaded.
>>> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
>>> ...
>>> Complete!
>>> Generating grub configuration file ...
>>> Adding boot menu entry for UEFI Firmware Settings ...
>>> done
>>> Updating and loading repositories:
>>> Repositories loaded.
>>> Failed to resolve the transaction:
>>> No match for argument: isc-dhcp-client
>>> No match for argument: ifupdown
>>> You can try to add to command line:
>>>   --skip-unavailable to skip unavailable packages
>>> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
>>> exited with an error
>>>
>>> If reporting bugs, run virt-builder with debugging enabled and include the
>>> complete output:
>>>
>>>   virt-builder -v -x [...]
>> This appears to come from the new base_image virt_builder template,
>> where we have:
>>
>> {% if distro_debian_based is defined and distro_debian_based %}
>> {# Ugh, debian has to be told to bring up the network and regenerate ssh
>> keys #}
>> {# Hope we get that interface name right! #}
>> install isc-dhcp-client,ifupdown
>> mkdir /etc/network/interfaces.d/
>>
>> Which I would expect to apply only when the /guest/ is Debian, not
>> the host. It could be that the distro detection logic works only when
>> both the host and guest are the same?
> 
> I think the problem arises because the new base_image role uses these
> distro checks instead of what bringup_guestfs.sh did, which was select
> these commands based on {{ virtbuilder_os_version }}
> 
> virt-builder.j2 needs to use virtbuilder_os_version for these checks.
virtbuild_os_version doesn't give us quite everything that is needed.
Looks like the template needs to add some more cruft if Trixie is in
use. So I've put together a possible fix, pushed to the cel-fixes branch
in linux-kdevops. Please have a look.

Passes some (very) light testing with a Fedora 41 host and all of a
Debian 12, Debian 13, or Fedora 42 guest. Bringup gets past the
virt-builder stage now for all three.


-- 
Chuck Lever

Re: [PATCH 0/2] Locales fixes

[Daniel Gomez]

On 17/10/2025 23.59, Chuck Lever wrote:
> On 10/16/25 2:56 PM, Chuck Lever wrote:
>> On 10/15/25 5:17 PM, Chuck Lever wrote:
>>>> Note, I can't fully test this or the patch, as it appears to be broken (at least
>>>> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
>>>>
>>>> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
>>>> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
>>>> ###################################################################################################################################################################################################### 100.0%
>>>> [ 140.6] Planning how to build this image
>>>> [ 140.6] Uncompressing
>>>> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
>>>> [ 171.5] Opening the new disk
>>>> [ 181.7] Setting a random seed
>>>> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
>>>> [ 194.7] Uninstalling packages: dracut-config-rescue
>>>> [ 195.0] Deleting: /boot/*rescue*
>>>> [ 195.0] Deleting: /boot/loader/entries/*rescue*
>>>> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
>>>> [ 195.1] Appending line to /etc/sudoers.d/kdevops
>>>> [ 195.1] Editing: /etc/default/grub
>>>> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
>>>> [ 195.9] Installing packages: isc-dhcp-client ifupdown
>>>> Updating and loading repositories:
>>>>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>>>>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>>>>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>>>>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
>>>> Repositories loaded.
>>>> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
>>>> ...
>>>> Complete!
>>>> Generating grub configuration file ...
>>>> Adding boot menu entry for UEFI Firmware Settings ...
>>>> done
>>>> Updating and loading repositories:
>>>> Repositories loaded.
>>>> Failed to resolve the transaction:
>>>> No match for argument: isc-dhcp-client
>>>> No match for argument: ifupdown
>>>> You can try to add to command line:
>>>>   --skip-unavailable to skip unavailable packages
>>>> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
>>>> exited with an error
>>>>
>>>> If reporting bugs, run virt-builder with debugging enabled and include the
>>>> complete output:
>>>>
>>>>   virt-builder -v -x [...]
>>> This appears to come from the new base_image virt_builder template,
>>> where we have:
>>>
>>> {% if distro_debian_based is defined and distro_debian_based %}
>>> {# Ugh, debian has to be told to bring up the network and regenerate ssh
>>> keys #}
>>> {# Hope we get that interface name right! #}
>>> install isc-dhcp-client,ifupdown
>>> mkdir /etc/network/interfaces.d/
>>>
>>> Which I would expect to apply only when the /guest/ is Debian, not
>>> the host. It could be that the distro detection logic works only when
>>> both the host and guest are the same?
>>
>> I think the problem arises because the new base_image role uses these
>> distro checks instead of what bringup_guestfs.sh did, which was select
>> these commands based on {{ virtbuilder_os_version }}
>>
>> virt-builder.j2 needs to use virtbuilder_os_version for these checks.
> virtbuild_os_version doesn't give us quite everything that is needed.
> Looks like the template needs to add some more cruft if Trixie is in
> use. So I've put together a possible fix, pushed to the cel-fixes branch
> in linux-kdevops. Please have a look.
> 
> Passes some (very) light testing with a Fedora 41 host and all of a
> Debian 12, Debian 13, or Fedora 42 guest. Bringup gets past the
> virt-builder stage now for all three.
> 

I was just preparing the final patch for the fix. I'm adding your patch on top
and test it.

Thanks!

Re: [PATCH 0/2] Locales fixes

[Daniel Gomez]

On 17/10/2025 23.59, Chuck Lever wrote:
> On 10/16/25 2:56 PM, Chuck Lever wrote:
>> On 10/15/25 5:17 PM, Chuck Lever wrote:
>>>> Note, I can't fully test this or the patch, as it appears to be broken (at least
>>>> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
>>>>
>>>> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
>>>> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
>>>> ###################################################################################################################################################################################################### 100.0%
>>>> [ 140.6] Planning how to build this image
>>>> [ 140.6] Uncompressing
>>>> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
>>>> [ 171.5] Opening the new disk
>>>> [ 181.7] Setting a random seed
>>>> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
>>>> [ 194.7] Uninstalling packages: dracut-config-rescue
>>>> [ 195.0] Deleting: /boot/*rescue*
>>>> [ 195.0] Deleting: /boot/loader/entries/*rescue*
>>>> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
>>>> [ 195.1] Appending line to /etc/sudoers.d/kdevops
>>>> [ 195.1] Editing: /etc/default/grub
>>>> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
>>>> [ 195.9] Installing packages: isc-dhcp-client ifupdown
>>>> Updating and loading repositories:
>>>>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>>>>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>>>>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>>>>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
>>>> Repositories loaded.
>>>> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
>>>> ...
>>>> Complete!
>>>> Generating grub configuration file ...
>>>> Adding boot menu entry for UEFI Firmware Settings ...
>>>> done
>>>> Updating and loading repositories:
>>>> Repositories loaded.
>>>> Failed to resolve the transaction:
>>>> No match for argument: isc-dhcp-client
>>>> No match for argument: ifupdown
>>>> You can try to add to command line:
>>>>   --skip-unavailable to skip unavailable packages
>>>> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
>>>> exited with an error
>>>>
>>>> If reporting bugs, run virt-builder with debugging enabled and include the
>>>> complete output:
>>>>
>>>>   virt-builder -v -x [...]
>>> This appears to come from the new base_image virt_builder template,
>>> where we have:
>>>
>>> {% if distro_debian_based is defined and distro_debian_based %}
>>> {# Ugh, debian has to be told to bring up the network and regenerate ssh
>>> keys #}
>>> {# Hope we get that interface name right! #}
>>> install isc-dhcp-client,ifupdown
>>> mkdir /etc/network/interfaces.d/
>>>
>>> Which I would expect to apply only when the /guest/ is Debian, not
>>> the host. It could be that the distro detection logic works only when
>>> both the host and guest are the same?
>>
>> I think the problem arises because the new base_image role uses these
>> distro checks instead of what bringup_guestfs.sh did, which was select
>> these commands based on {{ virtbuilder_os_version }}
>>
>> virt-builder.j2 needs to use virtbuilder_os_version for these checks.
> virtbuild_os_version doesn't give us quite everything that is needed.
> Looks like the template needs to add some more cruft if Trixie is in
> use. So I've put together a possible fix, pushed to the cel-fixes branch
> in linux-kdevops. Please have a look.
> 
> Passes some (very) light testing with a Fedora 41 host and all of a
> Debian 12, Debian 13, or Fedora 42 guest. Bringup gets past the
> virt-builder stage now for all three.
> 
> 

FYI, I tested this on Debian Trixie with a Fedora 41 guestfs image. It didn't
fully work, but it got further than before. The guest now runs, though with many
issues, likely due to low disk space. I suspect the resize task didn't run.

systemd[1]: tmp.mount: Start request repeated too quickly.
systemd[1]: tmp.mount: Failed with result 'exit-code'.
systemd[1]: Failed to mount tmp.mount - Temporary Directory /tmp.
systemd[1]: systemd-oomd.service: Start request repeated too quickly.
systemd[1]: systemd-oomd.service: Failed with result 'exit-code'.
systemd[1]: Failed to start systemd-oomd.service - Userspace Out-Of-Memory (OOM) Killer

update_ssh_config_guestfs : Check if the new fixed include directive was used ----------------- 0.00s
+ LIBVIRT_DEFAULT_URI=qemu:///system
+ .//scripts/update_ssh_config_guestfs.py
Traceback (most recent call last):
  File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 109, in <module>
    main()
    ~~~~^^
  File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 96, in main
    addr = get_addr(name)
  File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 40, in get_addr
    raise Exception(f"Unable to get an address for {name} after 60s")
Exception: Unable to get an address for dagomez-xarray after 60s
make: *** [scripts/guestfs.Makefile:59: .provisioned_once_ssh] Error 1

On the other hand, images look size-wise just fine:

ls -lhsta /var/lib/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw
1.8G -rw-r--r-- 1 root root 20G Oct 18 00:19 /var/lib/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw
$ ls -lhsta /var/lib/libvirt/images/kdevops/guestfs/dagomez-xarray/root.raw
1.8G -rw-r--r-- 1 libvirt-qemu libvirt-qemu 20G Oct 18 00:39 /var/lib/libvirt/images/kdevops/guestfs/dagomez-xarray/root.raw

Re: [PATCH 0/2] Locales fixes

[Daniel Gomez]

On 18/10/2025 00.53, Daniel Gomez wrote:
> 
> 
> On 17/10/2025 23.59, Chuck Lever wrote:
>> On 10/16/25 2:56 PM, Chuck Lever wrote:
>>> On 10/15/25 5:17 PM, Chuck Lever wrote:
>>>>> Note, I can't fully test this or the patch, as it appears to be broken (at least
>>>>> for me) when using a Debian host and a Fedora guest with kdevops/libguestfs:
>>>>>
>>>>> virt-builder fedora-41 --arch x86_64 -o /xfs1/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw --size 20G --format raw --commands-from-file /tmp/ansible.8r5zc3so
>>>>> [   3.7] Downloading: http://builder.libguestfs.org/fedora-41.xz
>>>>> ###################################################################################################################################################################################################### 100.0%
>>>>> [ 140.6] Planning how to build this image
>>>>> [ 140.6] Uncompressing
>>>>> [ 141.7] Resizing (using virt-resize) to expand the disk to 20.0G
>>>>> [ 171.5] Opening the new disk
>>>>> [ 181.7] Setting a random seed
>>>>> [ 181.7] Installing packages: sudo qemu-guest-agent python3 bash policycoreutils-python-utils
>>>>> [ 194.7] Uninstalling packages: dracut-config-rescue
>>>>> [ 195.0] Deleting: /boot/*rescue*
>>>>> [ 195.0] Deleting: /boot/loader/entries/*rescue*
>>>>> [ 195.0] Running: useradd -u 1012 -s /bin/bash -m kdevops
>>>>> [ 195.1] Appending line to /etc/sudoers.d/kdevops
>>>>> [ 195.1] Editing: /etc/default/grub
>>>>> [ 195.2] Running: /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg
>>>>> [ 195.9] Installing packages: isc-dhcp-client ifupdown
>>>>> Updating and loading repositories:
>>>>>  Fedora 41 - x86_64                     100% | 221.4 KiB/s |  30.3 KiB |  00m00s
>>>>>  Fedora 41 - x86_64 - Updates           100% | 289.6 KiB/s |  29.0 KiB |  00m00s
>>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   3.2 KiB/s | 986.0   B |  00m00s
>>>>>  Fedora 41 - x86_64                     100% | 241.7 KiB/s | 480.5 KiB |  00m02s
>>>>>  Fedora 41 - x86_64 - Updates           100% |   4.4 MiB/s |  11.5 MiB |  00m03s
>>>>>  Fedora 41 openh264 (From Cisco) - x86_ 100% |   7.3 KiB/s |   5.8 KiB |  00m01s
>>>>> Repositories loaded.
>>>>> Package "sudo-1.9.15-5.p5.fc41.x86_64" is already installed.
>>>>> ...
>>>>> Complete!
>>>>> Generating grub configuration file ...
>>>>> Adding boot menu entry for UEFI Firmware Settings ...
>>>>> done
>>>>> Updating and loading repositories:
>>>>> Repositories loaded.
>>>>> Failed to resolve the transaction:
>>>>> No match for argument: isc-dhcp-client
>>>>> No match for argument: ifupdown
>>>>> You can try to add to command line:
>>>>>   --skip-unavailable to skip unavailable packages
>>>>> virt-builder: error: dnf -y install 'isc-dhcp-client' 'ifupdown': command
>>>>> exited with an error
>>>>>
>>>>> If reporting bugs, run virt-builder with debugging enabled and include the
>>>>> complete output:
>>>>>
>>>>>   virt-builder -v -x [...]
>>>> This appears to come from the new base_image virt_builder template,
>>>> where we have:
>>>>
>>>> {% if distro_debian_based is defined and distro_debian_based %}
>>>> {# Ugh, debian has to be told to bring up the network and regenerate ssh
>>>> keys #}
>>>> {# Hope we get that interface name right! #}
>>>> install isc-dhcp-client,ifupdown
>>>> mkdir /etc/network/interfaces.d/
>>>>
>>>> Which I would expect to apply only when the /guest/ is Debian, not
>>>> the host. It could be that the distro detection logic works only when
>>>> both the host and guest are the same?
>>>
>>> I think the problem arises because the new base_image role uses these
>>> distro checks instead of what bringup_guestfs.sh did, which was select
>>> these commands based on {{ virtbuilder_os_version }}
>>>
>>> virt-builder.j2 needs to use virtbuilder_os_version for these checks.
>> virtbuild_os_version doesn't give us quite everything that is needed.
>> Looks like the template needs to add some more cruft if Trixie is in
>> use. So I've put together a possible fix, pushed to the cel-fixes branch
>> in linux-kdevops. Please have a look.
>>
>> Passes some (very) light testing with a Fedora 41 host and all of a
>> Debian 12, Debian 13, or Fedora 42 guest. Bringup gets past the
>> virt-builder stage now for all three.
>>
>>
> 
> FYI, I tested this on Debian Trixie with a Fedora 41 guestfs image. It didn't
> fully work, but it got further than before. The guest now runs, though with many
> issues, likely due to low disk space. I suspect the resize task didn't run.
> 
> systemd[1]: tmp.mount: Start request repeated too quickly.
> systemd[1]: tmp.mount: Failed with result 'exit-code'.
> systemd[1]: Failed to mount tmp.mount - Temporary Directory /tmp.
> systemd[1]: systemd-oomd.service: Start request repeated too quickly.
> systemd[1]: systemd-oomd.service: Failed with result 'exit-code'.
> systemd[1]: Failed to start systemd-oomd.service - Userspace Out-Of-Memory (OOM) Killer
> 
> update_ssh_config_guestfs : Check if the new fixed include directive was used ----------------- 0.00s
> + LIBVIRT_DEFAULT_URI=qemu:///system
> + .//scripts/update_ssh_config_guestfs.py
> Traceback (most recent call last):
>   File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 109, in <module>
>     main()
>     ~~~~^^
>   File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 96, in main
>     addr = get_addr(name)
>   File "/home/dagomez.linux/src/kdevops/.//scripts/update_ssh_config_guestfs.py", line 40, in get_addr
>     raise Exception(f"Unable to get an address for {name} after 60s")
> Exception: Unable to get an address for dagomez-xarray after 60s
> make: *** [scripts/guestfs.Makefile:59: .provisioned_once_ssh] Error 1
> 
> On the other hand, images look size-wise just fine:
> 
> ls -lhsta /var/lib/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw
> 1.8G -rw-r--r-- 1 root root 20G Oct 18 00:19 /var/lib/libvirt/images/kdevops/guestfs/base_images/fedora-41.raw
> $ ls -lhsta /var/lib/libvirt/images/kdevops/guestfs/dagomez-xarray/root.raw
> 1.8G -rw-r--r-- 1 libvirt-qemu libvirt-qemu 20G Oct 18 00:39 /var/lib/libvirt/images/kdevops/guestfs/dagomez-xarray/root.raw

I've added support to select the guestfs image (e.g. debian-13/fedora-41) from
GitHub manual actions when in use with branch: ci-testing/fedora-locales-fix.
The server (Debian) reproduces the error above. You have full kdevops logs here:

https://github.com/linux-kdevops/kdevops/actions/runs/18606712456/job/53057523581

The branch also includes the patch discussed and tested to fix fedora hosts
(controller node). I'll push to main when it passes.