From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 045BC2F0678 for ; Fri, 7 Nov 2025 23:24:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762557886; cv=none; b=odHdiY2lL3wKgXAgHnXTzxFn+Dc8MEYw7WMU+AOIwgoJYmMQDP4G2nxoHQTqnEoEpmUgwJ4oqwvLj6GaSaQaenHcPJ/tGvcenB0HEhcucN5aHE7+oiqt650LgqT8/f2o+ru6GpyE9tZ1mJ/2k7cBov9Rl9HaJmEe4GMFlQTW47o= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762557886; c=relaxed/simple; bh=ToJgHjHcmqy1gptmVOALmG0iL7JDbOIJfzx7UhvLAts=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=CrZW37EbP/3X/Yk5L68EE7yZefhdjqNsRVDAhaSi9IowRs2B8EQYRZacq94fVUkmsLI/dNIwwIWpBjvXAMuR6wAWjizA/uIdbt81Yy3fY2cPyjmLwfA/TBw76KOPzovQHbQxxUZ013RZmW91r3FJcKh/oYunv0w3m1ijywTWfxA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Arcpam2V; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Arcpam2V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4AF26C16AAE; Fri, 7 Nov 2025 23:24:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762557885; bh=ToJgHjHcmqy1gptmVOALmG0iL7JDbOIJfzx7UhvLAts=; h=Date:Reply-To:Subject:From:To:Cc:References:In-Reply-To:From; b=Arcpam2V2Bc/2TZPcEyQy649SBHmGA7bWXF64bw+WarvZh/Kbo5PY9244NXJ5EV5u oE4B6Z1lpUrXsR+RYeSPqWqpBUOoIgJRWXX2tvb+JQE3RZ/iTUbg6cv4Ojo3wSptGY Wbe4DBFnFKT22kxYciS4Nmv6evLWGcVn3tqYpYWX4uCyv+I5H/oUFoNH2rrjH0Hbxh PLxLoaq9YLJ6Myd7kKENbPHIvtsIEkujr9yNP+f2uANiDQUxRgKeBKpHNd5Br355Zw iIBVROYzJuz+U/kP0X7EennEzWJZ7sGZUrZ2cEF+iuivlxj2ehHWZELBLkLVRwszZ5 5RzOmDdwCuaOg== Message-ID: <426e2136-0e93-423c-b3dc-f41be6e61780@kernel.org> Date: Sat, 8 Nov 2025 00:24:42 +0100 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: Daniel Gomez Subject: Re: [PATCH v2 09/15] bootlinux: fix os detection for 9p build dependency installation From: Daniel Gomez To: Chuck Lever , Luis Chamberlain Cc: kdevops@lists.linux.dev, Daniel Gomez References: <20251029-fedora-on-debian-v2-0-ddc6e5bebc15@samsung.com> <20251029-fedora-on-debian-v2-9-ddc6e5bebc15@samsung.com> <7a126e42-a65c-4020-b7b4-3e8c784be593@oracle.com> <9657ff48-8cc0-4ce6-b52c-ef1ab40c7a74@kernel.org> <0439acaf-b96b-4553-9a3a-59cd7134e50b@oracle.com> <52e65c62-70eb-47da-b0e2-c92e5c514cbd@kernel.org> <0551827c-a0bd-4589-b9e4-073aa545ccb0@kernel.org> Content-Language: en-US Organization: kernel.org In-Reply-To: <0551827c-a0bd-4589-b9e4-073aa545ccb0@kernel.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 08/11/2025 00.20, Daniel Gomez wrote: > > > On 07/11/2025 21.53, Chuck Lever wrote: >> On 11/7/25 3:47 PM, Daniel Gomez wrote: >>> >>> >>> On 07/11/2025 21.17, Chuck Lever wrote: >>>> On 11/7/25 3:00 PM, Daniel Gomez wrote: >>>>> On 07/11/2025 20.22, Chuck Lever wrote: >>>>>> On 10/29/25 8:40 AM, Daniel Gomez wrote: >>>>>>> From: Daniel Gomez >>>>>>> >>>>>>> Dependency installation for 9P builds was checking ansible_os_family, >>>>>>> which references the target guest's OS instead of the control host where >>>>>>> packages are actually installed. This caused incorrect packages to be >>>>>>> installed when building Fedora guests on Debian hosts. >>>>>>> >>>>>>> Replace ansible_os_family checks with Kconfig distro variables >>>>>>> (distro_debian_based, distro_fedora, etc.) which correctly detect >>>>>>> the control host's distribution. Add fallback defaults to ensure the >>>>>>> variables are always defined for standalone role usage. >>>>>>> >>>>>>> Generated-by: Claude AI >>>>>>> Suggested-by: Chuck Lever >>>>>> >>>>>> Hrm. I might not have understood the whole picture. >>>>>> >>>>>> Yes, distro_yada_based does pick the controller's OS version. >>>>>> But so does ansible_os_family when the task is running on >>>>>> localhost. >>>>> >>>>> Just a reminder that we control where tasks are deployed with --limit and/or >>>>> hosts: field in bootlinux.yml. For this case, IIRC, the playbook was run for >>>>> baseline:dev but tasks were "delegated_to" localhost. >>>> >>>> That's new since I did the "build kernel on a separate target node" >>>> changes and the related clean-ups in the bootlinux playbook. Not a >>>> finger-point, just saying we could have collided somewhere. >>>> >>>> >>>>>> I just hit a problem with the "build linux on a separate >>>>>> target node" configuration, where the target is running >>>>>> Debian 11 and the controller is running Fedora 41. The >>>>>> bootlinux install-deps/main.yml checks were coming to >>>>>> the wrong conclusion. >>>>> >>>>> Can you share the output? And the --limit argument used in this case? >>>>> FYI, I normally enable CONFIG_KDEVOPS_MAKE_VERBOSE=y) which prints: >>>>> >>>>> make bringup >>>>> + make linux-clone >>>>> ==> [guestfs/kdevops_nodes.yaml] >>>>> + ansible-playbook playbooks/gen_nodes.yml --extra-vars=@./extra_vars.yaml >>>>> ... >>>>> ==> [linux-clone-9p] >>>>> + ansible-playbook --limit localhost playbooks/bootlinux.yml >>>>> '--extra-vars=target_linux_git=/mirror/linux.git ... >>>>> ... >>>> >>>> Here's from my scroll-back buffer earlier this afternoon: >>>> >>>> TASK [install-rust-deps : Install Rust build dependencies] >>>> ************************************************************************************* >>>> included: >>>> /home/cel/src/kdevops/buildbot-configs/playbooks/roles/install-rust-deps/tasks/install-deps/main.yml >>>> for kernel-builder >>>> >>>> TASK [install-rust-deps : Install Rust build dependencies] >>>> ************************************************************************************* >>>> changed: [kernel-builder] >>>> FAILED - RETRYING: [kernel-builder]: Install packages we care about (3 >>>> retries left). >>>> FAILED - RETRYING: [kernel-builder]: Install packages we care about (2 >>>> retries left). >>>> FAILED - RETRYING: [kernel-builder]: Install packages we care about (1 >>>> retries left). >>>> >>>> TASK [bootlinux : Install packages we care about] >>>> ********************************************************************************************** >>>> task path: >>>> /home/cel/src/kdevops/buildbot-configs/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml:8 >>>> fatal: [kernel-builder]: FAILED! => { >>>> "ansible_facts": { >>>> "pkg_mgr": "apt" >>>> }, >>>> "attempts": 3, >>>> "changed": false >>>> } >>>> >>>> MSG: >>>> >>>> ('Could not detect which major revision of dnf is in use, which is >>>> required to determine module backend.', 'You should manually specify >>>> use_backend to tell the module whether to use the dnf4 or dnf5 backend})') >>>> >>>> >>>> As you can see, bootlinux is trying to install the redhat deps on >>>> the target, but in fact the target is running Debian 11. This is >>>> because the install-deps play is now looking at distro_yada_based >>>> even though it is running on the target. >>> >>> I think we need to detect both cases based on the target selection >>> (BOOTLINUX_TARGETS). Can you give this a try/check/review? >>> >>> For 9p targets (controller node, ie BOOTLINUX_9P), we keep the >>> distro_yada_based. For builder targets (BOOTLINUX_BUILDER) we use the >>> ansible_os_family: >>> >>> diff --git a/playbooks/roles/bootlinux/tasks/install-deps/main.yml b/playbooks/roles/bootlinux/tasks/install-deps/main.yml >>> index 058f3926..88d2baad 100644 >>> --- a/playbooks/roles/bootlinux/tasks/install-deps/main.yml >>> +++ b/playbooks/roles/bootlinux/tasks/install-deps/main.yml >>> @@ -1,15 +1,36 @@ >>> --- >>> -- name: Debian-specific setup >>> +- name: Debian-specific setup (9p/controller node) >>> ansible.builtin.import_tasks: debian/main.yml >>> when: >>> - distro_debian_based|bool >>> + - bootlinux_9p|bool >>> >>> -- name: SuSE-specific setup >>> +- name: Debian-specific setup (builder node) >>> + ansible.builtin.import_tasks: debian/main.yml >>> + when: >>> + - bootlinux_builder|bool >>> + - ansible_os_family == "Debian" >>> + >>> +- name: SuSE-specific setup (9p/controller node) >>> ansible.builtin.import_tasks: suse/main.yml >>> when: >>> - distro_suse_based|bool >>> + - bootlinux_9p|bool >>> >>> -- name: Red Hat-specific setup >>> +- name: SuSE-specific setup (builder node) >>> + ansible.builtin.import_tasks: suse/main.yml >>> + when: >>> + - bootlinux_builder|bool >>> + - ansible_os_family == "Suse" >>> + >>> +- name: Red Hat-specific setup (controller node) >>> ansible.builtin.import_tasks: redhat/main.yml >>> when: >>> - distro_redhat_based|bool >>> + - bootlinux_9p|bool >>> + >>> +- name: Red Hat-specific setup (builder node) >>> + ansible.builtin.import_tasks: redhat/main.yml >>> + when: >>> + - bootlinux_builder|bool >>> + - ansible_os_family == "RedHat" >>> >>> IIUC, this should return the correct ansible_os_family value when builder target >>> is selected because the task is executed on the guest with --limit baseline:dev. >> Hi Daniel - >> >> None of these tasks will execute if bootlinux_targets == true, AFAICT. > > Sorry, I missed that target. I meant to add that condition too when we use > ansible_os_family case. > >> >> Why can't this work based on which host the install-deps tasks are >> running on? That was the original design I had in mind, and seems >> most Ansible-like. > > I can't replicate the issue. I think we can revert the patch. Here my tests: > > fedora: https://github.com/linux-kdevops/kdevops/actions/runs/19183690762 > debian: https://github.com/linux-kdevops/kdevops/actions/runs/19183543710 > > I sent this too soon. https://github.com/linux-kdevops/kdevops/actions/runs/19183690762/job/54845995905 + ansible-playbook --limit baseline:dev playbooks/bootlinux.yml '--extra-vars=target_linux_git=/mirror/linux.git target_linux_tree=linux target_linux_ref=v6.15 target_linux_config=config-v6.15 target_linux_kernelrelease= target_linux_localversion= target_linux_shallow_depth=1 bootlinux_9p_host_path='\''/data4-xfs/gh/actions-runner/kdevops-ci-0019/_work/kdevops/kdevops/linux'\'' bootlinux_9p_msize='\''131072'\'' bootlinux_9p_fsdev='\''kdevops_9p_fsdev'\'' bootlinux_9p_security_model='\''none'\'' bootlinux_9p_driver='\''virtio-9p-pci'\'' bootlinux_9p_mount_tag='\''kdevops_9p_bootlinux'\'' bootlinux_cxl_test=' ==> [linux] Warning: : Could not match supplied host pattern, ignoring: dev Warning: : Deprecation warnings can be disabled by setting `deprecation_warnings=False` in ansible.cfg. [DEPRECATION WARNING]: The 'community.general.diy' callback plugin implements deprecated method 'v2_on_any'. This feature will be removed from the callback plugin API in ansible-core version 2.23. Use event-specific callback methods instead. [DEPRECATION WARNING]: Passing `convert_data` to `template` is deprecated. This feature will be removed from ansible-core version 2.23. PLAY: BOOTLINUX TASK: Gathering Facts [kci-19183690762-181-nvme] ⠀⠀✓ [kci-19183690762-181-nvme] TASK: Import optional extra_args file [kci-19183690762-181-nvme] ⠀⠀✓ [kci-19183690762-181-nvme] TASK: Select the .config file for building the test kernel [kci-19183690762-181-nvme] ⠀⠀✓ [kci-19183690762-181-nvme] ... TASK: Build the Linux kernel on the controller host [kci-19183690762-181-nvme] ⠀⠀✓ [kci-19183690762-181-nvme] included: /data4-xfs/gh/actions-runner/kdevops-ci-0019/_work/kdevops/kdevops/playbooks/roles/bootlinux/tasks/build/9p.yml for kci-19183690762-181-nvme TASK: Update apt cache [kci-19183690762-181-nvme] ⠀⠀⊘ [kci-19183690762-181-nvme] TASK: Install Linux kernel build dependencies [kci-19183690762-181-nvme] ⠀⠀⊘ [kci-19183690762-181-nvme] TASK: Install Linux kernel build dependencies for SUSE sources [kci-19183690762-181-nvme] ⠀⠀⊘ [kci-19183690762-181-nvme] TASK: Enable installation of packages from EPEL [kci-19183690762-181-nvme] ⠀⠀⊘ [kci-19183690762-181-nvme] TASK: Install packages we care about [kci-19183690762-181-nvme] FAILED - RETRYING: [kci-19183690762-181-nvme -> localhost]: Install packages we care about (3 retries left). FAILED - RETRYING: [kci-19183690762-181-nvme -> localhost]: Install packages we care about (2 retries left). FAILED - RETRYING: [kci-19183690762-181-nvme -> localhost]: Install packages we care about (1 retries left). ⠀⠀FAILED: [kci-19183690762-181-nvme] => ['Could not detect which major revision of dnf is in use, which is required to determine module backend.', 'You should manually specify use_backend to tell the module whether to use the dnf4 or dnf5 backend})'] Error: : Task failed: Action failed: ('Could not detect which major revision of dnf is in use, which is required to determine module backend.', 'You should manually specify use_backend to tell the module whether to use the dnf4 or dnf5 backend})') Origin: /data4-xfs/gh/actions-runner/kdevops-ci-0019/_work/kdevops/kdevops/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml:8:3 6 - ansible_distribution != "Fedora" 7 8 - name: Install packages we care about ^ column 3 fatal: [kci-19183690762-181-nvme -> localhost]: FAILED! => {"attempts": 3, "changed": false, "msg": ["Could not detect which major revision of dnf is in use, which is required to determine module backend.", "You should manually specify use_backend to tell the module whether to use the dnf4 or dnf5 backend})"]}>> >>