From: Luis Chamberlain <mcgrof@kernel.org>
To: Chuck Lever <cel@kernel.org>, Daniel Gomez <da.gomez@kruces.com>,
kdevops@lists.linux.dev
Cc: Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v3 10/11] devconfig: enhance hop1 detection to support traditional sources.list
Date: Fri, 1 Aug 2025 12:46:34 -0700 [thread overview]
Message-ID: <20250801194635.1598544-11-mcgrof@kernel.org> (raw)
In-Reply-To: <20250801194635.1598544-1-mcgrof@kernel.org>
Enhance the hop1 mirror detection to work with both DEB822 and traditional
sources.list formats. When a hop1 mirror is detected on the control host
(regardless of format) and the guest is running Debian testing, automatically
convert to DEB822 format for consistency and modern package management.
Key improvements:
- Detect hop1 servers from both /etc/apt/sources.list.d/debian.sources
(DEB822) and /etc/apt/sources.list (traditional format)
- Extract mirror host and path correctly from both formats
- Automatically convert Debian testing guests to use DEB822 when a hop1
mirror is available and accessible
- Preserve the mirror path (e.g., /debian, /debian-mirror) from the
control host configuration
- Add connectivity checks before using the hop1 mirror
- Fall back to official Debian mirrors only if hop1 is unavailable or
inaccessible
This ensures that local hop1 mirrors are used efficiently even when the
control host hasn't migrated to DEB822 format yet, while modernizing
the guest configuration for Debian testing systems.
Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
.../devconfig/tasks/check-apt-mirrors.yml | 114 ++++++++++++++++++
.../templates/debian-hop1-mirror.sources | 13 ++
2 files changed, 127 insertions(+)
create mode 100644 playbooks/roles/devconfig/templates/debian-hop1-mirror.sources
diff --git a/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
index 96e4048015d1..d86196f15ebe 100644
--- a/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
+++ b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
@@ -1,5 +1,55 @@
---
# Only run mirror checks for Debian testing (trixie) where mirror issues are common
+- name: Check hop count on control host
+ delegate_to: localhost
+ shell: |
+ {{ topdir_path }}/scripts/get-distro-has-hop-count-sources.sh 1
+ register: localhost_hop_count
+ changed_when: false
+ ignore_errors: yes
+
+- name: Extract hop1 mirror info from control host (DEB822 format)
+ delegate_to: localhost
+ shell: |
+ if [ -f /etc/apt/sources.list.d/debian.sources ]; then
+ HOST=$(grep -E "^URIs:" /etc/apt/sources.list.d/debian.sources | head -1 | awk '{print $2}' | sed -E 's|https?://||')
+ echo "${HOST}"
+ fi
+ register: localhost_hop1_mirror_deb822
+ changed_when: false
+ ignore_errors: yes
+ when: localhost_hop_count.stdout == 'y'
+
+- name: Extract hop1 mirror info from control host (legacy format)
+ delegate_to: localhost
+ shell: |
+ if [ -f /etc/apt/sources.list ] && [ ! -f /etc/apt/sources.list.d/debian.sources ]; then
+ LINE=$(grep -v "^#" /etc/apt/sources.list | grep -E "^deb\s+http" | head -1)
+ if [ -n "$LINE" ]; then
+ URL=$(echo $LINE | awk '{print $2}')
+ echo "${URL}" | sed 's|http://||'
+ fi
+ fi
+ register: localhost_hop1_mirror_legacy
+ changed_when: false
+ ignore_errors: yes
+ when: localhost_hop_count.stdout == 'y'
+
+- name: Set hop1 mirror variables
+ set_fact:
+ has_hop1_mirror: "{{ localhost_hop_count.stdout == 'y' }}"
+ hop1_mirror_full: "{{ localhost_hop1_mirror_deb822.stdout if localhost_hop1_mirror_deb822.stdout != '' else localhost_hop1_mirror_legacy.stdout }}"
+ when: localhost_hop_count.stdout == 'y'
+
+- name: Parse hop1 mirror host and path
+ set_fact:
+ hop1_mirror_host: "{{ hop1_mirror_full.split('/')[0] }}"
+ hop1_mirror_path: "/{{ hop1_mirror_full.split('/', 1)[1] if '/' in hop1_mirror_full else 'debian' }}"
+ when:
+ - has_hop1_mirror is defined
+ - has_hop1_mirror | bool
+ - hop1_mirror_full != ''
+
- name: Check for DEB822-style sources
stat:
path: /etc/apt/sources.list.d/debian.sources
@@ -42,6 +92,69 @@
Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }}
when: apt_mirror_host.stdout != ""
+- name: Configure APT sources based on hop1 availability and debian testing
+ block:
+ - name: Check connectivity to hop1 mirror if available
+ wait_for:
+ host: "{{ hop1_mirror_host }}"
+ port: 80
+ timeout: 10
+ register: hop1_mirror_connectivity
+ ignore_errors: yes
+ when:
+ - has_hop1_mirror is defined
+ - has_hop1_mirror | bool
+ - hop1_mirror_host is defined
+
+ - name: Use hop1 mirror with DEB822 format for debian testing
+ block:
+ - name: Backup current sources
+ copy:
+ src: "{{ item }}"
+ dest: "{{ item }}.backup"
+ remote_src: yes
+ become: yes
+ loop:
+ - /etc/apt/sources.list
+ - /etc/apt/sources.list.d/debian.sources
+ ignore_errors: yes
+
+ - name: Apply hop1 mirror configuration using DEB822 format
+ template:
+ src: debian-hop1-mirror.sources
+ dest: /etc/apt/sources.list.d/debian.sources
+ owner: root
+ group: root
+ mode: '0644'
+ become: yes
+
+ - name: Remove legacy sources.list if migrating to DEB822
+ file:
+ path: /etc/apt/sources.list
+ state: absent
+ become: yes
+ when: not deb822_sources.stat.exists
+
+ - name: Update APT cache with hop1 mirror
+ apt:
+ update_cache: yes
+ cache_valid_time: 0
+ become: yes
+
+ - name: Inform user about hop1 mirror usage
+ debug:
+ msg: |
+ Using local hop1 mirror with DEB822 format:
+ Mirror: {{ hop1_mirror_host }}{{ hop1_mirror_path }}
+
+ This provides faster package downloads from your local mirror.
+ Sources have been converted to modern DEB822 format.
+ when:
+ - has_hop1_mirror is defined
+ - has_hop1_mirror | bool
+ - hop1_mirror_connectivity is not failed
+ - hop1_mirror_host is defined
+
- name: Fall back to official Debian mirrors if current mirror fails
block:
- name: Backup current sources (DEB822 format)
@@ -97,3 +210,4 @@
when:
- apt_mirror_host.stdout != ""
- mirror_connectivity is failed
+ - not (has_hop1_mirror is defined and has_hop1_mirror | bool and hop1_mirror_connectivity is not failed)
diff --git a/playbooks/roles/devconfig/templates/debian-hop1-mirror.sources b/playbooks/roles/devconfig/templates/debian-hop1-mirror.sources
new file mode 100644
index 000000000000..eff66706396e
--- /dev/null
+++ b/playbooks/roles/devconfig/templates/debian-hop1-mirror.sources
@@ -0,0 +1,13 @@
+Types: deb deb-src
+URIs: http://{{ hop1_mirror_host }}{{ hop1_mirror_path | default('/debian') }}
+Suites: testing testing-updates
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
+
+Types: deb deb-src
+URIs: https://security.debian.org/debian-security
+Suites: testing-security
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
\ No newline at end of file
--
2.47.2
next prev parent reply other threads:[~2025-08-01 19:46 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-01 19:46 [PATCH v3 00/11] kdevops: add support for A/B testing Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 01/11] roles/guestfs: add missing bootlinux_9p: False Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 02/11] Makefile: suppress Ansible warnings during configuration generation Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 03/11] playbooks: few space cleanups Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 04/11] style: add extensive code formatting checks to make style Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 05/11] Makefile: move styling to scripts/style.Makefile Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 06/11] CLAUDE.md: add instrucitons to verify commit Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 07/11] all: run black Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 08/11] scripts: enhance hop count detection to support DEB822 format Luis Chamberlain
2025-08-01 19:46 ` [PATCH v3 09/11] devconfig: add automatic APT mirror fallback with DEB822 modernization Luis Chamberlain
2025-08-01 19:46 ` Luis Chamberlain [this message]
2025-08-01 19:46 ` [PATCH v3 11/11] bootlinux: add support for A/B kernel testing Luis Chamberlain
2025-08-02 17:15 ` [PATCH v3 00/11] kdevops: add support for A/B testing Luis Chamberlain
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250801194635.1598544-11-mcgrof@kernel.org \
--to=mcgrof@kernel.org \
--cc=cel@kernel.org \
--cc=da.gomez@kruces.com \
--cc=kdevops@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox