[PATCH v3 09/11] devconfig: add automatic APT mirror fallback with DEB822 modernization

[Luis Chamberlain <mcgrof@kernel.org>]

Debian testing (trixie) VMs can fail to provision when configured APT
mirrors become unavailable or unresponsive. This is particularly common
with local or regional mirrors that may have intermittent connectivity
issues.

This fix adds automatic mirror health checking specifically for Debian
testing systems. The implementation:

1. Detects current APT format (legacy sources.list or DEB822)
2. Extracts the configured mirror hostname from either format
3. Tests connectivity to the mirror on port 80 with 10 second timeout
4. Falls back to official Debian mirrors if the test fails
5. Backs up the original sources before making changes
6. Applies the new sources in modern DEB822 format
7. Removes legacy sources.list when migrating from old format
8. Updates the APT cache after switching mirrors

The check only runs on Debian testing systems where devconfig_debian_testing
is set to true, avoiding any impact on stable Debian or other distributions.

By modernizing to DEB822 format during fallback, we align with Debian's
recommended configuration while ensuring VMs can successfully provision
even when the initially configured mirror is unavailable.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 .../devconfig/tasks/check-apt-mirrors.yml     | 99 +++++++++++++++++++
 playbooks/roles/devconfig/tasks/main.yml      |  8 ++
 .../templates/debian-testing-fallback.sources | 13 +++
 3 files changed, 120 insertions(+)
 create mode 100644 playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
 create mode 100644 playbooks/roles/devconfig/templates/debian-testing-fallback.sources

diff --git a/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
new file mode 100644
index 000000000000..96e4048015d1
--- /dev/null
+++ b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
@@ -0,0 +1,99 @@
+---
+# Only run mirror checks for Debian testing (trixie) where mirror issues are common
+- name: Check for DEB822-style sources
+  stat:
+    path: /etc/apt/sources.list.d/debian.sources
+  register: deb822_sources
+
+- name: Extract current APT mirror hostname (DEB822 format)
+  shell: |
+    grep -E "^URIs:" /etc/apt/sources.list.d/debian.sources | head -1 | awk '{print $2}' | sed -E 's|https?://||' | cut -d'/' -f1
+  register: apt_mirror_host_deb822
+  changed_when: false
+  ignore_errors: yes
+  when: deb822_sources.stat.exists
+
+- name: Extract current APT mirror hostname (legacy format)
+  shell: |
+    grep -E "^deb\s+http" /etc/apt/sources.list | head -1 | awk '{print $2}' | sed 's|http://||' | cut -d'/' -f1
+  register: apt_mirror_host_legacy
+  changed_when: false
+  ignore_errors: yes
+  when: not deb822_sources.stat.exists
+
+- name: Set unified mirror hostname
+  set_fact:
+    apt_mirror_host:
+      stdout: "{{ apt_mirror_host_deb822.stdout if deb822_sources.stat.exists else apt_mirror_host_legacy.stdout }}"
+
+- name: Check connectivity to current APT mirror
+  wait_for:
+    host: "{{ apt_mirror_host.stdout }}"
+    port: 80
+    timeout: 10
+  register: mirror_connectivity
+  ignore_errors: yes
+  when: apt_mirror_host.stdout != ""
+
+- name: Display mirror check results
+  debug:
+    msg: |
+      Current APT mirror: {{ apt_mirror_host.stdout | default('Not found') }}
+      Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }}
+  when: apt_mirror_host.stdout != ""
+
+- name: Fall back to official Debian mirrors if current mirror fails
+  block:
+    - name: Backup current sources (DEB822 format)
+      copy:
+        src: /etc/apt/sources.list.d/debian.sources
+        dest: /etc/apt/sources.list.d/debian.sources.backup
+        remote_src: yes
+      become: yes
+      when: deb822_sources.stat.exists
+
+    - name: Backup current sources (legacy format)
+      copy:
+        src: /etc/apt/sources.list
+        dest: /etc/apt/sources.list.backup
+        remote_src: yes
+      become: yes
+      when: not deb822_sources.stat.exists
+
+    - name: Apply Debian testing fallback sources using modern DEB822 format
+      template:
+        src: debian-testing-fallback.sources
+        dest: /etc/apt/sources.list.d/debian.sources
+        owner: root
+        group: root
+        mode: '0644'
+      become: yes
+
+    - name: Remove legacy sources.list if migrating to DEB822
+      file:
+        path: /etc/apt/sources.list
+        state: absent
+      become: yes
+      when: not deb822_sources.stat.exists
+
+    - name: Update APT cache after mirror change
+      apt:
+        update_cache: yes
+        cache_valid_time: 0
+      become: yes
+
+    - name: Inform user about mirror fallback
+      debug:
+        msg: |
+          WARNING: The configured APT mirror '{{ apt_mirror_host.stdout }}' is not accessible.
+          Falling back to official Debian testing mirrors using modern DEB822 format:
+          - deb.debian.org for main packages
+          - security.debian.org for security updates
+
+          Your sources have been migrated to /etc/apt/sources.list.d/debian.sources
+          This may result in slower package downloads depending on your location.
+          Consider configuring a local mirror for better performance.
+
+  when:
+    - apt_mirror_host.stdout != ""
+    - mirror_connectivity is failed
diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml
index 656d5389f685..ceb0f2e8b29c 100644
--- a/playbooks/roles/devconfig/tasks/main.yml
+++ b/playbooks/roles/devconfig/tasks/main.yml
@@ -30,6 +30,14 @@
   tags: hostname
 
 # Distro specific
+
+# Check and fix APT mirrors for Debian testing before installing dependencies
+- name: Check and fix APT mirrors for Debian testing
+  include_tasks: check-apt-mirrors.yml
+  when:
+    - devconfig_debian_testing is defined
+    - devconfig_debian_testing | bool
+
 - name: Install dependencies
   ansible.builtin.include_tasks: install-deps/main.yml
   tags: ['vars', 'vars_simple']
diff --git a/playbooks/roles/devconfig/templates/debian-testing-fallback.sources b/playbooks/roles/devconfig/templates/debian-testing-fallback.sources
new file mode 100644
index 000000000000..5358e3e61bee
--- /dev/null
+++ b/playbooks/roles/devconfig/templates/debian-testing-fallback.sources
@@ -0,0 +1,13 @@
+Types: deb deb-src
+URIs: https://deb.debian.org/debian
+Suites: testing testing-updates
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
+
+Types: deb deb-src
+URIs: https://security.debian.org/debian-security
+Suites: testing-security
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
\ No newline at end of file
-- 
2.47.2