From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7BE01E492D for ; Fri, 1 Aug 2025 19:46:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754077601; cv=none; b=FT5O2/T+1Fm7+zkFUPlP1uubmubMLFQ0eTmI4/Ezcg9Xy29lO3dmu7NnCGYRcXn7m+ic8l7hRn77MXZFN0dau9n/qz+MJBSP5yA006n07/0SUxa3bxBMqNhDvcytcmNJc7ZU+UCmo8t8Dmbvif/ReQxWqEjjAqjb8BreahaIdxo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754077601; c=relaxed/simple; bh=M11pxHHOx9o7tNAowx3YKbfxwGFPUerp8KAOVHLNDjI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fATOD0j/ohYNTDgAVBzjLGXBStIfGIyxGs2ZgIY82woPSvlPfHGbnNygZPKX1VcRGFq1JrOx0BXyeHIMm2/q957NjyhKoY47ANmPG2cuC8trYdiF8Lyc/0Zo4u/e6Nq1uK/WX1pQJPPc3gfhlvyrfg6+AZ7/9YIbB0Lv8JeDkF0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=OH310CzT; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OH310CzT" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=QbgroPYvvu6ca67WC5aZew9LMtbOSUgY90uJrtinqDw=; b=OH310CzTld0SdwutulM+4CHLkq IeGFm4WX1GsQkdn29FpQ7J6+2TqNuBzB43gTpQBGEuj1Gb8XunQbrHomaKIADE3zApOvaoNpIGDoE daJehlkoI6CuY6VTm+gy2GyK9btcV+2kx+w4Qwz5LbousHMy67dD3VO0gD5H5LwuAxRUxXPVFHpvQ vm9mauu87tKdLjyYH09MkIhFqhyDrWTmolPyvweRrchqr2PYqoCyxfEpqveHES1EgEXS25RjkrBbT gQ+WFGPDlOr1Vexj1nJlkousLj9FEqg2xPKgO/H2qD6+9lPSjfHBgftGWlaPG0QUu6OHw/1YotAfh nIr3qRZw==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uhvi9-00000006hrj-1Gqq; Fri, 01 Aug 2025 19:46:37 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v3 09/11] devconfig: add automatic APT mirror fallback with DEB822 modernization Date: Fri, 1 Aug 2025 12:46:33 -0700 Message-ID: <20250801194635.1598544-10-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250801194635.1598544-1-mcgrof@kernel.org> References: <20250801194635.1598544-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Debian testing (trixie) VMs can fail to provision when configured APT mirrors become unavailable or unresponsive. This is particularly common with local or regional mirrors that may have intermittent connectivity issues. This fix adds automatic mirror health checking specifically for Debian testing systems. The implementation: 1. Detects current APT format (legacy sources.list or DEB822) 2. Extracts the configured mirror hostname from either format 3. Tests connectivity to the mirror on port 80 with 10 second timeout 4. Falls back to official Debian mirrors if the test fails 5. Backs up the original sources before making changes 6. Applies the new sources in modern DEB822 format 7. Removes legacy sources.list when migrating from old format 8. Updates the APT cache after switching mirrors The check only runs on Debian testing systems where devconfig_debian_testing is set to true, avoiding any impact on stable Debian or other distributions. By modernizing to DEB822 format during fallback, we align with Debian's recommended configuration while ensuring VMs can successfully provision even when the initially configured mirror is unavailable. Generated-by: Claude AI Signed-off-by: Luis Chamberlain --- .../devconfig/tasks/check-apt-mirrors.yml | 99 +++++++++++++++++++ playbooks/roles/devconfig/tasks/main.yml | 8 ++ .../templates/debian-testing-fallback.sources | 13 +++ 3 files changed, 120 insertions(+) create mode 100644 playbooks/roles/devconfig/tasks/check-apt-mirrors.yml create mode 100644 playbooks/roles/devconfig/templates/debian-testing-fallback.sources diff --git a/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml new file mode 100644 index 000000000000..96e4048015d1 --- /dev/null +++ b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml @@ -0,0 +1,99 @@ +--- +# Only run mirror checks for Debian testing (trixie) where mirror issues are common +- name: Check for DEB822-style sources + stat: + path: /etc/apt/sources.list.d/debian.sources + register: deb822_sources + +- name: Extract current APT mirror hostname (DEB822 format) + shell: | + grep -E "^URIs:" /etc/apt/sources.list.d/debian.sources | head -1 | awk '{print $2}' | sed -E 's|https?://||' | cut -d'/' -f1 + register: apt_mirror_host_deb822 + changed_when: false + ignore_errors: yes + when: deb822_sources.stat.exists + +- name: Extract current APT mirror hostname (legacy format) + shell: | + grep -E "^deb\s+http" /etc/apt/sources.list | head -1 | awk '{print $2}' | sed 's|http://||' | cut -d'/' -f1 + register: apt_mirror_host_legacy + changed_when: false + ignore_errors: yes + when: not deb822_sources.stat.exists + +- name: Set unified mirror hostname + set_fact: + apt_mirror_host: + stdout: "{{ apt_mirror_host_deb822.stdout if deb822_sources.stat.exists else apt_mirror_host_legacy.stdout }}" + +- name: Check connectivity to current APT mirror + wait_for: + host: "{{ apt_mirror_host.stdout }}" + port: 80 + timeout: 10 + register: mirror_connectivity + ignore_errors: yes + when: apt_mirror_host.stdout != "" + +- name: Display mirror check results + debug: + msg: | + Current APT mirror: {{ apt_mirror_host.stdout | default('Not found') }} + Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }} + when: apt_mirror_host.stdout != "" + +- name: Fall back to official Debian mirrors if current mirror fails + block: + - name: Backup current sources (DEB822 format) + copy: + src: /etc/apt/sources.list.d/debian.sources + dest: /etc/apt/sources.list.d/debian.sources.backup + remote_src: yes + become: yes + when: deb822_sources.stat.exists + + - name: Backup current sources (legacy format) + copy: + src: /etc/apt/sources.list + dest: /etc/apt/sources.list.backup + remote_src: yes + become: yes + when: not deb822_sources.stat.exists + + - name: Apply Debian testing fallback sources using modern DEB822 format + template: + src: debian-testing-fallback.sources + dest: /etc/apt/sources.list.d/debian.sources + owner: root + group: root + mode: '0644' + become: yes + + - name: Remove legacy sources.list if migrating to DEB822 + file: + path: /etc/apt/sources.list + state: absent + become: yes + when: not deb822_sources.stat.exists + + - name: Update APT cache after mirror change + apt: + update_cache: yes + cache_valid_time: 0 + become: yes + + - name: Inform user about mirror fallback + debug: + msg: | + WARNING: The configured APT mirror '{{ apt_mirror_host.stdout }}' is not accessible. + Falling back to official Debian testing mirrors using modern DEB822 format: + - deb.debian.org for main packages + - security.debian.org for security updates + + Your sources have been migrated to /etc/apt/sources.list.d/debian.sources + This may result in slower package downloads depending on your location. + Consider configuring a local mirror for better performance. + + when: + - apt_mirror_host.stdout != "" + - mirror_connectivity is failed diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml index 656d5389f685..ceb0f2e8b29c 100644 --- a/playbooks/roles/devconfig/tasks/main.yml +++ b/playbooks/roles/devconfig/tasks/main.yml @@ -30,6 +30,14 @@ tags: hostname # Distro specific + +# Check and fix APT mirrors for Debian testing before installing dependencies +- name: Check and fix APT mirrors for Debian testing + include_tasks: check-apt-mirrors.yml + when: + - devconfig_debian_testing is defined + - devconfig_debian_testing | bool + - name: Install dependencies ansible.builtin.include_tasks: install-deps/main.yml tags: ['vars', 'vars_simple'] diff --git a/playbooks/roles/devconfig/templates/debian-testing-fallback.sources b/playbooks/roles/devconfig/templates/debian-testing-fallback.sources new file mode 100644 index 000000000000..5358e3e61bee --- /dev/null +++ b/playbooks/roles/devconfig/templates/debian-testing-fallback.sources @@ -0,0 +1,13 @@ +Types: deb deb-src +URIs: https://deb.debian.org/debian +Suites: testing testing-updates +Components: main contrib non-free non-free-firmware +Enabled: yes +Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg + +Types: deb deb-src +URIs: https://security.debian.org/debian-security +Suites: testing-security +Components: main contrib non-free non-free-firmware +Enabled: yes +Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg \ No newline at end of file -- 2.47.2