Re: [Qemu-devel] [PATCH] Fedora images: use URLs from stable "archives.fedoraproject.org"

["Daniel P. Berrangé" <berrange@redhat.com>]

On Tue, Sep 03, 2019 at 08:52:18PM -0400, Cleber Rosa wrote:
> The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test,
> from tests/acceptance/linux_initrd.py, is currently failing to fetch
> the "vmlinuz" file.  The reason for the failure is that the Fedora
> project retires older versions from the "dl.fedoraproject.org" URL,
> and keeps them in "archives.fedoraproject.org".  As an added note,
> that test uses a Fedora 28 image, because of the specific Linux kernel
> version requirements of the test.
> 
> For the sake of stability, let's use URLs from the archived and
> supposedely ever stable URLs.  The good news is that the currently
> supported versions are also hosted on the later.  This change limits
> itself to change the URLs, while keeping the fetched files the same
> (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using
is serviced by the Fedora mirrors which is very desirable as it
spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting
server(s). dl.fedoraproject.org is the same master hosting service
that is intended for use by the mirror sites to sync their content
from.  Projects really shouldn't use either of these URLs for getting
any content which is available via the mirror service as it places
uncessary load on the Fedora master servers.

> Documentation and the "vm tests" fedora definition were also updated.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  qemu-doc.texi                          |  6 +++---
>  tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
>  tests/acceptance/linux_initrd.py       |  5 +++--
>  tests/vm/fedora                        |  2 +-
>  4 files changed, 22 insertions(+), 16 deletions(-)
> 
> diff --git a/qemu-doc.texi b/qemu-doc.texi
> index 577d1e8376..37795f86fb 100644
> --- a/qemu-doc.texi
> +++ b/qemu-doc.texi
> @@ -440,15 +440,15 @@ of <protocol>.
>  
>  Example: boot from a remote Fedora 20 live ISO image
>  @example
> -qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
> +qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
>  
> -qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
> +qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
>  @end example
>  
>  Example: boot from a remote Fedora 20 cloud image using a local overlay for
>  writes, copy-on-read, and a readahead of 64k
>  @example
> -qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
> +qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
>  
>  qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
>  @end example

Nothing quite says "cutting edge" like examples pointing to 5+ years old
software. Or to put it another way, we should update this to point to a
modern OS release. ie Fedora 30.

These should also be changed to download instead of dl.


> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 2504ef0150..8a9a314ab4 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:x86_64
>          :avocado: tags=machine:pc
>          """
> -        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
> -                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> +                      '/vmlinuz')
>          kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>  
> @@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:aarch64
>          :avocado: tags=machine:virt
>          """
> -        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
> -                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
> +                      '/vmlinuz')
>          kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>  
> @@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:virt
>          """
> -        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
> -                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
> +                      '/vmlinuz')
>          kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>  
> @@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:s390x
>          :avocado: tags=machine:s390_ccw_virtio
>          """
> -        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
> -                      'releases/29/Everything/s390x/os/images/kernel.img')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> +                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
> +                      '/kernel.img')
>          kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>  
> @@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:ppc64
>          :avocado: tags=machine:pseries
>          """
> -        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
> -                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> +                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
> +                      '/ppc/ppc64/vmlinuz')
>          kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want
to.

>  
> diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py
> index 23be5a63aa..c61d9826a4 100644
> --- a/tests/acceptance/linux_initrd.py
> +++ b/tests/acceptance/linux_initrd.py
> @@ -54,8 +54,9 @@ class LinuxInitrd(Test):
>          QEMU has supported up to 4 GiB initrd for recent kernel
>          Expect guest can reach 'Unpacking initramfs...'
>          """
> -        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
> -                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
> +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
> +                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
> +                      'vmlinuz')
>          kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
>          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>          max_size = 2 * (1024 ** 3) + 1
> diff --git a/tests/vm/fedora b/tests/vm/fedora
> index e8fa5bf0d2..7fec1479fb 100755
> --- a/tests/vm/fedora
> +++ b/tests/vm/fedora
> @@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
>      name = "fedora"
>      arch = "x86_64"
>  
> -    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
> +    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
>      link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
>      repo = base + "Server/x86_64/os/"
>      full = base + "Everything/x86_64/os/"

This should be changed to download instead of dl.

> -- 
> 2.21.0
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|