From: "Philippe Mathieu-Daudé" <philmd@redhat.com>
To: Cleber Rosa <crosa@redhat.com>,
qemu-devel@nongnu.org,
"Daniel P . Berrange" <berrange@redhat.com>
Cc: "Peter Maydell" <peter.maydell@linaro.org>,
"Sarah Harris" <S.E.Harris@kent.ac.uk>,
"Mark Cave-Ayland" <mark.cave-ayland@ilande.co.uk>,
"Thomas Huth" <thuth@redhat.com>,
"Jiaxun Yang" <jiaxun.yang@flygoat.com>,
"KONRAD Frederic" <frederic.konrad@adacore.com>,
"Willian Rampazzo" <wrampazz@redhat.com>,
"Yoshinori Sato" <ysato@users.sourceforge.jp>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Aleksandar Markovic" <aleksandar.qemu.devel@gmail.com>,
"Hervé Poussineau" <hpoussin@reactos.org>,
"Antony Pavlov" <antonynpavlov@gmail.com>,
"Aleksandar Rikalo" <aleksandar.rikalo@syrmia.com>,
"Eduardo Habkost" <ehabkost@redhat.com>,
"Fabien Chouteau" <chouteau@adacore.com>,
qemu-arm@nongnu.org, "Michael Rolnik" <mrolnik@gmail.com>,
"Pavel Dovgalyuk" <pavel.dovgaluk@ispras.ru>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Beraldo Leal" <bleal@redhat.com>,
qemu-ppc@nongnu.org, "Paolo Bonzini" <pbonzini@redhat.com>,
"Aurelien Jarno" <aurelien@aurel32.net>
Subject: Re: [PATCH 1/4] Acceptance tests: use an available kernel image package for arm
Date: Mon, 7 Sep 2020 10:06:13 +0200 [thread overview]
Message-ID: <001385a1-92eb-d311-88b5-8e1e7f6f1f48@redhat.com> (raw)
In-Reply-To: <20200907042000.415931-2-crosa@redhat.com>
[Cc'ing Daniel who usually have good ideas for that
kind if project-wide problem]
On 9/7/20 6:19 AM, Cleber Rosa wrote:
> Which means a newer kernel version. Expected output was changed
> to match the new kernel too.
Nack.
Acceptance tests are not to test the latest Linux kernel,
they aim to assert a specific kernel tested by some developer
still works while QEMU evolves.
QEMU doesn't have to adapt to the latest kernel;
QEMU should keep boot an old kernel.
Testing new kernels is good, you are adding coverage. But
this break the acceptance testing contract "keep testing
the same thing over time".
The problem you are trying to fix is the "where to keep
assets from public locations where they are being removed?"
one. Two years ago [*] you suggested to use some storage on
the avocado-project.org:
For Avocado-VT, there are the JeOS images[1], which we
keep on a test "assets" directory. We have a lot of
storage/bandwidth availability, so it can be used for
other assets proven to be necessary for tests.
As long as distribution rights and licensing are not
issues, we can definitely use the same server for kernels,
u-boot images and what not.
[1] - https://avocado-project.org/data/assets/
It is time to have QEMU assets managed the same way.
[*] https://www.mail-archive.com/avocado-devel@redhat.com/msg00860.html
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> tests/acceptance/boot_linux_console.py | 44 +++++++++++++-------------
> tests/acceptance/replay_kernel.py | 8 ++---
> 2 files changed, 26 insertions(+), 26 deletions(-)
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index aaa781a581..751b47b8fd 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -490,12 +490,12 @@ class BootLinuxConsole(LinuxKernelTest):
> :avocado: tags=arch:arm
> :avocado: tags=machine:cubieboard
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
> initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -522,7 +522,7 @@ class BootLinuxConsole(LinuxKernelTest):
> exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
> 'Allwinner sun4i/sun5i')
> exec_command_and_wait_for_pattern(self, 'cat /proc/iomem',
> - 'system-control@1c00000')
> + '1c00000.system-control')
> # cubieboard's reboot is not functioning; omit reboot test.
>
> def test_arm_cubieboard_sata(self):
> @@ -530,12 +530,12 @@ class BootLinuxConsole(LinuxKernelTest):
> :avocado: tags=arch:arm
> :avocado: tags=machine:cubieboard
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
> rootfs_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -573,12 +573,12 @@ class BootLinuxConsole(LinuxKernelTest):
> :avocado: tags=arch:arm
> :avocado: tags=machine:orangepi-pc
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
>
> @@ -598,12 +598,12 @@ class BootLinuxConsole(LinuxKernelTest):
> :avocado: tags=arch:arm
> :avocado: tags=machine:orangepi-pc
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
> initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> @@ -629,7 +629,7 @@ class BootLinuxConsole(LinuxKernelTest):
> exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
> 'Allwinner sun8i Family')
> exec_command_and_wait_for_pattern(self, 'cat /proc/iomem',
> - 'system-control@1c00000')
> + '1c00000.system-control')
> exec_command_and_wait_for_pattern(self, 'reboot',
> 'reboot: Restarting system')
> # Wait for VM to shut down gracefully
> @@ -641,12 +641,12 @@ class BootLinuxConsole(LinuxKernelTest):
> :avocado: tags=machine:orangepi-pc
> :avocado: tags=device:sd
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun8i-h3-orangepi-pc.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
> rootfs_url = ('http://storage.kernelci.org/images/rootfs/buildroot/'
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b79fc8daf8..80a460814f 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -132,12 +132,12 @@ class ReplayKernel(LinuxKernelTest):
> :avocado: tags=arch:arm
> :avocado: tags=machine:cubieboard
> """
> - deb_url = ('https://apt.armbian.com/pool/main/l/'
> - 'linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb')
> - deb_hash = '1334c29c44d984ffa05ed10de8c3361f33d78315'
> + deb_url = ('https://apt.armbian.com/pool/main/l/linux-5.8.0-sunxi/'
> + 'linux-image-dev-sunxi_20.08_armhf.deb')
> + deb_hash = 'ae553a9f7d43b18abfa8f3e64bf2d31878b9be89'
> deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> kernel_path = self.extract_from_deb(deb_path,
> - '/boot/vmlinuz-4.20.7-sunxi')
> + '/boot/vmlinuz-5.8.0-sunxi')
> dtb_path = '/usr/lib/linux-image-dev-sunxi/sun4i-a10-cubieboard.dtb'
> dtb_path = self.extract_from_deb(deb_path, dtb_path)
> initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>
next prev parent reply other threads:[~2020-09-07 8:07 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-07 4:19 [PATCH 0/4] Acceptance Tests: update assets location and cancel tests if missing Cleber Rosa
2020-09-07 4:19 ` [PATCH 1/4] Acceptance tests: use an available kernel image package for arm Cleber Rosa
2020-09-07 8:06 ` Philippe Mathieu-Daudé [this message]
2020-09-07 9:39 ` Daniel P. Berrangé
2020-09-07 9:59 ` Philippe Mathieu-Daudé
2020-09-07 10:28 ` Daniel P. Berrangé
2020-09-07 10:37 ` Philippe Mathieu-Daudé
2020-09-08 13:20 ` Alex Bennée
2020-09-08 13:47 ` Philippe Mathieu-Daudé
2020-09-07 4:19 ` [PATCH 2/4] boot linux test: update arm bionic URL Cleber Rosa
2020-09-07 7:52 ` Philippe Mathieu-Daudé
2020-09-08 18:19 ` Willian Rampazzo
2020-09-07 4:19 ` [PATCH 3/4] tests: bump avocado version Cleber Rosa
2020-09-08 19:55 ` Philippe Mathieu-Daudé
2020-09-07 4:20 ` [PATCH 4/4] Acceptance tests: cancel tests on missing assets Cleber Rosa
2020-09-08 18:12 ` Willian Rampazzo
2020-09-08 20:21 ` [PATCH 0/4] Acceptance Tests: update assets location and cancel tests if missing Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=001385a1-92eb-d311-88b5-8e1e7f6f1f48@redhat.com \
--to=philmd@redhat.com \
--cc=S.E.Harris@kent.ac.uk \
--cc=aleksandar.qemu.devel@gmail.com \
--cc=aleksandar.rikalo@syrmia.com \
--cc=alex.bennee@linaro.org \
--cc=antonynpavlov@gmail.com \
--cc=aurelien@aurel32.net \
--cc=berrange@redhat.com \
--cc=bleal@redhat.com \
--cc=chouteau@adacore.com \
--cc=crosa@redhat.com \
--cc=ehabkost@redhat.com \
--cc=frederic.konrad@adacore.com \
--cc=hpoussin@reactos.org \
--cc=jiaxun.yang@flygoat.com \
--cc=mark.cave-ayland@ilande.co.uk \
--cc=mrolnik@gmail.com \
--cc=pavel.dovgaluk@ispras.ru \
--cc=pbonzini@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=qemu-arm@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=qemu-ppc@nongnu.org \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=wrampazz@redhat.com \
--cc=ysato@users.sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).