* [Qemu-devel] [PATCH] BootLinuxSshTest: Only run the tests when explicitly requested
@ 2019-09-18 12:27 Philippe Mathieu-Daudé
2019-09-19 1:23 ` Cleber Rosa
0 siblings, 1 reply; 3+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-09-18 12:27 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Gerd Hoffmann, Cleber Rosa,
Philippe Mathieu-Daudé, Aurelien Jarno, David Gibson
Currently the Avocado framework does not distinct the time spent
downloading assets vs. the time spent running a test. With big
assets (like a full VM image) the tests likely fail.
This is a limitation known by the Avocado team.
Until this issue get fixed, do not run this tests automatically.
Tests can still be run setting the AVOCADO_TIMEOUT_EXPECTED
environment variable.
Reported-by: Gerd Hoffmann <kraxel@redhat.com>
Reported-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
tests/acceptance/linux_ssh_mips_malta.py | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index 134f10cac3..7200507a3a 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -12,7 +12,7 @@ import logging
import paramiko
import time
-from avocado import skipIf
+from avocado import skipUnless
from avocado_qemu import Test
from avocado.utils import process
from avocado.utils import archive
@@ -171,7 +171,7 @@ class LinuxSSH(Test):
self.run_common_commands()
self.shutdown_via_ssh()
- @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
+ @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
def test_mips_malta32eb_kernel3_2_0(self):
"""
:avocado: tags=arch:mips
@@ -186,7 +186,7 @@ class LinuxSSH(Test):
self.check_mips_malta('be', kernel_path, 'mips')
- @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
+ @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
def test_mips_malta32el_kernel3_2_0(self):
"""
:avocado: tags=arch:mipsel
@@ -201,7 +201,7 @@ class LinuxSSH(Test):
self.check_mips_malta('le', kernel_path, 'mips')
- @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
+ @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
def test_mips_malta64eb_kernel3_2_0(self):
"""
:avocado: tags=arch:mips64
@@ -215,7 +215,7 @@ class LinuxSSH(Test):
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
self.check_mips_malta('be', kernel_path, 'mips64')
- @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
+ @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
def test_mips_malta64el_kernel3_2_0(self):
"""
:avocado: tags=arch:mips64el
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [Qemu-devel] [PATCH] BootLinuxSshTest: Only run the tests when explicitly requested
2019-09-18 12:27 [Qemu-devel] [PATCH] BootLinuxSshTest: Only run the tests when explicitly requested Philippe Mathieu-Daudé
@ 2019-09-19 1:23 ` Cleber Rosa
2019-09-19 13:52 ` Cleber Rosa
0 siblings, 1 reply; 3+ messages in thread
From: Cleber Rosa @ 2019-09-19 1:23 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Eduardo Habkost, Aleksandar Rikalo, qemu-devel, Gerd Hoffmann,
Aurelien Jarno, David Gibson
On Wed, Sep 18, 2019 at 02:27:48PM +0200, Philippe Mathieu-Daudé wrote:
> Currently the Avocado framework does not distinct the time spent
> downloading assets vs. the time spent running a test. With big
> assets (like a full VM image) the tests likely fail.
>
> This is a limitation known by the Avocado team.
> Until this issue get fixed, do not run this tests automatically.
>
> Tests can still be run setting the AVOCADO_TIMEOUT_EXPECTED
> environment variable.
>
> Reported-by: Gerd Hoffmann <kraxel@redhat.com>
> Reported-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> tests/acceptance/linux_ssh_mips_malta.py | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
> index 134f10cac3..7200507a3a 100644
> --- a/tests/acceptance/linux_ssh_mips_malta.py
> +++ b/tests/acceptance/linux_ssh_mips_malta.py
> @@ -12,7 +12,7 @@ import logging
> import paramiko
> import time
>
> -from avocado import skipIf
> +from avocado import skipUnless
> from avocado_qemu import Test
> from avocado.utils import process
> from avocado.utils import archive
> @@ -171,7 +171,7 @@ class LinuxSSH(Test):
> self.run_common_commands()
> self.shutdown_via_ssh()
>
> - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
This is definitely an improvement, specially because IMO a "make
check-acceptance" command should be more similar across executions,
wether on Travis or not. The problem is (which is not new) is that
users have to resort to the test to learn how to run those tests.
FIY, I'm working on a RFC to have a handful of "make
check-acceptance-$(TYPE)" targets that hopefully can represent most
users' needs. The avocado command line would still be available for
more advanced users.
> def test_mips_malta32eb_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips
> @@ -186,7 +186,7 @@ class LinuxSSH(Test):
>
> self.check_mips_malta('be', kernel_path, 'mips')
>
> - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> def test_mips_malta32el_kernel3_2_0(self):
> """
> :avocado: tags=arch:mipsel
> @@ -201,7 +201,7 @@ class LinuxSSH(Test):
>
> self.check_mips_malta('le', kernel_path, 'mips')
>
> - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> def test_mips_malta64eb_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips64
> @@ -215,7 +215,7 @@ class LinuxSSH(Test):
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> self.check_mips_malta('be', kernel_path, 'mips64')
>
> - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> def test_mips_malta64el_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips64el
> --
> 2.20.1
>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [Qemu-devel] [PATCH] BootLinuxSshTest: Only run the tests when explicitly requested
2019-09-19 1:23 ` Cleber Rosa
@ 2019-09-19 13:52 ` Cleber Rosa
0 siblings, 0 replies; 3+ messages in thread
From: Cleber Rosa @ 2019-09-19 13:52 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Eduardo Habkost, Aleksandar Rikalo, qemu-devel, Gerd Hoffmann,
Aurelien Jarno, David Gibson
On Wed, Sep 18, 2019 at 09:23:59PM -0400, Cleber Rosa wrote:
> On Wed, Sep 18, 2019 at 02:27:48PM +0200, Philippe Mathieu-Daudé wrote:
> > Currently the Avocado framework does not distinct the time spent
> > downloading assets vs. the time spent running a test. With big
> > assets (like a full VM image) the tests likely fail.
> >
> > This is a limitation known by the Avocado team.
> > Until this issue get fixed, do not run this tests automatically.
> >
> > Tests can still be run setting the AVOCADO_TIMEOUT_EXPECTED
> > environment variable.
> >
> > Reported-by: Gerd Hoffmann <kraxel@redhat.com>
> > Reported-by: David Gibson <david@gibson.dropbear.id.au>
> > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> > ---
> > tests/acceptance/linux_ssh_mips_malta.py | 10 +++++-----
> > 1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
> > index 134f10cac3..7200507a3a 100644
> > --- a/tests/acceptance/linux_ssh_mips_malta.py
> > +++ b/tests/acceptance/linux_ssh_mips_malta.py
> > @@ -12,7 +12,7 @@ import logging
> > import paramiko
> > import time
> >
> > -from avocado import skipIf
> > +from avocado import skipUnless
> > from avocado_qemu import Test
> > from avocado.utils import process
> > from avocado.utils import archive
> > @@ -171,7 +171,7 @@ class LinuxSSH(Test):
> > self.run_common_commands()
> > self.shutdown_via_ssh()
> >
> > - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> > + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
>
> This is definitely an improvement, specially because IMO a "make
> check-acceptance" command should be more similar across executions,
> wether on Travis or not. The problem is (which is not new) is that
> users have to resort to the test to learn how to run those tests.
>
> FIY, I'm working on a RFC to have a handful of "make
> check-acceptance-$(TYPE)" targets that hopefully can represent most
> users' needs. The avocado command line would still be available for
> more advanced users.
>
> > def test_mips_malta32eb_kernel3_2_0(self):
> > """
> > :avocado: tags=arch:mips
> > @@ -186,7 +186,7 @@ class LinuxSSH(Test):
> >
> > self.check_mips_malta('be', kernel_path, 'mips')
> >
> > - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> > + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> > def test_mips_malta32el_kernel3_2_0(self):
> > """
> > :avocado: tags=arch:mipsel
> > @@ -201,7 +201,7 @@ class LinuxSSH(Test):
> >
> > self.check_mips_malta('le', kernel_path, 'mips')
> >
> > - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> > + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> > def test_mips_malta64eb_kernel3_2_0(self):
> > """
> > :avocado: tags=arch:mips64
> > @@ -215,7 +215,7 @@ class LinuxSSH(Test):
> > kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> > self.check_mips_malta('be', kernel_path, 'mips64')
> >
> > - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
> > + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
> > def test_mips_malta64el_kernel3_2_0(self):
> > """
> > :avocado: tags=arch:mips64el
> > --
> > 2.20.1
> >
>
> Reviewed-by: Cleber Rosa <crosa@redhat.com>
BTW, also queueing this one on my python-next branch.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-09-19 14:13 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-09-18 12:27 [Qemu-devel] [PATCH] BootLinuxSshTest: Only run the tests when explicitly requested Philippe Mathieu-Daudé
2019-09-19 1:23 ` Cleber Rosa
2019-09-19 13:52 ` Cleber Rosa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).