biosbits test failing on origin/master

biosbits test failing on origin/master

[John Snow]

Hiya, on today's origin/master
(2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
check-avocado" is failing on the new biosbits test on my local
development machine:

 (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
FAIL: True is not false : The VM seems to have failed to shutdown in
time (83.65 s)

Is this a known issue, or should I begin to investigate it?

--js

Re: biosbits test failing on origin/master

[Ani Sinha]

On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
>
> Hiya, on today's origin/master
> (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> check-avocado" is failing on the new biosbits test on my local
> development machine:
>
>  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> FAIL: True is not false : The VM seems to have failed to shutdown in
> time (83.65 s)
>
> Is this a known issue, or should I begin to investigate it?

In my test environment it does pass.

$ ./tests/venv/bin/avocado run -t acpi tests/avocado
Fetching asset from
tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
 (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
PASS (57.57 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 63.82 s

However, I have seen that on certain slower test machines or when run
within a virtual machine, the test can take longer to complete and 60
secs may not always be enough. In those cases raising the maximum
completion time to 90 secs helps. Perhaps you can try this and let me
know if it helps:

diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
index 8745a58a76..b11fe39350 100644
--- a/tests/avocado/acpi-bits.py
+++ b/tests/avocado/acpi-bits.py
@@ -385,8 +385,9 @@ def test_acpi_smbios_bits(self):
         self._vm.launch()
         # biosbits has been configured to run all the specified test suites
         # in batch mode and then automatically initiate a vm shutdown.
-        # sleep for maximum of one minute
-        max_sleep_time = time.monotonic() + 60
+        # sleep for a maximum of one and half minutes to accommodate
running this
+        # even on slower machines.
+        max_sleep_time = time.monotonic() + 90
         while self._vm.is_running() and time.monotonic() < max_sleep_time:
             time.sleep(1)

Re: biosbits test failing on origin/master

[Ani Sinha]

On Fri, Nov 11, 2022 at 9:52 AM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> >
> > Hiya, on today's origin/master
> > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > check-avocado" is failing on the new biosbits test on my local
> > development machine:
> >
> >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > FAIL: True is not false : The VM seems to have failed to shutdown in
> > time (83.65 s)
> >
> > Is this a known issue, or should I begin to investigate it?
>
> In my test environment it does pass.
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> Fetching asset from
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
>  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> PASS (57.57 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> | CANCEL 0
> JOB TIME   : 63.82 s
>
> However, I have seen that on certain slower test machines or when run
> within a virtual machine, the test can take longer to complete and 60
> secs may not always be enough.

Here is an interesting data point. I re-ran on a Centos 8 VM running
on Ubuntu bare metal. Just like the Ubuntu bare metal, it passed fine.
Next, I tried to run it on a Centos 7.9.2009 VM running on the same
Ubuntu bare metal. I was able to reproduce this consistently. So I did
some digging, gdb into qemu and it looked like the VM was just slow
but not stuck. So I timed the QEMU command line that was being used in
the test using the same iso that was being generated by the test. Here
it is:

# time ./qemu-system-x86_64 -display none -vga none -chardev
file,path=/var/tmp/debugcon-log.txt,id=debugcon -device
isa-debugcon,iobase=0x403,chardev=debugcon -cdrom
/var/tmp/acpi-bits-oogd8wp9.tmp/bits-2020.iso

real 2m34.052s
user 2m33.858s
sys 0m0.467s

On bare metal Ubuntu, I see this:

$ time ./qemu-system-x86_64 -display none -vga none -chardev
file,path=/var/tmp/debugcon-log.txt,id=debugcon -device
isa-debugcon,iobase=0x403,chardev=debugcon -cdrom ~/temp/bits-2020.iso

real 1m15.318s
user 1m15.136s
sys 0m0.345s

With "-icount auto" added in the command line,  both bare metal and VM
environments were slower. However, we need this command line for some
latency tests.

avocado framework has a 2 min timeout for any test. This would
definitely time out even without my internal timeout check. I verified
that the test indeed passed by looking into the debug con logs pushed
out by the test framework. If the system is just slow/overloaded I am
not sure what we can do.

In those cases raising the maximum
> completion time to 90 secs helps. Perhaps you can try this and let me
> know if it helps:

Maybe I will make the timeout 90 secs and hope that this will be enough.

>
> diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
> index 8745a58a76..b11fe39350 100644
> --- a/tests/avocado/acpi-bits.py
> +++ b/tests/avocado/acpi-bits.py
> @@ -385,8 +385,9 @@ def test_acpi_smbios_bits(self):
>          self._vm.launch()
>          # biosbits has been configured to run all the specified test suites
>          # in batch mode and then automatically initiate a vm shutdown.
> -        # sleep for maximum of one minute
> -        max_sleep_time = time.monotonic() + 60
> +        # sleep for a maximum of one and half minutes to accommodate
> running this
> +        # even on slower machines.
> +        max_sleep_time = time.monotonic() + 90
>          while self._vm.is_running() and time.monotonic() < max_sleep_time:
>              time.sleep(1)

Re: biosbits test failing on origin/master

[Ani Sinha]

On Fri, Nov 11, 2022 at 4:39 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Fri, Nov 11, 2022 at 9:52 AM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > >
> > > Hiya, on today's origin/master
> > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > check-avocado" is failing on the new biosbits test on my local
> > > development machine:
> > >
> > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > time (83.65 s)
> > >
> > > Is this a known issue, or should I begin to investigate it?
> >
> > In my test environment it does pass.
> >
> > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > Fetching asset from
> > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > PASS (57.57 s)
> > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > | CANCEL 0
> > JOB TIME   : 63.82 s
> >
> > However, I have seen that on certain slower test machines or when run
> > within a virtual machine, the test can take longer to complete and 60
> > secs may not always be enough.
>
> Here is an interesting data point. I re-ran on a Centos 8 VM running
> on Ubuntu bare metal. Just like the Ubuntu bare metal, it passed fine.
> Next, I tried to run it on a Centos 7.9.2009 VM running on the same
> Ubuntu bare metal. I was able to reproduce this consistently. So I did
> some digging, gdb into qemu and it looked like the VM was just slow
> but not stuck. So I timed the QEMU command line that was being used in
> the test using the same iso that was being generated by the test. Here
> it is:
>
> # time ./qemu-system-x86_64 -display none -vga none -chardev
> file,path=/var/tmp/debugcon-log.txt,id=debugcon -device
> isa-debugcon,iobase=0x403,chardev=debugcon -cdrom
> /var/tmp/acpi-bits-oogd8wp9.tmp/bits-2020.iso
>
> real 2m34.052s
> user 2m33.858s
> sys 0m0.467s
>
> On bare metal Ubuntu, I see this:
>
> $ time ./qemu-system-x86_64 -display none -vga none -chardev
> file,path=/var/tmp/debugcon-log.txt,id=debugcon -device
> isa-debugcon,iobase=0x403,chardev=debugcon -cdrom ~/temp/bits-2020.iso
>
> real 1m15.318s
> user 1m15.136s
> sys 0m0.345s
>
> With "-icount auto" added in the command line,  both bare metal and VM
> environments were slower. However, we need this command line for some
> latency tests.
>
> avocado framework has a 2 min timeout for any test.

Just realized that this value is coming from
https://gitlab.com/anisinha/qemu/-/blob/master/tests/avocado/avocado_qemu/__init__.py#L232
and can be overiden.
So I will override this to 200 seconds. If we still see timeouts, we
can raise this further by a bit more.

Thanks for bringing this to my attention.

This would
> definitely time out even without my internal timeout check. I verified
> that the test indeed passed by looking into the debug con logs pushed
> out by the test framework. If the system is just slow/overloaded I am
> not sure what we can do.
>
> In those cases raising the maximum
> > completion time to 90 secs helps. Perhaps you can try this and let me
> > know if it helps:
>
> Maybe I will make the timeout 90 secs and hope that this will be enough.
>
> >
> > diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
> > index 8745a58a76..b11fe39350 100644
> > --- a/tests/avocado/acpi-bits.py
> > +++ b/tests/avocado/acpi-bits.py
> > @@ -385,8 +385,9 @@ def test_acpi_smbios_bits(self):
> >          self._vm.launch()
> >          # biosbits has been configured to run all the specified test suites
> >          # in batch mode and then automatically initiate a vm shutdown.
> > -        # sleep for maximum of one minute
> > -        max_sleep_time = time.monotonic() + 60
> > +        # sleep for a maximum of one and half minutes to accommodate
> > running this
> > +        # even on slower machines.
> > +        max_sleep_time = time.monotonic() + 90
> >          while self._vm.is_running() and time.monotonic() < max_sleep_time:
> >              time.sleep(1)

Re: biosbits test failing on origin/master

[John Snow]

On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> >
> > Hiya, on today's origin/master
> > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > check-avocado" is failing on the new biosbits test on my local
> > development machine:
> >
> >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > FAIL: True is not false : The VM seems to have failed to shutdown in
> > time (83.65 s)
> >
> > Is this a known issue, or should I begin to investigate it?
>
> In my test environment it does pass.
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> Fetching asset from
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
>  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> PASS (57.57 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> | CANCEL 0
> JOB TIME   : 63.82 s
>
> However, I have seen that on certain slower test machines or when run
> within a virtual machine, the test can take longer to complete and 60
> secs may not always be enough. In those cases raising the maximum
> completion time to 90 secs helps. Perhaps you can try this and let me
> know if it helps:

Hmm - I'm running on a fairly modern machine and not in a VM. Do you
have an invocation to share that exists outside of the avocado
machinery where I could test this individually and see how long it
might take to complete if I just let it run? I am worried that it's
getting wedged instead of just taking a long time, but it's hard to
tell.

--js

>
> diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
> index 8745a58a76..b11fe39350 100644
> --- a/tests/avocado/acpi-bits.py
> +++ b/tests/avocado/acpi-bits.py
> @@ -385,8 +385,9 @@ def test_acpi_smbios_bits(self):
>          self._vm.launch()
>          # biosbits has been configured to run all the specified test suites
>          # in batch mode and then automatically initiate a vm shutdown.
> -        # sleep for maximum of one minute
> -        max_sleep_time = time.monotonic() + 60
> +        # sleep for a maximum of one and half minutes to accommodate
> running this
> +        # even on slower machines.
> +        max_sleep_time = time.monotonic() + 90
>          while self._vm.is_running() and time.monotonic() < max_sleep_time:
>              time.sleep(1)
>

Re: biosbits test failing on origin/master

[Ani Sinha]

On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
>
> On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > >
> > > Hiya, on today's origin/master
> > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > check-avocado" is failing on the new biosbits test on my local
> > > development machine:
> > >
> > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > time (83.65 s)
> > >
> > > Is this a known issue, or should I begin to investigate it?
> >
> > In my test environment it does pass.
> >
> > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > Fetching asset from
> > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > PASS (57.57 s)
> > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > | CANCEL 0
> > JOB TIME   : 63.82 s
> >
> > However, I have seen that on certain slower test machines or when run
> > within a virtual machine, the test can take longer to complete and 60
> > secs may not always be enough. In those cases raising the maximum
> > completion time to 90 secs helps. Perhaps you can try this and let me
> > know if it helps:
>
> Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> have an invocation to share that exists outside of the avocado
> machinery

If you pass V=1 in the environment then it dumps the QEMU command line
that was used to run the test. You also need to comment out the line
> shutil.rmtree(self._workDir)
in tearDown() so that the iso is not cleaned up.

diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
index a67d30d583..2060e3b84f 100644
--- a/tests/avocado/acpi-bits.py
+++ b/tests/avocado/acpi-bits.py
@@ -357,7 +357,7 @@ def tearDown(self):
         if self._vm:
             self.assertFalse(not self._vm.is_running)
         self.logger.info('removing the work directory %s', self._workDir)
-        shutil.rmtree(self._workDir)
+        # shutil.rmtree(self._workDir)
         super().tearDown()

     def test_acpi_smbios_bits(self):

while you are at it, it might makes sense to check the vnc for the VM
to see what it is doing.

 where I could test this individually and see how long it
> might take to complete if I just let it run? I am worried that it's
> getting wedged instead of just taking a long time, but it's hard to
> tell.
>
> --js
>
> >
> > diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
> > index 8745a58a76..b11fe39350 100644
> > --- a/tests/avocado/acpi-bits.py
> > +++ b/tests/avocado/acpi-bits.py
> > @@ -385,8 +385,9 @@ def test_acpi_smbios_bits(self):
> >          self._vm.launch()
> >          # biosbits has been configured to run all the specified test suites
> >          # in batch mode and then automatically initiate a vm shutdown.
> > -        # sleep for maximum of one minute
> > -        max_sleep_time = time.monotonic() + 60
> > +        # sleep for a maximum of one and half minutes to accommodate
> > running this
> > +        # even on slower machines.
> > +        max_sleep_time = time.monotonic() + 90
> >          while self._vm.is_running() and time.monotonic() < max_sleep_time:
> >              time.sleep(1)
> >
>

Re: biosbits test failing on origin/master

[Ani Sinha]

On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> >
> > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > >
> > > > Hiya, on today's origin/master
> > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > check-avocado" is failing on the new biosbits test on my local
> > > > development machine:
> > > >
> > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > time (83.65 s)
> > > >
> > > > Is this a known issue, or should I begin to investigate it?
> > >
> > > In my test environment it does pass.
> > >
> > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > Fetching asset from
> > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > PASS (57.57 s)
> > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > | CANCEL 0
> > > JOB TIME   : 63.82 s
> > >
> > > However, I have seen that on certain slower test machines or when run
> > > within a virtual machine, the test can take longer to complete and 60
> > > secs may not always be enough. In those cases raising the maximum
> > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > know if it helps:
> >
> > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > have an invocation to share that exists outside of the avocado
> > machinery
>
> If you pass V=1 in the environment then it dumps the QEMU command line
> that was used to run the test. You also need to comment out the line
> > shutil.rmtree(self._workDir)
> in tearDown() so that the iso is not cleaned up.

Maybe I will send out this patch once we have figured out what is
going on with your environment:
https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e

I ran the test on another box sitting in my office running centos7.9
and it passed as well.

Re: biosbits test failing on origin/master

[Ani Sinha]

On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > >
> > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > >
> > > > > Hiya, on today's origin/master
> > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > development machine:
> > > > >
> > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > time (83.65 s)
> > > > >
> > > > > Is this a known issue, or should I begin to investigate it?
> > > >
> > > > In my test environment it does pass.
> > > >
> > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > Fetching asset from
> > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > PASS (57.57 s)
> > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > | CANCEL 0
> > > > JOB TIME   : 63.82 s
> > > >
> > > > However, I have seen that on certain slower test machines or when run
> > > > within a virtual machine, the test can take longer to complete and 60
> > > > secs may not always be enough. In those cases raising the maximum
> > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > know if it helps:
> > >
> > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > have an invocation to share that exists outside of the avocado
> > > machinery
> >
> > If you pass V=1 in the environment then it dumps the QEMU command line
> > that was used to run the test. You also need to comment out the line
> > > shutil.rmtree(self._workDir)
> > in tearDown() so that the iso is not cleaned up.
>
> Maybe I will send out this patch once we have figured out what is
> going on with your environment:
> https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
>
> I ran the test on another box sitting in my office running centos7.9
> and it passed as well.

For the records,
I ran make check-avocado on my Ubuntu 222.04 laptop:
https://pastebin.com/0ZKEEQds
On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4

Re: biosbits test failing on origin/master

[John Snow]

On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > >
> > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > >
> > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > > >
> > > > > > Hiya, on today's origin/master
> > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > development machine:
> > > > > >
> > > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > > time (83.65 s)
> > > > > >
> > > > > > Is this a known issue, or should I begin to investigate it?
> > > > >
> > > > > In my test environment it does pass.
> > > > >
> > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > Fetching asset from
> > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > PASS (57.57 s)
> > > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > > | CANCEL 0
> > > > > JOB TIME   : 63.82 s
> > > > >
> > > > > However, I have seen that on certain slower test machines or when run
> > > > > within a virtual machine, the test can take longer to complete and 60
> > > > > secs may not always be enough. In those cases raising the maximum
> > > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > > know if it helps:
> > > >
> > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > have an invocation to share that exists outside of the avocado
> > > > machinery
> > >
> > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > that was used to run the test. You also need to comment out the line
> > > > shutil.rmtree(self._workDir)
> > > in tearDown() so that the iso is not cleaned up.
> >
> > Maybe I will send out this patch once we have figured out what is
> > going on with your environment:
> > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> >
> > I ran the test on another box sitting in my office running centos7.9
> > and it passed as well.
>
> For the records,
> I ran make check-avocado on my Ubuntu 222.04 laptop:
> https://pastebin.com/0ZKEEQds
> On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4

As a question: Is it necessary to implement your own timeout here?
What's wrong with relying on Avocado's timeout?
My hunch is that you'll get greater flexibility by leaning into the
tool suite's configuration instead of hardcoding your own...

For what it's worth, I am now trying to run this test manually by doing:

> time ./qemu-system-x86_64 -cdrom /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto

This is not a quick test:

________________________________________________________
Executed in   86.50 secs    fish           external
   usr time   86.57 secs    0.00 micros   86.57 secs
   sys time    0.30 secs  903.00 micros    0.29 secs


This isn't the *most* cutting edge machine, but it's a Intel(R)
Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
Is your machine really passing this test in under 60 seconds flat?


--js

Re: biosbits test failing on origin/master

[John Snow]

On Tue, Nov 15, 2022 at 1:47 PM John Snow <jsnow@redhat.com> wrote:
>
> On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > > >
> > > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > >
> > > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > > > >
> > > > > > > Hiya, on today's origin/master
> > > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > > development machine:
> > > > > > >
> > > > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > > > time (83.65 s)
> > > > > > >
> > > > > > > Is this a known issue, or should I begin to investigate it?
> > > > > >
> > > > > > In my test environment it does pass.
> > > > > >
> > > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > > Fetching asset from
> > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > PASS (57.57 s)
> > > > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > > > | CANCEL 0
> > > > > > JOB TIME   : 63.82 s
> > > > > >
> > > > > > However, I have seen that on certain slower test machines or when run
> > > > > > within a virtual machine, the test can take longer to complete and 60
> > > > > > secs may not always be enough. In those cases raising the maximum
> > > > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > > > know if it helps:
> > > > >
> > > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > > have an invocation to share that exists outside of the avocado
> > > > > machinery
> > > >
> > > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > > that was used to run the test. You also need to comment out the line
> > > > > shutil.rmtree(self._workDir)
> > > > in tearDown() so that the iso is not cleaned up.
> > >
> > > Maybe I will send out this patch once we have figured out what is
> > > going on with your environment:
> > > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> > >
> > > I ran the test on another box sitting in my office running centos7.9
> > > and it passed as well.
> >
> > For the records,
> > I ran make check-avocado on my Ubuntu 222.04 laptop:
> > https://pastebin.com/0ZKEEQds
> > On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4
>
> As a question: Is it necessary to implement your own timeout here?
> What's wrong with relying on Avocado's timeout?
> My hunch is that you'll get greater flexibility by leaning into the
> tool suite's configuration instead of hardcoding your own...
>
> For what it's worth, I am now trying to run this test manually by doing:
>
> > time ./qemu-system-x86_64 -cdrom /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto
>
> This is not a quick test:
>
> ________________________________________________________
> Executed in   86.50 secs    fish           external
>    usr time   86.57 secs    0.00 micros   86.57 secs
>    sys time    0.30 secs  903.00 micros    0.29 secs
>
>
> This isn't the *most* cutting edge machine, but it's a Intel(R)
> Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
> Is your machine really passing this test in under 60 seconds flat?
>

FWIW, I sent a patch to just disable the test-specific timeout. I
figure you had a good reason for putting it there, but I suppose
you'll tell me tomorrow what that reason was :)

I couldn't get the test to finish in under 60 seconds on this machine,
but I did manage to get it to finish in under 60 on my ... 16 core AMD
5950X. Just needed some very heavy firepower, I guess!

--js

Re: biosbits test failing on origin/master

[Ani Sinha]

On Wed, Nov 16, 2022 at 3:22 AM John Snow <jsnow@redhat.com> wrote:
>
> On Tue, Nov 15, 2022 at 1:47 PM John Snow <jsnow@redhat.com> wrote:
> >
> > On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > > >
> > > > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > > >
> > > > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > > > > >
> > > > > > > > Hiya, on today's origin/master
> > > > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > > > development machine:
> > > > > > > >
> > > > > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > > > > time (83.65 s)
> > > > > > > >
> > > > > > > > Is this a known issue, or should I begin to investigate it?
> > > > > > >
> > > > > > > In my test environment it does pass.
> > > > > > >
> > > > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > > > Fetching asset from
> > > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > PASS (57.57 s)
> > > > > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > > > > | CANCEL 0
> > > > > > > JOB TIME   : 63.82 s
> > > > > > >
> > > > > > > However, I have seen that on certain slower test machines or when run
> > > > > > > within a virtual machine, the test can take longer to complete and 60
> > > > > > > secs may not always be enough. In those cases raising the maximum
> > > > > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > > > > know if it helps:
> > > > > >
> > > > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > > > have an invocation to share that exists outside of the avocado
> > > > > > machinery
> > > > >
> > > > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > > > that was used to run the test. You also need to comment out the line
> > > > > > shutil.rmtree(self._workDir)
> > > > > in tearDown() so that the iso is not cleaned up.
> > > >
> > > > Maybe I will send out this patch once we have figured out what is
> > > > going on with your environment:
> > > > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> > > >
> > > > I ran the test on another box sitting in my office running centos7.9
> > > > and it passed as well.
> > >
> > > For the records,
> > > I ran make check-avocado on my Ubuntu 222.04 laptop:
> > > https://pastebin.com/0ZKEEQds
> > > On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4
> >
> > As a question: Is it necessary to implement your own timeout here?
> > What's wrong with relying on Avocado's timeout?
> > My hunch is that you'll get greater flexibility by leaning into the
> > tool suite's configuration instead of hardcoding your own...
> >
> > For what it's worth, I am now trying to run this test manually by doing:
> >
> > > time ./qemu-system-x86_64 -cdrom /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto
> >
> > This is not a quick test:
> >
> > ________________________________________________________
> > Executed in   86.50 secs    fish           external
> >    usr time   86.57 secs    0.00 micros   86.57 secs
> >    sys time    0.30 secs  903.00 micros    0.29 secs
> >
> >
> > This isn't the *most* cutting edge machine, but it's a Intel(R)
> > Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
> > Is your machine really passing this test in under 60 seconds flat?
> >
>
> FWIW, I sent a patch to just disable the test-specific timeout. I
> figure you had a good reason for putting it there, but I suppose
> you'll tell me tomorrow what that reason was :)

We need to wait until the VM terminates. The reason for providing the
timeout was that
(a) I figure it is a bad idea to wait indefinitely in case the VM gets
wedged or stuck.
(b) I was not aware of the framework level timeout mechanism.

I am ok if we want to get rid of the internal timeout. However, we
need to wait for the VM to terminate.

>
> I couldn't get the test to finish in under 60 seconds on this machine,
> but I did manage to get it to finish in under 60 on my ... 16 core AMD
> 5950X. Just needed some very heavy firepower, I guess!
>
> --js
>

Re: biosbits test failing on origin/master

[Ani Sinha]

On Wed, Nov 16, 2022 at 12:18 AM John Snow <jsnow@redhat.com> wrote:
>
> On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > > >
> > > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > >
> > > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > > > >
> > > > > > > Hiya, on today's origin/master
> > > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > > development machine:
> > > > > > >
> > > > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > > > time (83.65 s)
> > > > > > >
> > > > > > > Is this a known issue, or should I begin to investigate it?
> > > > > >
> > > > > > In my test environment it does pass.
> > > > > >
> > > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > > Fetching asset from
> > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > PASS (57.57 s)
> > > > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > > > | CANCEL 0
> > > > > > JOB TIME   : 63.82 s
> > > > > >
> > > > > > However, I have seen that on certain slower test machines or when run
> > > > > > within a virtual machine, the test can take longer to complete and 60
> > > > > > secs may not always be enough. In those cases raising the maximum
> > > > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > > > know if it helps:
> > > > >
> > > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > > have an invocation to share that exists outside of the avocado
> > > > > machinery
> > > >
> > > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > > that was used to run the test. You also need to comment out the line
> > > > > shutil.rmtree(self._workDir)
> > > > in tearDown() so that the iso is not cleaned up.
> > >
> > > Maybe I will send out this patch once we have figured out what is
> > > going on with your environment:
> > > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> > >
> > > I ran the test on another box sitting in my office running centos7.9
> > > and it passed as well.
> >
> > For the records,
> > I ran make check-avocado on my Ubuntu 222.04 laptop:
> > https://pastebin.com/0ZKEEQds
> > On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4
>
> As a question: Is it necessary to implement your own timeout here?
> What's wrong with relying on Avocado's timeout?

When I wrote the test I was not aware of the avocado timeout. Hence
implemented mine. However, I kind of think that timing out from the
test itself rather than from the framework provides an opportunity to
provide more meaningful information to the user when the timeout
happens. Of Course we can improve upon the current  "FAIL: True is not
false" assertion message.

> My hunch is that you'll get greater flexibility by leaning into the
> tool suite's configuration instead of hardcoding your own...
>
> For what it's worth, I am now trying to run this test manually by doing:
>
> > time ./qemu-system-x86_64 -cdrom /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto
>
> This is not a quick test:
>
> ________________________________________________________
> Executed in   86.50 secs    fish           external
>    usr time   86.57 secs    0.00 micros   86.57 secs
>    sys time    0.30 secs  903.00 micros    0.29 secs
>
>
> This isn't the *most* cutting edge machine, but it's a Intel(R)
> Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
> Is your machine really passing this test in under 60 seconds flat?

Yes it is both on my 3 year old lenovo laptop with 16 Gib memory:

Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz
    CPU family:          6
    Model:               142
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            12
    CPU max MHz:         4100.0000
    CPU min MHz:         400.0000

as well as on a desktop machine with 32 Gib of memory:

Vendor ID:             GenuineIntel
CPU family:            6
Model:                 158
Model name:            Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz
Stepping:              13
CPU MHz:               799.987
CPU max MHz:           4700.0000
CPU min MHz:           800.0000
BogoMIPS:              6000.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K
NUMA node0 CPU(s):     0-7

Re: biosbits test failing on origin/master

[Daniel P. Berrangé]

On Wed, Nov 16, 2022 at 08:34:00AM +0530, Ani Sinha wrote:
> On Wed, Nov 16, 2022 at 12:18 AM John Snow <jsnow@redhat.com> wrote:
> >
> > On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > > >
> > > > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > > >
> > > > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com> wrote:
> > > > > > > >
> > > > > > > > Hiya, on today's origin/master
> > > > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that "make
> > > > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > > > development machine:
> > > > > > > >
> > > > > > > >  (001/193) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > > FAIL: True is not false : The VM seems to have failed to shutdown in
> > > > > > > > time (83.65 s)
> > > > > > > >
> > > > > > > > Is this a known issue, or should I begin to investigate it?
> > > > > > >
> > > > > > > In my test environment it does pass.
> > > > > > >
> > > > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > > > Fetching asset from
> > > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > > > JOB ID     : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > > > JOB LOG    : /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > > > >  (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > PASS (57.57 s)
> > > > > > > RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
> > > > > > > | CANCEL 0
> > > > > > > JOB TIME   : 63.82 s
> > > > > > >
> > > > > > > However, I have seen that on certain slower test machines or when run
> > > > > > > within a virtual machine, the test can take longer to complete and 60
> > > > > > > secs may not always be enough. In those cases raising the maximum
> > > > > > > completion time to 90 secs helps. Perhaps you can try this and let me
> > > > > > > know if it helps:
> > > > > >
> > > > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > > > have an invocation to share that exists outside of the avocado
> > > > > > machinery
> > > > >
> > > > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > > > that was used to run the test. You also need to comment out the line
> > > > > > shutil.rmtree(self._workDir)
> > > > > in tearDown() so that the iso is not cleaned up.
> > > >
> > > > Maybe I will send out this patch once we have figured out what is
> > > > going on with your environment:
> > > > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> > > >
> > > > I ran the test on another box sitting in my office running centos7.9
> > > > and it passed as well.
> > >
> > > For the records,
> > > I ran make check-avocado on my Ubuntu 222.04 laptop:
> > > https://pastebin.com/0ZKEEQds
> > > On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4
> >
> > As a question: Is it necessary to implement your own timeout here?
> > What's wrong with relying on Avocado's timeout?
> 
> When I wrote the test I was not aware of the avocado timeout. Hence
> implemented mine. However, I kind of think that timing out from the
> test itself rather than from the framework provides an opportunity to
> provide more meaningful information to the user when the timeout
> happens. Of Course we can improve upon the current  "FAIL: True is not
> false" assertion message.
> 
> > My hunch is that you'll get greater flexibility by leaning into the
> > tool suite's configuration instead of hardcoding your own...
> >
> > For what it's worth, I am now trying to run this test manually by doing:
> >
> > > time ./qemu-system-x86_64 -cdrom /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto
> >
> > This is not a quick test:
> >
> > ________________________________________________________
> > Executed in   86.50 secs    fish           external
> >    usr time   86.57 secs    0.00 micros   86.57 secs
> >    sys time    0.30 secs  903.00 micros    0.29 secs
> >
> >
> > This isn't the *most* cutting edge machine, but it's a Intel(R)
> > Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
> > Is your machine really passing this test in under 60 seconds flat?
> 
> Yes it is both on my 3 year old lenovo laptop with 16 Gib memory:

Note gitlab.com public CI shared runners are *very* small VM instances.

IIUC, currently we can expect the runner to have as little as 1 vCPU,
and 3.75 GB of RAM.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|