check-functional skipUnless failure

check-functional skipUnless failure

[Richard Henderson]

[rth@cfarm120 bld]$ make check-functional
[1/2] Generating tests/functional/func-precache-m68k-m68k_nextcube with a custom command 
(wrapped by meson to set env)
FAILED: tests/functional/m68k-m68k_nextcube.tstamp
env QEMU_TEST_PRECACHE=/home/rth/qemu/bld/tests/functional/m68k-m68k_nextcube.tstamp 
PYTHONPATH=/home/rth/qemu/src/python:/home/rth/qemu/src/tests/functional 
/home/rth/qemu/bld/pyvenv/bin/python3 
/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py
Traceback (most recent call last):
   File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", line 25, in <module>
     class NextCubeMachine(QemuSystemTest):
   File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", line 60, in 
NextCubeMachine
     @skipUnless(tesseract_available(4), 'tesseract OCR tool not available')
   File "/home/rth/qemu/src/tests/functional/qemu_test/tesseract.py", line 16, in 
tesseract_available
     (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
   File "/home/rth/qemu/src/tests/functional/qemu_test/cmd.py", line 69, in run_cmd
     subp = subprocess.Popen(args,
   File "/usr/lib64/python3.9/subprocess.py", line 951, in __init__
     self._execute_child(args, executable, preexec_fn, close_fds,
   File "/usr/lib64/python3.9/subprocess.py", line 1837, in _execute_child
     raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'tesseract'


On this host, tesseract is not available.
But skipUnless is not skipping, but erroring out.


r~

Re: check-functional skipUnless failure

[Philippe Mathieu-Daudé]

On 10/9/24 05:58, Richard Henderson wrote:
> [rth@cfarm120 bld]$ make check-functional
> [1/2] Generating tests/functional/func-precache-m68k-m68k_nextcube with 
> a custom command (wrapped by meson to set env)
> FAILED: tests/functional/m68k-m68k_nextcube.tstamp
> env 
> QEMU_TEST_PRECACHE=/home/rth/qemu/bld/tests/functional/m68k-m68k_nextcube.tstamp PYTHONPATH=/home/rth/qemu/src/python:/home/rth/qemu/src/tests/functional /home/rth/qemu/bld/pyvenv/bin/python3 /home/rth/qemu/src/tests/functional/test_m68k_nextcube.py
> Traceback (most recent call last):
>    File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", 
> line 25, in <module>
>      class NextCubeMachine(QemuSystemTest):
>    File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", 
> line 60, in NextCubeMachine
>      @skipUnless(tesseract_available(4), 'tesseract OCR tool not 
> available')
>    File "/home/rth/qemu/src/tests/functional/qemu_test/tesseract.py", 
> line 16, in tesseract_available
>      (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
>    File "/home/rth/qemu/src/tests/functional/qemu_test/cmd.py", line 69, 
> in run_cmd
>      subp = subprocess.Popen(args,
>    File "/usr/lib64/python3.9/subprocess.py", line 951, in __init__
>      self._execute_child(args, executable, preexec_fn, close_fds,
>    File "/usr/lib64/python3.9/subprocess.py", line 1837, in _execute_child
>      raise child_exception_type(errno_num, err_msg, err_filename)
> FileNotFoundError: [Errno 2] No such file or directory: 'tesseract'
> 
> 
> On this host, tesseract is not available.
> But skipUnless is not skipping, but erroring out.

While looking at this, I'm surprised to notice has_cmd() actually
*runs* the command by calling run_cmd(). I'd have expected it solely
checks for binary presence and RX perms, avoiding unexpected side
effects.

Re: check-functional skipUnless failure

[Thomas Huth]

On 10/09/2024 08.41, Philippe Mathieu-Daudé wrote:
> On 10/9/24 05:58, Richard Henderson wrote:
>> [rth@cfarm120 bld]$ make check-functional
>> [1/2] Generating tests/functional/func-precache-m68k-m68k_nextcube with a 
>> custom command (wrapped by meson to set env)
>> FAILED: tests/functional/m68k-m68k_nextcube.tstamp
>> env 
>> QEMU_TEST_PRECACHE=/home/rth/qemu/bld/tests/functional/m68k-m68k_nextcube.tstamp PYTHONPATH=/home/rth/qemu/src/python:/home/rth/qemu/src/tests/functional /home/rth/qemu/bld/pyvenv/bin/python3 /home/rth/qemu/src/tests/functional/test_m68k_nextcube.py
>> Traceback (most recent call last):
>>    File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", line 
>> 25, in <module>
>>      class NextCubeMachine(QemuSystemTest):
>>    File "/home/rth/qemu/src/tests/functional/test_m68k_nextcube.py", line 
>> 60, in NextCubeMachine
>>      @skipUnless(tesseract_available(4), 'tesseract OCR tool not available')
>>    File "/home/rth/qemu/src/tests/functional/qemu_test/tesseract.py", line 
>> 16, in tesseract_available
>>      (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
>>    File "/home/rth/qemu/src/tests/functional/qemu_test/cmd.py", line 69, 
>> in run_cmd
>>      subp = subprocess.Popen(args,
>>    File "/usr/lib64/python3.9/subprocess.py", line 951, in __init__
>>      self._execute_child(args, executable, preexec_fn, close_fds,
>>    File "/usr/lib64/python3.9/subprocess.py", line 1837, in _execute_child
>>      raise child_exception_type(errno_num, err_msg, err_filename)
>> FileNotFoundError: [Errno 2] No such file or directory: 'tesseract'
>>
>>
>> On this host, tesseract is not available.
>> But skipUnless is not skipping, but erroring out.

Ouch, it's a stupid mistake that I made while converting the test: has_cmd 
returns a list, not a boolean value. This should fix the bug:

diff --git a/tests/functional/qemu_test/tesseract.py 
b/tests/functional/qemu_test/tesseract.py
--- a/tests/functional/qemu_test/tesseract.py
+++ b/tests/functional/qemu_test/tesseract.py
@@ -11,7 +11,8 @@
  from . import has_cmd, run_cmd

  def tesseract_available(expected_version):
-    if not has_cmd('tesseract'):
+    (has_tesseract, errmsg) = has_cmd('tesseract')
+    if not has_tesseract:
          return False
      (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
      if ret:

I'll send it out as a proper patch ASAP.

> While looking at this, I'm surprised to notice has_cmd() actually
> *runs* the command by calling run_cmd(). I'd have expected it solely
> checks for binary presence and RX perms, avoiding unexpected side
> effects.

It uses "which" to search for the binary ... that could be done better for 
sure (especially since "which" might not be available on all systems), but 
it already was done this way in the Avocado tests, so at least that's not a 
regression.

  Thomas

Re: check-functional skipUnless failure

[Philippe Mathieu-Daudé]

On 10/9/24 09:36, Thomas Huth wrote:
> On 10/09/2024 08.41, Philippe Mathieu-Daudé wrote:
>>> line 16, in tesseract_available
>>>      (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
>>>    File "/home/rth/qemu/src/tests/functional/qemu_test/cmd.py", line 
>>> 69, in run_cmd
>>>      subp = subprocess.Popen(args,
>>>    File "/usr/lib64/python3.9/subprocess.py", line 951, in __init__
>>>      self._execute_child(args, executable, preexec_fn, close_fds,
>>>    File "/usr/lib64/python3.9/subprocess.py", line 1837, in 
>>> _execute_child
>>>      raise child_exception_type(errno_num, err_msg, err_filename)
>>> FileNotFoundError: [Errno 2] No such file or directory: 'tesseract'


>> While looking at this, I'm surprised to notice has_cmd() actually
>> *runs* the command by calling run_cmd(). I'd have expected it solely
>> checks for binary presence and RX perms, avoiding unexpected side
>> effects.
> 
> It uses "which" to search for the binary ... that could be done better 
> for sure (especially since "which" might not be available on all 
> systems), but it already was done this way in the Avocado tests, so at 
> least that's not a regression.

Sure, I just wanted to share my surprise before forgetting ;)

Re: check-functional skipUnless failure

[Peter Maydell]

On Tue, 10 Sept 2024 at 08:37, Thomas Huth <thuth@redhat.com> wrote:
>
> On 10/09/2024 08.41, Philippe Mathieu-Daudé wrote:
> > While looking at this, I'm surprised to notice has_cmd() actually
> > *runs* the command by calling run_cmd(). I'd have expected it solely
> > checks for binary presence and RX perms, avoiding unexpected side
> > effects.
>
> It uses "which" to search for the binary ... that could be done better for
> sure (especially since "which" might not be available on all systems), but
> it already was done this way in the Avocado tests, so at least that's not a
> regression.

So it runs an external binary, but it doesn't run the command
being searched for; "which" won't have any side effects.

-- PMM