make vm-build-freebsd appears to require . in PATH

make vm-build-freebsd appears to require . in PATH

[Markus Armbruster]

Look at this:

    $ make vm-build-freebsd
    [...]
    ### Preparing iso and disk image ...
    /home/armbru/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
      100 %       684.3 MiB / 959.4 MiB = 0.713   222 MiB/s       0:04             
    Failed to prepare guest environment
    Traceback (most recent call last):
      File "/work/armbru/qemu/tests/vm/basevm.py", line 640, in main
        return vm.build_image(args.image)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/work/armbru/qemu/tests/vm/freebsd", line 100, in build_image
        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
      File "/work/armbru/qemu/tests/vm/basevm.py", line 270, in exec_qemu_img
        subprocess.check_call(cmd)
      File "/usr/lib64/python3.11/subprocess.py", line 408, in check_call
        retcode = call(*popenargs, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/subprocess.py", line 389, in call
        with Popen(*popenargs, **kwargs) as p:
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/subprocess.py", line 1024, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/lib64/python3.11/subprocess.py", line 1901, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'qemu-img'
    make: *** [/work/armbru/qemu/tests/vm/Makefile.include:97: /home/armbru/.cache/qemu-vm/images/freebsd.img] Error 2
    make: Target 'vm-build-freebsd' not remade because of errors.
    make: Leaving directory '/work/armbru/qemu/bld'

Or similar errors at a later stage when the "guest environment" doesn't
need preparing anymore (I guess).

If I add PATH="$PATH:.", I don't get such errors.

Re: make vm-build-freebsd appears to require . in PATH

[Peter Maydell]

On Tue, 10 Jan 2023 at 15:44, Markus Armbruster <armbru@redhat.com> wrote:
>
> Look at this:
>
>     $ make vm-build-freebsd
>     [...]
>     ### Preparing iso and disk image ...
>     /home/armbru/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
>       100 %       684.3 MiB / 959.4 MiB = 0.713   222 MiB/s       0:04
>     Failed to prepare guest environment
>     Traceback (most recent call last):
>       File "/work/armbru/qemu/tests/vm/basevm.py", line 640, in main
>         return vm.build_image(args.image)
>                ^^^^^^^^^^^^^^^^^^^^^^^^^^
>       File "/work/armbru/qemu/tests/vm/freebsd", line 100, in build_image
>         self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
>       File "/work/armbru/qemu/tests/vm/basevm.py", line 270, in exec_qemu_img
>         subprocess.check_call(cmd)
>       File "/usr/lib64/python3.11/subprocess.py", line 408, in check_call
>         retcode = call(*popenargs, **kwargs)
>                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
>       File "/usr/lib64/python3.11/subprocess.py", line 389, in call
>         with Popen(*popenargs, **kwargs) as p:
>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>       File "/usr/lib64/python3.11/subprocess.py", line 1024, in __init__
>         self._execute_child(args, executable, preexec_fn, close_fds,
>       File "/usr/lib64/python3.11/subprocess.py", line 1901, in _execute_child
>         raise child_exception_type(errno_num, err_msg, err_filename)
>     FileNotFoundError: [Errno 2] No such file or directory: 'qemu-img'
>     make: *** [/work/armbru/qemu/tests/vm/Makefile.include:97: /home/armbru/.cache/qemu-vm/images/freebsd.img] Error 2
>     make: Target 'vm-build-freebsd' not remade because of errors.
>     make: Leaving directory '/work/armbru/qemu/bld'
>
> Or similar errors at a later stage when the "guest environment" doesn't
> need preparing anymore (I guess).

Does it actually require '.' on the PATH, or does it just want
a qemu-img binary on the PATH? (eg your distro one in /usr/bin).
I don't have '.' on my PATH and it works for me.

(Alternatively you can set QEMU_IMG to an absolute path to a
qemu-img, as documented in the 'make vm-help' output.)

thanks
-- PMM

Re: make vm-build-freebsd appears to require . in PATH

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 10 Jan 2023 at 15:44, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Look at this:
>>
>>     $ make vm-build-freebsd
>>     [...]
>>     ### Preparing iso and disk image ...
>>     /home/armbru/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
>>       100 %       684.3 MiB / 959.4 MiB = 0.713   222 MiB/s       0:04
>>     Failed to prepare guest environment
>>     Traceback (most recent call last):
>>       File "/work/armbru/qemu/tests/vm/basevm.py", line 640, in main
>>         return vm.build_image(args.image)
>>                ^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       File "/work/armbru/qemu/tests/vm/freebsd", line 100, in build_image
>>         self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
>>       File "/work/armbru/qemu/tests/vm/basevm.py", line 270, in exec_qemu_img
>>         subprocess.check_call(cmd)
>>       File "/usr/lib64/python3.11/subprocess.py", line 408, in check_call
>>         retcode = call(*popenargs, **kwargs)
>>                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       File "/usr/lib64/python3.11/subprocess.py", line 389, in call
>>         with Popen(*popenargs, **kwargs) as p:
>>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>       File "/usr/lib64/python3.11/subprocess.py", line 1024, in __init__
>>         self._execute_child(args, executable, preexec_fn, close_fds,
>>       File "/usr/lib64/python3.11/subprocess.py", line 1901, in _execute_child
>>         raise child_exception_type(errno_num, err_msg, err_filename)
>>     FileNotFoundError: [Errno 2] No such file or directory: 'qemu-img'
>>     make: *** [/work/armbru/qemu/tests/vm/Makefile.include:97: /home/armbru/.cache/qemu-vm/images/freebsd.img] Error 2
>>     make: Target 'vm-build-freebsd' not remade because of errors.
>>     make: Leaving directory '/work/armbru/qemu/bld'
>>
>> Or similar errors at a later stage when the "guest environment" doesn't
>> need preparing anymore (I guess).
>
> Does it actually require '.' on the PATH, or does it just want
> a qemu-img binary on the PATH? (eg your distro one in /usr/bin).
> I don't have '.' on my PATH and it works for me.

Do we want to use qemu-img, qemu-system-x86_64 and so forth from PATH,
or the one in the build tree?

The former could well be old, which feels like a potential source of
problems.

> (Alternatively you can set QEMU_IMG to an absolute path to a
> qemu-img, as documented in the 'make vm-help' output.)

Thanks.

Re: make vm-build-freebsd appears to require . in PATH

[Peter Maydell]

On Tue, 10 Jan 2023 at 16:26, Markus Armbruster <armbru@redhat.com> wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> > Does it actually require '.' on the PATH, or does it just want
> > a qemu-img binary on the PATH? (eg your distro one in /usr/bin).
> > I don't have '.' on my PATH and it works for me.
>
> Do we want to use qemu-img, qemu-system-x86_64 and so forth from PATH,
> or the one in the build tree?

There's no guarantee there is one in the build tree at all.
I usually use these like
 (cd build && ../configure)
 make -C build  vm-build-openbsd

in which case it doesn't need to build anything in the build
tree at all (neither qemu-system-x86_64 nor qemu-img).

It's nice to be able to do "test this build on *BSD" with
a known-good QEMU running the VM rather than having the
code-under-test affecting both the outer QEMU and the
build-and-make-check running inside the VM.

> The former could well be old, which feels like a potential source of
> problems.

In practice we only use it for very simple operations
("create a qcow2 image" and "resize this qcow2 file"),
so using the distro qemu-img has never been an issue for me.

I think I have in the past run into problems because the
system's qemu-system-x86_64 was super-old, but it was easy
to just build a known-good QEMU version and put that on
the PATH. (And now that system has had a host distro
upgrade, so I have gone back to using the system binary.)

thanks
-- PMM

Re: make vm-build-freebsd appears to require . in PATH

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 10 Jan 2023 at 16:26, Markus Armbruster <armbru@redhat.com> wrote:
>> Peter Maydell <peter.maydell@linaro.org> writes:
>> > Does it actually require '.' on the PATH, or does it just want
>> > a qemu-img binary on the PATH? (eg your distro one in /usr/bin).
>> > I don't have '.' on my PATH and it works for me.
>>
>> Do we want to use qemu-img, qemu-system-x86_64 and so forth from PATH,
>> or the one in the build tree?
>
> There's no guarantee there is one in the build tree at all.
> I usually use these like
>  (cd build && ../configure)
>  make -C build  vm-build-openbsd
>
> in which case it doesn't need to build anything in the build
> tree at all (neither qemu-system-x86_64 nor qemu-img).
>
> It's nice to be able to do "test this build on *BSD" with
> a known-good QEMU running the VM rather than having the
> code-under-test affecting both the outer QEMU and the
> build-and-make-check running inside the VM.

True.

>> The former could well be old, which feels like a potential source of
>> problems.
>
> In practice we only use it for very simple operations
> ("create a qcow2 image" and "resize this qcow2 file"),
> so using the distro qemu-img has never been an issue for me.
>
> I think I have in the past run into problems because the
> system's qemu-system-x86_64 was super-old, but it was easy
> to just build a known-good QEMU version and put that on
> the PATH. (And now that system has had a host distro
> upgrade, so I have gone back to using the system binary.)

I since came to understand this line in output of vm-help:

    QEMU_LOCAL=1                 - Use QEMU binary local to this build.

So the intent appears to be "use (presumably known-good) QEMU tooling
from $PATH by default, pass QEMU_LOCAL=1 to use the build tree instead,
and pass QEMU=... QEMU_IMG=... QEMU_CONFIG=... when you need even more
control."

Thanks again!