running a single functional test: where do the logs go?

running a single functional test: where do the logs go?

[Peter Maydell]

The functional test documentation suggests running a single test
with the build/run script. But if you do this where do the logfiles go?
As you can see from this transcript, they don't seem to get written
into the place that a "make check-functional" run puts them:

$ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
total 16
-rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
-rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
-rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
$ date
Mon Mar  9 11:20:13 GMT 2026
$ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm
./build/san/run tests/functional/arm/test_emcraft_sf2.py
TAP version 13
ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
1..1

real    3m24.394s
user    3m21.314s
sys     0m3.100s
$ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
total 16
-rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
-rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
-rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch

thanks
-- PMM

Re: running a single functional test: where do the logs go?

[Thomas Huth]

On 09/03/2026 12.55, Peter Maydell wrote:
> The functional test documentation suggests running a single test
> with the build/run script. But if you do this where do the logfiles go?
> As you can see from this transcript, they don't seem to get written
> into the place that a "make check-functional" run puts them:
> 
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> total 16
> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
> $ date
> Mon Mar  9 11:20:13 GMT 2026
> $ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm
> ./build/san/run tests/functional/arm/test_emcraft_sf2.py
> TAP version 13
> ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
> 1..1
> 
> real    3m24.394s
> user    3m21.314s
> sys     0m3.100s
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/

I think the functional tests log their output according to the current 
working directory, so have a look at 
$PWD/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/ 
instead of 
$PWD/build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/ 
?

  HTH,
   Thomas

Re: running a single functional test: where do the logs go?

[Peter Maydell]

On Mon, 9 Mar 2026 at 12:03, Thomas Huth <thuth@redhat.com> wrote:
>
> On 09/03/2026 12.55, Peter Maydell wrote:
> > The functional test documentation suggests running a single test
> > with the build/run script. But if you do this where do the logfiles go?
> > As you can see from this transcript, they don't seem to get written
> > into the place that a "make check-functional" run puts them:
> >
> > $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> > total 16
> > -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> > -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> > -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> > drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
> > $ date
> > Mon Mar  9 11:20:13 GMT 2026
> > $ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm
> > ./build/san/run tests/functional/arm/test_emcraft_sf2.py
> > TAP version 13
> > ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
> > 1..1
> >
> > real    3m24.394s
> > user    3m21.314s
> > sys     0m3.100s
> > $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
>
> I think the functional tests log their output according to the current
> working directory, so have a look at
> $PWD/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> instead of
> $PWD/build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/

No, there's nothing there (no directories created under
tests/functional/arm/). Also having build/test write to
the source directory rather than the build directory would be
surprising -- if it does that then we should definitely
alter the recommended command line in the docs to not
suggest running the 'run' script from the source directory.

thanks
-- PMM

Re: running a single functional test: where do the logs go?

[Thomas Huth]

On 09/03/2026 13.06, Peter Maydell wrote:
> On Mon, 9 Mar 2026 at 12:03, Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 09/03/2026 12.55, Peter Maydell wrote:
>>> The functional test documentation suggests running a single test
>>> with the build/run script. But if you do this where do the logfiles go?
>>> As you can see from this transcript, they don't seem to get written
>>> into the place that a "make check-functional" run puts them:
>>>
>>> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
>>> total 16
>>> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
>>> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
>>> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
>>> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
>>> $ date
>>> Mon Mar  9 11:20:13 GMT 2026
>>> $ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm
>>> ./build/san/run tests/functional/arm/test_emcraft_sf2.py
>>> TAP version 13
>>> ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
>>> 1..1
>>>
>>> real    3m24.394s
>>> user    3m21.314s
>>> sys     0m3.100s
>>> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
>>
>> I think the functional tests log their output according to the current
>> working directory, so have a look at
>> $PWD/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
>> instead of
>> $PWD/build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> 
> No, there's nothing there (no directories created under
> tests/functional/arm/). Also having build/test write to
> the source directory rather than the build directory would be
> surprising -- if it does that then we should definitely
> alter the recommended command line in the docs to not
> suggest running the 'run' script from the source directory.

Agreed, this sounds fishy. I can have a look at it after the softfreeze if 
nobody else beats me to it.

  Thomas

Re: running a single functional test: where do the logs go?

[Daniel P. Berrangé]

On Mon, Mar 09, 2026 at 11:55:58AM +0000, Peter Maydell wrote:
> The functional test documentation suggests running a single test
> with the build/run script. But if you do this where do the logfiles go?

They should always go in the build directory

> As you can see from this transcript, they don't seem to get written
> into the place that a "make check-functional" run puts them:
> 
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> total 16
> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
> $ date
> Mon Mar  9 11:20:13 GMT 2026
> $ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm
> ./build/san/run tests/functional/arm/test_emcraft_sf2.py
> TAP version 13
> ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
> 1..1
> 
> real    3m24.394s
> user    3m21.314s
> sys     0m3.100s
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> total 16
> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch

That seems different from the behaviour I get


$ ls -al build/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
total 28
drwxr-xr-x. 2 berrange berrange 4096 Mar  9 12:31 .
drwxr-xr-x. 3 berrange berrange 4096 Mar  9 12:30 ..
-rw-r--r--. 1 berrange berrange 6449 Mar  9 12:31 base.log
-rw-r--r--. 1 berrange berrange 8200 Mar  9 12:31 console.log
-rw-r--r--. 1 berrange berrange    0 Mar  9 12:30 default.log

$ time QEMU_TEST_QEMU_BINARY=qemu-system-arm ./build/run tests/functional/arm/test_emcraft_sf2.py
TAP version 13
ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
1..1

real	0m10.731s
user	0m10.597s
sys	0m0.130s
$ ls -al build/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
total 28
drwxr-xr-x. 2 berrange berrange 4096 Mar  9 12:32 .
drwxr-xr-x. 3 berrange berrange 4096 Mar  9 12:30 ..
-rw-r--r--. 1 berrange berrange 7253 Mar  9 12:32 base.log
-rw-r--r--. 1 berrange berrange 8200 Mar  9 12:32 console.log
-rw-r--r--. 1 berrange berrange    0 Mar  9 12:32 default.log


but in my case I'm just using ./build, where as you are using
./build/san, and I think that's tripping us up


def _build_dir():
    root = os.getenv('QEMU_BUILD_ROOT')
    if root is not None:
        return Path(root)
    # Makefile.mtest only exists in build dir, so if it is available, use CWD
    if os.path.exists('Makefile.mtest'):
        return Path(os.getcwd())

    root = os.path.join(_source_dir(), 'build')
    if os.path.exists(root):
        return Path(root)

    raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT")


The $CWD is the directory from which you are invoking the scripts,
so that Makefile.mtest check won't pass.

The _source_dir() / 'build'  test will pass, but that's not your
actual build dir, so that's not really correct.

The QEMU_BUILD_ROOT env was a special hack to allow overrides, but
we never set that by default.

These days, we do, however, set MESON_BUILD_ROOT in the meson devenv,
so the 'run' script should see that.

IOW, we should fix the _build_dir() method to look at MESON_BUILD_ROOT
instead of QEMU_BUILD_ROOT.

Meanwhile your logs are likely in build/tests/functional/... instead
of build/san/tests/functional.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: running a single functional test: where do the logs go?

[Peter Maydell]

On Mon, 9 Mar 2026 at 12:40, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Mon, Mar 09, 2026 at 11:55:58AM +0000, Peter Maydell wrote:
> > The functional test documentation suggests running a single test
> > with the build/run script. But if you do this where do the logfiles go?

> Meanwhile your logs are likely in build/tests/functional/... instead
> of build/san/tests/functional.

Thanks, yes, that's exactly where they ended up.

-- PMM

Re: running a single functional test: where do the logs go?

[Daniel P. Berrangé]

On Mon, Mar 09, 2026 at 11:55:58AM +0000, Peter Maydell wrote:
> The functional test documentation suggests running a single test
> with the build/run script. But if you do this where do the logfiles go?
> As you can see from this transcript, they don't seem to get written
> into the place that a "make check-functional" run puts them:
> 
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> total 16
> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
> $ date
> Mon Mar  9 11:20:13 GMT 2026
> $ time QEMU_TEST_QEMU_BINARY=./build/san/qemu-system-arm

BTW this is previously needed, but is now overkill wit the 'run' script.
QEMU_TEST_QEMU_BINARY=qemu-system-arm is sufficient, as meson's devenv
sets $PATH to include the locally built binary dirs first.

> ./build/san/run tests/functional/arm/test_emcraft_sf2.py
> TAP version 13
> ok 1 test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2
> 1..1
> 
> real    3m24.394s
> user    3m21.314s
> sys     0m3.100s
> $ ls -l build/san/tests/functional/arm/test_emcraft_sf2.EmcraftSf2Machine.test_arm_emcraft_sf2/
> total 16
> -rw-r--r-- 1 pm215 pm215 3084 Mar  9 10:51 base.log
> -rw-r--r-- 1 pm215 pm215  825 Mar  9 10:51 console.log
> -rw-r--r-- 1 pm215 pm215  235 Mar  9 10:54 default.log
> drwxr-xr-x 3 pm215 pm215 4096 Mar  9 10:51 scratch
> 
> thanks
> -- PMM
> 

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|