Re: check-tcg HOWTO?

[Claudio Fontana <cfontana@suse.de>]

Ciao Alex,

thanks for your answer,

On 1/11/21 2:35 PM, Alex Bennée wrote:
> 
> Claudio Fontana <cfontana@suse.de> writes:
> 
>> Hi Alex,
>>
>> happy new year,
>>
>> I am trying to get check-tcg to run reliably,
>> as I am doing some substantial refactoring of tcg cpu operations, so I need to verify that TCG is fine.
>>
>> This is an overall getting started question, is there a how-to on how
>> to use check-tcg and how to fix things when things don't go smoothly?
> 
> Not really but I could certainly add something. Generally I just run the
> tests manually in gdb when I'm trying to debug stuff.

Right, I plan to do the same, if I get to the command line to run.
I think it does make sense to add something similar to what was explained here in documentation, with a pointer maybe from README?


> 
>> I get different results on different machines for check-tcg, although the runs are containerized,
>> on one machine the tests for aarch64 tcg are SKIPPED completely (so no
>> errors),
> 
> The compiles *may* be containerized - the runs are always in your host
> environment. It's one of the reasons the binaries are built as static
> images so you don't need to mess about with dynamic linking and
> libraries.
> 

Ah good to know, thanks. So everything is actually run in the host environment in the end.

> The only reason some tests get skipped is if you have a locally
> installed cross compiler which doesn't support some architecture
> features (e.g. CROSS_CC_HAS_SVE).


hmm I will have to check then how to make sure that the test does not see these cross compilers...?


> 
>> on the other machine I get:
>>
>> qemu-system-aarch64: terminating on signal 15 from pid 18583 (timeout)
>> qemu-system-aarch64: terminating on signal 15 from pid 18584 (timeout)
>> qemu-system-aarch64: terminating on signal 15 from pid 18585 (timeout)
>> make[2]: *** [../Makefile.target:162: run-hello] Error 124
>> make[2]: *** Waiting for unfinished jobs....
>> make[2]: *** [../Makefile.target:162: run-pauth-3] Error 124
>> make[2]: *** [../Makefile.target:162: run-memory] Error 124
> 
> Given it's timing out on hello I guess it's the shutdown deadlocking.
> Running with V=1 will give you the command line but the semihosting
> config is setup for redirect. So I usually build my own command line. e.g.:
> 
>   ./qemu-system-aarch64 -monitor none -display none \
>     -chardev stdio,id=output  \
>     -M virt -cpu max -display none \
>     -semihosting-config enable=on,target=native,chardev=output \
>     -kernel tests/tcg/aarch64-softmmu/hello
> 


Would it be possible for check-tcg (and possibly even make check in general where applicable)
to automatically spew the command line to reproduce the error, similar to what you have shown here?


I think this is would be of great value for anyone to be able to act on the errors reported.


> There is nothing particularly special apart from making sure semihosting
> is wired up for the output. Apart from some special cases like
> test-mmap-XXXX most tests don't take any arguments.
> 
>>
>> Both are configured with 
>>
>> configure --enable-tcg
>>
>> Anything more than V=1 to get more output?
> 
> The output is normally dumped in $TESTNAME.out in the appropriate
> $BUILDDIR/tests/tcg/$TARGET/ directory.
> 
>> How do I debug and get logs and cores out of containers?
> 
> As I mentioned above the tests are not run in containers, just the
> compiles (if local compilers are missing).

Thanks for clearing this up!

> 
>>
>> in tests/tcg/ there is:
>>
>> a README (with no hint unfortunately) ,
> 
> Woefully out of date I'm afraid. What docs we have are in docs/devel/testing.rst

maybe a

+ Please see docs/devel/testing.rst for hints on how to run make-tcg and reproduce its results from the cmdline.

> 
>> Makefile.qemu
> 
> Links into the main tests/Makefile.include - invoked for each target
> 
>> Makefile.prereqs
> 
> This ensures docker images are built (if required) for each set of tests.
> 
>> Makefile.target
> 
> This is the main (rather simple) makefile which provides the build and
> run targets. You can run directly if you are in the right build dir, eg:
> 
>   13:58:10 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ + pwd
>   /home/alex/lsrc/qemu.git/builds/arm.all/tests/tcg/aarch64-softmmu
>   13:58:57 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ +
>   make  -f ~/lsrc/qemu.git/tests/tcg/Makefile.target TARGET="aarch64-softmmu" SRC_PATH="/home/alex/lsrc/qemu.git" run
> 
> But TBH this is functionally equivalent to calling:
> 
>   make run-tcg-tests-aarch64-softmmu
> 
> in your main build directory.

Thanks, that's helpful.

> 
>> There are a bunch of variables in these files, which seem to be
>> possible to configure, am I expected to set some of those?
> 
> Not really. Most of the magic variables from:
> 
>   tests/tcg/config-$TARGET.mak
> 
> which is built by tests/tcg/configure.sh during the configure step.
> 
>> I think that it would be beneficial to have either more documentation
>> or more immediately actionable information out of make check failures;
> 
> V=1 should show the command lines run and then you should be able to run
> the tests directly yourself.


Hmm V=1 did not show the command line to me, but maybe I just missed it somehow?
Or it contained some sockets that cannot be manually connected?

> 
>> Any help you could give me to make some progess?
> 
> Hope that helps.

It does, I wonder if this could be fed into docs/ with a pointer to it from README,
and also if this new feature for the tests could be developed, ie, producing a command line useful for reproducing the error,
something that can be run directly without further editing, sanitizing etc...

Thanks a lot,

Claudio