Re: [Qemu-devel] Who is running QEMU automated tests, and when?

["Alex Bennée" <alex.bennee@linaro.org>]

Michael Clark <mjc@sifive.com> writes:

> On Fri, Apr 27, 2018 at 2:04 AM, Peter Maydell <peter.maydell@linaro.org>
> wrote:
>
>> On 26 April 2018 at 14:57, Eduardo Habkost <ehabkost@redhat.com> wrote:
>> > Peter, do you have additional tests you run before merging a pull
>> > request?  Additional test sets run before tagging a release?
>>
>> I run make && make check, on a variety of hosts, before merging
>> any pull request. That's the only testing I do, and I don't
>> do anything more before I do a tag.
>>
>
> This is what I do on riscv, however it is not yet automated:
>
<snip>
>
> We could bring a snapshot of riscv-tests into the QEMU tree however it
> would be best if they retained the BSD license so we can share locally
> added tests with upstream as upstream is adding tests periodically.

Certainly it would be worth considering some smoke tests for
tests/tcg/riscv that at least verify the basics CPU features work as
expected.

I've got some ideas for packaging more extensive test suites like LTP
and kvm-unit-tests but would like to get simple linux-user stuff up and
running first.

<snip>
>
> We would like to automate all of this and we are working towards that goal.
> One of the dependencies for docker tests are the riscv toolchain packages
> and we are working on this...

I got riscv64 working with the SID toolchain:

  https://github.com/stsquad/qemu/commit/db0a108b9a0efdeeb2f63e670595c5e28e236a97

Once I've fixed it up to use snapshots.debian.org it will be in v4 of
the TCG tests.

> Adding port specific tests, like riscv tests to QEMU not only benefits the
> particular port but also exercises a log of generic code such as the TCG,
> the IO system, IO mutex, Virt IO and many other shared code paths.

I agree it would be worthwhile testing.

System tests are a bit more complicated in terms of code required
although we have some for CRIS/MIPS/Xtensa/lm32 which need re-enabling -
although toolchains for the more esoteric architectures is a pain.

--
Alex Bennée