[Qemu-devel] Question: How to integrate simple tests for a new platform so that they fit in well within QEMU

[Qemu-devel] Question: How to integrate simple tests for a new platform so that they fit in well within QEMU

[Aleksandar Markovic]

Hello, folks.

We are introducing a new MIPS platform - called nanoMIPS - to the QEMU.

Besides providing new functionality, we would like to integrate into QEMU some basic testing capabilities for that new platform. Something like this:

    Enable user to specify cross compiler and, using QEMU Linux user mode:
        a. execute several tests that are just almost minimal tests of emulation of selected instructions.
        b. execute several tests that are just almost minimal tests of translation of selected system calls.

In other words, this would enable a user to run "smoke tests" for nanoMIPS platform, and perhaps to use these tests as a starting point for some other tests.

My question is:

1. Is this a good idea at all?
2. If yes, what would be the best way to integrate these tests into current QEMU test organization?

I appreciate any opinion of yours.

Regards,
Aleksandar

Re: [Qemu-devel] Question: How to integrate simple tests for a new platform so that they fit in well within QEMU

[Richard Henderson]

On 08/07/2018 05:34 AM, Aleksandar Markovic wrote:
> We are introducing a new MIPS platform - called nanoMIPS - to the QEMU.
> 
> Besides providing new functionality, we would like to integrate into QEMU
> some basic testing capabilities for that new platform. Something like this:
> 
> Enable user to specify cross compiler and, using QEMU Linux user mode: 
> a. execute several tests that are just almost minimal tests of emulation of
> selected instructions.
> b. execute several tests that are just almost minimal tests of translation 
> of selected system calls.
QEMU has some infrastructure for this now.
I'm not clear on all of the pieces, but Alex will be able to help you.

In configure, there is a section setting cross_cc_foo and cross_cc_cflags_foo;
there is another section that tests target_compiler and target_compiler_cflags
from that.  At minimum you'll need to add a new section that sets
target_compiler etc for nanomips.

In tests/tcg/mips/Makefile.include, one can set up a docker cross-compile
definition.  This is correlated to files in tests/docker/dockerfiles/.

I don't know much about docker, but I know it should be possible to build and
distribute a cross-compiler, for a base system such as debian 9, that other
folk can pull and use.  For a brand new target like nanomips, requiring a brand
new tool chain, this is certainly something that you will have to do yourself.
But having that available for CI should pay dividends.


r~

Re: [Qemu-devel] Question: How to integrate simple tests for a new platform so that they fit in well within QEMU

[Thomas Huth]

On 08/07/2018 02:34 PM, Aleksandar Markovic wrote:
> Hello, folks.
> 
> We are introducing a new MIPS platform - called nanoMIPS - to the QEMU.
> 
> Besides providing new functionality, we would like to integrate into QEMU some basic testing capabilities for that new platform. Something like this:
> 
>     Enable user to specify cross compiler and, using QEMU Linux user mode:
>         a. execute several tests that are just almost minimal tests of emulation of selected instructions.
>         b. execute several tests that are just almost minimal tests of translation of selected system calls.
> 
> In other words, this would enable a user to run "smoke tests" for nanoMIPS platform, and perhaps to use these tests as a starting point for some other tests.
> 
> My question is:
> 
> 1. Is this a good idea at all?

Yes, it certainly is!

> 2. If yes, what would be the best way to integrate these tests into current QEMU test organization?

Best way is to use the test infrastructure in tests/tcg (see mail from
Richard).
Another possibility is to add a simple test to tests/boot-serial-test.c
- but this only works if your machine can output something to the serial
console with some few lines of assembly code.

 Thomas

Re: [Qemu-devel] Question: How to integrate simple tests for a new platform so that they fit in well within QEMU

[Alex Bennée]

Aleksandar Markovic <amarkovic@wavecomp.com> writes:

> Hello, folks.
>
> We are introducing a new MIPS platform - called nanoMIPS - to the QEMU.
>
> Besides providing new functionality, we would like to integrate into QEMU some basic testing capabilities for that new platform. Something like this:
>
>     Enable user to specify cross compiler and, using QEMU Linux user mode:
>         a. execute several tests that are just almost minimal tests of emulation of selected instructions.
>         b. execute several tests that are just almost minimal tests of
>     translation of selected system calls.
>
> In other words, this would enable a user to run "smoke tests" for nanoMIPS platform, and perhaps to use these tests as a starting point for some other tests.
>
> My question is:
>
> 1. Is this a good idea at all?

Totally. The test/tcg section languished for awhile as most people don't
have cross compilers set up. Thanks to docker we can now have a fall-back to
known good cross-compilers while still supporting local setups with
--cross-cc-ARCH and --cross-cc-flags-ARCH.

> 2. If yes, what would be the best way to integrate these tests into
> current QEMU test organization?

So currently tests/tcg is setup for linux-user testing. It will run
everything in tests/tcg/multiarch and any additional user-mode tests in
tests/tcg/ARCH as governed by the Makefile.include.

For the existing MIPS it is basically multiarch however there are
sub-directories that are system tests that need to be re-integrated. We
have cross compilers for a bunch of the existing MIPS platforms in the
form of:

  debian-mips-cross.docker
  debian-mips64-cross.docker
  debian-mipsel-cross.docker
  debian-mipsn32-cross.docker
  debian-mips64el-cross.docker

If you would like to help resurrect the system emulation testing for the
existing MIPS tests I would welcome the help. I think all the
infrastructure for compilers is already there so it should just be a
case of tweaking the Makefile structure a little.

> I appreciate any opinion of yours.

So the first question that needs to be answered is where can you get
nanoMIPS compilers?

For some of the more esoteric architectures QEMU supports we ended up
using Debian's rolling Sid distribution as it still contains cross
compilers for unofficial architectures that are in ports. If there is
any push for getting a nanoMIPS Debian release that might be the
quickest way. Otherwise you would need to write a dockerfile to install
the toolchain on some reasonable base system - see for example
debian-tricore-cross.docker which basically builds Bastien's fork of
binutils.

You don't have to build the toolchain from scratch - however it would
nice to have a clear idea from of where the source for the toolchain
comes from. At least with Debian images we know any binary is only an
apt-get source away from it's code.

If you have any more questions please ask away and also CC me on any
patches (MAINTAINERS should take care of this).

--
Alex Bennée