Re: [Qemu-devel] [PATCH v2 0/2] softfloat tests based on berkeley's testfloat

["Emilio G. Cota" <cota@braap.org>]

On Mon, Sep 10, 2018 at 12:26:58 +0100, Alex Bennée wrote:
> 
> Emilio G. Cota <cota@braap.org> writes:
> 
> > A few fixes since yesterday's v1:
> >   https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg00884.html
> >
> > - Convert copy_qemu_to_soft80 to qemu_to_soft80, just like the other
> >   conversion functions
> > - Set fp-test as the program name as reported by itself
> > - Fix Makefile to include .d files so that dependencies are
> >   properly tracked
> > - Update commit log
> 
> Just some general comments:
> 
>  - I think this is a better way to go than the IBM test suite
>  - I'm ambivalent about maintaining our fp-test.c in close alignment to
>    softfloat unless we expect much upstreaming of changes.

I don't think there'll be any, to be honest. They have a git repo but
I doubt they'll take any patches. This was just a minimum attempt to
get some tests working (I don't have a lot of time to work on this)

>  - the coverage seems a bit low. Rebuilding everything with
>    --enable-gcov and running -all1 -all2 I get:
> 
>   tests/fp/fp-test.c - 53.5 % coverage 43.3 % branch coverage
>   fpu/softfloat.c - 32.5 % coverage 25.1 % branch coverage
> 
> But maybe I didn't pass enough options to fp-test? I could really do
> with a --just-run-everything-and-summarise-failing-functions option so I
> can then go through in more detail with fp-test fFOO_BAR.

Yes right now to get better coverage you need to run it several times.
After a few runs I got it to 58%, but still without testing some
rounding modes. So considering we're coming from 0% coverage, I'd
say coverage is pretty good!

But yes, we should add a flag to just test -all.

There are some functions that we are not testing yet though, but
that can be fixed over time (we can add more tests to fp-test even
though they're not in testfloat, such as testing flush-to-zero/
denormals-are-zero, or the muladd variants that we have).

I have basically no time left to work on this. What do you think about
the following plan?

1. Have our own clones (forks) of testfloat/softfloat in qemu servers.
2. Add those as submodules
3. Add fp-test with as you said an -all flag that reports all
   errors to get decent coverage.
4. Add hardfloat patches, with tests. This requires a small
   change to testfloat:
   https://github.com/cota/berkeley-testfloat-3/commit/ca9fa2ba05

I'd then leave adding further tests to increase coverage and fixing
the existing bugs (prior to hardfloat) to someone else with
more time/resources.

Thanks,

		Emilio