[Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[amateur]

Hi, All

I'm trying to run benchmarks from SPEC CPU2006 compiled for SPARC on
top of qemu-sparc32plus. However, several benchmarks failed to get the
correct result.

One of the failed benchmarks, 403.gcc, runs for a while and then print
an error message and exit. However, the same compiled gcc binary for
SPARC runs quite well in Simics(a commercial emulator) and give the
correct result. So I think there may be a bug in Qemu.

Has anyone meet the same kind of problem before?

I can do some debug, but I have no clue on how to do it. Could anyone
please give me some hint on this? I have read the source code, but
find it really hard to track down guest program miss-behavior problem.

BTW, when I use sparc-linux-gdb with qemu-sparc32plus, it connect, but
give the wrong value when print local variables. So, I'm really lost.
Can anyone confirm on this, or I'm doing it the wrong way?

-- 
As of next Tuesday, C will be flushed in favor of COBOL.
Please update your programs.

Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[Artyom Tarasenko]

2010/7/29 amateur <tianlei.zhao@gmail.com>:
> Hi, All
>
> I'm trying to run benchmarks from SPEC CPU2006 compiled for SPARC on
> top of qemu-sparc32plus. However, several benchmarks failed to get the
> correct result.

Can you try to compile them for sparc32 (no plus)? Are the tests
multi-threaded? If yes, can you try to run them under
qemu-system-sparc? There are some known problems in user mode
emulation.

> One of the failed benchmarks, 403.gcc, runs for a while and then print
> an error message and exit.

Can you be more specific here? What message? From the description of
403.gcc, it looks like it does some floating point operations. Is the
error FPU specific?

What other tests are failing? Are there any pure integer tests among them?

> However, the same compiled gcc binary for
> SPARC runs quite well in Simics(a commercial emulator) and give the
> correct result.

What CPU does it emulate in your test?

> So I think there may be a bug in Qemu.
>
> Has anyone meet the same kind of problem before?

Well, there are still some bugs in qemu. I suspect there may be one
bug in the integer math which prevents NeXTStep from booting, but
haven't had a chance to track it down yet.

> I can do some debug, but I have no clue on how to do it. Could anyone
> please give me some hint on this? I have read the source code, but
> find it really hard to track down guest program miss-behavior problem.
>
> BTW, when I use sparc-linux-gdb with qemu-sparc32plus, it connect, but
> give the wrong value when print local variables. So, I'm really lost.
> Can anyone confirm on this, or I'm doing it the wrong way?

Haven't tried it with qemu-sparc32plus, but it seems to work fine with
qemu-system-sparc.


-- 
Regards,
Artyom Tarasenko

solaris/sparc under qemu blog: http://tyom.blogspot.com/

Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[amateur]

On Thu. 2010-07-29 22:20:28 +0200, Artyom Tarasenko wrote:
> 2010/7/29 amateur <tianlei.zhao@gmail.com>:
> > Hi, All
> >
> > I'm trying to run benchmarks from SPEC CPU2006 compiled for SPARC on
> > top of qemu-sparc32plus. However, several benchmarks failed to get the
> > correct result.
> 
> Can you try to compile them for sparc32 (no plus)? Are the tests
> multi-threaded? If yes, can you try to run them under
> qemu-system-sparc? There are some known problems in user mode
> emulation.

The benchmarks were compiled for sparc32. Qemu was configured for
target sparc32plus-linux-user and host is x86_64. All benchmarks are
single threaded.

Host system is Debian testing for x86_64. Host compiler is gcc-4.4.3
from debian repository.

I tried use qemu-sparc instead of qemu-sparc32plus as you suggested.
To my surprise, that works for 403.gcc which run to end and get the
correct result.

I'm wondering why qemu-sparc works while qemu-sparc32plus doesn't.
Maybe that has something to do with the CPU model, which is "TI
UltraSparc II" for qemu-sparc32plus and "Fujitsu MB86904" for
qemu-sparc.

I will try to do some more research on this. Thanks for the hint.

> > One of the failed benchmarks, 403.gcc, runs for a while and then print
> > an error message and exit.
> 
> Can you be more specific here? What message? From the description of
> 403.gcc, it looks like it does some floating point operations. Is the
> error FPU specific?

After registered binfmt_misc:

[quote]
workstation:~/403.gcc.train$ ./gcc integrate.i -o integrate.s 
integrate.i:1936: unrecognizable insn:
integrate.i:1936: Internal compiler error in extract_insn, at recog.c:2148
It is possible that you may be trying to use SPEC's version of gcc 
without first defining the appropriate flags.  Please check the flags 
that are in the config files from recently-published results on your 
platform, and check that you are using an up-to-date compiler.  If
you still need help, please contact SPEC, reporting your hw/os 
platform, your compiler version, and your compilation flags.
Contact SPEC at <URL:http://www.spec.org/>
[/quote]

Although it complains, there is no other compilation flags than
SPEC_CPU which I've already set properly. And the result binray can
run flawlessly on Simics.

> What other tests are failing? Are there any pure integer tests among them?

447.dealII(compiled for sparc32) from SPEC CPU 2006 fails on
qemu-sparc32plus too, while the same binary runs well on Simics. The
error message is:

[quote]
workstation:~/447.dealII.train$ ./dealII 10
Refinement cycle: 0
   Number of degrees of freedom=79
   Point value=0.111419
   Estimated error=-0.0202358
Refinement cycle: 1


----------------------------------------------------
Exception on processing: 
--------------------------------------------------------
An error occurred in line <2591> of file <data_out_base.cc> in function
    unknown
The violated condition was: 
    out
The name and call sequence of the exception was:
    ExcIO()
Additional Information: 
(none)
--------------------------------------------------------

Aborting!
----------------------------------------------------
[/quote]

I've made a mistake. Only these two benchmarks in SPEC CPU 2006 fail
on qemu-sparc32plus.

> > However, the same compiled gcc binary for
> > SPARC runs quite well in Simics(a commercial emulator) and give the
> > correct result.
> 
> What CPU does it emulate in your test?

I'm using the default CPU model set by qemu. For sparc32plus, that is
"TI UltraSparc II".


-- 
When in this world the headlines read
Of those whose hearts are filled with greed
Who rob and steal from those who need
The cry goes up with blinding speed for Underdog (UNDERDOG!)
Underdog (UNDERDOG!)
Speed of lightning, roar of thunder
Fighting all who rob or plunder
Underdog (ah-ah-ah-ah)
Underdog
UNDERDOG!

Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[Vince Weaver]

Hello

have you tried older versions of qemu, in an attempt to see if this is a 
recent problem?

As of about a year ago I had qemu-sparc32plus running most of SPEC2006 
properly, validated against hardware perf counters.

gcc definitely worked.  I had issues with calculix, dealII, soplex, wrf, 
zeusmp and astar but the rest all worked and gave retired instruction 
counts close to that of real hardware.  I can probably look up what 
version of qemu I used for that work.

Vince

Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[Blue Swirl]

On Thu, Jul 29, 2010 at 3:40 PM, amateur <tianlei.zhao@gmail.com> wrote:
> Hi, All
>
> I'm trying to run benchmarks from SPEC CPU2006 compiled for SPARC on
> top of qemu-sparc32plus. However, several benchmarks failed to get the
> correct result.
>
> One of the failed benchmarks, 403.gcc, runs for a while and then print
> an error message and exit. However, the same compiled gcc binary for
> SPARC runs quite well in Simics(a commercial emulator) and give the
> correct result. So I think there may be a bug in Qemu.
>
> Has anyone meet the same kind of problem before?
>
> I can do some debug, but I have no clue on how to do it. Could anyone
> please give me some hint on this? I have read the source code, but
> find it really hard to track down guest program miss-behavior problem.

There's '-d in_asm,op,out_asm' but it may not help much.

Once I singlestepped a program under GDB, simultaneously both in
native host and in QEMU. After each step, both GDBs calculated and
displayed a XOR checksum from all registers. I had to do some hacks to
make address layout and register values match in QEMU.

> BTW, when I use sparc-linux-gdb with qemu-sparc32plus, it connect, but
> give the wrong value when print local variables. So, I'm really lost.
> Can anyone confirm on this, or I'm doing it the wrong way?

You may need to use sparc64-linux-gdb.

Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006

[amateur]

On Fri. 2010-07-30 14:21:42 +0000, Blue Swirl wrote:
> On Thu, Jul 29, 2010 at 3:40 PM, amateur <tianlei.zhao@gmail.com> wrote:
> > Hi, All
> There's '-d in_asm,op,out_asm' but it may not help much.

Since the program runs a while before fail, that counts a lot of
instructions(hundreds of millions). With this volume of instructions
to trace, the log file would be two large. Anyway, thanks for the
suggestion.

> Once I singlestepped a program under GDB, simultaneously both in
> native host and in QEMU. After each step, both GDBs calculated and
> displayed a XOR checksum from all registers. I had to do some hacks to
> make address layout and register values match in QEMU.

This sounds feasible. I've managed to make the address layout match
for guest program running in qemu and in Simics. I'll try it.

> > BTW, when I use sparc-linux-gdb with qemu-sparc32plus, it connect, but
> > give the wrong value when print local variables. So, I'm really lost.
> > Can anyone confirm on this, or I'm doing it the wrong way?
> 
> You may need to use sparc64-linux-gdb.

Tried both sparc64-linux-gdb and sparc-linux-gdb, same result.