[Qemu-devel] [PATCH] target-i386/FPU: wrong conversion infinity from float80 to int32/int64

[Dmitry Poletaev <poletaev-qemu@yandex.ru>]

I executed test-i386 from tests folder from QEMU rep and according to the result, instructions fistl and fistpll returns maximum positive result (0x7fff...), if a FPU register stores a positive infinity, and minimum negative result (0x80...), if a negative infinity stores in a register. Real processor in both cases returns minimum negative result (0x80...). An Intel manual(Vol. 2A 3-299 (http://www.intel.ru/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-2a-manual.pdf)) tells us "If the invalid-operation exception is masked, the integer indefinite value is stored in memory." I not found what "integer indefinite value" obviously means, but I make conclusion that "integer indefinite value" is 0x8000...

This patch fixes behaviour of fistl/fistpll instructions correct, and at least not adds new bugs(according to tcg tests), but I am not shure it doesn't breaks anything.

From: Dmitry Poletaev <poletaev-qemu@yandex.ru>
Signed-off-by: Dmitry Poletaev <poletaev-qemu@yandex.ru>

---
 fpu/softfloat.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 9274ebf..580c322 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -133,7 +133,7 @@ static int32 roundAndPackInt32( flag zSign, uint64_t absZ STATUS_PARAM)
     if ( zSign ) z = - z;
     if ( ( absZ>>32 ) || ( z && ( ( z < 0 ) ^ zSign ) ) ) {
         float_raise( float_flag_invalid STATUS_VAR);
-        return zSign ? (int32_t) 0x80000000 : 0x7FFFFFFF;
+        return (int32_t) 0x80000000;
     }
     if ( roundBits ) STATUS(float_exception_flags) |= float_flag_inexact;
     return z;
@@ -4681,12 +4681,6 @@ int64 floatx80_to_int64( floatx80 a STATUS_PARAM )
     if ( shiftCount <= 0 ) {
         if ( shiftCount ) {
             float_raise( float_flag_invalid STATUS_VAR);
-            if (    ! aSign
-                 || (    ( aExp == 0x7FFF )
-                      && ( aSig != LIT64( 0x8000000000000000 ) ) )
-               ) {
-                return LIT64( 0x7FFFFFFFFFFFFFFF );
-            }
             return (int64_t) LIT64( 0x8000000000000000 );
         }
         aSigExtra = 0;
-- 
1.8.4.msysgit.0