[Qemu-devel] [PATCH] softfloat: Fix factor 2 error for scalbn on denormal inputs

[Qemu-devel] [PATCH] softfloat: Fix factor 2 error for scalbn on denormal inputs

[Peter Maydell]

If the input to float*_scalbn() is denormal then it represents
a number 0.[mantissabits] * 2^(1-exponentbias) (and the actual
exponent field is all zeroes). This means that when we convert
it to our unpacked encoding the unpacked exponent must be one
greater than for a normal number, which represents
1.[mantissabits] * 2^(e-exponentbias) for an exponent field e.

This meant we were giving answers too small by a factor of 2 for
all denormal inputs.

Note that the float-to-int routines also have this behaviour
of not adjusting the exponent for denormals; however there it is
harmless because denormals will all convert to integer zero anyway.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
Found this while going through and fixing "wrong exception flags"
bugs in the 32 bit ARM VCVT instructions. Error is most obvious for
scale factor of n==0, in which case we should return the same value
we were passed, but were halving it instead. Since all denormals
are so small they round to zero in the second half of the fixpoint
conversion the only effect for ARM was erroneously setting Inexact
if the input had a 1 in the least significant bit of the mantissa
and the scale factor was n==0.

This patch is licensed under either the softfloat -2a or -2b licenses,
at your option.

 fpu/softfloat.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index dbda61b..db77b07 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -6795,10 +6795,13 @@ float32 float32_scalbn( float32 a, int n STATUS_PARAM )
         }
         return a;
     }
-    if ( aExp != 0 )
+    if (aExp != 0) {
         aSig |= 0x00800000;
-    else if ( aSig == 0 )
+    } else if (aSig == 0) {
         return a;
+    } else {
+        aExp++;
+    }
 
     if (n > 0x200) {
         n = 0x200;
@@ -6828,10 +6831,13 @@ float64 float64_scalbn( float64 a, int n STATUS_PARAM )
         }
         return a;
     }
-    if ( aExp != 0 )
+    if (aExp != 0) {
         aSig |= LIT64( 0x0010000000000000 );
-    else if ( aSig == 0 )
+    } else if (aSig == 0) {
         return a;
+    } else {
+        aExp++;
+    }
 
     if (n > 0x1000) {
         n = 0x1000;
-- 
1.8.5

Re: [Qemu-devel] [PATCH] softfloat: Fix factor 2 error for scalbn on denormal inputs

[Richard Henderson]

On 12/19/2013 01:57 PM, Peter Maydell wrote:
> If the input to float*_scalbn() is denormal then it represents
> a number 0.[mantissabits] * 2^(1-exponentbias) (and the actual
> exponent field is all zeroes). This means that when we convert
> it to our unpacked encoding the unpacked exponent must be one
> greater than for a normal number, which represents
> 1.[mantissabits] * 2^(e-exponentbias) for an exponent field e.
> 
> This meant we were giving answers too small by a factor of 2 for
> all denormal inputs.
> 
> Note that the float-to-int routines also have this behaviour
> of not adjusting the exponent for denormals; however there it is
> harmless because denormals will all convert to integer zero anyway.
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---

At least the float128 routine needs the same fix.

I'm less certain about the floatx80 routine, since IIRC
that format has no implicit 1 bit -- it's always explicit.


r~

Re: [Qemu-devel] [PATCH] softfloat: Fix factor 2 error for scalbn on denormal inputs

[Peter Maydell]

On 20 December 2013 17:34, Richard Henderson <rth@twiddle.net> wrote:
> On 12/19/2013 01:57 PM, Peter Maydell wrote:
>> If the input to float*_scalbn() is denormal then it represents
>> a number 0.[mantissabits] * 2^(1-exponentbias) (and the actual
>> exponent field is all zeroes). This means that when we convert
>> it to our unpacked encoding the unpacked exponent must be one
>> greater than for a normal number, which represents
>> 1.[mantissabits] * 2^(e-exponentbias) for an exponent field e.
>>
>> This meant we were giving answers too small by a factor of 2 for
>> all denormal inputs.
>>
>> Note that the float-to-int routines also have this behaviour
>> of not adjusting the exponent for denormals; however there it is
>> harmless because denormals will all convert to integer zero anyway.

> At least the float128 routine needs the same fix.
>
> I'm less certain about the floatx80 routine, since IIRC
> that format has no implicit 1 bit -- it's always explicit.

It still needs the increment, because according to
http://en.wikipedia.org/wiki/Extended_precision#x86_Extended_Precision_Format
the value for a denormal is m * 2^(1-exponentbias)
versus m * (2^(e-exponentbias), so the off-by-one
effect still holds; the explicitness of the integer bit
isn't relevant here (it just means 'm' is the integer
and mantissa bits, rather than either 0.mantissa
or 1.mantissa as in the commit message above).

thanks
-- PMM