linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] Fix rounding bug in emulation for double float operating
@ 2007-12-10  5:00 Liu Yu
  2007-12-10  4:56 ` David Gibson
  2007-12-14  5:01 ` [PATCH] Fix rounding bug in emulation for double float operating Kumar Gala
  0 siblings, 2 replies; 13+ messages in thread
From: Liu Yu @ 2007-12-10  5:00 UTC (permalink / raw)
  To: linuxppc-dev


This patch fixes rounding bug in emulation for double float operating on PowerPC platform.

When pack double float operand, it need to truncate the tail due to the limited precision.
If the truncated part is not zero, the last bit of work bit (totally 3 bits) need to '|' 1.

This patch is completed in _FP_FRAC_SRS_2(X,N,sz) (arch/powerpc/math-emu/op-2.h).
Originally the code leftwards rotates the operand to just keep the truncated part,
then check whether it is zero. However, the number it rotates is not correct when
N is not smaller than _FP_W_TYPE_SIZE, and it will cause the work bit '|' 1 in the improper case.

This patch fixes this issue.

Signed-off-by: Liu Yu <b13201@freescale.com>

---
 arch/powerpc/math-emu/op-2.h |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/math-emu/op-2.h b/arch/powerpc/math-emu/op-2.h
index b9b06b4..7d6f17c 100644
--- a/arch/powerpc/math-emu/op-2.h
+++ b/arch/powerpc/math-emu/op-2.h
@@ -59,7 +59,8 @@
     else								\
       {									\
 	X##_f0 = (X##_f1 >> ((N) - _FP_W_TYPE_SIZE) |			\
-	          (((X##_f1 << (sz - (N))) | X##_f0) != 0));		\
+	          (((X##_f1 << (2 * _FP_W_TYPE_SIZE - (N))) |		\
+		   X##_f0) != 0));					\
 	X##_f1 = 0;							\
       }									\
   } while (0)
-- 
1.5.2

^ permalink raw reply related	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2007-12-14  6:14 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-12-10  5:00 [PATCH] Fix rounding bug in emulation for double float operating Liu Yu
2007-12-10  4:56 ` David Gibson
2007-12-10  5:25   ` [PATCH] Fix rounding bug in emulation for double floatoperating Liu Yu
2007-12-10 15:00     ` Kumar Gala
2007-12-11  1:38       ` Liu Yu
2007-12-11  3:19       ` Zang Roy-r61911
2007-12-11 15:26         ` Kumar Gala
2007-12-12  8:30           ` Zang Roy-r61911
2007-12-12 13:36             ` Kumar Gala
2007-12-14  5:46             ` Kumar Gala
2007-12-14  6:06               ` Zang Roy-r61911
2007-12-14  6:14                 ` Kumar Gala
2007-12-14  5:01 ` [PATCH] Fix rounding bug in emulation for double float operating Kumar Gala

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).