public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] MPILIB: Fix comparison of negative MPIs
@ 2014-12-06  0:00 Rasmus Villemoes
  2014-12-06  0:00 ` [PATCH 2/2] MPILIB: Deobfuscate mpi_cmp Rasmus Villemoes
  2015-01-09 10:58 ` [PATCH 1/2] MPILIB: Fix comparison of negative MPIs David Howells
  0 siblings, 2 replies; 8+ messages in thread
From: Rasmus Villemoes @ 2014-12-06  0:00 UTC (permalink / raw)
  To: David Howells, Rusty Russell; +Cc: Rasmus Villemoes, linux-kernel

If u and v both represent negative integers and their limb counts
happen to differ, mpi_cmp will always return a positive value - this
is obviously bogus. u is smaller than v if and only if it is larger in
absolute value.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 lib/mpi/mpi-cmp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/mpi/mpi-cmp.c b/lib/mpi/mpi-cmp.c
index 1871e7b61ca0..3801694240d8 100644
--- a/lib/mpi/mpi-cmp.c
+++ b/lib/mpi/mpi-cmp.c
@@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v)
 	if (usize != vsize && !u->sign && !v->sign)
 		return usize - vsize;
 	if (usize != vsize && u->sign && v->sign)
-		return vsize + usize;
+		return vsize - usize;
 	if (!usize)
 		return 0;
 	cmp = mpihelp_cmp(u->d, v->d, usize);
-- 
2.1.3


^ permalink raw reply related	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2015-01-12 11:47 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-12-06  0:00 [PATCH 1/2] MPILIB: Fix comparison of negative MPIs Rasmus Villemoes
2014-12-06  0:00 ` [PATCH 2/2] MPILIB: Deobfuscate mpi_cmp Rasmus Villemoes
2015-01-09 11:00   ` David Howells
2015-01-10 10:29     ` [Keyrings] " Dmitry Kasatkin
2015-01-12 11:43       ` David Howells
2015-01-12 11:47         ` Dmitry Kasatkin
2015-01-09 10:58 ` [PATCH 1/2] MPILIB: Fix comparison of negative MPIs David Howells
2015-01-10 10:27   ` [Keyrings] " Dmitry Kasatkin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox