Re: [PATCH 1/2] MPILIB: Fix comparison of negative MPIs

[David Howells <dhowells@redhat.com>]

I think you're right - *adding* the two sizes makes no sense.  cc'ing Dmitry
also for his check.

David


Rasmus Villemoes <linux@rasmusvillemoes.dk> wrote:

> If u and v both represent negative integers and their limb counts
> happen to differ, mpi_cmp will always return a positive value - this
> is obviously bogus. u is smaller than v if and only if it is larger in
> absolute value.
> 
> Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> ---
>  lib/mpi/mpi-cmp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/mpi/mpi-cmp.c b/lib/mpi/mpi-cmp.c
> index 1871e7b61ca0..3801694240d8 100644
> --- a/lib/mpi/mpi-cmp.c
> +++ b/lib/mpi/mpi-cmp.c
> @@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v)
>  	if (usize != vsize && !u->sign && !v->sign)
>  		return usize - vsize;
>  	if (usize != vsize && u->sign && v->sign)
> -		return vsize + usize;
> +		return vsize - usize;
>  	if (!usize)
>  		return 0;
>  	cmp = mpihelp_cmp(u->d, v->d, usize);
> -- 
> 2.1.3
>