Re: [PATCH] avoid negative shifts in radix-tree.c, take 2

[Andreas Schwab <schwab@suse.de>]

Peter Lund <firefly@vax64.dk> writes:

> Shifting by more than the width of the value on the left is also not allowed.

Shifting by the width of the value is not allowed as well.

> --- linux-2.6.22/lib/radix-tree.c.orig	2007-08-27 15:42:37.000000000 +0200
> +++ linux-2.6.22/lib/radix-tree.c	2007-08-29 13:19:19.000000000 +0200
> @@ -980,12 +980,13 @@
>  
>  static __init unsigned long __maxindex(unsigned int height)
>  {
> -	unsigned int tmp = height * RADIX_TREE_MAP_SHIFT;
> -	unsigned long index = (~0UL >> (RADIX_TREE_INDEX_BITS - tmp - 1)) >> 1;
> +	unsigned int width = height * RADIX_TREE_MAP_SHIFT;
> +	int          shift = RADIX_TREE_INDEX_BITS - width;
>  
> -	if (tmp >= RADIX_TREE_INDEX_BITS)
> -		index = ~0UL;
> -	return index;
> +	if (shift < 0)
> +		return ~0UL;
> +	else
> +		return ~0UL >> shift;

Since height can be zero, you still have undefined behaviour.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."