[PATCH] fix integer overflow in groups_search

[PATCH] fix integer overflow in groups_search

[Jerome Marchand]

gid_t is a unsigned int. If group_info contains a gid greater than
MAX_INT, groups_search() function may look on the wrong side of the 
search tree.
This solves some unfair "permission denied" problems.

Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
---
diff --git a/kernel/groups.c b/kernel/groups.c
index 53b1916..253dc0f 100644
--- a/kernel/groups.c
+++ b/kernel/groups.c
@@ -143,10 +143,9 @@ int groups_search(const struct group_info *group_info, gid_t grp)
 	right = group_info->ngroups;
 	while (left < right) {
 		unsigned int mid = (left+right)/2;
-		int cmp = grp - GROUP_AT(group_info, mid);
-		if (cmp > 0)
+		if (grp > GROUP_AT(group_info, mid))
 			left = mid + 1;
-		else if (cmp < 0)
+		else if (grp < GROUP_AT(group_info, mid))
 			right = mid;
 		else
 			return 1;

Re: [PATCH] fix integer overflow in groups_search

[Andrew Morton]

On Thu, 02 Sep 2010 17:20:20 +0200
Jerome Marchand <jmarchan@redhat.com> wrote:

> 
> gid_t is a unsigned int. If group_info contains a gid greater than
> MAX_INT, groups_search() function may look on the wrong side of the 
> search tree.
> This solves some unfair "permission denied" problems.
> 
> Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
> ---
> diff --git a/kernel/groups.c b/kernel/groups.c
> index 53b1916..253dc0f 100644
> --- a/kernel/groups.c
> +++ b/kernel/groups.c
> @@ -143,10 +143,9 @@ int groups_search(const struct group_info *group_info, gid_t grp)
>  	right = group_info->ngroups;
>  	while (left < right) {
>  		unsigned int mid = (left+right)/2;
> -		int cmp = grp - GROUP_AT(group_info, mid);
> -		if (cmp > 0)
> +		if (grp > GROUP_AT(group_info, mid))
>  			left = mid + 1;
> -		else if (cmp < 0)
> +		else if (grp < GROUP_AT(group_info, mid))
>  			right = mid;
>  		else
>  			return 1;

hah, that's some pretty ancient code there.  You must have a lot of
groups.

The patch is a no-brainer and I guess you've encountered the bug in
real kernels so I think we'll wave the fix at the -stable guys for
backporting, OK?