[PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

[PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

[Anatoli Antonovitch]

Description
(max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more
than 4GB memory, and hence cause vmap to fail every time.


Signed-off-by: Michael Chen <micche@ati.com>

Patch
diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
linux-2.4.21-40.EL.diff/mm/vmalloc.c
--- linux-2.4.21-40.EL/mm/vmalloc.c     2006-02-02 21:13:20.000000000
-0600
+++ linux-2.4.21-40.EL.diff/mm/vmalloc.c        2006-09-04
11:29:33.000000000 -0500
@@ -298,8 +298,8 @@
        struct vm_struct *area;
        unsigned long size = count << PAGE_SHIFT;
 
-       if (!size || size > (max_mapnr << PAGE_SHIFT))
-               return NULL;
+    if (!count || count > max_mapnr)
+        return NULL;
        area = get_vm_area(size, flags);
        if (!area) {
                return NULL;

Re: [PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

[Hugh Dickins]

This is a 2.4 fix (not needed in 2.6): let's CC maintainer Willy Tarreau.

On Fri, 15 Sep 2006, Anatoli Antonovitch wrote:

> Description
> (max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more
> than 4GB memory, and hence cause vmap to fail every time.

Good point, thanks for the patch.  Sorry I'm so slow to get to it.

> 
> Signed-off-by: Michael Chen <micche@ati.com>
> 
> Patch
> diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
> linux-2.4.21-40.EL.diff/mm/vmalloc.c
> --- linux-2.4.21-40.EL/mm/vmalloc.c     2006-02-02 21:13:20.000000000
> -0600
> +++ linux-2.4.21-40.EL.diff/mm/vmalloc.c        2006-09-04

And still needs fixing in latest mainline 2.4.

> 11:29:33.000000000 -0500
> @@ -298,8 +298,8 @@
>         struct vm_struct *area;
>         unsigned long size = count << PAGE_SHIFT;
>  
> -       if (!size || size > (max_mapnr << PAGE_SHIFT))
> -               return NULL;
> +    if (!count || count > max_mapnr)
> +        return NULL;

I'm afraid the tabs got messed up in both the old and new lines.
Also, count is a signed int (whereas size and max_mapnr are both
unsigned longs), so best reject "count <= 0" rather than just "!count".

>         area = get_vm_area(size, flags);
>         if (!area) {
>                 return NULL;

Here's a replacement patch for Willy.  Anatoli, you didn't sign
off the patch yourself: so I'm assuming Michael is the originator.


From: Michael Chen <micche@ati.com>

(max_mapnr << PAGE_SHIFT) would overflow on a system which has
4GB memory or more, and so could cause vmap to fail every time.

Signed-off-by: Michael Chen <micche@ati.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
---

 mm/vmalloc.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- 2.4.34-pre3/mm/vmalloc.c	2004-04-14 14:05:41.000000000 +0100
+++ linux/mm/vmalloc.c	2006-09-23 17:52:59.000000000 +0100
@@ -293,7 +293,7 @@ void * vmap(struct page **pages, int cou
 	struct vm_struct *area;
 	unsigned long size = count << PAGE_SHIFT;
 
-	if (!size || size > (max_mapnr << PAGE_SHIFT))
+	if (count <= 0 || count > max_mapnr)
 		return NULL;
 	area = get_vm_area(size, flags);
 	if (!area) {

Re: [PATCH]i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

[Willy Tarreau]

Hi Hugh !

On Sat, Sep 23, 2006 at 06:26:26PM +0100, Hugh Dickins wrote:
> This is a 2.4 fix (not needed in 2.6): let's CC maintainer Willy Tarreau.
> 
> On Fri, 15 Sep 2006, Anatoli Antonovitch wrote:
> 
> > Description
> > (max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more
> > than 4GB memory, and hence cause vmap to fail every time.
> 
> Good point, thanks for the patch.  Sorry I'm so slow to get to it.

Don't worry, Andrew already forwarded it to me. BTW, thanks for your
review and comment, I'll finally apply yours since it's better.

Cheers,
Willy