Re: Arithmetic overflow in may_expand_vm()

[Johannes Weiner <hannes@cmpxchg.org>]

Hi,

On Thu, Oct 15, 2009 at 10:24:51AM -0700, Earl Chew wrote:
> This code currently reads:
> 
> >int may_expand_vm(struct mm_struct *mm, unsigned long npages)
> >{
> >        unsigned long cur = mm->total_vm;       /* pages */
> >        unsigned long lim;
> >
> >        lim = current->signal->rlim[RLIMIT_AS].rlim_cur >> PAGE_SHIFT;
> >
> >        if (cur + npages > lim)
> >                return 0;
> >        return 1;
> >}
> 
> If npages is stupendously large, the failure predicate may
> return a false negative due to (cur + npages) overflowing and
> wrapping.

Can this really happen?

npages always originates in a value of byte granularity, giving a
theoretical maximum of ~0UL >> PAGE_SHIFT (checking for more than the
number of addressable bytes just makes no sense).

And mm->total_vm is always PAGE_SIZE times smaller than total user
address space (which in turn is always less than ~0UL).

So I can not see this overflow being possible with PAGE_SHIFT > 0.