Re: [PATCH v3] hugetlbfs: Fix integer overflow check in hugetlbfs_file_mmap()

[Mike Kravetz <mike.kravetz@oracle.com>]

On 07/24/23 04:47, Matthew Wilcox wrote:
> On Thu, Jul 20, 2023 at 10:49:52PM +0800, Linke Li wrote:
> > +++ b/fs/hugetlbfs/inode.c
> > @@ -154,10 +154,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
> >  	if (vma->vm_pgoff & (~huge_page_mask(h) >> PAGE_SHIFT))
> >  		return -EINVAL;
> >  
> > -	vma_len = (loff_t)(vma->vm_end - vma->vm_start);
> > -	len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
> > -	/* check for overflow */
> > -	if (len < vma_len)
> > +	if (check_add_overflow(vma_len, (loff_t)vma->vm_pgoff << PAGE_SHIFT, &len))
> >  		return -EINVAL;
> 
> Doesn't this check duplicate that performed by file_mmap_ok()?  Can't we
> just delete the check, or is there a code path that leads here while
> avoiding file_mmap_ok()?

Thanks for pointing that out.
Yes, from my reading/understanding that is a repeat.

It looks like most of the overflow checking in hugetlbfs_file_mmap is a
repeat of checks done previously.  I remember adding this code in
response to a checker or someone pointing out the potential for overflow:

	/*
	 * page based offset in vm_pgoff could be sufficiently large to
	 * overflow a loff_t when converted to byte offset.  This can
	 * only happen on architectures where sizeof(loff_t) ==
	 * sizeof(unsigned long).  So, only check in those instances.
	 */
	if (sizeof(unsigned long) == sizeof(loff_t)) {
		if (vma->vm_pgoff & PGOFF_LOFFT_MAX)
			return -EINVAL;
	}

However, file_mmap_ok seems to handle this as well.  The important thing that
needs to be done in hugetlbfs_file_mmap is checking for huge page alignment.

I have added this code cleanup to my list if someone does not do it first.
-- 
Mike Kravetz