Re: [PATCH v4] mm/userfaultfd: fix hugetlb fault mutex hash calculation

[jane.chu@oracle.com]

On 3/10/2026 4:05 AM, Jianhui Zhou wrote:
> In mfill_atomic_hugetlb(), linear_page_index() is used to calculate the
> page index for hugetlb_fault_mutex_hash(). However, linear_page_index()
> returns the index in PAGE_SIZE units, while hugetlb_fault_mutex_hash()
> expects the index in huge page units. This mismatch means that different
> addresses within the same huge page can produce different hash values,
> leading to the use of different mutexes for the same huge page. This can
> cause races between faulting threads, which can corrupt the reservation
> map and trigger the BUG_ON in resv_map_release().
> 
> Fix this by introducing hugetlb_linear_page_index(), which returns the
> page index in huge page granularity, and using it in place of
> linear_page_index().
> 
> Fixes: a08c7193e4f1 ("mm/filemap: remove hugetlb special casing in filemap.c")
> Reported-by: syzbot+f525fd79634858f478e7@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=f525fd79634858f478e7
> Cc: stable@vger.kernel.org
> Signed-off-by: Jianhui Zhou <jianhuizzzzz@gmail.com>
> ---
> v4:
> - Introduce hugetlb_linear_page_index() instead of exposing
>    vma_hugecache_offset(); call hstate_vma() internally to simplify
>    the API (David Hildenbrand)
> 
> v3:
> - Fix Fixes tag to a08c7193e4f1 (Hugh Dickins)
> 
> v2:
> - Remove unnecessary !CONFIG_HUGETLB_PAGE stub for vma_hugecache_offset()
>    (Peter Xu, SeongJae Park)
> 
>   include/linux/hugetlb.h | 17 +++++++++++++++++
>   mm/userfaultfd.c        |  2 +-
>   2 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 65910437be1c..67d4f0924646 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -796,6 +796,23 @@ static inline unsigned huge_page_shift(struct hstate *h)
>   	return h->order + PAGE_SHIFT;
>   }
>   
> +/**
> + * hugetlb_linear_page_index() - linear_page_index() but in hugetlb
> + *				 page size granularity.
> + * @vma: the hugetlb VMA
> + * @address: the virtual address within the VMA
> + *
> + * Return: the page offset within the mapping in huge page units.
> + */
> +static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma,
> +		unsigned long address)
> +{
> +	struct hstate *h = hstate_vma(vma);
> +
> +	return ((address - vma->vm_start) >> huge_page_shift(h)) +
> +		(vma->vm_pgoff >> huge_page_order(h));
> +}
> +
>   static inline bool order_is_gigantic(unsigned int order)
>   {
>   	return order > MAX_PAGE_ORDER;
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index 927086bb4a3c..5590989e18c7 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -573,7 +573,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(
>   		 * in the case of shared pmds.  fault mutex prevents
>   		 * races with other faulting threads.
>   		 */
> -		idx = linear_page_index(dst_vma, dst_addr);
> +		idx = hugetlb_linear_page_index(dst_vma, dst_addr);

Just wondering whether making the shift explicit here instead of 
introducing another hugetlb helper might be sufficient?

     idx >>= huge_page_order(hstate_vma(vma));

I mean huge_page_order() is already explicitly called in several places 
outside hugetlb.


>   		mapping = dst_vma->vm_file->f_mapping;
>   		hash = hugetlb_fault_mutex_hash(mapping, idx);
>   		mutex_lock(&hugetlb_fault_mutex_table[hash]);

thanks,
-jane