Re: [PATCH 3.10 060/268] xen/gntdev: Use VM_MIXEDMAP instead of VM_IO to avoid NUMA balancing

Hugh Dickins
2017-06-20T02:57:39Z

On Mon, 19 Jun 2017, Willy Tarreau wrote:

> From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>
> commit 30faaafdfa0c754c91bac60f216c9f34a2bfdf7e upstream.
>
> Commit 9c17d96500f7 ("xen/gntdev: Grant maps should not be subject to
> NUMA balancing") set VM_IO flag to prevent grant maps from being
> subjected to NUMA balancing.
>
> It was discovered recently that this flag causes get_user_pages() to
> always fail with -EFAULT.
>
> check_vma_flags
> __get_user_pages
> __get_user_pages_locked
> __get_user_pages_unlocked
> get_user_pages_fast
> iov_iter_get_pages
> dio_refill_pages
> do_direct_IO
> do_blockdev_direct_IO
> do_blockdev_direct_IO
> ext4_direct_IO_read
> generic_file_read_iter
> aio_run_iocb
>
> (which can happen if guest's vdisk has direct-io-safe option).
>
> To avoid this let's use VM_MIXEDMAP flag instead --- it prevents
> NUMA balancing just as VM_IO does and has no effect on
> check_vma_flags().

This is only valid if kernel/sched/fair.c is checking for VM_MIXEDMAP,
and that came in v4.1's 8e76d4eecf7a ("sched, numa: do not hint for
NUMA balancing on VM_MIXEDMAP mappings"), which I don't see in your
tree nor in this series (please double check, I may have missed it).

It would be good to have that one in too, and it was marked for
stable; but maybe it didn't apply, because of depending on another
commit adding the is_vm_hugetlb_page(vma) check there? Which I
expect would also be good to have, but I haven't looked it up.

Maybe drop this one for this round, and gather up its dependencies
for the next round.

Ben's 3.16 tree appeared to be in the same position,
I didn't look at the EOL 3.18.

(I've not yet checked through backports of the "larger stack guard gap"
- thank you all for those - will do so, but won't get through them
tonight - I must look into DaveJ's trinity VM_BUG_ON now.)

Hugh

>
> Reported-by: Olaf Hering <olaf@aepfle.de>
> Suggested-by: Hugh Dickins <hughd@google.com>
> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Acked-by: Hugh Dickins <hughd@google.com>
> Tested-by: Olaf Hering <olaf@aepfle.de>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Signed-off-by: Willy Tarreau <w@1wt.eu>
> ---
> drivers/xen/gntdev.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
> index 3d8e609..6c6d253 100644
> --- a/drivers/xen/gntdev.c
> +++ b/drivers/xen/gntdev.c
> @@ -770,7 +770,7 @@ static int gntdev_mmap(struct file *flip, struct vm_area_struct *vma)
>
> vma->vm_ops = &gntdev_vmops;
>
> - vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP | VM_IO;
> + vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP | VM_MIXEDMAP;
>
> if (use_ptemod)
> vma->vm_flags |= VM_DONTCOPY;
> --
> 2.8.0.rc2.1.gbe9624a