From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4BE55C54E5D for ; Thu, 14 Mar 2024 03:23:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 11C3910F88E; Thu, 14 Mar 2024 03:23:56 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="bEhMd+BR"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) by gabe.freedesktop.org (Postfix) with ESMTPS id D1E6C10F82F for ; Thu, 14 Mar 2024 03:23:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710386629; x=1741922629; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hFsAD+ZI0orQxpm9XEc+BoRKAY8ZkA+CGuFLkJL3ecU=; b=bEhMd+BR7aEs+ax1KbsoPSj1O0CXSJNl9A+P6jsJ5pE8K6MxQBQZx/gH +0HXFzcOSfNf6k/aLvyaPicZM6+2/P1HC9BsOx0f5vtRpVP5GcfRYEsye 0sDmFjKXV+3H15xO8duhZS/kGVYHZv3ducuNJRZMmxq63gMheNAWpwZ41 3vVI0o0tVlLZckbFpvfWsgGeaqxip08HaMAoD6DZsoWwqNJ/QDzcfYcne JixaYUInGqo3Uu8qhpTFJdUN1Sn8zlig8BW+HwS4lb6PUxKV6Ahr7zpbx orgrHvwIpemCGJ62zv4mTYsCasb8x0B9uCJHRTH8iodORYHwN3JIaYOyw w==; X-IronPort-AV: E=McAfee;i="6600,9927,11012"; a="8966484" X-IronPort-AV: E=Sophos;i="6.07,124,1708416000"; d="scan'208";a="8966484" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2024 20:23:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,124,1708416000"; d="scan'208";a="12084629" Received: from szeng-desk.jf.intel.com ([10.165.21.149]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2024 20:23:48 -0700 From: Oak Zeng To: intel-xe@lists.freedesktop.org Cc: thomas.hellstrom@intel.com, matthew.brost@intel.com, airlied@gmail.com, brian.welty@intel.com, himal.prasad.ghimiray@intel.com Subject: [PATCH 5/5] drm/xe: Use hmm_range_fault to populate user pages Date: Wed, 13 Mar 2024 23:35:53 -0400 Message-Id: <20240314033553.1379444-6-oak.zeng@intel.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20240314033553.1379444-1-oak.zeng@intel.com> References: <20240314033553.1379444-1-oak.zeng@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" This is an effort to unify hmmptr (aka system allocator) and userptr code. hmm_range_fault is used to populate a virtual address range for both hmmptr and userptr, instead of hmmptr using hmm_range_fault and userptr using get_user_pages_fast. This also aligns with AMD gpu driver's behavior. In long term, we plan to put some common helpers in this area to drm layer so it can be re-used by different vendors. Signed-off-by: Oak Zeng --- drivers/gpu/drm/xe/xe_vm.c | 105 ++----------------------------------- 1 file changed, 4 insertions(+), 101 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index db3f049a47dc..d6088dcac74a 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -38,6 +38,7 @@ #include "xe_sync.h" #include "xe_trace.h" #include "xe_wa.h" +#include "xe_hmm.h" static struct drm_gem_object *xe_vm_obj(struct xe_vm *vm) { @@ -65,113 +66,15 @@ int xe_vma_userptr_check_repin(struct xe_userptr_vma *uvma) int xe_vma_userptr_pin_pages(struct xe_userptr_vma *uvma) { - struct xe_userptr *userptr = &uvma->userptr; struct xe_vma *vma = &uvma->vma; struct xe_vm *vm = xe_vma_vm(vma); struct xe_device *xe = vm->xe; - const unsigned long num_pages = xe_vma_size(vma) >> PAGE_SHIFT; - struct page **pages; - bool in_kthread = !current->mm; - unsigned long notifier_seq; - int pinned, ret, i; - bool read_only = xe_vma_read_only(vma); + bool write = !xe_vma_read_only(vma); + struct hmm_range hmm_range; lockdep_assert_held(&vm->lock); xe_assert(xe, xe_vma_is_userptr(vma)); -retry: - if (vma->gpuva.flags & XE_VMA_DESTROYED) - return 0; - - notifier_seq = mmu_interval_read_begin(&userptr->notifier); - if (notifier_seq == userptr->notifier_seq) - return 0; - - pages = kvmalloc_array(num_pages, sizeof(*pages), GFP_KERNEL); - if (!pages) - return -ENOMEM; - - if (userptr->sg) { - dma_unmap_sgtable(xe->drm.dev, - userptr->sg, - read_only ? DMA_TO_DEVICE : - DMA_BIDIRECTIONAL, 0); - sg_free_table(userptr->sg); - userptr->sg = NULL; - } - - pinned = ret = 0; - if (in_kthread) { - if (!mmget_not_zero(userptr->notifier.mm)) { - ret = -EFAULT; - goto mm_closed; - } - kthread_use_mm(userptr->notifier.mm); - } - - while (pinned < num_pages) { - ret = get_user_pages_fast(xe_vma_userptr(vma) + - pinned * PAGE_SIZE, - num_pages - pinned, - read_only ? 0 : FOLL_WRITE, - &pages[pinned]); - if (ret < 0) - break; - - pinned += ret; - ret = 0; - } - - if (in_kthread) { - kthread_unuse_mm(userptr->notifier.mm); - mmput(userptr->notifier.mm); - } -mm_closed: - if (ret) - goto out; - - ret = sg_alloc_table_from_pages_segment(&userptr->sgt, pages, - pinned, 0, - (u64)pinned << PAGE_SHIFT, - xe_sg_segment_size(xe->drm.dev), - GFP_KERNEL); - if (ret) { - userptr->sg = NULL; - goto out; - } - userptr->sg = &userptr->sgt; - - ret = dma_map_sgtable(xe->drm.dev, userptr->sg, - read_only ? DMA_TO_DEVICE : - DMA_BIDIRECTIONAL, - DMA_ATTR_SKIP_CPU_SYNC | - DMA_ATTR_NO_KERNEL_MAPPING); - if (ret) { - sg_free_table(userptr->sg); - userptr->sg = NULL; - goto out; - } - - for (i = 0; i < pinned; ++i) { - if (!read_only) { - lock_page(pages[i]); - set_page_dirty(pages[i]); - unlock_page(pages[i]); - } - - mark_page_accessed(pages[i]); - } - -out: - release_pages(pages, pinned); - kvfree(pages); - - if (!(ret < 0)) { - userptr->notifier_seq = notifier_seq; - if (xe_vma_userptr_check_repin(uvma) == -EAGAIN) - goto retry; - } - - return ret < 0 ? ret : 0; + return xe_hmm_populate_range(vma, &hmm_range, write); } static bool preempt_fences_waiting(struct xe_vm *vm) -- 2.26.3