* (no subject)
@ 2024-04-24 9:42 Farah Kassabri
2024-04-24 9:42 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Farah Kassabri @ 2024-04-24 9:42 UTC (permalink / raw)
To: intel-xe; +Cc: ofir1.bitton, Farah Kassabri
Current Xe driver supports one TLB invaliation flow, which is
for the compute TLB going through the Guc.
This driver will be used for future products which will have multiple
TLBs each has it's own invalidation flow.
In order to prepare the driver for those new flows, these patches here
will add infrastructure to easily allow each TLB to register it's own
function keeping the common code generic.
Farah Kassabri (2):
drm/xe: rename gt invalidation functions
drm/xe: preparations for multi instance TLB invalidations
drivers/gpu/drm/xe/Makefile | 1 +
drivers/gpu/drm/xe/abi/guc_actions_abi.h | 2 +-
drivers/gpu/drm/xe/xe_device.c | 7 +
drivers/gpu/drm/xe/xe_ggtt.c | 6 +-
drivers/gpu/drm/xe/xe_gt.c | 9 +-
.../gpu/drm/xe/xe_gt_guc_tlb_invalidation.c | 459 +++++++++++++++++
.../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 26 +
...s.h => xe_gt_guc_tlb_invalidation_types.h} | 15 +-
drivers/gpu/drm/xe/xe_gt_pagefault.c | 2 +-
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 484 ++++--------------
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 66 ++-
drivers/gpu/drm/xe/xe_gt_types.h | 6 +-
drivers/gpu/drm/xe/xe_guc_ct.c | 6 +-
drivers/gpu/drm/xe/xe_pt.c | 12 +-
drivers/gpu/drm/xe/xe_trace.h | 38 +-
drivers/gpu/drm/xe/xe_vm.c | 8 +-
16 files changed, 695 insertions(+), 452 deletions(-)
create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c
create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h
rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation_types.h => xe_gt_guc_tlb_invalidation_types.h} (54%)
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/2] drm/xe: rename gt invalidation functions 2024-04-24 9:42 Farah Kassabri @ 2024-04-24 9:42 ` Farah Kassabri 2024-04-29 1:29 ` Matthew Brost 2024-04-24 9:42 ` [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations Farah Kassabri 2024-04-28 22:02 ` ✗ CI.Patch_applied: failure for series starting with [1/2] drm/xe: rename gt invalidation functions Patchwork 2 siblings, 1 reply; 7+ messages in thread From: Farah Kassabri @ 2024-04-24 9:42 UTC (permalink / raw) To: intel-xe; +Cc: ofir1.bitton, Farah Kassabri Future products will have multi instance TLBs, Current Xe driver handles only the compute TLBs invalidaiton through the Guc. In preparation for Multi instance TLB support, we need to rename current invalidation as Guc specific invalidation functions, and not general gt functions. This patch just renames the current functions, files and variables. follow up patches will start adding the multi instance TLB support for future XE products, where each will have its own invalidation flow. Signed-off-by: Farah Kassabri <fkassabri@habana.ai> --- drivers/gpu/drm/xe/Makefile | 2 +- drivers/gpu/drm/xe/abi/guc_actions_abi.h | 2 +- drivers/gpu/drm/xe/xe_ggtt.c | 6 +- drivers/gpu/drm/xe/xe_gt.c | 6 +- ...idation.c => xe_gt_guc_tlb_invalidation.c} | 72 +++++++++---------- .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 26 +++++++ ...s.h => xe_gt_guc_tlb_invalidation_types.h} | 15 ++-- drivers/gpu/drm/xe/xe_gt_pagefault.c | 4 +- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 4 +- drivers/gpu/drm/xe/xe_gt_types.h | 2 +- drivers/gpu/drm/xe/xe_guc_ct.c | 6 +- drivers/gpu/drm/xe/xe_pt.c | 12 ++-- drivers/gpu/drm/xe/xe_trace.h | 38 +++++----- drivers/gpu/drm/xe/xe_vm.c | 6 +- 14 files changed, 115 insertions(+), 86 deletions(-) rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation.c => xe_gt_guc_tlb_invalidation.c} (83%) create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation_types.h => xe_gt_guc_tlb_invalidation_types.h} (54%) diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile index 8321ec4f9b46..63e757406595 100644 --- a/drivers/gpu/drm/xe/Makefile +++ b/drivers/gpu/drm/xe/Makefile @@ -90,7 +90,7 @@ xe-y += xe_bb.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \ xe_gt_throttle_sysfs.o \ - xe_gt_tlb_invalidation.o \ + xe_gt_guc_tlb_invalidation.o \ xe_gt_topology.o \ xe_guc.o \ xe_guc_ads.o \ diff --git a/drivers/gpu/drm/xe/abi/guc_actions_abi.h b/drivers/gpu/drm/xe/abi/guc_actions_abi.h index 79ba98a169f9..ad4cdd5af80e 100644 --- a/drivers/gpu/drm/xe/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/xe/abi/guc_actions_abi.h @@ -187,7 +187,7 @@ enum xe_guc_sleep_state_status { /* Flush PPC or SMRO caches along with TLB invalidation request */ #define XE_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) -enum xe_guc_tlb_invalidation_type { +enum xe_gt_guc_tlb_invalidation_type { XE_GUC_TLB_INVAL_FULL = 0x0, XE_GUC_TLB_INVAL_PAGE_SELECTIVE = 0x1, XE_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX = 0x2, diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c index 0d541f55b4fc..e0c1bfa92730 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.c +++ b/drivers/gpu/drm/xe/xe_ggtt.c @@ -19,7 +19,7 @@ #include "xe_device.h" #include "xe_gt.h" #include "xe_gt_printk.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_map.h" #include "xe_pm.h" #include "xe_sriov.h" @@ -252,9 +252,9 @@ static void ggtt_invalidate_gt_tlb(struct xe_gt *gt) if (!gt) return; - err = xe_gt_tlb_invalidation_ggtt(gt); + err = xe_gt_guc_tlb_invalidation_ggtt(gt); if (err) - drm_warn(>_to_xe(gt)->drm, "xe_gt_tlb_invalidation_ggtt error=%d", err); + drm_warn(>_to_xe(gt)->drm, "xe_gt_guc_tlb_invalidation_ggtt error=%d", err); } static void xe_ggtt_invalidate(struct xe_ggtt *ggtt) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index 491d0413de15..b162f10902ea 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -31,7 +31,7 @@ #include "xe_gt_printk.h" #include "xe_gt_sriov_pf.h" #include "xe_gt_sysfs.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_gt_topology.h" #include "xe_guc_exec_queue_types.h" #include "xe_guc_pc.h" @@ -536,7 +536,7 @@ int xe_gt_init(struct xe_gt *gt) xe_hw_fence_irq_init(>->fence_irq[i]); } - err = xe_gt_tlb_invalidation_init(gt); + err = xe_gt_guc_tlb_invalidation_init(gt); if (err) return err; @@ -659,7 +659,7 @@ static int gt_reset(struct xe_gt *gt) if (err) goto err_out; - xe_gt_tlb_invalidation_reset(gt); + xe_gt_guc_tlb_invalidation_reset(gt); err = do_gt_reset(gt); if (err) diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c similarity index 83% rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c index 93df2d7969b3..f994c64c95f4 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c @@ -3,7 +3,7 @@ * Copyright © 2023 Intel Corporation */ -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "abi/guc_actions_abi.h" #include "xe_device.h" @@ -17,11 +17,11 @@ #define TLB_TIMEOUT (HZ / 4) -static void xe_gt_tlb_fence_timeout(struct work_struct *work) +static void xe_gt_guc_tlb_fence_timeout(struct work_struct *work) { struct xe_gt *gt = container_of(work, struct xe_gt, tlb_invalidation.fence_tdr.work); - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; spin_lock_irq(>->tlb_invalidation.pending_lock); list_for_each_entry_safe(fence, next, @@ -32,7 +32,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) if (msecs_to_jiffies(since_inval_ms) < TLB_TIMEOUT) break; - trace_xe_gt_tlb_invalidation_fence_timeout(fence); + trace_xe_gt_guc_tlb_invalidation_fence_timeout(fence); xe_gt_err(gt, "TLB invalidation fence timeout, seqno=%d recv=%d", fence->seqno, gt->tlb_invalidation.seqno_recv); @@ -49,7 +49,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) } /** - * xe_gt_tlb_invalidation_init - Initialize GT TLB invalidation state + * xe_gt_guc_tlb_invalidation_init - Initialize GT TLB invalidation state * @gt: graphics tile * * Initialize GT TLB invalidation state, purely software initialization, should @@ -57,42 +57,42 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) * * Return: 0 on success, negative error code on error. */ -int xe_gt_tlb_invalidation_init(struct xe_gt *gt) +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt) { gt->tlb_invalidation.seqno = 1; INIT_LIST_HEAD(>->tlb_invalidation.pending_fences); spin_lock_init(>->tlb_invalidation.pending_lock); spin_lock_init(>->tlb_invalidation.lock); INIT_DELAYED_WORK(>->tlb_invalidation.fence_tdr, - xe_gt_tlb_fence_timeout); + xe_gt_guc_tlb_fence_timeout); return 0; } static void -__invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) +__invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) { - trace_xe_gt_tlb_invalidation_fence_signal(fence); + trace_xe_gt_guc_tlb_invalidation_fence_signal(fence); dma_fence_signal(&fence->base); dma_fence_put(&fence->base); } static void -invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) +invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) { list_del(&fence->link); __invalidation_fence_signal(fence); } /** - * xe_gt_tlb_invalidation_reset - Initialize GT TLB invalidation reset + * xe_gt_guc_tlb_invalidation_reset - Initialize GT TLB invalidation reset * @gt: graphics tile * * Signal any pending invalidation fences, should be called during a GT reset */ -void xe_gt_tlb_invalidation_reset(struct xe_gt *gt) +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt) { - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; struct xe_guc *guc = >->uc.guc; int pending_seqno; @@ -139,7 +139,7 @@ static bool tlb_invalidation_seqno_past(struct xe_gt *gt, int seqno) } static int send_tlb_invalidation(struct xe_guc *guc, - struct xe_gt_tlb_invalidation_fence *fence, + struct xe_gt_guc_tlb_invalidation_fence *fence, u32 *action, int len) { struct xe_gt *gt = guc_to_gt(guc); @@ -156,7 +156,7 @@ static int send_tlb_invalidation(struct xe_guc *guc, seqno = gt->tlb_invalidation.seqno; if (fence) { fence->seqno = seqno; - trace_xe_gt_tlb_invalidation_fence_send(fence); + trace_xe_gt_guc_tlb_invalidation_fence_send(fence); } action[1] = seqno; ret = xe_guc_ct_send_locked(&guc->ct, action, len, @@ -202,16 +202,16 @@ static int send_tlb_invalidation(struct xe_guc *guc, XE_GUC_TLB_INVAL_FLUSH_CACHE) /** - * xe_gt_tlb_invalidation_guc - Issue a TLB invalidation on this GT for the GuC + * xe_gt_guc_tlb_invalidation - Issue a TLB invalidation on this GT for the GuC * @gt: graphics tile * * Issue a TLB invalidation for the GuC. Completion of TLB is asynchronous and - * caller can use seqno + xe_gt_tlb_invalidation_wait to wait for completion. + * caller can use seqno + xe_gt_guc_tlb_invalidation_wait to wait for completion. * * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, * negative error code on error. */ -static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) +static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) { u32 action[] = { XE_GUC_ACTION_TLB_INVALIDATION, @@ -224,7 +224,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) } /** - * xe_gt_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT + * xe_gt_guc_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT * @gt: graphics tile * * Issue a TLB invalidation for the GGTT. Completion of TLB invalidation is @@ -232,7 +232,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) * * Return: 0 on success, negative error code on error */ -int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt) { struct xe_device *xe = gt_to_xe(gt); @@ -240,11 +240,11 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) gt->uc.guc.submission_state.enabled) { int seqno; - seqno = xe_gt_tlb_invalidation_guc(gt); + seqno = xe_gt_guc_tlb_invalidation(gt); if (seqno <= 0) return seqno; - xe_gt_tlb_invalidation_wait(gt, seqno); + xe_gt_guc_tlb_invalidation_wait(gt, seqno); } else if (xe_device_uc_enabled(xe)) { xe_gt_WARN_ON(gt, xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)); if (xe->info.platform == XE_PVC || GRAPHICS_VER(xe) >= 20) { @@ -263,23 +263,23 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) } /** - * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA - * @gt: graphics tile + * xe_gt_guc_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA + * @gt: graphics technology * @fence: invalidation fence which will be signal on TLB invalidation * completion, can be NULL * @vma: VMA to invalidate * * Issue a range based TLB invalidation if supported, if not fallback to a full * TLB invalidation. Completion of TLB is asynchronous and caller can either use - * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for + * the invalidation fence or seqno + xe_gt_guc_tlb_invalidation_wait to wait for * completion. * - * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, + * Return: Seqno which can be passed to xe_gt_guc_tlb_invalidation_wait on success, * negative error code on error. */ -int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, - struct xe_gt_tlb_invalidation_fence *fence, - struct xe_vma *vma) +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, + struct xe_gt_guc_tlb_invalidation_fence *fence, + struct xe_vma *vma) { struct xe_device *xe = gt_to_xe(gt); #define MAX_TLB_INVALIDATION_LEN 7 @@ -350,16 +350,16 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, } /** - * xe_gt_tlb_invalidation_wait - Wait for TLB to complete + * xe_gt_guc_tlb_invalidation_wait - Wait for the gt guc TLB invalidation to complete * @gt: graphics tile - * @seqno: seqno to wait which was returned from xe_gt_tlb_invalidation + * @seqno: seqno to wait which was returned from xe_gt_guc_tlb_invalidation * * Wait for 200ms for a TLB invalidation to complete, in practice we always * should receive the TLB invalidation within 200ms. * * Return: 0 on success, -ETIME on TLB invalidation timeout */ -int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno) { struct xe_guc *guc = >->uc.guc; int ret; @@ -388,7 +388,7 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) } /** - * xe_guc_tlb_invalidation_done_handler - TLB invalidation done handler + * xe_gt_guc_tlb_invalidation_done_handler - Guc TLB invalidation done handler * @guc: guc * @msg: message indicating TLB invalidation done * @len: length of message @@ -399,10 +399,10 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) * * Return: 0 on success, -EPROTO for malformed messages. */ -int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) { struct xe_gt *gt = guc_to_gt(guc); - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; unsigned long flags; if (unlikely(len != 1)) @@ -438,7 +438,7 @@ int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) list_for_each_entry_safe(fence, next, >->tlb_invalidation.pending_fences, link) { - trace_xe_gt_tlb_invalidation_fence_recv(fence); + trace_xe_gt_guc_tlb_invalidation_fence_recv(fence); if (!tlb_invalidation_seqno_past(gt, fence->seqno)) break; diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h new file mode 100644 index 000000000000..f7402dd0bec3 --- /dev/null +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2023 Intel Corporation + */ + +#ifndef _XE_GT_GUC_TLB_INVALIDATION_H_ +#define _XE_GT_GUC_TLB_INVALIDATION_H_ + +#include <linux/types.h> + +#include "xe_gt_guc_tlb_invalidation_types.h" + +struct xe_gt; +struct xe_guc; +struct xe_vma; + +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt); +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt); +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt); +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, + struct xe_gt_guc_tlb_invalidation_fence *fence, + struct xe_vma *vma); +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno); +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); + +#endif /* _XE_GT_GUC_TLB_INVALIDATION_ */ diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h similarity index 54% rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h index 934c828efe31..9e3d1daa51ba 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h @@ -3,18 +3,21 @@ * Copyright © 2023 Intel Corporation */ -#ifndef _XE_GT_TLB_INVALIDATION_TYPES_H_ -#define _XE_GT_TLB_INVALIDATION_TYPES_H_ +#ifndef _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ +#define _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ #include <linux/dma-fence.h> +struct xe_vma; +struct xe_gt; + /** - * struct xe_gt_tlb_invalidation_fence - XE GT TLB invalidation fence + * struct xe_gt_guc_tlb_invalidation_fence - XE GUC TLB invalidation fence * - * Optionally passed to xe_gt_tlb_invalidation and will be signaled upon TLB + * Optionally passed to xe_gt_guc_tlb_invalidation and will be signaled upon TLB * invalidation completion. */ -struct xe_gt_tlb_invalidation_fence { +struct xe_gt_guc_tlb_invalidation_fence { /** @base: dma fence base */ struct dma_fence base; /** @link: link into list of pending tlb fences */ @@ -25,4 +28,4 @@ struct xe_gt_tlb_invalidation_fence { ktime_t invalidation_time; }; -#endif +#endif /* _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ */ diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c index fa9e9853c53b..8c47c60f3cd3 100644 --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c @@ -15,7 +15,7 @@ #include "abi/guc_actions_abi.h" #include "xe_bo.h" #include "xe_gt.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_guc.h" #include "xe_guc_ct.h" #include "xe_migrate.h" @@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) goto retry_userptr; if (!ret) { - ret = xe_gt_tlb_invalidation_vma(gt, NULL, vma); + ret = xe_gt_guc_tlb_invalidation_vma(gt, NULL, vma); if (ret >= 0) ret = 0; } diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h index fbb743d80d2c..fab65b53cb24 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h @@ -8,7 +8,7 @@ #include <linux/types.h> -#include "xe_gt_tlb_invalidation_types.h" +#include "xe_gt_guc_tlb_invalidation_types.h" struct xe_gt; struct xe_guc; @@ -18,7 +18,7 @@ int xe_gt_tlb_invalidation_init(struct xe_gt *gt); void xe_gt_tlb_invalidation_reset(struct xe_gt *gt); int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, - struct xe_gt_tlb_invalidation_fence *fence, + struct xe_gt_guc_tlb_invalidation_fence *fence, struct xe_vma *vma); int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h index cfdc761ff7f4..c38e48c039e6 100644 --- a/drivers/gpu/drm/xe/xe_gt_types.h +++ b/drivers/gpu/drm/xe/xe_gt_types.h @@ -185,7 +185,7 @@ struct xe_gt { spinlock_t pending_lock; /** * @tlb_invalidation.fence_tdr: schedules a delayed call to - * xe_gt_tlb_fence_timeout after the timeut interval is over. + * xe_gt_guc_tlb_fence_timeout after the timeut interval is over. */ struct delayed_work fence_tdr; /** @tlb_invalidation.lock: protects TLB invalidation fences */ diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c index 8ac819a7061e..73cbb7a8d12f 100644 --- a/drivers/gpu/drm/xe/xe_guc_ct.c +++ b/drivers/gpu/drm/xe/xe_guc_ct.c @@ -22,7 +22,7 @@ #include "xe_gt_pagefault.h" #include "xe_gt_printk.h" #include "xe_gt_sriov_pf_control.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_guc.h" #include "xe_guc_relay.h" #include "xe_guc_submit.h" @@ -1050,7 +1050,7 @@ static int process_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len) ret = xe_guc_pagefault_handler(guc, payload, adj_len); break; case XE_GUC_ACTION_TLB_INVALIDATION_DONE: - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, adj_len); break; case XE_GUC_ACTION_ACCESS_COUNTER_NOTIFY: @@ -1184,7 +1184,7 @@ static void g2h_fast_path(struct xe_guc_ct *ct, u32 *msg, u32 len) break; case XE_GUC_ACTION_TLB_INVALIDATION_DONE: __g2h_release_space(ct, len); - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, adj_len); break; default: diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c index 5b7930f46cf3..a0f63f457342 100644 --- a/drivers/gpu/drm/xe/xe_pt.c +++ b/drivers/gpu/drm/xe/xe_pt.c @@ -10,7 +10,7 @@ #include "xe_device.h" #include "xe_drm_client.h" #include "xe_gt.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_migrate.h" #include "xe_pt_types.h" #include "xe_pt_walk.h" @@ -1073,7 +1073,7 @@ static const struct xe_migrate_pt_update_ops userptr_bind_ops = { }; struct invalidation_fence { - struct xe_gt_tlb_invalidation_fence base; + struct xe_gt_guc_tlb_invalidation_fence base; struct xe_gt *gt; struct xe_vma *vma; struct dma_fence *fence; @@ -1104,7 +1104,7 @@ static void invalidation_fence_cb(struct dma_fence *fence, struct invalidation_fence *ifence = container_of(cb, struct invalidation_fence, cb); - trace_xe_gt_tlb_invalidation_fence_cb(&ifence->base); + trace_xe_gt_guc_tlb_invalidation_fence_cb(&ifence->base); if (!ifence->fence->error) { queue_work(system_wq, &ifence->work); } else { @@ -1120,8 +1120,8 @@ static void invalidation_fence_work_func(struct work_struct *w) struct invalidation_fence *ifence = container_of(w, struct invalidation_fence, work); - trace_xe_gt_tlb_invalidation_fence_work_func(&ifence->base); - xe_gt_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); + trace_xe_gt_guc_tlb_invalidation_fence_work_func(&ifence->base); + xe_gt_guc_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); } static int invalidation_fence_init(struct xe_gt *gt, @@ -1131,7 +1131,7 @@ static int invalidation_fence_init(struct xe_gt *gt, { int ret; - trace_xe_gt_tlb_invalidation_fence_create(&ifence->base); + trace_xe_gt_guc_tlb_invalidation_fence_create(&ifence->base); spin_lock_irq(>->tlb_invalidation.lock); dma_fence_init(&ifence->base.base, &invalidation_fence_ops, diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h index 2d56cfc09e42..f3adde117811 100644 --- a/drivers/gpu/drm/xe/xe_trace.h +++ b/drivers/gpu/drm/xe/xe_trace.h @@ -16,18 +16,18 @@ #include "xe_bo_types.h" #include "xe_exec_queue_types.h" #include "xe_gpu_scheduler_types.h" -#include "xe_gt_tlb_invalidation_types.h" +#include "xe_gt_guc_tlb_invalidation_types.h" #include "xe_gt_types.h" #include "xe_guc_exec_queue_types.h" #include "xe_sched_job.h" #include "xe_vm.h" -DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DECLARE_EVENT_CLASS(xe_gt_guc_tlb_invalidation_fence, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence), TP_STRUCT__entry( - __field(struct xe_gt_tlb_invalidation_fence *, fence) + __field(struct xe_gt_guc_tlb_invalidation_fence *, fence) __field(int, seqno) ), @@ -40,39 +40,39 @@ DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, __entry->fence, __entry->seqno) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_create, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_create, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, - xe_gt_tlb_invalidation_fence_work_func, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, + xe_gt_guc_tlb_invalidation_fence_work_func, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_cb, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_cb, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_send, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_send, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_recv, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_recv, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_signal, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_signal, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_timeout, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_timeout, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 7ae2b0300db6..965e17c640c9 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -28,7 +28,7 @@ #include "xe_drm_client.h" #include "xe_exec_queue.h" #include "xe_gt_pagefault.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_migrate.h" #include "xe_pat.h" #include "xe_pm.h" @@ -3179,7 +3179,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) * FIXME: We potentially need to invalidate multiple * GTs within the tile */ - seqno[id] = xe_gt_tlb_invalidation_vma(tile->primary_gt, NULL, vma); + seqno[id] = xe_gt_guc_tlb_invalidation_vma(tile->primary_gt, NULL, vma); if (seqno[id] < 0) return seqno[id]; } @@ -3187,7 +3187,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) for_each_tile(tile, xe, id) { if (tile_needs_invalidate & BIT(id)) { - ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); + ret = xe_gt_guc_tlb_invalidation_wait(tile->primary_gt, seqno[id]); if (ret < 0) return ret; } -- 2.34.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] drm/xe: rename gt invalidation functions 2024-04-24 9:42 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri @ 2024-04-29 1:29 ` Matthew Brost 0 siblings, 0 replies; 7+ messages in thread From: Matthew Brost @ 2024-04-29 1:29 UTC (permalink / raw) To: Farah Kassabri; +Cc: intel-xe, ofir1.bitton On Wed, Apr 24, 2024 at 12:42:43PM +0300, Farah Kassabri wrote: > Future products will have multi instance TLBs, > Current Xe driver handles only the compute TLBs invalidaiton > through the Guc. > In preparation for Multi instance TLB support, > we need to rename current invalidation as Guc specific > invalidation functions, and not general gt functions. > This patch just renames the current functions, files and variables. > follow up patches will start adding the multi instance TLB > support for future XE products, where each will have > its own invalidation flow. > I've been meaning to fix this layering violation for a while (GT file directly calling into GuC). Thanks for doing this. Comments below. BTW - I think you will have to rebase this series I merged a change to this code late last week [1]. [1] https://patchwork.freedesktop.org/patch/591281/?series=132246&rev=3 > Signed-off-by: Farah Kassabri <fkassabri@habana.ai> > --- > drivers/gpu/drm/xe/Makefile | 2 +- > drivers/gpu/drm/xe/abi/guc_actions_abi.h | 2 +- > drivers/gpu/drm/xe/xe_ggtt.c | 6 +- > drivers/gpu/drm/xe/xe_gt.c | 6 +- > ...idation.c => xe_gt_guc_tlb_invalidation.c} | 72 +++++++++---------- > .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 26 +++++++ s/xe_gt_guc_tlb_invalidation/xe_guc_tlb_invalidation > ...s.h => xe_gt_guc_tlb_invalidation_types.h} | 15 ++-- > drivers/gpu/drm/xe/xe_gt_pagefault.c | 4 +- > drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 4 +- > drivers/gpu/drm/xe/xe_gt_types.h | 2 +- > drivers/gpu/drm/xe/xe_guc_ct.c | 6 +- > drivers/gpu/drm/xe/xe_pt.c | 12 ++-- > drivers/gpu/drm/xe/xe_trace.h | 38 +++++----- > drivers/gpu/drm/xe/xe_vm.c | 6 +- > 14 files changed, 115 insertions(+), 86 deletions(-) > rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation.c => xe_gt_guc_tlb_invalidation.c} (83%) > create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation_types.h => xe_gt_guc_tlb_invalidation_types.h} (54%) > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile > index 8321ec4f9b46..63e757406595 100644 > --- a/drivers/gpu/drm/xe/Makefile > +++ b/drivers/gpu/drm/xe/Makefile > @@ -90,7 +90,7 @@ xe-y += xe_bb.o \ > xe_gt_pagefault.o \ > xe_gt_sysfs.o \ > xe_gt_throttle_sysfs.o \ > - xe_gt_tlb_invalidation.o \ > + xe_gt_guc_tlb_invalidation.o \ > xe_gt_topology.o \ > xe_guc.o \ > xe_guc_ads.o \ > diff --git a/drivers/gpu/drm/xe/abi/guc_actions_abi.h b/drivers/gpu/drm/xe/abi/guc_actions_abi.h > index 79ba98a169f9..ad4cdd5af80e 100644 > --- a/drivers/gpu/drm/xe/abi/guc_actions_abi.h > +++ b/drivers/gpu/drm/xe/abi/guc_actions_abi.h > @@ -187,7 +187,7 @@ enum xe_guc_sleep_state_status { > /* Flush PPC or SMRO caches along with TLB invalidation request */ > #define XE_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) > > -enum xe_guc_tlb_invalidation_type { > +enum xe_gt_guc_tlb_invalidation_type { > XE_GUC_TLB_INVAL_FULL = 0x0, > XE_GUC_TLB_INVAL_PAGE_SELECTIVE = 0x1, > XE_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX = 0x2, > diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c > index 0d541f55b4fc..e0c1bfa92730 100644 > --- a/drivers/gpu/drm/xe/xe_ggtt.c > +++ b/drivers/gpu/drm/xe/xe_ggtt.c > @@ -19,7 +19,7 @@ > #include "xe_device.h" > #include "xe_gt.h" > #include "xe_gt_printk.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_map.h" > #include "xe_pm.h" > #include "xe_sriov.h" > @@ -252,9 +252,9 @@ static void ggtt_invalidate_gt_tlb(struct xe_gt *gt) > if (!gt) > return; > > - err = xe_gt_tlb_invalidation_ggtt(gt); > + err = xe_gt_guc_tlb_invalidation_ggtt(gt); > if (err) > - drm_warn(>_to_xe(gt)->drm, "xe_gt_tlb_invalidation_ggtt error=%d", err); > + drm_warn(>_to_xe(gt)->drm, "xe_gt_guc_tlb_invalidation_ggtt error=%d", err); > } > > static void xe_ggtt_invalidate(struct xe_ggtt *ggtt) > diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c > index 491d0413de15..b162f10902ea 100644 > --- a/drivers/gpu/drm/xe/xe_gt.c > +++ b/drivers/gpu/drm/xe/xe_gt.c > @@ -31,7 +31,7 @@ > #include "xe_gt_printk.h" > #include "xe_gt_sriov_pf.h" > #include "xe_gt_sysfs.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_gt_topology.h" > #include "xe_guc_exec_queue_types.h" > #include "xe_guc_pc.h" > @@ -536,7 +536,7 @@ int xe_gt_init(struct xe_gt *gt) > xe_hw_fence_irq_init(>->fence_irq[i]); > } > > - err = xe_gt_tlb_invalidation_init(gt); > + err = xe_gt_guc_tlb_invalidation_init(gt); > if (err) > return err; > > @@ -659,7 +659,7 @@ static int gt_reset(struct xe_gt *gt) > if (err) > goto err_out; > > - xe_gt_tlb_invalidation_reset(gt); > + xe_gt_guc_tlb_invalidation_reset(gt); > > err = do_gt_reset(gt); > if (err) > diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > similarity index 83% > rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c > rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > index 93df2d7969b3..f994c64c95f4 100644 > --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c > +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > @@ -3,7 +3,7 @@ > * Copyright © 2023 Intel Corporation > */ > > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > > #include "abi/guc_actions_abi.h" > #include "xe_device.h" > @@ -17,11 +17,11 @@ > > #define TLB_TIMEOUT (HZ / 4) > > -static void xe_gt_tlb_fence_timeout(struct work_struct *work) > +static void xe_gt_guc_tlb_fence_timeout(struct work_struct *work) > { > struct xe_gt *gt = container_of(work, struct xe_gt, > tlb_invalidation.fence_tdr.work); > - struct xe_gt_tlb_invalidation_fence *fence, *next; > + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; > > spin_lock_irq(>->tlb_invalidation.pending_lock); > list_for_each_entry_safe(fence, next, > @@ -32,7 +32,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) > if (msecs_to_jiffies(since_inval_ms) < TLB_TIMEOUT) > break; > > - trace_xe_gt_tlb_invalidation_fence_timeout(fence); > + trace_xe_gt_guc_tlb_invalidation_fence_timeout(fence); > xe_gt_err(gt, "TLB invalidation fence timeout, seqno=%d recv=%d", > fence->seqno, gt->tlb_invalidation.seqno_recv); > > @@ -49,7 +49,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) > } > > /** > - * xe_gt_tlb_invalidation_init - Initialize GT TLB invalidation state > + * xe_gt_guc_tlb_invalidation_init - Initialize GT TLB invalidation state > * @gt: graphics tile > * > * Initialize GT TLB invalidation state, purely software initialization, should > @@ -57,42 +57,42 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) > * > * Return: 0 on success, negative error code on error. > */ > -int xe_gt_tlb_invalidation_init(struct xe_gt *gt) > +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt) > { > gt->tlb_invalidation.seqno = 1; > INIT_LIST_HEAD(>->tlb_invalidation.pending_fences); > spin_lock_init(>->tlb_invalidation.pending_lock); > spin_lock_init(>->tlb_invalidation.lock); > INIT_DELAYED_WORK(>->tlb_invalidation.fence_tdr, > - xe_gt_tlb_fence_timeout); > + xe_gt_guc_tlb_fence_timeout); > > return 0; > } > > static void > -__invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) > +__invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) > { > - trace_xe_gt_tlb_invalidation_fence_signal(fence); > + trace_xe_gt_guc_tlb_invalidation_fence_signal(fence); > dma_fence_signal(&fence->base); > dma_fence_put(&fence->base); > } > > static void > -invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) > +invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) > { > list_del(&fence->link); > __invalidation_fence_signal(fence); > } > > /** > - * xe_gt_tlb_invalidation_reset - Initialize GT TLB invalidation reset > + * xe_gt_guc_tlb_invalidation_reset - Initialize GT TLB invalidation reset > * @gt: graphics tile > * > * Signal any pending invalidation fences, should be called during a GT reset > */ > -void xe_gt_tlb_invalidation_reset(struct xe_gt *gt) > +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt) > { > - struct xe_gt_tlb_invalidation_fence *fence, *next; > + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; > struct xe_guc *guc = >->uc.guc; > int pending_seqno; > > @@ -139,7 +139,7 @@ static bool tlb_invalidation_seqno_past(struct xe_gt *gt, int seqno) > } > > static int send_tlb_invalidation(struct xe_guc *guc, > - struct xe_gt_tlb_invalidation_fence *fence, > + struct xe_gt_guc_tlb_invalidation_fence *fence, > u32 *action, int len) > { > struct xe_gt *gt = guc_to_gt(guc); > @@ -156,7 +156,7 @@ static int send_tlb_invalidation(struct xe_guc *guc, > seqno = gt->tlb_invalidation.seqno; > if (fence) { > fence->seqno = seqno; > - trace_xe_gt_tlb_invalidation_fence_send(fence); > + trace_xe_gt_guc_tlb_invalidation_fence_send(fence); > } > action[1] = seqno; > ret = xe_guc_ct_send_locked(&guc->ct, action, len, > @@ -202,16 +202,16 @@ static int send_tlb_invalidation(struct xe_guc *guc, > XE_GUC_TLB_INVAL_FLUSH_CACHE) > > /** > - * xe_gt_tlb_invalidation_guc - Issue a TLB invalidation on this GT for the GuC > + * xe_gt_guc_tlb_invalidation - Issue a TLB invalidation on this GT for the GuC > * @gt: graphics tile > * > * Issue a TLB invalidation for the GuC. Completion of TLB is asynchronous and > - * caller can use seqno + xe_gt_tlb_invalidation_wait to wait for completion. > + * caller can use seqno + xe_gt_guc_tlb_invalidation_wait to wait for completion. > * > * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, > * negative error code on error. > */ > -static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) > +static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) > { > u32 action[] = { > XE_GUC_ACTION_TLB_INVALIDATION, > @@ -224,7 +224,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) > } > > /** > - * xe_gt_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT > + * xe_gt_guc_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT > * @gt: graphics tile > * > * Issue a TLB invalidation for the GGTT. Completion of TLB invalidation is > @@ -232,7 +232,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) > * > * Return: 0 on success, negative error code on error > */ > -int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) > +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt) > { > struct xe_device *xe = gt_to_xe(gt); > > @@ -240,11 +240,11 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) > gt->uc.guc.submission_state.enabled) { > int seqno; > > - seqno = xe_gt_tlb_invalidation_guc(gt); > + seqno = xe_gt_guc_tlb_invalidation(gt); > if (seqno <= 0) > return seqno; > > - xe_gt_tlb_invalidation_wait(gt, seqno); > + xe_gt_guc_tlb_invalidation_wait(gt, seqno); > } else if (xe_device_uc_enabled(xe)) { > xe_gt_WARN_ON(gt, xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)); > if (xe->info.platform == XE_PVC || GRAPHICS_VER(xe) >= 20) { > @@ -263,23 +263,23 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) > } > > /** > - * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA > - * @gt: graphics tile > + * xe_gt_guc_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA > + * @gt: graphics technology > * @fence: invalidation fence which will be signal on TLB invalidation > * completion, can be NULL > * @vma: VMA to invalidate > * > * Issue a range based TLB invalidation if supported, if not fallback to a full > * TLB invalidation. Completion of TLB is asynchronous and caller can either use > - * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for > + * the invalidation fence or seqno + xe_gt_guc_tlb_invalidation_wait to wait for > * completion. > * > - * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, > + * Return: Seqno which can be passed to xe_gt_guc_tlb_invalidation_wait on success, > * negative error code on error. > */ > -int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, > - struct xe_gt_tlb_invalidation_fence *fence, > - struct xe_vma *vma) > +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, > + struct xe_gt_guc_tlb_invalidation_fence *fence, > + struct xe_vma *vma) > { > struct xe_device *xe = gt_to_xe(gt); > #define MAX_TLB_INVALIDATION_LEN 7 > @@ -350,16 +350,16 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, > } > > /** > - * xe_gt_tlb_invalidation_wait - Wait for TLB to complete > + * xe_gt_guc_tlb_invalidation_wait - Wait for the gt guc TLB invalidation to complete > * @gt: graphics tile > - * @seqno: seqno to wait which was returned from xe_gt_tlb_invalidation > + * @seqno: seqno to wait which was returned from xe_gt_guc_tlb_invalidation > * > * Wait for 200ms for a TLB invalidation to complete, in practice we always > * should receive the TLB invalidation within 200ms. > * > * Return: 0 on success, -ETIME on TLB invalidation timeout > */ > -int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) > +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno) > { > struct xe_guc *guc = >->uc.guc; > int ret; > @@ -388,7 +388,7 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) > } > > /** > - * xe_guc_tlb_invalidation_done_handler - TLB invalidation done handler > + * xe_gt_guc_tlb_invalidation_done_handler - Guc TLB invalidation done handler > * @guc: guc > * @msg: message indicating TLB invalidation done > * @len: length of message > @@ -399,10 +399,10 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) > * > * Return: 0 on success, -EPROTO for malformed messages. > */ > -int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) > +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) > { > struct xe_gt *gt = guc_to_gt(guc); > - struct xe_gt_tlb_invalidation_fence *fence, *next; > + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; > unsigned long flags; > > if (unlikely(len != 1)) > @@ -438,7 +438,7 @@ int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) > > list_for_each_entry_safe(fence, next, > >->tlb_invalidation.pending_fences, link) { > - trace_xe_gt_tlb_invalidation_fence_recv(fence); > + trace_xe_gt_guc_tlb_invalidation_fence_recv(fence); > > if (!tlb_invalidation_seqno_past(gt, fence->seqno)) > break; > diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > new file mode 100644 > index 000000000000..f7402dd0bec3 > --- /dev/null > +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > @@ -0,0 +1,26 @@ > +/* SPDX-License-Identifier: MIT */ > +/* > + * Copyright © 2023 Intel Corporation > + */ > + > +#ifndef _XE_GT_GUC_TLB_INVALIDATION_H_ > +#define _XE_GT_GUC_TLB_INVALIDATION_H_ > + > +#include <linux/types.h> > + > +#include "xe_gt_guc_tlb_invalidation_types.h" > + > +struct xe_gt; > +struct xe_guc; > +struct xe_vma; > + > +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt); > +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt); > +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt); > +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, > + struct xe_gt_guc_tlb_invalidation_fence *fence, > + struct xe_vma *vma); > +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno); > +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); > + > +#endif /* _XE_GT_GUC_TLB_INVALIDATION_ */ > diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h > similarity index 54% > rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h > rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h > index 934c828efe31..9e3d1daa51ba 100644 > --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h > +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h > @@ -3,18 +3,21 @@ > * Copyright © 2023 Intel Corporation > */ > > -#ifndef _XE_GT_TLB_INVALIDATION_TYPES_H_ > -#define _XE_GT_TLB_INVALIDATION_TYPES_H_ > +#ifndef _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ > +#define _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ > > #include <linux/dma-fence.h> > > +struct xe_vma; > +struct xe_gt; > + > /** > - * struct xe_gt_tlb_invalidation_fence - XE GT TLB invalidation fence > + * struct xe_gt_guc_tlb_invalidation_fence - XE GUC TLB invalidation fence > * > - * Optionally passed to xe_gt_tlb_invalidation and will be signaled upon TLB > + * Optionally passed to xe_gt_guc_tlb_invalidation and will be signaled upon TLB > * invalidation completion. > */ > -struct xe_gt_tlb_invalidation_fence { > +struct xe_gt_guc_tlb_invalidation_fence { This is still probably a GT layer concept. e.g. We invalidate all invalidation backends with all global seqno, when all backends complete, we signal this fence. With that, this patch by itself doesn't really work. I think you need to combine this patch the following patch in the series. A few more tips to get the layering right. - All of the code external to this file shouldn't change (e.g. public functions rename the same) - xe_gt_tlb_invalidation calls into vfuncs to issue invalidations - xe_gt_tlb_invalidation layer managers seqno, fences, and resets - Invalidation backends export 1 function which returns the vops for the backend (something kinda like we do in xe_ring_ops.h) Make sense? Matt > /** @base: dma fence base */ > struct dma_fence base; > /** @link: link into list of pending tlb fences */ > @@ -25,4 +28,4 @@ struct xe_gt_tlb_invalidation_fence { > ktime_t invalidation_time; > }; > > -#endif > +#endif /* _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ */ > diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c > index fa9e9853c53b..8c47c60f3cd3 100644 > --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c > +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c > @@ -15,7 +15,7 @@ > #include "abi/guc_actions_abi.h" > #include "xe_bo.h" > #include "xe_gt.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_guc.h" > #include "xe_guc_ct.h" > #include "xe_migrate.h" > @@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) > goto retry_userptr; > > if (!ret) { > - ret = xe_gt_tlb_invalidation_vma(gt, NULL, vma); > + ret = xe_gt_guc_tlb_invalidation_vma(gt, NULL, vma); > if (ret >= 0) > ret = 0; > } > diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > index fbb743d80d2c..fab65b53cb24 100644 > --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > @@ -8,7 +8,7 @@ > > #include <linux/types.h> > > -#include "xe_gt_tlb_invalidation_types.h" > +#include "xe_gt_guc_tlb_invalidation_types.h" > > struct xe_gt; > struct xe_guc; > @@ -18,7 +18,7 @@ int xe_gt_tlb_invalidation_init(struct xe_gt *gt); > void xe_gt_tlb_invalidation_reset(struct xe_gt *gt); > int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); > int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, > - struct xe_gt_tlb_invalidation_fence *fence, > + struct xe_gt_guc_tlb_invalidation_fence *fence, > struct xe_vma *vma); > int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); > int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); > diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h > index cfdc761ff7f4..c38e48c039e6 100644 > --- a/drivers/gpu/drm/xe/xe_gt_types.h > +++ b/drivers/gpu/drm/xe/xe_gt_types.h > @@ -185,7 +185,7 @@ struct xe_gt { > spinlock_t pending_lock; > /** > * @tlb_invalidation.fence_tdr: schedules a delayed call to > - * xe_gt_tlb_fence_timeout after the timeut interval is over. > + * xe_gt_guc_tlb_fence_timeout after the timeut interval is over. > */ > struct delayed_work fence_tdr; > /** @tlb_invalidation.lock: protects TLB invalidation fences */ > diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c > index 8ac819a7061e..73cbb7a8d12f 100644 > --- a/drivers/gpu/drm/xe/xe_guc_ct.c > +++ b/drivers/gpu/drm/xe/xe_guc_ct.c > @@ -22,7 +22,7 @@ > #include "xe_gt_pagefault.h" > #include "xe_gt_printk.h" > #include "xe_gt_sriov_pf_control.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_guc.h" > #include "xe_guc_relay.h" > #include "xe_guc_submit.h" > @@ -1050,7 +1050,7 @@ static int process_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len) > ret = xe_guc_pagefault_handler(guc, payload, adj_len); > break; > case XE_GUC_ACTION_TLB_INVALIDATION_DONE: > - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, > + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, > adj_len); > break; > case XE_GUC_ACTION_ACCESS_COUNTER_NOTIFY: > @@ -1184,7 +1184,7 @@ static void g2h_fast_path(struct xe_guc_ct *ct, u32 *msg, u32 len) > break; > case XE_GUC_ACTION_TLB_INVALIDATION_DONE: > __g2h_release_space(ct, len); > - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, > + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, > adj_len); > break; > default: > diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c > index 5b7930f46cf3..a0f63f457342 100644 > --- a/drivers/gpu/drm/xe/xe_pt.c > +++ b/drivers/gpu/drm/xe/xe_pt.c > @@ -10,7 +10,7 @@ > #include "xe_device.h" > #include "xe_drm_client.h" > #include "xe_gt.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_migrate.h" > #include "xe_pt_types.h" > #include "xe_pt_walk.h" > @@ -1073,7 +1073,7 @@ static const struct xe_migrate_pt_update_ops userptr_bind_ops = { > }; > > struct invalidation_fence { > - struct xe_gt_tlb_invalidation_fence base; > + struct xe_gt_guc_tlb_invalidation_fence base; > struct xe_gt *gt; > struct xe_vma *vma; > struct dma_fence *fence; > @@ -1104,7 +1104,7 @@ static void invalidation_fence_cb(struct dma_fence *fence, > struct invalidation_fence *ifence = > container_of(cb, struct invalidation_fence, cb); > > - trace_xe_gt_tlb_invalidation_fence_cb(&ifence->base); > + trace_xe_gt_guc_tlb_invalidation_fence_cb(&ifence->base); > if (!ifence->fence->error) { > queue_work(system_wq, &ifence->work); > } else { > @@ -1120,8 +1120,8 @@ static void invalidation_fence_work_func(struct work_struct *w) > struct invalidation_fence *ifence = > container_of(w, struct invalidation_fence, work); > > - trace_xe_gt_tlb_invalidation_fence_work_func(&ifence->base); > - xe_gt_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); > + trace_xe_gt_guc_tlb_invalidation_fence_work_func(&ifence->base); > + xe_gt_guc_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); > } > > static int invalidation_fence_init(struct xe_gt *gt, > @@ -1131,7 +1131,7 @@ static int invalidation_fence_init(struct xe_gt *gt, > { > int ret; > > - trace_xe_gt_tlb_invalidation_fence_create(&ifence->base); > + trace_xe_gt_guc_tlb_invalidation_fence_create(&ifence->base); > > spin_lock_irq(>->tlb_invalidation.lock); > dma_fence_init(&ifence->base.base, &invalidation_fence_ops, > diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h > index 2d56cfc09e42..f3adde117811 100644 > --- a/drivers/gpu/drm/xe/xe_trace.h > +++ b/drivers/gpu/drm/xe/xe_trace.h > @@ -16,18 +16,18 @@ > #include "xe_bo_types.h" > #include "xe_exec_queue_types.h" > #include "xe_gpu_scheduler_types.h" > -#include "xe_gt_tlb_invalidation_types.h" > +#include "xe_gt_guc_tlb_invalidation_types.h" > #include "xe_gt_types.h" > #include "xe_guc_exec_queue_types.h" > #include "xe_sched_job.h" > #include "xe_vm.h" > > -DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DECLARE_EVENT_CLASS(xe_gt_guc_tlb_invalidation_fence, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence), > > TP_STRUCT__entry( > - __field(struct xe_gt_tlb_invalidation_fence *, fence) > + __field(struct xe_gt_guc_tlb_invalidation_fence *, fence) > __field(int, seqno) > ), > > @@ -40,39 +40,39 @@ DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, > __entry->fence, __entry->seqno) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_create, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_create, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, > - xe_gt_tlb_invalidation_fence_work_func, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, > + xe_gt_guc_tlb_invalidation_fence_work_func, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_cb, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_cb, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_send, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_send, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_recv, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_recv, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_signal, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_signal, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_timeout, > - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), > +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_timeout, > + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), > TP_ARGS(fence) > ); > > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c > index 7ae2b0300db6..965e17c640c9 100644 > --- a/drivers/gpu/drm/xe/xe_vm.c > +++ b/drivers/gpu/drm/xe/xe_vm.c > @@ -28,7 +28,7 @@ > #include "xe_drm_client.h" > #include "xe_exec_queue.h" > #include "xe_gt_pagefault.h" > -#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > #include "xe_migrate.h" > #include "xe_pat.h" > #include "xe_pm.h" > @@ -3179,7 +3179,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) > * FIXME: We potentially need to invalidate multiple > * GTs within the tile > */ > - seqno[id] = xe_gt_tlb_invalidation_vma(tile->primary_gt, NULL, vma); > + seqno[id] = xe_gt_guc_tlb_invalidation_vma(tile->primary_gt, NULL, vma); > if (seqno[id] < 0) > return seqno[id]; > } > @@ -3187,7 +3187,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) > > for_each_tile(tile, xe, id) { > if (tile_needs_invalidate & BIT(id)) { > - ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); > + ret = xe_gt_guc_tlb_invalidation_wait(tile->primary_gt, seqno[id]); > if (ret < 0) > return ret; > } > -- > 2.34.1 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations 2024-04-24 9:42 Farah Kassabri 2024-04-24 9:42 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri @ 2024-04-24 9:42 ` Farah Kassabri 2024-04-29 2:01 ` Matthew Brost 2024-04-28 22:02 ` ✗ CI.Patch_applied: failure for series starting with [1/2] drm/xe: rename gt invalidation functions Patchwork 2 siblings, 1 reply; 7+ messages in thread From: Farah Kassabri @ 2024-04-24 9:42 UTC (permalink / raw) To: intel-xe; +Cc: ofir1.bitton, Farah Kassabri This patch prepares the xe driver for supporting multi instance TLB invalidations. Currently xe driver support one invalidation flow for the compute engines which passes through the Guc submission flow. In the near future this driver should invalidate other TLB instances, each has its own flow for performing the invalidation. This patch will allow the engine to register its own functions for invalidation and wait, keeping the invalidation calls from the common code generic. Signed-off-by: Farah Kassabri <fkassabri@habana.ai> --- drivers/gpu/drm/xe/Makefile | 1 + drivers/gpu/drm/xe/xe_device.c | 7 + drivers/gpu/drm/xe/xe_ggtt.c | 8 +- drivers/gpu/drm/xe/xe_gt.c | 5 +- .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.c | 72 ++++---- .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 2 +- drivers/gpu/drm/xe/xe_gt_pagefault.c | 4 +- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 167 ++++++++++++++++++ drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 66 +++++-- drivers/gpu/drm/xe/xe_gt_types.h | 4 + drivers/gpu/drm/xe/xe_vm.c | 12 +- 11 files changed, 281 insertions(+), 67 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile index 63e757406595..d279cec7bb1d 100644 --- a/drivers/gpu/drm/xe/Makefile +++ b/drivers/gpu/drm/xe/Makefile @@ -91,6 +91,7 @@ xe-y += xe_bb.o \ xe_gt_sysfs.o \ xe_gt_throttle_sysfs.o \ xe_gt_guc_tlb_invalidation.o \ + xe_gt_tlb_invalidation.o \ xe_gt_topology.o \ xe_guc.o \ xe_guc_ads.o \ diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c index 55bbc8b8df15..5b438f0e8a7d 100644 --- a/drivers/gpu/drm/xe/xe_device.c +++ b/drivers/gpu/drm/xe/xe_device.c @@ -45,6 +45,7 @@ #include "xe_ttm_sys_mgr.h" #include "xe_vm.h" #include "xe_wait_user_fence.h" +#include "xe_gt_tlb_invalidation.h" static int xe_file_open(struct drm_device *dev, struct drm_file *file) { @@ -568,6 +569,12 @@ int xe_device_probe(struct xe_device *xe) } } + for_each_gt(gt, xe, id) { + err = xe_gt_tlb_invalidation_init(gt); + if (err) + return err; + } + for_each_gt(gt, xe, id) { err = xe_gt_init_hwconfig(gt); if (err) diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c index e0c1bfa92730..3031a45957e9 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.c +++ b/drivers/gpu/drm/xe/xe_ggtt.c @@ -19,7 +19,7 @@ #include "xe_device.h" #include "xe_gt.h" #include "xe_gt_printk.h" -#include "xe_gt_guc_tlb_invalidation.h" +#include "xe_gt_tlb_invalidation.h" #include "xe_map.h" #include "xe_pm.h" #include "xe_sriov.h" @@ -247,14 +247,14 @@ int xe_ggtt_init(struct xe_ggtt *ggtt) static void ggtt_invalidate_gt_tlb(struct xe_gt *gt) { - int err; + int seqno[TLB_INV_MAX_INS] = {0}, err; if (!gt) return; - err = xe_gt_guc_tlb_invalidation_ggtt(gt); + err = xe_gt_tlb_invalidation_all(gt, seqno); if (err) - drm_warn(>_to_xe(gt)->drm, "xe_gt_guc_tlb_invalidation_ggtt error=%d", err); + drm_warn(>_to_xe(gt)->drm, "xe_gt_tlb_invalidation_all error=%d", err); } static void xe_ggtt_invalidate(struct xe_ggtt *ggtt) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index b162f10902ea..7554da5c7e19 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -504,6 +504,7 @@ int xe_gt_init_hwconfig(struct xe_gt *gt) goto out; xe_gt_topology_init(gt); + xe_gt_mcr_init(gt); xe_pat_init(gt); @@ -536,10 +537,6 @@ int xe_gt_init(struct xe_gt *gt) xe_hw_fence_irq_init(>->fence_irq[i]); } - err = xe_gt_guc_tlb_invalidation_init(gt); - if (err) - return err; - err = xe_gt_pagefault_init(gt); if (err) return err; diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c index f994c64c95f4..469e57426a60 100644 --- a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c @@ -4,6 +4,7 @@ */ #include "xe_gt_guc_tlb_invalidation.h" +#include "xe_gt_tlb_invalidation.h" #include "abi/guc_actions_abi.h" #include "xe_device.h" @@ -17,6 +18,8 @@ #define TLB_TIMEOUT (HZ / 4) +static void xe_gt_guc_init_tlb_inv_funcs(struct xe_gt *gt); + static void xe_gt_guc_tlb_fence_timeout(struct work_struct *work) { struct xe_gt *gt = container_of(work, struct xe_gt, @@ -66,6 +69,8 @@ int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt) INIT_DELAYED_WORK(>->tlb_invalidation.fence_tdr, xe_gt_guc_tlb_fence_timeout); + xe_gt_guc_init_tlb_inv_funcs(gt); + return 0; } @@ -224,7 +229,7 @@ static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) } /** - * xe_gt_guc_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT + * xe_gt_guc_tlb_invalidation_all - Issue a TLB invalidation on this GT for the GGTT * @gt: graphics tile * * Issue a TLB invalidation for the GGTT. Completion of TLB invalidation is @@ -232,7 +237,7 @@ static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) * * Return: 0 on success, negative error code on error */ -int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt) +int xe_gt_guc_tlb_invalidation_all(struct xe_gt *gt) { struct xe_device *xe = gt_to_xe(gt); @@ -301,41 +306,9 @@ int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, if (!xe->info.has_range_tlb_invalidation) { action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_FULL); } else { - u64 start = xe_vma_start(vma); - u64 length = xe_vma_size(vma); - u64 align, end; + u64 start, length; - if (length < SZ_4K) - length = SZ_4K; - - /* - * We need to invalidate a higher granularity if start address - * is not aligned to length. When start is not aligned with - * length we need to find the length large enough to create an - * address mask covering the required range. - */ - align = roundup_pow_of_two(length); - start = ALIGN_DOWN(xe_vma_start(vma), align); - end = ALIGN(xe_vma_end(vma), align); - length = align; - while (start + length < end) { - length <<= 1; - start = ALIGN_DOWN(xe_vma_start(vma), length); - } - - /* - * Minimum invalidation size for a 2MB page that the hardware - * expects is 16MB - */ - if (length >= SZ_2M) { - length = max_t(u64, SZ_16M, length); - start = ALIGN_DOWN(xe_vma_start(vma), length); - } - - xe_gt_assert(gt, length >= SZ_4K); - xe_gt_assert(gt, is_power_of_2(length)); - xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1))); - xe_gt_assert(gt, IS_ALIGNED(start, length)); + xe_gt_tlb_invalidation_vma_alignments(gt, vma, &start, &length); action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_PAGE_SELECTIVE); action[len++] = xe_vma_vm(vma)->usm.asid; @@ -457,3 +430,30 @@ int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 le return 0; } + +/** + * xe_gt_guc_tlb_invalidation_vma_wrapper - wrapper for gt guc tlb range invalidation. + * @gt: graphics technology the invalidation was initiated on. + * @vma: VMA to invalidate + * @data: data used for the invalidation. + * + * Issue a range based TLB invalidation. + * + * Return: Seqno which can be passed to xe_gt_guc_tlb_invalidation_wait on success, + * negative error code on error. + */ +static int xe_gt_guc_tlb_invalidation_vma_wrapper(struct xe_gt *gt, struct xe_vma *vma, void *data) +{ + struct xe_gt_guc_tlb_invalidation_fence *fence = data; + + return xe_gt_guc_tlb_invalidation_vma(gt, fence, vma); +} + +static void xe_gt_guc_init_tlb_inv_funcs(struct xe_gt *gt) +{ + struct xe_gt_tlb_ins_invalidation_funcs *guc_ins = >->tlb_invalidation.inv_funcs[GUC_TLB]; + + guc_ins->inv_range = xe_gt_guc_tlb_invalidation_vma_wrapper; + guc_ins->inv_all = xe_gt_guc_tlb_invalidation_all; + guc_ins->inv_wait = xe_gt_guc_tlb_invalidation_wait; +} diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h index f7402dd0bec3..a02ee007c642 100644 --- a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h @@ -16,7 +16,7 @@ struct xe_vma; int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt); void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt); -int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt); +int xe_gt_guc_tlb_invalidation_all(struct xe_gt *gt); int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, struct xe_gt_guc_tlb_invalidation_fence *fence, struct xe_vma *vma); diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c index 8c47c60f3cd3..dba49f3ae933 100644 --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c @@ -15,7 +15,7 @@ #include "abi/guc_actions_abi.h" #include "xe_bo.h" #include "xe_gt.h" -#include "xe_gt_guc_tlb_invalidation.h" +#include "xe_gt_tlb_invalidation.h" #include "xe_guc.h" #include "xe_guc_ct.h" #include "xe_migrate.h" @@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) goto retry_userptr; if (!ret) { - ret = xe_gt_guc_tlb_invalidation_vma(gt, NULL, vma); + ret = xe_gt_tlb_invalidation_vma(gt, vma, NULL, NULL); if (ret >= 0) ret = 0; } diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c new file mode 100644 index 000000000000..84dec631aeb1 --- /dev/null +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2024 Intel Corporation + */ + +#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" +#include "xe_gt.h" +#include "xe_gt_printk.h" +#include "xe_vm.h" + +/** + * xe_gt_tlb_invalidation_init - TLB invalidation init generic function. + * @gt: graphics technology the invalidation was initiated on. + * + * Initialize all TLBs in the system, including setting the invalidation + * proprietary function pointer. + * + * Return: 0 on success, negative error code on error. + */ +int xe_gt_tlb_invalidation_init(struct xe_gt *gt) +{ + int ret; + + gt->tlb_invalidation.inv_funcs = kcalloc(TLB_INV_MAX_INS, + sizeof(struct xe_gt_tlb_ins_invalidation_funcs), + GFP_KERNEL); + if (!gt->tlb_invalidation.inv_funcs) + return -ENOMEM; + + /* Initialize gt Guc TLB */ + ret = xe_gt_guc_tlb_invalidation_init(gt); + if (ret) { + kfree(gt->tlb_invalidation.inv_funcs); + return ret; + } + + return 0; +} + +/** + * xe_gt_tlb_invalidation_all - TLB invalidation all generic function. + * @gt: graphics technology the invalidation was initiated on. + * @seq: sequence numbers of the TLB invalidations. + * This is the output of the invalidation functions, can be used when calling + * xe_gt_tlb_invalidation_wait. + * + * Issue a TLB invalidation when the GGTT is changed. + * This will make invalidation for all TLBs in the system, for mappings in GGTT and all + * PPGTT tables. + * + * Return: 0 on success, negative error code on error. + */ +int xe_gt_tlb_invalidation_all(struct xe_gt *gt, int *seq) +{ + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { + seq[ins] = gt->tlb_invalidation.inv_funcs[ins].inv_all(gt); + if (seq[ins] < 0) + return seq[ins]; + } + + return 0; +} + +/** + * xe_gt_tlb_invalidation_vma - TLB range invalidation generic function. + * @gt: graphics technology the invalidation was initiated on. + * @vma: VMA to invalidate + * @data: private data if needed for some TLB instances. + * @seq_out: sequence numbers of the TLB invalidations. + * This is the output of the invalidation functions, can be used when calling + * xe_gt_tlb_invalidation_wait, in case it's NULL then the caller doesn't need to wait for the + * invalidation. + * + * Issue a PPGTT related TLB invalidation, which will invalidate a range defined in the vma. + * + * Return: 0 on success, negative error code on error. + */ +int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, struct xe_vma *vma, void *data, int *seq_out) +{ + int seq[TLB_INV_MAX_INS] = {0}; + + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { + seq[ins] = gt->tlb_invalidation.inv_funcs[ins].inv_range(gt, vma, data); + if (seq[ins] < 0) + return seq[ins]; + } + + if (seq_out) + memcpy(seq_out, seq, sizeof(seq)); + + return 0; +} + +/** + * xe_gt_tlb_invalidation_wait - TLB invalidation wait generic function. + * @gt: graphics technology the invalidation was initiated on. + * @seq: sequence numbers which was returned from the TLB invalidation functions. + * + * Wait for a TLB invalidation to complete. + * + * Return: 0 on success, negative value on timeout/failure. + */ +int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int *seq) +{ + int ret; + + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { + ret = gt->tlb_invalidation.inv_funcs[ins].inv_wait(gt, seq[ins]); + if (ret < 0) + return ret; + } + + return 0; +} + +/** + * xe_gt_tlb_invalidation_vma_alignments - Make alignments on vma. + * @gt: graphics technology the invalidation was initiated on. + * @vma: VMA to invalidate + * @start_aligned: aligned start address (output). + * @length_aligned: aligned length (output). + * + * Return: 0 on success, negative value on timeout/failure. + */ +void xe_gt_tlb_invalidation_vma_alignments(struct xe_gt *gt, struct xe_vma *vma, + u64 *start_aligned, u64 *length_aligned) +{ + u64 start = xe_vma_start(vma); + u64 length = xe_vma_size(vma); + u64 align, end; + + if (length < SZ_4K) + length = SZ_4K; + + /* + * We need to invalidate a higher granularity if start address + * is not aligned to length. When start is not aligned with + * length we need to find the length large enough to create an + * address mask covering the required range. + */ + align = roundup_pow_of_two(length); + start = ALIGN_DOWN(xe_vma_start(vma), align); + end = ALIGN(xe_vma_end(vma), align); + length = align; + while (start + length < end) { + length <<= 1; + start = ALIGN_DOWN(xe_vma_start(vma), length); + } + + /* + * Minimum invalidation size for a 2MB page that the hardware + * expects is 16MB + */ + if (length >= SZ_2M) { + length = max_t(u64, SZ_16M, length); + start = ALIGN_DOWN(xe_vma_start(vma), length); + } + + xe_gt_assert(gt, length >= SZ_4K); + xe_gt_assert(gt, is_power_of_2(length)); + xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1))); + xe_gt_assert(gt, IS_ALIGNED(start, length)); + + *start_aligned = start; + *length_aligned = length; +} diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h index fab65b53cb24..406e2820e8c1 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: MIT */ /* - * Copyright © 2023 Intel Corporation + * Copyright © 2024 Intel Corporation */ #ifndef _XE_GT_TLB_INVALIDATION_H_ @@ -8,19 +8,57 @@ #include <linux/types.h> -#include "xe_gt_guc_tlb_invalidation_types.h" - -struct xe_gt; -struct xe_guc; struct xe_vma; +struct xe_device; +struct xe_gt; + +enum xe_gt_tlb_instances { + GUC_TLB = 0, + TLB_INV_MAX_INS +}; + +struct xe_gt_tlb_ins_invalidation_funcs { + /** + * struct xe_gt_tlb_invalidation_funcs member inv_range + * + * @gt: graphics technology the invalidation was initiated on. + * @vma: Pointer to vma data. + * @data: private data of the specific invalidation. + * + * This function should invalidate range of virtual addresses. + * The function return sequence number > 0 in case we need to wait for the invalidation, + * otherwise 0 which indicate success and no wait is needed, negative number upon failure. + */ + int (*inv_range)(struct xe_gt *gt, struct xe_vma *vma, void *data); + + /** + * struct xe_gt_tlb_invalidation_funcs member inv_all + * + * @gt: graphics technology the invalidation was initiated on. + * + * This function should invalidate the TLB instance when GGTT table is changed. + * When GGTT is changed driver must invalidate all mappings for both GGTT/PPGTT in the TLB + * instance. + * The function return sequence number > 0 in case we need to wait for the invalidation, + * otherwise 0 which indicate success and no wait is needed, negative number upon failure. + */ + int (*inv_all)(struct xe_gt *gt); + + /** + * struct xe_gt_tlb_invalidation_funcs member inv_wait + * + * @gt: graphics technology the invalidation was initiated on. + * @seq: the sequence number returned from the invalidation function. + * + * This function should wait till the invalidation is done. + */ + int (*inv_wait)(struct xe_gt *gt, int seq); +}; int xe_gt_tlb_invalidation_init(struct xe_gt *gt); -void xe_gt_tlb_invalidation_reset(struct xe_gt *gt); -int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); -int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, - struct xe_gt_guc_tlb_invalidation_fence *fence, - struct xe_vma *vma); -int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); -int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); - -#endif /* _XE_GT_TLB_INVALIDATION_ */ +int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, struct xe_vma *vma, void *data, int *seq_out); +int xe_gt_tlb_invalidation_all(struct xe_gt *gt, int *seq_out); +int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int *seq); +void xe_gt_tlb_invalidation_vma_alignments(struct xe_gt *gt, struct xe_vma *vma, + u64 *start_aligned, u64 *length_aligned); +#endif /* _XE_GT_TLB_INVALIDATION_H_ */ diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h index c38e48c039e6..49a88b0d0d25 100644 --- a/drivers/gpu/drm/xe/xe_gt_types.h +++ b/drivers/gpu/drm/xe/xe_gt_types.h @@ -190,6 +190,10 @@ struct xe_gt { struct delayed_work fence_tdr; /** @tlb_invalidation.lock: protects TLB invalidation fences */ spinlock_t lock; + /** + * @tlb_invalidation.inv_funcs: invalidation functions for TLB instances + */ + struct xe_gt_tlb_ins_invalidation_funcs *inv_funcs; } tlb_invalidation; /** diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 965e17c640c9..b2825ed0f1fa 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -28,7 +28,7 @@ #include "xe_drm_client.h" #include "xe_exec_queue.h" #include "xe_gt_pagefault.h" -#include "xe_gt_guc_tlb_invalidation.h" +#include "xe_gt_tlb_invalidation.h" #include "xe_migrate.h" #include "xe_pat.h" #include "xe_pm.h" @@ -3146,7 +3146,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) struct xe_device *xe = xe_vma_vm(vma)->xe; struct xe_tile *tile; u32 tile_needs_invalidate = 0; - int seqno[XE_MAX_TILES_PER_DEVICE]; + int seqno[XE_MAX_TILES_PER_DEVICE][TLB_INV_MAX_INS] = {{0}}; u8 id; int ret; @@ -3179,15 +3179,15 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) * FIXME: We potentially need to invalidate multiple * GTs within the tile */ - seqno[id] = xe_gt_guc_tlb_invalidation_vma(tile->primary_gt, NULL, vma); - if (seqno[id] < 0) - return seqno[id]; + ret = xe_gt_tlb_invalidation_vma(tile->primary_gt, vma, NULL, seqno[id]); + if (ret < 0) + return ret; } } for_each_tile(tile, xe, id) { if (tile_needs_invalidate & BIT(id)) { - ret = xe_gt_guc_tlb_invalidation_wait(tile->primary_gt, seqno[id]); + ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); if (ret < 0) return ret; } -- 2.34.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations 2024-04-24 9:42 ` [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations Farah Kassabri @ 2024-04-29 2:01 ` Matthew Brost 0 siblings, 0 replies; 7+ messages in thread From: Matthew Brost @ 2024-04-29 2:01 UTC (permalink / raw) To: Farah Kassabri; +Cc: intel-xe, ofir1.bitton On Wed, Apr 24, 2024 at 12:42:44PM +0300, Farah Kassabri wrote: > This patch prepares the xe driver for supporting multi instance TLB > invalidations. > Currently xe driver support one invalidation flow for the compute > engines which passes through the Guc submission flow. > In the near future this driver should invalidate other TLB instances, > each has its own flow for performing the invalidation. > This patch will allow the engine to register its own functions > for invalidation and wait, keeping the invalidation calls from the > common code generic. > Same comments in the previous patch here apply wrt to layering, won't repeat. > Signed-off-by: Farah Kassabri <fkassabri@habana.ai> > --- > drivers/gpu/drm/xe/Makefile | 1 + > drivers/gpu/drm/xe/xe_device.c | 7 + > drivers/gpu/drm/xe/xe_ggtt.c | 8 +- > drivers/gpu/drm/xe/xe_gt.c | 5 +- > .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.c | 72 ++++---- > .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 2 +- > drivers/gpu/drm/xe/xe_gt_pagefault.c | 4 +- > drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 167 ++++++++++++++++++ > drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 66 +++++-- > drivers/gpu/drm/xe/xe_gt_types.h | 4 + > drivers/gpu/drm/xe/xe_vm.c | 12 +- > 11 files changed, 281 insertions(+), 67 deletions(-) > create mode 100644 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile > index 63e757406595..d279cec7bb1d 100644 > --- a/drivers/gpu/drm/xe/Makefile > +++ b/drivers/gpu/drm/xe/Makefile > @@ -91,6 +91,7 @@ xe-y += xe_bb.o \ > xe_gt_sysfs.o \ > xe_gt_throttle_sysfs.o \ > xe_gt_guc_tlb_invalidation.o \ > + xe_gt_tlb_invalidation.o \ > xe_gt_topology.o \ > xe_guc.o \ > xe_guc_ads.o \ > diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c > index 55bbc8b8df15..5b438f0e8a7d 100644 > --- a/drivers/gpu/drm/xe/xe_device.c > +++ b/drivers/gpu/drm/xe/xe_device.c > @@ -45,6 +45,7 @@ > #include "xe_ttm_sys_mgr.h" > #include "xe_vm.h" > #include "xe_wait_user_fence.h" > +#include "xe_gt_tlb_invalidation.h" > > static int xe_file_open(struct drm_device *dev, struct drm_file *file) > { > @@ -568,6 +569,12 @@ int xe_device_probe(struct xe_device *xe) > } > } > > + for_each_gt(gt, xe, id) { > + err = xe_gt_tlb_invalidation_init(gt); > + if (err) > + return err; > + } > + This is probably not the right place for this. Lucas is doing some work [1] on the probe order. This is just a SW init, right? There should be place in the xe_gt layer that initializes the SW, that would be right place to hook in the new xe_gt_tlb_invalidation_init. [1] https://patchwork.freedesktop.org/series/132920/ > for_each_gt(gt, xe, id) { > err = xe_gt_init_hwconfig(gt); > if (err) > diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c > index e0c1bfa92730..3031a45957e9 100644 > --- a/drivers/gpu/drm/xe/xe_ggtt.c > +++ b/drivers/gpu/drm/xe/xe_ggtt.c > @@ -19,7 +19,7 @@ > #include "xe_device.h" > #include "xe_gt.h" > #include "xe_gt_printk.h" > -#include "xe_gt_guc_tlb_invalidation.h" > +#include "xe_gt_tlb_invalidation.h" > #include "xe_map.h" > #include "xe_pm.h" > #include "xe_sriov.h" > @@ -247,14 +247,14 @@ int xe_ggtt_init(struct xe_ggtt *ggtt) > > static void ggtt_invalidate_gt_tlb(struct xe_gt *gt) > { > - int err; > + int seqno[TLB_INV_MAX_INS] = {0}, err; > > if (!gt) > return; > > - err = xe_gt_guc_tlb_invalidation_ggtt(gt); > + err = xe_gt_tlb_invalidation_all(gt, seqno); Why s/ggtt/all? Without know what the other backends will look like it is hard to say if I agree with naming change. Can you elaborate on reasing for the name change? > if (err) > - drm_warn(>_to_xe(gt)->drm, "xe_gt_guc_tlb_invalidation_ggtt error=%d", err); > + drm_warn(>_to_xe(gt)->drm, "xe_gt_tlb_invalidation_all error=%d", err); > } > > static void xe_ggtt_invalidate(struct xe_ggtt *ggtt) > diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c > index b162f10902ea..7554da5c7e19 100644 > --- a/drivers/gpu/drm/xe/xe_gt.c > +++ b/drivers/gpu/drm/xe/xe_gt.c > @@ -504,6 +504,7 @@ int xe_gt_init_hwconfig(struct xe_gt *gt) > goto out; > > xe_gt_topology_init(gt); > + > xe_gt_mcr_init(gt); > xe_pat_init(gt); > > @@ -536,10 +537,6 @@ int xe_gt_init(struct xe_gt *gt) > xe_hw_fence_irq_init(>->fence_irq[i]); > } > > - err = xe_gt_guc_tlb_invalidation_init(gt); > - if (err) > - return err; > - > err = xe_gt_pagefault_init(gt); > if (err) > return err; > diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > index f994c64c95f4..469e57426a60 100644 > --- a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c > @@ -4,6 +4,7 @@ > */ > > #include "xe_gt_guc_tlb_invalidation.h" > +#include "xe_gt_tlb_invalidation.h" > > #include "abi/guc_actions_abi.h" > #include "xe_device.h" > @@ -17,6 +18,8 @@ > > #define TLB_TIMEOUT (HZ / 4) > > +static void xe_gt_guc_init_tlb_inv_funcs(struct xe_gt *gt); > + > static void xe_gt_guc_tlb_fence_timeout(struct work_struct *work) > { > struct xe_gt *gt = container_of(work, struct xe_gt, > @@ -66,6 +69,8 @@ int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt) > INIT_DELAYED_WORK(>->tlb_invalidation.fence_tdr, > xe_gt_guc_tlb_fence_timeout); > > + xe_gt_guc_init_tlb_inv_funcs(gt); > + > return 0; > } > > @@ -224,7 +229,7 @@ static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) > } > > /** > - * xe_gt_guc_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT > + * xe_gt_guc_tlb_invalidation_all - Issue a TLB invalidation on this GT for the GGTT > * @gt: graphics tile > * > * Issue a TLB invalidation for the GGTT. Completion of TLB invalidation is > @@ -232,7 +237,7 @@ static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) > * > * Return: 0 on success, negative error code on error > */ > -int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt) > +int xe_gt_guc_tlb_invalidation_all(struct xe_gt *gt) > { > struct xe_device *xe = gt_to_xe(gt); > > @@ -301,41 +306,9 @@ int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, > if (!xe->info.has_range_tlb_invalidation) { > action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_FULL); > } else { > - u64 start = xe_vma_start(vma); > - u64 length = xe_vma_size(vma); > - u64 align, end; > + u64 start, length; > > - if (length < SZ_4K) > - length = SZ_4K; > - > - /* > - * We need to invalidate a higher granularity if start address > - * is not aligned to length. When start is not aligned with > - * length we need to find the length large enough to create an > - * address mask covering the required range. > - */ > - align = roundup_pow_of_two(length); > - start = ALIGN_DOWN(xe_vma_start(vma), align); > - end = ALIGN(xe_vma_end(vma), align); > - length = align; > - while (start + length < end) { > - length <<= 1; > - start = ALIGN_DOWN(xe_vma_start(vma), length); > - } > - > - /* > - * Minimum invalidation size for a 2MB page that the hardware > - * expects is 16MB > - */ > - if (length >= SZ_2M) { > - length = max_t(u64, SZ_16M, length); > - start = ALIGN_DOWN(xe_vma_start(vma), length); > - } > - > - xe_gt_assert(gt, length >= SZ_4K); > - xe_gt_assert(gt, is_power_of_2(length)); > - xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1))); > - xe_gt_assert(gt, IS_ALIGNED(start, length)); > + xe_gt_tlb_invalidation_vma_alignments(gt, vma, &start, &length); > > action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_PAGE_SELECTIVE); > action[len++] = xe_vma_vm(vma)->usm.asid; > @@ -457,3 +430,30 @@ int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 le > > return 0; > } > + > +/** > + * xe_gt_guc_tlb_invalidation_vma_wrapper - wrapper for gt guc tlb range invalidation. > + * @gt: graphics technology the invalidation was initiated on. > + * @vma: VMA to invalidate > + * @data: data used for the invalidation. > + * > + * Issue a range based TLB invalidation. > + * > + * Return: Seqno which can be passed to xe_gt_guc_tlb_invalidation_wait on success, > + * negative error code on error. > + */ > +static int xe_gt_guc_tlb_invalidation_vma_wrapper(struct xe_gt *gt, struct xe_vma *vma, void *data) > +{ > + struct xe_gt_guc_tlb_invalidation_fence *fence = data; > + > + return xe_gt_guc_tlb_invalidation_vma(gt, fence, vma); > +} > + > +static void xe_gt_guc_init_tlb_inv_funcs(struct xe_gt *gt) > +{ > + struct xe_gt_tlb_ins_invalidation_funcs *guc_ins = >->tlb_invalidation.inv_funcs[GUC_TLB]; > + > + guc_ins->inv_range = xe_gt_guc_tlb_invalidation_vma_wrapper; > + guc_ins->inv_all = xe_gt_guc_tlb_invalidation_all; > + guc_ins->inv_wait = xe_gt_guc_tlb_invalidation_wait; So the proper way to do this is have a 'static const struct xe_gt_tlb_ins_invalidation_funcs' locally and return that rather than setting the functions individually. See xe_ring_ops.c and 'static const struct xe_ring_ops' for an example. > +} > diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > index f7402dd0bec3..a02ee007c642 100644 > --- a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h > @@ -16,7 +16,7 @@ struct xe_vma; > > int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt); > void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt); > -int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt); > +int xe_gt_guc_tlb_invalidation_all(struct xe_gt *gt); > int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, > struct xe_gt_guc_tlb_invalidation_fence *fence, > struct xe_vma *vma); > diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c > index 8c47c60f3cd3..dba49f3ae933 100644 > --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c > +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c > @@ -15,7 +15,7 @@ > #include "abi/guc_actions_abi.h" > #include "xe_bo.h" > #include "xe_gt.h" > -#include "xe_gt_guc_tlb_invalidation.h" > +#include "xe_gt_tlb_invalidation.h" > #include "xe_guc.h" > #include "xe_guc_ct.h" > #include "xe_migrate.h" > @@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) > goto retry_userptr; > > if (!ret) { > - ret = xe_gt_guc_tlb_invalidation_vma(gt, NULL, vma); > + ret = xe_gt_tlb_invalidation_vma(gt, vma, NULL, NULL); > if (ret >= 0) > ret = 0; > } > diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c > new file mode 100644 > index 000000000000..84dec631aeb1 > --- /dev/null > +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c > @@ -0,0 +1,167 @@ > +// SPDX-License-Identifier: MIT > +/* > + * Copyright © 2024 Intel Corporation > + */ > + > +#include "xe_gt_tlb_invalidation.h" > +#include "xe_gt_guc_tlb_invalidation.h" > +#include "xe_gt.h" > +#include "xe_gt_printk.h" > +#include "xe_vm.h" > + > +/** > + * xe_gt_tlb_invalidation_init - TLB invalidation init generic function. > + * @gt: graphics technology the invalidation was initiated on. > + * > + * Initialize all TLBs in the system, including setting the invalidation > + * proprietary function pointer. > + * > + * Return: 0 on success, negative error code on error. > + */ > +int xe_gt_tlb_invalidation_init(struct xe_gt *gt) > +{ > + int ret; > + > + gt->tlb_invalidation.inv_funcs = kcalloc(TLB_INV_MAX_INS, > + sizeof(struct xe_gt_tlb_ins_invalidation_funcs), > + GFP_KERNEL); See my comments below in xe_gt_types.h, I don't think we need a malloc here. > + if (!gt->tlb_invalidation.inv_funcs) > + return -ENOMEM; > + > + /* Initialize gt Guc TLB */ > + ret = xe_gt_guc_tlb_invalidation_init(gt); See my comments in the previous patch, I think the init function should return the inv_funcs rather than setting these in the GT. > + if (ret) { > + kfree(gt->tlb_invalidation.inv_funcs); > + return ret; > + } > + > + return 0; > +} > + > +/** > + * xe_gt_tlb_invalidation_all - TLB invalidation all generic function. > + * @gt: graphics technology the invalidation was initiated on. > + * @seq: sequence numbers of the TLB invalidations. > + * This is the output of the invalidation functions, can be used when calling > + * xe_gt_tlb_invalidation_wait. > + * > + * Issue a TLB invalidation when the GGTT is changed. > + * This will make invalidation for all TLBs in the system, for mappings in GGTT and all > + * PPGTT tables. > + * > + * Return: 0 on success, negative error code on error. > + */ > +int xe_gt_tlb_invalidation_all(struct xe_gt *gt, int *seq) > +{ > + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { > + seq[ins] = gt->tlb_invalidation.inv_funcs[ins].inv_all(gt); Wondering if we should support backends having NULL vfuncs? e.g. if (!inv_funcs[ins].inv_all) seq[ins] = not present SEQNO else seq[ins] = gt->tlb_invalidation.inv_funcs[ins].inv_all(gt); > + if (seq[ins] < 0) > + return seq[ins]; Here if one the backend fails the seqno will get out sync. Maybe that is ok? I guess the interface accepts an array of seqnos. I'm wondering if we could make this a single seqno so the number of backends is opaque to the other layers. > + } > + > + return 0; > +} > + > +/** > + * xe_gt_tlb_invalidation_vma - TLB range invalidation generic function. > + * @gt: graphics technology the invalidation was initiated on. > + * @vma: VMA to invalidate > + * @data: private data if needed for some TLB instances. > + * @seq_out: sequence numbers of the TLB invalidations. > + * This is the output of the invalidation functions, can be used when calling > + * xe_gt_tlb_invalidation_wait, in case it's NULL then the caller doesn't need to wait for the > + * invalidation. > + * > + * Issue a PPGTT related TLB invalidation, which will invalidate a range defined in the vma. > + * > + * Return: 0 on success, negative error code on error. > + */ > +int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, struct xe_vma *vma, void *data, int *seq_out) > +{ > + int seq[TLB_INV_MAX_INS] = {0}; > + > + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { > + seq[ins] = gt->tlb_invalidation.inv_funcs[ins].inv_range(gt, vma, data); > + if (seq[ins] < 0) > + return seq[ins]; > + } > + > + if (seq_out) > + memcpy(seq_out, seq, sizeof(seq)); > + > + return 0; > +} > + > +/** > + * xe_gt_tlb_invalidation_wait - TLB invalidation wait generic function. > + * @gt: graphics technology the invalidation was initiated on. > + * @seq: sequence numbers which was returned from the TLB invalidation functions. > + * > + * Wait for a TLB invalidation to complete. > + * > + * Return: 0 on success, negative value on timeout/failure. > + */ > +int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int *seq) > +{ Same comment as above, it would be great if we could have a single seq to wait on. > + int ret; > + > + for (int ins = 0 ; ins < TLB_INV_MAX_INS ; ins++) { > + ret = gt->tlb_invalidation.inv_funcs[ins].inv_wait(gt, seq[ins]); > + if (ret < 0) > + return ret; > + } > + > + return 0; > +} > + > +/** > + * xe_gt_tlb_invalidation_vma_alignments - Make alignments on vma. > + * @gt: graphics technology the invalidation was initiated on. > + * @vma: VMA to invalidate > + * @start_aligned: aligned start address (output). > + * @length_aligned: aligned length (output). > + * > + * Return: 0 on success, negative value on timeout/failure. > + */ > +void xe_gt_tlb_invalidation_vma_alignments(struct xe_gt *gt, struct xe_vma *vma, > + u64 *start_aligned, u64 *length_aligned) > +{ I already made some changes here in [2]. Take a look at that. [2] https://patchwork.freedesktop.org/patch/591281/?series=132246&rev=3 > + u64 start = xe_vma_start(vma); > + u64 length = xe_vma_size(vma); > + u64 align, end; > + > + if (length < SZ_4K) > + length = SZ_4K; > + > + /* > + * We need to invalidate a higher granularity if start address > + * is not aligned to length. When start is not aligned with > + * length we need to find the length large enough to create an > + * address mask covering the required range. > + */ > + align = roundup_pow_of_two(length); > + start = ALIGN_DOWN(xe_vma_start(vma), align); > + end = ALIGN(xe_vma_end(vma), align); > + length = align; > + while (start + length < end) { > + length <<= 1; > + start = ALIGN_DOWN(xe_vma_start(vma), length); > + } > + > + /* > + * Minimum invalidation size for a 2MB page that the hardware > + * expects is 16MB > + */ > + if (length >= SZ_2M) { > + length = max_t(u64, SZ_16M, length); > + start = ALIGN_DOWN(xe_vma_start(vma), length); > + } > + > + xe_gt_assert(gt, length >= SZ_4K); > + xe_gt_assert(gt, is_power_of_2(length)); > + xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1))); > + xe_gt_assert(gt, IS_ALIGNED(start, length)); > + > + *start_aligned = start; > + *length_aligned = length; > +} > diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > index fab65b53cb24..406e2820e8c1 100644 > --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h > @@ -1,6 +1,6 @@ > /* SPDX-License-Identifier: MIT */ > /* > - * Copyright © 2023 Intel Corporation > + * Copyright © 2024 Intel Corporation > */ > > #ifndef _XE_GT_TLB_INVALIDATION_H_ > @@ -8,19 +8,57 @@ > > #include <linux/types.h> > > -#include "xe_gt_guc_tlb_invalidation_types.h" > - > -struct xe_gt; > -struct xe_guc; > struct xe_vma; > +struct xe_device; > +struct xe_gt; > + > +enum xe_gt_tlb_instances { > + GUC_TLB = 0, > + TLB_INV_MAX_INS > +}; > + > +struct xe_gt_tlb_ins_invalidation_funcs { > + /** > + * struct xe_gt_tlb_invalidation_funcs member inv_range > + * > + * @gt: graphics technology the invalidation was initiated on. > + * @vma: Pointer to vma data. > + * @data: private data of the specific invalidation. > + * > + * This function should invalidate range of virtual addresses. > + * The function return sequence number > 0 in case we need to wait for the invalidation, > + * otherwise 0 which indicate success and no wait is needed, negative number upon failure. > + */ > + int (*inv_range)(struct xe_gt *gt, struct xe_vma *vma, void *data); > + > + /** > + * struct xe_gt_tlb_invalidation_funcs member inv_all > + * > + * @gt: graphics technology the invalidation was initiated on. > + * > + * This function should invalidate the TLB instance when GGTT table is changed. > + * When GGTT is changed driver must invalidate all mappings for both GGTT/PPGTT in the TLB > + * instance. > + * The function return sequence number > 0 in case we need to wait for the invalidation, > + * otherwise 0 which indicate success and no wait is needed, negative number upon failure. > + */ > + int (*inv_all)(struct xe_gt *gt); > + > + /** > + * struct xe_gt_tlb_invalidation_funcs member inv_wait > + * > + * @gt: graphics technology the invalidation was initiated on. > + * @seq: the sequence number returned from the invalidation function. > + * > + * This function should wait till the invalidation is done. > + */ > + int (*inv_wait)(struct xe_gt *gt, int seq); > +}; This should probably go in a *_types.h file. > > int xe_gt_tlb_invalidation_init(struct xe_gt *gt); > -void xe_gt_tlb_invalidation_reset(struct xe_gt *gt); > -int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); > -int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, > - struct xe_gt_guc_tlb_invalidation_fence *fence, > - struct xe_vma *vma); > -int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); > -int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); > - See my comment in previous reply, would be great if we can get this work without changing any existing exported funcion. To me that indicates we have gotten the layering correct. > -#endif /* _XE_GT_TLB_INVALIDATION_ */ > +int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, struct xe_vma *vma, void *data, int *seq_out); > +int xe_gt_tlb_invalidation_all(struct xe_gt *gt, int *seq_out); > +int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int *seq); > +void xe_gt_tlb_invalidation_vma_alignments(struct xe_gt *gt, struct xe_vma *vma, > + u64 *start_aligned, u64 *length_aligned); > +#endif /* _XE_GT_TLB_INVALIDATION_H_ */ > diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h > index c38e48c039e6..49a88b0d0d25 100644 > --- a/drivers/gpu/drm/xe/xe_gt_types.h > +++ b/drivers/gpu/drm/xe/xe_gt_types.h > @@ -190,6 +190,10 @@ struct xe_gt { > struct delayed_work fence_tdr; > /** @tlb_invalidation.lock: protects TLB invalidation fences */ > spinlock_t lock; > + /** > + * @tlb_invalidation.inv_funcs: invalidation functions for TLB instances > + */ > + struct xe_gt_tlb_ins_invalidation_funcs *inv_funcs; const struct xe_gt_tlb_ins_invalidation_funcs *inv_funcs[MAX_INV_BACKEND] Fix MAX_INV_BACKEND to a better name but you get the idea. > } tlb_invalidation; > > /** > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c > index 965e17c640c9..b2825ed0f1fa 100644 > --- a/drivers/gpu/drm/xe/xe_vm.c > +++ b/drivers/gpu/drm/xe/xe_vm.c > @@ -28,7 +28,7 @@ > #include "xe_drm_client.h" > #include "xe_exec_queue.h" > #include "xe_gt_pagefault.h" > -#include "xe_gt_guc_tlb_invalidation.h" > +#include "xe_gt_tlb_invalidation.h" > #include "xe_migrate.h" > #include "xe_pat.h" > #include "xe_pm.h" > @@ -3146,7 +3146,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) > struct xe_device *xe = xe_vma_vm(vma)->xe; > struct xe_tile *tile; > u32 tile_needs_invalidate = 0; > - int seqno[XE_MAX_TILES_PER_DEVICE]; > + int seqno[XE_MAX_TILES_PER_DEVICE][TLB_INV_MAX_INS] = {{0}}; Again in a perfect world we have a single seqno per GT. Matt > u8 id; > int ret; > > @@ -3179,15 +3179,15 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) > * FIXME: We potentially need to invalidate multiple > * GTs within the tile > */ > - seqno[id] = xe_gt_guc_tlb_invalidation_vma(tile->primary_gt, NULL, vma); > - if (seqno[id] < 0) > - return seqno[id]; > + ret = xe_gt_tlb_invalidation_vma(tile->primary_gt, vma, NULL, seqno[id]); > + if (ret < 0) > + return ret; > } > } > > for_each_tile(tile, xe, id) { > if (tile_needs_invalidate & BIT(id)) { > - ret = xe_gt_guc_tlb_invalidation_wait(tile->primary_gt, seqno[id]); > + ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); > if (ret < 0) > return ret; > } > -- > 2.34.1 > ^ permalink raw reply [flat|nested] 7+ messages in thread
* ✗ CI.Patch_applied: failure for series starting with [1/2] drm/xe: rename gt invalidation functions 2024-04-24 9:42 Farah Kassabri 2024-04-24 9:42 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri 2024-04-24 9:42 ` [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations Farah Kassabri @ 2024-04-28 22:02 ` Patchwork 2 siblings, 0 replies; 7+ messages in thread From: Patchwork @ 2024-04-28 22:02 UTC (permalink / raw) To: Farah Kassabri; +Cc: intel-xe == Series Details == Series: series starting with [1/2] drm/xe: rename gt invalidation functions URL : https://patchwork.freedesktop.org/series/133001/ State : failure == Summary == === Applying kernel patches on branch 'drm-tip' with base: === Base commit: 94a0eea182f0 drm-tip: 2024y-04m-27d-07h-26m-48s UTC integration manifest === git am output follows === error: patch failed: drivers/gpu/drm/xe/xe_gt.c:659 error: drivers/gpu/drm/xe/xe_gt.c: patch does not apply error: patch failed: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c:240 error: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c: patch does not apply error: patch failed: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h:18 error: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h: patch does not apply error: patch failed: drivers/gpu/drm/xe/xe_pt.c:1073 error: drivers/gpu/drm/xe/xe_pt.c: patch does not apply hint: Use 'git am --show-current-patch=diff' to see the failed patch Applying: drm/xe: rename gt invalidation functions Patch failed at 0001 drm/xe: rename gt invalidation functions When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". ^ permalink raw reply [flat|nested] 7+ messages in thread
* (no subject)
@ 2024-04-25 9:35 Farah Kassabri
2024-04-25 9:35 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri
0 siblings, 1 reply; 7+ messages in thread
From: Farah Kassabri @ 2024-04-25 9:35 UTC (permalink / raw)
To: intel-xe; +Cc: ofir1.bitton
Current Xe driver supports one TLB invaliation flow, which is
for the compute TLB going through the Guc.
This driver will be used for future products which will have multiple
TLBs each has it's own invalidation flow.
In order to prepare the driver for those new flows, these patches here
will add infrastructure to easily allow each TLB to register it's own
function keeping the common code generic.
Farah Kassabri (2):
drm/xe: rename gt invalidation functions
drm/xe: preparations for multi instance TLB invalidations
drivers/gpu/drm/xe/Makefile | 1 +
drivers/gpu/drm/xe/abi/guc_actions_abi.h | 2 +-
drivers/gpu/drm/xe/xe_device.c | 7 +
drivers/gpu/drm/xe/xe_ggtt.c | 6 +-
drivers/gpu/drm/xe/xe_gt.c | 9 +-
.../gpu/drm/xe/xe_gt_guc_tlb_invalidation.c | 459 +++++++++++++++++
.../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 26 +
...s.h => xe_gt_guc_tlb_invalidation_types.h} | 15 +-
drivers/gpu/drm/xe/xe_gt_pagefault.c | 2 +-
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 484 ++++--------------
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 66 ++-
drivers/gpu/drm/xe/xe_gt_types.h | 6 +-
drivers/gpu/drm/xe/xe_guc_ct.c | 6 +-
drivers/gpu/drm/xe/xe_pt.c | 12 +-
drivers/gpu/drm/xe/xe_trace.h | 38 +-
drivers/gpu/drm/xe/xe_vm.c | 8 +-
16 files changed, 695 insertions(+), 452 deletions(-)
create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c
create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h
rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation_types.h => xe_gt_guc_tlb_invalidation_types.h} (54%)
--
2.34.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/2] drm/xe: rename gt invalidation functions 2024-04-25 9:35 Farah Kassabri @ 2024-04-25 9:35 ` Farah Kassabri 0 siblings, 0 replies; 7+ messages in thread From: Farah Kassabri @ 2024-04-25 9:35 UTC (permalink / raw) To: intel-xe; +Cc: ofir1.bitton Future products will have multi instance TLBs, Current Xe driver handles only the compute TLBs invalidaiton through the Guc. In preparation for Multi instance TLB support, we need to rename current invalidation as Guc specific invalidation functions, and not general gt functions. This patch just renames the current functions, files and variables. follow up patches will start adding the multi instance TLB support for future XE products, where each will have its own invalidation flow. Signed-off-by: Farah Kassabri <fkassabri@habana.ai> --- drivers/gpu/drm/xe/Makefile | 2 +- drivers/gpu/drm/xe/abi/guc_actions_abi.h | 2 +- drivers/gpu/drm/xe/xe_ggtt.c | 6 +- drivers/gpu/drm/xe/xe_gt.c | 6 +- ...idation.c => xe_gt_guc_tlb_invalidation.c} | 72 +++++++++---------- .../gpu/drm/xe/xe_gt_guc_tlb_invalidation.h | 26 +++++++ ...s.h => xe_gt_guc_tlb_invalidation_types.h} | 15 ++-- drivers/gpu/drm/xe/xe_gt_pagefault.c | 4 +- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 4 +- drivers/gpu/drm/xe/xe_gt_types.h | 2 +- drivers/gpu/drm/xe/xe_guc_ct.c | 6 +- drivers/gpu/drm/xe/xe_pt.c | 12 ++-- drivers/gpu/drm/xe/xe_trace.h | 38 +++++----- drivers/gpu/drm/xe/xe_vm.c | 6 +- 14 files changed, 115 insertions(+), 86 deletions(-) rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation.c => xe_gt_guc_tlb_invalidation.c} (83%) create mode 100644 drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h rename drivers/gpu/drm/xe/{xe_gt_tlb_invalidation_types.h => xe_gt_guc_tlb_invalidation_types.h} (54%) diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile index 8321ec4f9b46..63e757406595 100644 --- a/drivers/gpu/drm/xe/Makefile +++ b/drivers/gpu/drm/xe/Makefile @@ -90,7 +90,7 @@ xe-y += xe_bb.o \ xe_gt_pagefault.o \ xe_gt_sysfs.o \ xe_gt_throttle_sysfs.o \ - xe_gt_tlb_invalidation.o \ + xe_gt_guc_tlb_invalidation.o \ xe_gt_topology.o \ xe_guc.o \ xe_guc_ads.o \ diff --git a/drivers/gpu/drm/xe/abi/guc_actions_abi.h b/drivers/gpu/drm/xe/abi/guc_actions_abi.h index 79ba98a169f9..ad4cdd5af80e 100644 --- a/drivers/gpu/drm/xe/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/xe/abi/guc_actions_abi.h @@ -187,7 +187,7 @@ enum xe_guc_sleep_state_status { /* Flush PPC or SMRO caches along with TLB invalidation request */ #define XE_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) -enum xe_guc_tlb_invalidation_type { +enum xe_gt_guc_tlb_invalidation_type { XE_GUC_TLB_INVAL_FULL = 0x0, XE_GUC_TLB_INVAL_PAGE_SELECTIVE = 0x1, XE_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX = 0x2, diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c index 0d541f55b4fc..e0c1bfa92730 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.c +++ b/drivers/gpu/drm/xe/xe_ggtt.c @@ -19,7 +19,7 @@ #include "xe_device.h" #include "xe_gt.h" #include "xe_gt_printk.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_map.h" #include "xe_pm.h" #include "xe_sriov.h" @@ -252,9 +252,9 @@ static void ggtt_invalidate_gt_tlb(struct xe_gt *gt) if (!gt) return; - err = xe_gt_tlb_invalidation_ggtt(gt); + err = xe_gt_guc_tlb_invalidation_ggtt(gt); if (err) - drm_warn(>_to_xe(gt)->drm, "xe_gt_tlb_invalidation_ggtt error=%d", err); + drm_warn(>_to_xe(gt)->drm, "xe_gt_guc_tlb_invalidation_ggtt error=%d", err); } static void xe_ggtt_invalidate(struct xe_ggtt *ggtt) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index 491d0413de15..b162f10902ea 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -31,7 +31,7 @@ #include "xe_gt_printk.h" #include "xe_gt_sriov_pf.h" #include "xe_gt_sysfs.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_gt_topology.h" #include "xe_guc_exec_queue_types.h" #include "xe_guc_pc.h" @@ -536,7 +536,7 @@ int xe_gt_init(struct xe_gt *gt) xe_hw_fence_irq_init(>->fence_irq[i]); } - err = xe_gt_tlb_invalidation_init(gt); + err = xe_gt_guc_tlb_invalidation_init(gt); if (err) return err; @@ -659,7 +659,7 @@ static int gt_reset(struct xe_gt *gt) if (err) goto err_out; - xe_gt_tlb_invalidation_reset(gt); + xe_gt_guc_tlb_invalidation_reset(gt); err = do_gt_reset(gt); if (err) diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c similarity index 83% rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c index 93df2d7969b3..f994c64c95f4 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.c @@ -3,7 +3,7 @@ * Copyright © 2023 Intel Corporation */ -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "abi/guc_actions_abi.h" #include "xe_device.h" @@ -17,11 +17,11 @@ #define TLB_TIMEOUT (HZ / 4) -static void xe_gt_tlb_fence_timeout(struct work_struct *work) +static void xe_gt_guc_tlb_fence_timeout(struct work_struct *work) { struct xe_gt *gt = container_of(work, struct xe_gt, tlb_invalidation.fence_tdr.work); - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; spin_lock_irq(>->tlb_invalidation.pending_lock); list_for_each_entry_safe(fence, next, @@ -32,7 +32,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) if (msecs_to_jiffies(since_inval_ms) < TLB_TIMEOUT) break; - trace_xe_gt_tlb_invalidation_fence_timeout(fence); + trace_xe_gt_guc_tlb_invalidation_fence_timeout(fence); xe_gt_err(gt, "TLB invalidation fence timeout, seqno=%d recv=%d", fence->seqno, gt->tlb_invalidation.seqno_recv); @@ -49,7 +49,7 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) } /** - * xe_gt_tlb_invalidation_init - Initialize GT TLB invalidation state + * xe_gt_guc_tlb_invalidation_init - Initialize GT TLB invalidation state * @gt: graphics tile * * Initialize GT TLB invalidation state, purely software initialization, should @@ -57,42 +57,42 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) * * Return: 0 on success, negative error code on error. */ -int xe_gt_tlb_invalidation_init(struct xe_gt *gt) +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt) { gt->tlb_invalidation.seqno = 1; INIT_LIST_HEAD(>->tlb_invalidation.pending_fences); spin_lock_init(>->tlb_invalidation.pending_lock); spin_lock_init(>->tlb_invalidation.lock); INIT_DELAYED_WORK(>->tlb_invalidation.fence_tdr, - xe_gt_tlb_fence_timeout); + xe_gt_guc_tlb_fence_timeout); return 0; } static void -__invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) +__invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) { - trace_xe_gt_tlb_invalidation_fence_signal(fence); + trace_xe_gt_guc_tlb_invalidation_fence_signal(fence); dma_fence_signal(&fence->base); dma_fence_put(&fence->base); } static void -invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence) +invalidation_fence_signal(struct xe_gt_guc_tlb_invalidation_fence *fence) { list_del(&fence->link); __invalidation_fence_signal(fence); } /** - * xe_gt_tlb_invalidation_reset - Initialize GT TLB invalidation reset + * xe_gt_guc_tlb_invalidation_reset - Initialize GT TLB invalidation reset * @gt: graphics tile * * Signal any pending invalidation fences, should be called during a GT reset */ -void xe_gt_tlb_invalidation_reset(struct xe_gt *gt) +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt) { - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; struct xe_guc *guc = >->uc.guc; int pending_seqno; @@ -139,7 +139,7 @@ static bool tlb_invalidation_seqno_past(struct xe_gt *gt, int seqno) } static int send_tlb_invalidation(struct xe_guc *guc, - struct xe_gt_tlb_invalidation_fence *fence, + struct xe_gt_guc_tlb_invalidation_fence *fence, u32 *action, int len) { struct xe_gt *gt = guc_to_gt(guc); @@ -156,7 +156,7 @@ static int send_tlb_invalidation(struct xe_guc *guc, seqno = gt->tlb_invalidation.seqno; if (fence) { fence->seqno = seqno; - trace_xe_gt_tlb_invalidation_fence_send(fence); + trace_xe_gt_guc_tlb_invalidation_fence_send(fence); } action[1] = seqno; ret = xe_guc_ct_send_locked(&guc->ct, action, len, @@ -202,16 +202,16 @@ static int send_tlb_invalidation(struct xe_guc *guc, XE_GUC_TLB_INVAL_FLUSH_CACHE) /** - * xe_gt_tlb_invalidation_guc - Issue a TLB invalidation on this GT for the GuC + * xe_gt_guc_tlb_invalidation - Issue a TLB invalidation on this GT for the GuC * @gt: graphics tile * * Issue a TLB invalidation for the GuC. Completion of TLB is asynchronous and - * caller can use seqno + xe_gt_tlb_invalidation_wait to wait for completion. + * caller can use seqno + xe_gt_guc_tlb_invalidation_wait to wait for completion. * * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, * negative error code on error. */ -static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) +static int xe_gt_guc_tlb_invalidation(struct xe_gt *gt) { u32 action[] = { XE_GUC_ACTION_TLB_INVALIDATION, @@ -224,7 +224,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) } /** - * xe_gt_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT + * xe_gt_guc_tlb_invalidation_ggtt - Issue a TLB invalidation on this GT for the GGTT * @gt: graphics tile * * Issue a TLB invalidation for the GGTT. Completion of TLB invalidation is @@ -232,7 +232,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) * * Return: 0 on success, negative error code on error */ -int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt) { struct xe_device *xe = gt_to_xe(gt); @@ -240,11 +240,11 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) gt->uc.guc.submission_state.enabled) { int seqno; - seqno = xe_gt_tlb_invalidation_guc(gt); + seqno = xe_gt_guc_tlb_invalidation(gt); if (seqno <= 0) return seqno; - xe_gt_tlb_invalidation_wait(gt, seqno); + xe_gt_guc_tlb_invalidation_wait(gt, seqno); } else if (xe_device_uc_enabled(xe)) { xe_gt_WARN_ON(gt, xe_force_wake_get(gt_to_fw(gt), XE_FW_GT)); if (xe->info.platform == XE_PVC || GRAPHICS_VER(xe) >= 20) { @@ -263,23 +263,23 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) } /** - * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA - * @gt: graphics tile + * xe_gt_guc_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA + * @gt: graphics technology * @fence: invalidation fence which will be signal on TLB invalidation * completion, can be NULL * @vma: VMA to invalidate * * Issue a range based TLB invalidation if supported, if not fallback to a full * TLB invalidation. Completion of TLB is asynchronous and caller can either use - * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for + * the invalidation fence or seqno + xe_gt_guc_tlb_invalidation_wait to wait for * completion. * - * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, + * Return: Seqno which can be passed to xe_gt_guc_tlb_invalidation_wait on success, * negative error code on error. */ -int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, - struct xe_gt_tlb_invalidation_fence *fence, - struct xe_vma *vma) +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, + struct xe_gt_guc_tlb_invalidation_fence *fence, + struct xe_vma *vma) { struct xe_device *xe = gt_to_xe(gt); #define MAX_TLB_INVALIDATION_LEN 7 @@ -350,16 +350,16 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, } /** - * xe_gt_tlb_invalidation_wait - Wait for TLB to complete + * xe_gt_guc_tlb_invalidation_wait - Wait for the gt guc TLB invalidation to complete * @gt: graphics tile - * @seqno: seqno to wait which was returned from xe_gt_tlb_invalidation + * @seqno: seqno to wait which was returned from xe_gt_guc_tlb_invalidation * * Wait for 200ms for a TLB invalidation to complete, in practice we always * should receive the TLB invalidation within 200ms. * * Return: 0 on success, -ETIME on TLB invalidation timeout */ -int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno) { struct xe_guc *guc = >->uc.guc; int ret; @@ -388,7 +388,7 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) } /** - * xe_guc_tlb_invalidation_done_handler - TLB invalidation done handler + * xe_gt_guc_tlb_invalidation_done_handler - Guc TLB invalidation done handler * @guc: guc * @msg: message indicating TLB invalidation done * @len: length of message @@ -399,10 +399,10 @@ int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) * * Return: 0 on success, -EPROTO for malformed messages. */ -int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) { struct xe_gt *gt = guc_to_gt(guc); - struct xe_gt_tlb_invalidation_fence *fence, *next; + struct xe_gt_guc_tlb_invalidation_fence *fence, *next; unsigned long flags; if (unlikely(len != 1)) @@ -438,7 +438,7 @@ int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) list_for_each_entry_safe(fence, next, >->tlb_invalidation.pending_fences, link) { - trace_xe_gt_tlb_invalidation_fence_recv(fence); + trace_xe_gt_guc_tlb_invalidation_fence_recv(fence); if (!tlb_invalidation_seqno_past(gt, fence->seqno)) break; diff --git a/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h new file mode 100644 index 000000000000..f7402dd0bec3 --- /dev/null +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2023 Intel Corporation + */ + +#ifndef _XE_GT_GUC_TLB_INVALIDATION_H_ +#define _XE_GT_GUC_TLB_INVALIDATION_H_ + +#include <linux/types.h> + +#include "xe_gt_guc_tlb_invalidation_types.h" + +struct xe_gt; +struct xe_guc; +struct xe_vma; + +int xe_gt_guc_tlb_invalidation_init(struct xe_gt *gt); +void xe_gt_guc_tlb_invalidation_reset(struct xe_gt *gt); +int xe_gt_guc_tlb_invalidation_ggtt(struct xe_gt *gt); +int xe_gt_guc_tlb_invalidation_vma(struct xe_gt *gt, + struct xe_gt_guc_tlb_invalidation_fence *fence, + struct xe_vma *vma); +int xe_gt_guc_tlb_invalidation_wait(struct xe_gt *gt, int seqno); +int xe_gt_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); + +#endif /* _XE_GT_GUC_TLB_INVALIDATION_ */ diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h similarity index 54% rename from drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h rename to drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h index 934c828efe31..9e3d1daa51ba 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h +++ b/drivers/gpu/drm/xe/xe_gt_guc_tlb_invalidation_types.h @@ -3,18 +3,21 @@ * Copyright © 2023 Intel Corporation */ -#ifndef _XE_GT_TLB_INVALIDATION_TYPES_H_ -#define _XE_GT_TLB_INVALIDATION_TYPES_H_ +#ifndef _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ +#define _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ #include <linux/dma-fence.h> +struct xe_vma; +struct xe_gt; + /** - * struct xe_gt_tlb_invalidation_fence - XE GT TLB invalidation fence + * struct xe_gt_guc_tlb_invalidation_fence - XE GUC TLB invalidation fence * - * Optionally passed to xe_gt_tlb_invalidation and will be signaled upon TLB + * Optionally passed to xe_gt_guc_tlb_invalidation and will be signaled upon TLB * invalidation completion. */ -struct xe_gt_tlb_invalidation_fence { +struct xe_gt_guc_tlb_invalidation_fence { /** @base: dma fence base */ struct dma_fence base; /** @link: link into list of pending tlb fences */ @@ -25,4 +28,4 @@ struct xe_gt_tlb_invalidation_fence { ktime_t invalidation_time; }; -#endif +#endif /* _XE_GT_GUC_TLB_INVALIDATION_TYPES_H_ */ diff --git a/drivers/gpu/drm/xe/xe_gt_pagefault.c b/drivers/gpu/drm/xe/xe_gt_pagefault.c index fa9e9853c53b..8c47c60f3cd3 100644 --- a/drivers/gpu/drm/xe/xe_gt_pagefault.c +++ b/drivers/gpu/drm/xe/xe_gt_pagefault.c @@ -15,7 +15,7 @@ #include "abi/guc_actions_abi.h" #include "xe_bo.h" #include "xe_gt.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_guc.h" #include "xe_guc_ct.h" #include "xe_migrate.h" @@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) goto retry_userptr; if (!ret) { - ret = xe_gt_tlb_invalidation_vma(gt, NULL, vma); + ret = xe_gt_guc_tlb_invalidation_vma(gt, NULL, vma); if (ret >= 0) ret = 0; } diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h index fbb743d80d2c..fab65b53cb24 100644 --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h @@ -8,7 +8,7 @@ #include <linux/types.h> -#include "xe_gt_tlb_invalidation_types.h" +#include "xe_gt_guc_tlb_invalidation_types.h" struct xe_gt; struct xe_guc; @@ -18,7 +18,7 @@ int xe_gt_tlb_invalidation_init(struct xe_gt *gt); void xe_gt_tlb_invalidation_reset(struct xe_gt *gt); int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, - struct xe_gt_tlb_invalidation_fence *fence, + struct xe_gt_guc_tlb_invalidation_fence *fence, struct xe_vma *vma); int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h index cfdc761ff7f4..c38e48c039e6 100644 --- a/drivers/gpu/drm/xe/xe_gt_types.h +++ b/drivers/gpu/drm/xe/xe_gt_types.h @@ -185,7 +185,7 @@ struct xe_gt { spinlock_t pending_lock; /** * @tlb_invalidation.fence_tdr: schedules a delayed call to - * xe_gt_tlb_fence_timeout after the timeut interval is over. + * xe_gt_guc_tlb_fence_timeout after the timeut interval is over. */ struct delayed_work fence_tdr; /** @tlb_invalidation.lock: protects TLB invalidation fences */ diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c index 8ac819a7061e..73cbb7a8d12f 100644 --- a/drivers/gpu/drm/xe/xe_guc_ct.c +++ b/drivers/gpu/drm/xe/xe_guc_ct.c @@ -22,7 +22,7 @@ #include "xe_gt_pagefault.h" #include "xe_gt_printk.h" #include "xe_gt_sriov_pf_control.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_guc.h" #include "xe_guc_relay.h" #include "xe_guc_submit.h" @@ -1050,7 +1050,7 @@ static int process_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len) ret = xe_guc_pagefault_handler(guc, payload, adj_len); break; case XE_GUC_ACTION_TLB_INVALIDATION_DONE: - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, adj_len); break; case XE_GUC_ACTION_ACCESS_COUNTER_NOTIFY: @@ -1184,7 +1184,7 @@ static void g2h_fast_path(struct xe_guc_ct *ct, u32 *msg, u32 len) break; case XE_GUC_ACTION_TLB_INVALIDATION_DONE: __g2h_release_space(ct, len); - ret = xe_guc_tlb_invalidation_done_handler(guc, payload, + ret = xe_gt_guc_tlb_invalidation_done_handler(guc, payload, adj_len); break; default: diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c index 5b7930f46cf3..a0f63f457342 100644 --- a/drivers/gpu/drm/xe/xe_pt.c +++ b/drivers/gpu/drm/xe/xe_pt.c @@ -10,7 +10,7 @@ #include "xe_device.h" #include "xe_drm_client.h" #include "xe_gt.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_migrate.h" #include "xe_pt_types.h" #include "xe_pt_walk.h" @@ -1073,7 +1073,7 @@ static const struct xe_migrate_pt_update_ops userptr_bind_ops = { }; struct invalidation_fence { - struct xe_gt_tlb_invalidation_fence base; + struct xe_gt_guc_tlb_invalidation_fence base; struct xe_gt *gt; struct xe_vma *vma; struct dma_fence *fence; @@ -1104,7 +1104,7 @@ static void invalidation_fence_cb(struct dma_fence *fence, struct invalidation_fence *ifence = container_of(cb, struct invalidation_fence, cb); - trace_xe_gt_tlb_invalidation_fence_cb(&ifence->base); + trace_xe_gt_guc_tlb_invalidation_fence_cb(&ifence->base); if (!ifence->fence->error) { queue_work(system_wq, &ifence->work); } else { @@ -1120,8 +1120,8 @@ static void invalidation_fence_work_func(struct work_struct *w) struct invalidation_fence *ifence = container_of(w, struct invalidation_fence, work); - trace_xe_gt_tlb_invalidation_fence_work_func(&ifence->base); - xe_gt_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); + trace_xe_gt_guc_tlb_invalidation_fence_work_func(&ifence->base); + xe_gt_guc_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma); } static int invalidation_fence_init(struct xe_gt *gt, @@ -1131,7 +1131,7 @@ static int invalidation_fence_init(struct xe_gt *gt, { int ret; - trace_xe_gt_tlb_invalidation_fence_create(&ifence->base); + trace_xe_gt_guc_tlb_invalidation_fence_create(&ifence->base); spin_lock_irq(>->tlb_invalidation.lock); dma_fence_init(&ifence->base.base, &invalidation_fence_ops, diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h index 2d56cfc09e42..f3adde117811 100644 --- a/drivers/gpu/drm/xe/xe_trace.h +++ b/drivers/gpu/drm/xe/xe_trace.h @@ -16,18 +16,18 @@ #include "xe_bo_types.h" #include "xe_exec_queue_types.h" #include "xe_gpu_scheduler_types.h" -#include "xe_gt_tlb_invalidation_types.h" +#include "xe_gt_guc_tlb_invalidation_types.h" #include "xe_gt_types.h" #include "xe_guc_exec_queue_types.h" #include "xe_sched_job.h" #include "xe_vm.h" -DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DECLARE_EVENT_CLASS(xe_gt_guc_tlb_invalidation_fence, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence), TP_STRUCT__entry( - __field(struct xe_gt_tlb_invalidation_fence *, fence) + __field(struct xe_gt_guc_tlb_invalidation_fence *, fence) __field(int, seqno) ), @@ -40,39 +40,39 @@ DECLARE_EVENT_CLASS(xe_gt_tlb_invalidation_fence, __entry->fence, __entry->seqno) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_create, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_create, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, - xe_gt_tlb_invalidation_fence_work_func, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, + xe_gt_guc_tlb_invalidation_fence_work_func, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_cb, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_cb, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_send, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_send, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_recv, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_recv, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_signal, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_signal, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); -DEFINE_EVENT(xe_gt_tlb_invalidation_fence, xe_gt_tlb_invalidation_fence_timeout, - TP_PROTO(struct xe_gt_tlb_invalidation_fence *fence), +DEFINE_EVENT(xe_gt_guc_tlb_invalidation_fence, xe_gt_guc_tlb_invalidation_fence_timeout, + TP_PROTO(struct xe_gt_guc_tlb_invalidation_fence *fence), TP_ARGS(fence) ); diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 7ae2b0300db6..965e17c640c9 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -28,7 +28,7 @@ #include "xe_drm_client.h" #include "xe_exec_queue.h" #include "xe_gt_pagefault.h" -#include "xe_gt_tlb_invalidation.h" +#include "xe_gt_guc_tlb_invalidation.h" #include "xe_migrate.h" #include "xe_pat.h" #include "xe_pm.h" @@ -3179,7 +3179,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) * FIXME: We potentially need to invalidate multiple * GTs within the tile */ - seqno[id] = xe_gt_tlb_invalidation_vma(tile->primary_gt, NULL, vma); + seqno[id] = xe_gt_guc_tlb_invalidation_vma(tile->primary_gt, NULL, vma); if (seqno[id] < 0) return seqno[id]; } @@ -3187,7 +3187,7 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) for_each_tile(tile, xe, id) { if (tile_needs_invalidate & BIT(id)) { - ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); + ret = xe_gt_guc_tlb_invalidation_wait(tile->primary_gt, seqno[id]); if (ret < 0) return ret; } -- 2.34.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-04-29 2:01 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-04-24 9:42 Farah Kassabri 2024-04-24 9:42 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri 2024-04-29 1:29 ` Matthew Brost 2024-04-24 9:42 ` [PATCH 2/2] drm/xe: preparations for multi instance TLB invalidations Farah Kassabri 2024-04-29 2:01 ` Matthew Brost 2024-04-28 22:02 ` ✗ CI.Patch_applied: failure for series starting with [1/2] drm/xe: rename gt invalidation functions Patchwork -- strict thread matches above, loose matches on Subject: below -- 2024-04-25 9:35 Farah Kassabri 2024-04-25 9:35 ` [PATCH 1/2] drm/xe: rename gt invalidation functions Farah Kassabri
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox