From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 093F0C87FDA for ; Wed, 6 Aug 2025 22:24:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3B66E10E7CD; Wed, 6 Aug 2025 22:24:08 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="cdS1InI/"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 88F4310E31E for ; Wed, 6 Aug 2025 22:24:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1754519048; x=1786055048; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IuOPe8XXWf33FmaNTwZjTXb2iICRRn+At7W/UJChh/w=; b=cdS1InI/3ssyIhH5ZgAu/ewCIAf3UIIc89RI9ZpPgMHSGbTmN6wkw/Yq GDr4YlgJeUYiQgJ3rDf/UIqbXvPeuljziwCgRkrJukUqNalcXWsvFU1c8 pR6LrqjIaMYpWifecfFCSDp/srF3nv8GXMBkO8Rd5mJ3Qprg/0vMwYryG mpcCVlqtzN05CYdkaNKWrX/0YD8YahwJz7iyHyktF+1E0TaQC3NM+WGJm vl3EMhizEDrmSybBcNAEHTlSGiNh7m9/FWGpssbX9/ZJJ9XS7LvjMNgbn WCY+G2VwrQfN9t/BotsTPGChgAB4II4nM+1kC6z5J21E7HGPlrNODQdYw Q==; X-CSE-ConnectionGUID: p9z0i5mpS5e4yUrEgQCsIw== X-CSE-MsgGUID: BddPyNLkQGqLzPfCXAX4Hw== X-IronPort-AV: E=McAfee;i="6800,10657,11514"; a="56928154" X-IronPort-AV: E=Sophos;i="6.17,271,1747724400"; d="scan'208";a="56928154" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Aug 2025 15:24:07 -0700 X-CSE-ConnectionGUID: n2sdBo7/S/e8pHbA2iYKBA== X-CSE-MsgGUID: ufq3wvU2RPe+L//Cgr1QTQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,271,1747724400"; d="scan'208";a="188560834" Received: from dut136arlu.fm.intel.com ([10.105.23.73]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Aug 2025 15:24:07 -0700 From: stuartsummers To: Cc: matthew.brost@intel.com, farah.kassabri@intel.com, intel-xe@lists.freedesktop.org, Stuart Summers Subject: [PATCH 6/8] drm/xe: Prep TLB invalidation fence before sending Date: Wed, 6 Aug 2025 22:24:02 +0000 Message-Id: <20250806222404.30333-7-stuart.summers@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250806222404.30333-1-stuart.summers@intel.com> References: <20250806222404.30333-1-stuart.summers@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" From: Matthew Brost It is a bit backwards to add a TLB invalidation fence to the pending list after issuing the invalidation. Perform this step before issuing the TLB invalidation in a helper function. Signed-off-by: Matthew Brost Reviewed-by: Stuart Summers --- drivers/gpu/drm/xe/xe_tlb_inval.c | 103 +++++++++++++++--------------- 1 file changed, 51 insertions(+), 52 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_tlb_inval.c b/drivers/gpu/drm/xe/xe_tlb_inval.c index 288960ea0caa..2ffd677ee180 100644 --- a/drivers/gpu/drm/xe/xe_tlb_inval.c +++ b/drivers/gpu/drm/xe/xe_tlb_inval.c @@ -65,19 +65,19 @@ __inval_fence_signal(struct xe_device *xe, struct xe_tlb_inval_fence *fence) static void inval_fence_signal(struct xe_device *xe, struct xe_tlb_inval_fence *fence) { + lockdep_assert_held(&fence->tlb_inval->pending_lock); + list_del(&fence->link); __inval_fence_signal(xe, fence); } -void xe_tlb_inval_fence_signal(struct xe_tlb_inval_fence *fence) +static void +inval_fence_signal_unlocked(struct xe_device *xe, + struct xe_tlb_inval_fence *fence) { - struct xe_gt *gt; - - if (WARN_ON_ONCE(!fence->tlb_inval)) - return; - - gt = fence->tlb_inval->private; - __inval_fence_signal(gt_to_xe(gt), fence); + spin_lock_irq(&fence->tlb_inval->pending_lock); + inval_fence_signal(xe, fence); + spin_unlock_irq(&fence->tlb_inval->pending_lock); } static void xe_gt_tlb_fence_timeout(struct work_struct *work) @@ -199,14 +199,10 @@ static bool tlb_inval_seqno_past(struct xe_gt *gt, int seqno) return seqno_recv >= seqno; } -static int send_tlb_inval(struct xe_guc *guc, - struct xe_tlb_inval_fence *fence, +static int send_tlb_inval(struct xe_guc *guc, struct xe_tlb_inval_fence *fence, u32 *action, int len) { struct xe_gt *gt = guc_to_gt(guc); - struct xe_device *xe = gt_to_xe(gt); - int seqno; - int ret; xe_gt_assert(gt, fence); @@ -216,45 +212,36 @@ static int send_tlb_inval(struct xe_guc *guc, * need to be updated. */ - seqno = gt->tlb_inval.seqno; - fence->seqno = seqno; - trace_xe_tlb_inval_fence_send(xe, fence); - action[1] = seqno; - ret = xe_guc_ct_send(&guc->ct, action, len, - G2H_LEN_DW_TLB_INVALIDATE, 1); - if (!ret) { - spin_lock_irq(>->tlb_inval.pending_lock); - /* - * We haven't actually published the TLB fence as per - * pending_fences, but in theory our seqno could have already - * been written as we acquired the pending_lock. In such a case - * we can just go ahead and signal the fence here. - */ - if (tlb_inval_seqno_past(gt, seqno)) { - __inval_fence_signal(xe, fence); - } else { - fence->inval_time = ktime_get(); - list_add_tail(&fence->link, - >->tlb_inval.pending_fences); - - if (list_is_singular(>->tlb_inval.pending_fences)) - queue_delayed_work(system_wq, - >->tlb_inval.fence_tdr, - tlb_timeout_jiffies(gt)); - } - spin_unlock_irq(>->tlb_inval.pending_lock); - } else { - __inval_fence_signal(xe, fence); - } - if (!ret) { - gt->tlb_inval.seqno = (gt->tlb_inval.seqno + 1) % - TLB_INVALIDATION_SEQNO_MAX; - if (!gt->tlb_inval.seqno) - gt->tlb_inval.seqno = 1; - } xe_gt_stats_incr(gt, XE_GT_STATS_ID_TLB_INVAL, 1); + action[1] = fence->seqno; - return ret; + return xe_guc_ct_send(&guc->ct, action, len, + G2H_LEN_DW_TLB_INVALIDATE, 1); +} + +static void xe_tlb_inval_fence_prep(struct xe_tlb_inval_fence *fence) +{ + struct xe_tlb_inval *tlb_inval = fence->tlb_inval; + struct xe_gt *gt = tlb_inval->private; + struct xe_device *xe = gt_to_xe(gt); + + fence->seqno = tlb_inval->seqno; + trace_xe_tlb_inval_fence_send(xe, fence); + + spin_lock_irq(&tlb_inval->pending_lock); + fence->inval_time = ktime_get(); + list_add_tail(&fence->link, &tlb_inval->pending_fences); + + if (list_is_singular(&tlb_inval->pending_fences)) + queue_delayed_work(system_wq, + &tlb_inval->fence_tdr, + tlb_timeout_jiffies(gt)); + spin_unlock_irq(&tlb_inval->pending_lock); + + tlb_inval->seqno = (tlb_inval->seqno + 1) % + TLB_INVALIDATION_SEQNO_MAX; + if (!tlb_inval->seqno) + tlb_inval->seqno = 1; } #define MAKE_INVAL_OP(type) ((type << XE_GUC_TLB_INVAL_TYPE_SHIFT) | \ @@ -282,7 +269,12 @@ static int xe_tlb_inval_guc(struct xe_gt *gt, }; int ret; + xe_tlb_inval_fence_prep(fence); + ret = send_tlb_inval(>->uc.guc, fence, action, ARRAY_SIZE(action)); + if (ret < 0) + inval_fence_signal_unlocked(gt_to_xe(gt), fence); + /* * -ECANCELED indicates the CT is stopped for a GT reset. TLB caches * should be nuked on a GT reset so this error can be ignored. @@ -409,7 +401,7 @@ int xe_tlb_inval_range(struct xe_tlb_inval *tlb_inval, #define MAX_TLB_INVALIDATION_LEN 7 u32 action[MAX_TLB_INVALIDATION_LEN]; u64 length = end - start; - int len = 0; + int len = 0, ret; xe_gt_assert(gt, fence); @@ -470,7 +462,14 @@ int xe_tlb_inval_range(struct xe_tlb_inval *tlb_inval, xe_gt_assert(gt, len <= MAX_TLB_INVALIDATION_LEN); - return send_tlb_inval(>->uc.guc, fence, action, len); + xe_tlb_inval_fence_prep(fence); + + ret = send_tlb_inval(>->uc.guc, fence, action, + ARRAY_SIZE(action)); + if (ret < 0) + inval_fence_signal_unlocked(xe, fence); + + return ret; } /** -- 2.34.1