From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEFC4C87FCF for ; Thu, 7 Aug 2025 20:39:40 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2D3E910E8B0; Thu, 7 Aug 2025 20:39:40 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="B4XiQmJ5"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id 90D4410E498 for ; Thu, 7 Aug 2025 20:39:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1754599175; x=1786135175; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xMe2dOVFK/CFcf4cPeh0rFpBWEQ3Inw4m9ub/GQqDRk=; b=B4XiQmJ5QgJsA8GbwcjNsAyfxZ5ID9pvVwWhjgcYFsUhX+zYpxjhlkKb zLUXrdZTrobgVj/mZg5wksaxJp9R9UwARAz3c9kPGt8N6YR3QEbNp4okz VmWtnQWOzBB5/Hoj/l10Wn+hQO7YihKkq4ZA1kLfNl/3Cnl+ACmjvVTJa It0a2j5UKZpKZCR5sqsar0MZ7uF0xA8kDbrySiaNEXcLg6T+cgJ8KWAYP dYT2hCFmuIgtqiudtS1/TH4D1C8IUZcemVwMrltzT/Y3Y1ZrGYeJ4ARvq yCHfPQLbFgMhgcgdI6bZEda2JkqkZ9Qo4c3h9nHzwVwyhNgaOuUsfhz6F g==; X-CSE-ConnectionGUID: YoKZAURCRFKkxV57TflGDA== X-CSE-MsgGUID: 1DvLmB64QdyA0zHeaAu9mg== X-IronPort-AV: E=McAfee;i="6800,10657,11514"; a="60575148" X-IronPort-AV: E=Sophos;i="6.17,274,1747724400"; d="scan'208";a="60575148" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2025 13:39:34 -0700 X-CSE-ConnectionGUID: I01NQv9hR1GnJrdSzQuDBg== X-CSE-MsgGUID: hSBiPbvAQW6i2qyYn2nICA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,274,1747724400"; d="scan'208";a="164380354" Received: from dut136arlu.fm.intel.com ([10.105.23.75]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2025 13:39:35 -0700 From: stuartsummers To: Cc: matthew.brost@intel.com, farah.kassabri@intel.com, intel-xe@lists.freedesktop.org, Stuart Summers Subject: [PATCH 6/8] drm/xe: Prep TLB invalidation fence before sending Date: Thu, 7 Aug 2025 20:39:30 +0000 Message-Id: <20250807203932.103341-7-stuart.summers@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250807203932.103341-1-stuart.summers@intel.com> References: <20250807203932.103341-1-stuart.summers@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" From: Matthew Brost It is a bit backwards to add a TLB invalidation fence to the pending list after issuing the invalidation. Perform this step before issuing the TLB invalidation in a helper function. v2: Make sure the seqno_lock mutex covers the send as well (Matt) Signed-off-by: Matthew Brost Signed-off-by: Stuart Summers Reviewed-by: Stuart Summers --- drivers/gpu/drm/xe/xe_tlb_inval.c | 109 +++++++++++++++--------------- 1 file changed, 55 insertions(+), 54 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_tlb_inval.c b/drivers/gpu/drm/xe/xe_tlb_inval.c index 995699108bcb..c554a33b7bc3 100644 --- a/drivers/gpu/drm/xe/xe_tlb_inval.c +++ b/drivers/gpu/drm/xe/xe_tlb_inval.c @@ -65,19 +65,19 @@ __inval_fence_signal(struct xe_device *xe, struct xe_tlb_inval_fence *fence) static void inval_fence_signal(struct xe_device *xe, struct xe_tlb_inval_fence *fence) { + lockdep_assert_held(&fence->tlb_inval->pending_lock); + list_del(&fence->link); __inval_fence_signal(xe, fence); } -void xe_tlb_inval_fence_signal(struct xe_tlb_inval_fence *fence) +static void +inval_fence_signal_unlocked(struct xe_device *xe, + struct xe_tlb_inval_fence *fence) { - struct xe_gt *gt; - - if (WARN_ON_ONCE(!fence->tlb_inval)) - return; - - gt = fence->tlb_inval->private; - __inval_fence_signal(gt_to_xe(gt), fence); + spin_lock_irq(&fence->tlb_inval->pending_lock); + inval_fence_signal(xe, fence); + spin_unlock_irq(&fence->tlb_inval->pending_lock); } static void xe_gt_tlb_fence_timeout(struct work_struct *work) @@ -208,14 +208,10 @@ static bool tlb_inval_seqno_past(struct xe_gt *gt, int seqno) return seqno_recv >= seqno; } -static int send_tlb_inval(struct xe_guc *guc, - struct xe_tlb_inval_fence *fence, +static int send_tlb_inval(struct xe_guc *guc, struct xe_tlb_inval_fence *fence, u32 *action, int len) { struct xe_gt *gt = guc_to_gt(guc); - struct xe_device *xe = gt_to_xe(gt); - int seqno; - int ret; xe_gt_assert(gt, fence); @@ -225,47 +221,36 @@ static int send_tlb_inval(struct xe_guc *guc, * need to be updated. */ - mutex_lock(>->tlb_inval.seqno_lock); - seqno = gt->tlb_inval.seqno; - fence->seqno = seqno; - trace_xe_tlb_inval_fence_send(xe, fence); - action[1] = seqno; - ret = xe_guc_ct_send(&guc->ct, action, len, - G2H_LEN_DW_TLB_INVALIDATE, 1); - if (!ret) { - spin_lock_irq(>->tlb_inval.pending_lock); - /* - * We haven't actually published the TLB fence as per - * pending_fences, but in theory our seqno could have already - * been written as we acquired the pending_lock. In such a case - * we can just go ahead and signal the fence here. - */ - if (tlb_inval_seqno_past(gt, seqno)) { - __inval_fence_signal(xe, fence); - } else { - fence->inval_time = ktime_get(); - list_add_tail(&fence->link, - >->tlb_inval.pending_fences); - - if (list_is_singular(>->tlb_inval.pending_fences)) - queue_delayed_work(system_wq, - >->tlb_inval.fence_tdr, - tlb_timeout_jiffies(gt)); - } - spin_unlock_irq(>->tlb_inval.pending_lock); - } else { - __inval_fence_signal(xe, fence); - } - if (!ret) { - gt->tlb_inval.seqno = (gt->tlb_inval.seqno + 1) % - TLB_INVALIDATION_SEQNO_MAX; - if (!gt->tlb_inval.seqno) - gt->tlb_inval.seqno = 1; - } - mutex_unlock(>->tlb_inval.seqno_lock); xe_gt_stats_incr(gt, XE_GT_STATS_ID_TLB_INVAL, 1); + action[1] = fence->seqno; - return ret; + return xe_guc_ct_send(&guc->ct, action, len, + G2H_LEN_DW_TLB_INVALIDATE, 1); +} + +static void xe_tlb_inval_fence_prep(struct xe_tlb_inval_fence *fence) +{ + struct xe_tlb_inval *tlb_inval = fence->tlb_inval; + struct xe_gt *gt = tlb_inval->private; + struct xe_device *xe = gt_to_xe(gt); + + fence->seqno = tlb_inval->seqno; + trace_xe_tlb_inval_fence_send(xe, fence); + + spin_lock_irq(&tlb_inval->pending_lock); + fence->inval_time = ktime_get(); + list_add_tail(&fence->link, &tlb_inval->pending_fences); + + if (list_is_singular(&tlb_inval->pending_fences)) + queue_delayed_work(system_wq, + &tlb_inval->fence_tdr, + tlb_timeout_jiffies(gt)); + spin_unlock_irq(&tlb_inval->pending_lock); + + tlb_inval->seqno = (tlb_inval->seqno + 1) % + TLB_INVALIDATION_SEQNO_MAX; + if (!tlb_inval->seqno) + tlb_inval->seqno = 1; } #define MAKE_INVAL_OP(type) ((type << XE_GUC_TLB_INVAL_TYPE_SHIFT) | \ @@ -293,7 +278,14 @@ static int xe_tlb_inval_guc(struct xe_gt *gt, }; int ret; + mutex_lock(>->tlb_inval.seqno_lock); + xe_tlb_inval_fence_prep(fence); + ret = send_tlb_inval(>->uc.guc, fence, action, ARRAY_SIZE(action)); + if (ret < 0) + inval_fence_signal_unlocked(gt_to_xe(gt), fence); + mutex_unlock(>->tlb_inval.seqno_lock); + /* * -ECANCELED indicates the CT is stopped for a GT reset. TLB caches * should be nuked on a GT reset so this error can be ignored. @@ -420,7 +412,7 @@ int xe_tlb_inval_range(struct xe_tlb_inval *tlb_inval, #define MAX_TLB_INVALIDATION_LEN 7 u32 action[MAX_TLB_INVALIDATION_LEN]; u64 length = end - start; - int len = 0; + int len = 0, ret; xe_gt_assert(gt, fence); @@ -481,7 +473,16 @@ int xe_tlb_inval_range(struct xe_tlb_inval *tlb_inval, xe_gt_assert(gt, len <= MAX_TLB_INVALIDATION_LEN); - return send_tlb_inval(>->uc.guc, fence, action, len); + mutex_lock(>->tlb_inval.seqno_lock); + xe_tlb_inval_fence_prep(fence); + + ret = send_tlb_inval(>->uc.guc, fence, action, + ARRAY_SIZE(action)); + if (ret < 0) + inval_fence_signal_unlocked(xe, fence); + mutex_unlock(>->tlb_inval.seqno_lock); + + return ret; } /** -- 2.34.1