[Intel-gfx] [PATCH v8 0/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Intel-gfx] [PATCH v8 0/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Jonathan Cavitt]

Implement GuC-based TLB invalidations and use them on MTL.

v2:
- Add missing supporting patches.

v3:
- Split suspend/resume changes and multi-gt support into separate
  patches.
- Only perform GuC TLB invalidation functions when supported.
- Move intel_guc_is_enabled check function to usage location.
- Address comments.

v4:
- Change conditions for GuC-based tlb invalidation support
  to a pci tag that's only active for MTL.
- Address some FIXMEs and formatting issues.
- Move suspend/resume changes to helper functions in intel_gt.h
- Improve comment for ct_handle_event change.
- Use cleaner if-else conditions.
- Address comments.

v5:
- Reintroduce missing change to selftest msleep duration
- Move suspend/resume loops from intel_gt.h to intel_tlb.c,
  making them no longer static inlines.
- Remove superfluous blocking and error checks.
- Move ct_handle_event exception to general case in
  ct_process_request.
- Explain usage of xa_alloc_cyclic_irq.
- Modify explanation of purpose of
  OUTSTANDING_GUC_TIMEOUT_PERIOD macro.
- Explain purpose of performing tlb invalidation twice in
  intel_gt_tlb_resume_all.

v6:
- Add this cover letter.
- Fix explanation of purpose of
  OUTSTANDING_GUC_TIMEOUT_PERIOD macro again.
- s/pci tags/pci flags
- Enable GuC TLB Invalidations separately from adding the
  flags to do so.

v7:
- Eliminate pci terminology from patches.
- Order new device info flag correctly.
- Run gen8_ggtt_invalidate in more cases, specifically when
  GuC-based TLB invalidation is not supported.
- Use intel_uncore_write_fw instead of intel_uncore_write
  during guc_ggtt_invalidate.
- Remove duplicate request message clear in ct_process_request.
- Remove faulty tag from series.

v8:
- Simplify cover letter contents.
- Fix miscellaneous formatting and typos.
- Reorder device info flags and defines.
- Reword commit message.
- Rename TLB invalidation enums and functions.
- Add comments explaining confusing points.
- Add helper function getting expected delay of CT buffer.
- Simplify intel_guc_tlb_invalidation_done by passing computed
  values.
- Remove helper functions for tlb suspend and resume.
- Move tlb suspend and resume paths to uc.
- Split suspend/resume and wedged into two patches.
- Clarify purpose of sleep change in tlb selftest.

Jonathan Cavitt (6):
  drm/i915: Add GuC TLB Invalidation device info flags
  drm/i915/guc: Add CT size delay helper
  drm/i915: No TLB invalidation on suspended GT
  drm/i915: No TLB invalidation on wedged GT
  drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
  drm/i915: Enable GuC TLB invalidations for MTL

Prathap Kumar Valsan (1):
  drm/i915: Define and use GuC and CTB TLB invalidation routines

 drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
 drivers/gpu/drm/i915/gt/intel_gt.h            |   1 +
 drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
 drivers/gpu/drm/i915/gt/selftest_tlb.c        |  11 +-
 .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
 drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  23 ++
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h     |  13 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 225 +++++++++++++++++-
 drivers/gpu/drm/i915/gt/uc/intel_uc.c         |   8 +
 drivers/gpu/drm/i915/i915_driver.c            |   2 +
 drivers/gpu/drm/i915/i915_drv.h               |   3 +-
 drivers/gpu/drm/i915/i915_pci.c               |   1 +
 drivers/gpu/drm/i915/intel_device_info.h      |   1 +
 15 files changed, 360 insertions(+), 15 deletions(-)

-- 
2.25.1

[Intel-gfx] [PATCH v8 1/7] drm/i915: Add GuC TLB Invalidation device info flags

[Jonathan Cavitt]

Add device info flags for if GuC TLB Invalidation is enabled.

Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
---
 drivers/gpu/drm/i915/i915_drv.h          | 3 ++-
 drivers/gpu/drm/i915/intel_device_info.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index cb60fc9cf8737..b7933e1120aa7 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -794,11 +794,12 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
 #define HAS_GUC_DEPRIVILEGE(i915) \
 	(INTEL_INFO(i915)->has_guc_deprivilege)
 
+#define HAS_GUC_TLB_INVALIDATION(i915)	(INTEL_INFO(i915)->has_guc_tlb_invalidation)
+
 #define HAS_3D_PIPELINE(i915)	(INTEL_INFO(i915)->has_3d_pipeline)
 
 #define HAS_ONE_EU_PER_FUSE_BIT(i915)	(INTEL_INFO(i915)->has_one_eu_per_fuse_bit)
 
 #define HAS_LMEMBAR_SMEM_STOLEN(i915) (!HAS_LMEM(i915) && \
 				       GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
-
 #endif
diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h
index 39817490b13fd..eba2f0b919c87 100644
--- a/drivers/gpu/drm/i915/intel_device_info.h
+++ b/drivers/gpu/drm/i915/intel_device_info.h
@@ -153,6 +153,7 @@ enum intel_ppgtt_type {
 	func(has_heci_pxp); \
 	func(has_heci_gscfi); \
 	func(has_guc_deprivilege); \
+	func(has_guc_tlb_invalidation); \
 	func(has_l3_ccs_read); \
 	func(has_l3_dpf); \
 	func(has_llc); \
-- 
2.25.1

[Intel-gfx] [PATCH v8 2/7] drm/i915/guc: Add CT size delay helper

[Jonathan Cavitt]

Add a helper function to the GuC CT buffer that reports the expected
time to process all outstanding requests.  As of now, there is no
functionality to check number of requests in the buffer, so the helper
function just reports 2 seconds, or 1ms per request up to the maximum
number of requests the CT buffer can store.

Suggested-by: John Harrison <john.c.harrison@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
---
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
index 58e42901ff498..36afc1ce9fabd 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
@@ -120,6 +120,19 @@ static inline bool intel_guc_ct_enabled(struct intel_guc_ct *ct)
 	return ct->enabled;
 }
 
+/*
+ * GuC has a timeout of 1ms for a TLB invalidation response from GAM.  On a
+ * timeout GuC drops the request and has no mechanism to notify the host about
+ * the timeout.  There is also no mechanism for determining the number of
+ * outstanding requests in the CT buffer.  Ergo, keep a larger timeout that accounts
+ * for this individual timeout and the max number of outstanding requests that
+ * can be queued in CT buffer.
+ */
+static inline long intel_guc_ct_expected_delay(struct intel_guc_ct *ct)
+{
+	return HZ * 2;
+}
+
 #define INTEL_GUC_CT_SEND_NB		BIT(31)
 #define INTEL_GUC_CT_SEND_G2H_DW_SHIFT	0
 #define INTEL_GUC_CT_SEND_G2H_DW_MASK	(0xff << INTEL_GUC_CT_SEND_G2H_DW_SHIFT)
-- 
2.25.1

[Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Jonathan Cavitt]

From: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>

The GuC firmware had defined the interface for Translation Look-Aside
Buffer (TLB) invalidation.  We should use this interface when
invalidating the engine and GuC TLBs.
Add additional functionality to intel_gt_invalidate_tlb, invalidating
the GuC TLBs and falling back to GT invalidation when the GuC is
disabled.
The invalidation is done by sending a request directly to the GuC
tlb_lookup that invalidates the table.  The invalidation is submitted as
a wait request and is performed in the CT event handler.  This means we
cannot perform this TLB invalidation path if the CT is not enabled.
If the request isn't fulfilled in two seconds, this would constitute
an error in the invalidation as that would constitute either a lost
request or a severe GuC overload.

With this new invalidation routine, we can perform GuC-based GGTT
invalidations.  GuC-based GGTT invalidation is incompatible with
MMIO invalidation so we should not perform MMIO invalidation when
GuC-based GGTT invalidation is expected.

Purpose of xarray:
The tlb_lookup table is allocated as an xarray because the set of
pending TLB invalidations may have no upper bound.  The consequence of
this is that all actions interfacing with this table need to use the
xarray functions, such as xa_alloc_cyclic_irq for array insertion.

Purpose of must_wait_woken:
Our wait for the G2H ack for the completion of a TLB invalidation is
mandatory; we must wait for the HW to confirm that the physical
addresses are no longer accessible before we return those to the system.

On switching to using the wait_woken() convenience routine, we
introduced ourselves to an issue where wait_woken() may complete early
under a kthread that is stopped. Since we send a TLB invalidation when
we try to release pages from the shrinker, we can be called from any
process; including kthreads.

Using wait_woken() from any process context causes another issue. The
use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
set was made by kthread_create() and has called set_kthread_struct().
This is not true for the raw kernel_thread():

BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 3089.759660] Call Trace:
[ 3089.762110]  wait_woken+0x4f/0x80
[ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
[ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
[ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
[ 3089.779430]  ? __switch_to_asm+0x40/0x70
[ 3089.783349]  ? __switch_to_asm+0x34/0x70
[ 3089.787273]  ? __switch_to+0x7a/0x3e0
[ 3089.790930]  ? __switch_to_asm+0x34/0x70
[ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
[ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
[ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
[ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
[ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
[ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
[ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
[ 3089.832230]  do_shrink_slab+0x12c/0x2a0
[ 3089.836065]  shrink_slab+0xad/0x2b0
[ 3089.839550]  shrink_node+0xcc/0x410
[ 3089.843035]  do_try_to_free_pages+0xc6/0x380
[ 3089.847306]  try_to_free_pages+0xec/0x1c0
[ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
[ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
[ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
[ 3089.864737]  new_slab+0x265/0xa80
[ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
[ 3089.871798]  ? copy_process+0x1e5/0x1a00
[ 3089.875717]  ? load_balance+0x165/0xb20
[ 3089.879555]  __slab_alloc+0x1c/0x30
[ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
[ 3089.887397]  ? copy_process+0x1e5/0x1a00
[ 3089.891314]  copy_process+0x1e5/0x1a00
[ 3089.895058]  ? __switch_to_asm+0x40/0x70
[ 3089.879555]  __slab_alloc+0x1c/0x30
[ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
[ 3089.887397]  ? copy_process+0x1e5/0x1a00
[ 3089.891314]  copy_process+0x1e5/0x1a00
[ 3089.895058]  ? __switch_to_asm+0x40/0x70
[ 3089.898977]  ? __switch_to_asm+0x34/0x70
[ 3089.902903]  ? __switch_to_asm+0x40/0x70
[ 3089.906828]  ? __switch_to_asm+0x34/0x70
[ 3089.910745]  _do_fork+0x83/0x350
[ 3089.913969]  ? __switch_to+0x7a/0x3e0
[ 3089.917626]  ? __switch_to_asm+0x34/0x70
[ 3089.921545]  kernel_thread+0x58/0x80
[ 3089.925124]  ? kthread_park+0x80/0x80
[ 3089.928788]  kthreadd+0x162/0x1b0
[ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
[ 3089.936538]  ret_from_fork+0x1f/0x40

Signed-off-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
Signed-off-by: Bruce Chang <yu.bruce.chang@intel.com>
Signed-off-by: Chris Wilson <chris.p.wilson@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@intel.com>
Signed-off-by: Fei Yang <fei.yang@intel.com>
CC: Andi Shyti <andi.shyti@linux.intel.com>
---
 drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
 drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
 .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
 drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
 drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
 7 files changed, 297 insertions(+), 12 deletions(-)

diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
index 4d7d88b92632b..a1f7bdc602996 100644
--- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
+++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
@@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct i915_ggtt *ggtt)
 	intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
 }
 
+static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
+{
+	struct intel_uncore *uncore = gt->uncore;
+	intel_wakeref_t wakeref;
+
+	with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
+		struct intel_guc *guc = &gt->uc.guc;
+
+		intel_guc_invalidate_tlb_guc(guc);
+	}
+}
+
 static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
 {
 	struct drm_i915_private *i915 = ggtt->vm.i915;
+	struct intel_gt *gt;
 
-	gen8_ggtt_invalidate(ggtt);
-
-	if (GRAPHICS_VER(i915) >= 12) {
-		struct intel_gt *gt;
+	if (!HAS_GUC_TLB_INVALIDATION(i915))
+		gen8_ggtt_invalidate(ggtt);
 
-		list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
+	list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
+		if (HAS_GUC_TLB_INVALIDATION(i915) &&
+		    intel_guc_is_ready(&gt->uc.guc)) {
+			guc_ggtt_ct_invalidate(gt);
+		} else if (GRAPHICS_VER(i915) >= 12) {
 			intel_uncore_write_fw(gt->uncore,
 					      GEN12_GUC_TLB_INV_CR,
 					      GEN12_GUC_TLB_INV_CR_INVALIDATE);
-	} else {
-		intel_uncore_write_fw(ggtt->vm.gt->uncore,
-				      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
+		} else {
+			intel_uncore_write_fw(gt->uncore,
+					      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
+		}
 	}
 }
 
@@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
 		ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
 	}
 
-	if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
+	if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
 		ggtt->invalidate = guc_ggtt_invalidate;
 	else
 		ggtt->invalidate = gen8_ggtt_invalidate;
diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c
index 139608c30d978..1278622b9554d 100644
--- a/drivers/gpu/drm/i915/gt/intel_tlb.c
+++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
@@ -12,6 +12,7 @@
 #include "intel_gt_print.h"
 #include "intel_gt_regs.h"
 #include "intel_tlb.h"
+#include "uc/intel_guc.h"
 
 /*
  * HW architecture suggest typical invalidation time at 40us,
@@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
 		return;
 
 	with_intel_gt_pm_if_awake(gt, wakeref) {
+		struct intel_guc *guc = &gt->uc.guc;
+
 		mutex_lock(&gt->tlb.invalidate_lock);
 		if (tlb_seqno_passed(gt, seqno))
 			goto unlock;
 
-		mmio_invalidate_full(gt);
+		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
+			/*
+			 * Only perform GuC TLB invalidation if GuC is ready.
+			 * If GuC is not ready, then there are no TLBs to
+			 * invalidate.  Ergo, skip invalidation.
+			 */
+			if (intel_guc_is_ready(guc))
+				intel_guc_invalidate_tlb_engines(guc);
+		} else {
+			mmio_invalidate_full(gt);
+		}
 
 		write_seqcount_invalidate(&gt->tlb.seqno);
 unlock:
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
index f359bef046e0b..33f253410d0c8 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
@@ -138,6 +138,8 @@ enum intel_guc_action {
 	INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
 	INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
 	INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
+	INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
+	INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
 	INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
 	INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
 	INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
@@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
 
 #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK      0x000000FF
 
+#define INTEL_GUC_TLB_INVAL_TYPE_MASK	REG_GENMASK(7, 0)
+#define INTEL_GUC_TLB_INVAL_MODE_MASK	REG_GENMASK(11, 8)
+#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
+
+enum intel_guc_tlb_invalidation_type {
+	INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
+	INTEL_GUC_TLB_INVAL_GUC = 0x3,
+};
+
+/*
+ * 0: Heavy mode of Invalidation:
+ * The pipeline of the engine(s) for which the invalidation is targeted to is
+ * blocked, and all the in-flight transactions are guaranteed to be Globally
+ * Observed before completing the TLB invalidation
+ * 1: Lite mode of Invalidation:
+ * TLBs of the targeted engine(s) are immediately invalidated.
+ * In-flight transactions are NOT guaranteed to be Globally Observed before
+ * completing TLB invalidation.
+ * Light Invalidation Mode is to be used only when
+ * it can be guaranteed (by SW) that the address translations remain invariant
+ * for the in-flight transactions across the TLB invalidation. In other words,
+ * this mode can be used when the TLB invalidation is intended to clear out the
+ * stale cached translations that are no longer in use. Light Invalidation Mode
+ * is much faster than the Heavy Invalidation Mode, as it does not wait for the
+ * in-flight transactions to be GOd.
+ */
+enum intel_guc_tlb_inval_mode {
+	INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
+	INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
+};
+
 #endif /* _ABI_GUC_ACTIONS_ABI_H */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 6c392bad29c19..88deb43bbdc48 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -79,6 +79,18 @@ struct intel_guc {
 	 */
 	atomic_t outstanding_submission_g2h;
 
+	/** @tlb_lookup: xarray to store all pending TLB invalidation requests */
+	struct xarray tlb_lookup;
+
+	/**
+	 * @serial_slot: id to the initial waiter created in tlb_lookup,
+	 * which is used only when failed to allocate new waiter.
+	 */
+	u32 serial_slot;
+
+	/** @next_seqno: the next id (sequence no.) to allocate. */
+	u32 next_seqno;
+
 	/** @interrupts: pointers to GuC interrupt-managing functions. */
 	struct {
 		bool enabled;
@@ -296,6 +308,11 @@ struct intel_guc {
 #define MAKE_GUC_VER_STRUCT(ver)	MAKE_GUC_VER((ver).major, (ver).minor, (ver).patch)
 #define GUC_SUBMIT_VER(guc)		MAKE_GUC_VER_STRUCT((guc)->submission_version)
 
+struct intel_guc_tlb_wait {
+	struct wait_queue_head wq;
+	bool busy;
+};
+
 static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
 {
 	return container_of(log, struct intel_guc, log);
@@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct intel_guc *guc)
 	return intel_uc_fw_is_supported(&guc->fw);
 }
 
+int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
+int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
+int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
+				    u32 len, u32 seqno);
+
 static inline bool intel_guc_is_wanted(struct intel_guc *guc)
 {
 	return intel_uc_fw_is_enabled(&guc->fw);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
index 6e22af31513a5..e7ac8402245b7 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
@@ -1115,6 +1115,9 @@ static int ct_process_request(struct intel_guc_ct *ct, struct ct_incoming_msg *r
 	case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
 		ret = intel_guc_crash_process_msg(guc, action);
 		break;
+	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
+		ret = intel_guc_tlb_invalidation_done(guc, request->size, len, payload[0]);
+		break;
 	default:
 		ret = -EOPNOTSUPP;
 		break;
@@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct *ct, struct ct_incoming_msg *requ
 	switch (action) {
 	case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
 	case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
+	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
 		g2h_release_space(ct, request->size);
 	}
 
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index b4d56eccfb1f0..a7c9874e122a3 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -22,6 +22,7 @@
 /* Payload length only i.e. don't include G2H header length */
 #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET	2
 #define G2H_LEN_DW_DEREGISTER_CONTEXT		1
+#define G2H_LEN_DW_INVALIDATE_TLB		1
 
 #define GUC_CONTEXT_DISABLE		0
 #define GUC_CONTEXT_ENABLE		1
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 2cce5ec1ff00d..0c5ccda1b3e87 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
 
 void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
 {
+	struct intel_guc_tlb_wait *wait;
 	struct intel_context *ce;
 	unsigned long index;
 	unsigned long flags;
+	unsigned long i;
 
 	if (unlikely(!guc_submission_initialized(guc))) {
 		/* Reset called during driver load? GuC not yet initialised! */
@@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
 
 	/* GuC is blown away, drop all references to contexts */
 	xa_destroy(&guc->context_lookup);
+
+	/*
+	 * The full GT reset will have cleared the TLB caches and flushed the
+	 * G2H message queue; we can release all the blocked waiters.
+	 *
+	 * This is safe to do unlocked because the xarray is not dependent
+	 * on the GT reset, and there's a separate execution path for TLB
+	 * invalidations on GT reset, and there's a large window of time
+	 * between the GT reset and GuC becoming available.
+	 */
+	xa_for_each(&guc->tlb_lookup, i, wait)
+		wake_up(&wait->wq);
 }
 
 static void guc_cancel_context_requests(struct intel_context *ce)
@@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
 static void destroyed_worker_func(struct work_struct *w);
 static void reset_fail_worker_func(struct work_struct *w);
 
+static int init_tlb_lookup(struct intel_guc *guc)
+{
+	struct intel_guc_tlb_wait *wait;
+	int err;
+
+	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
+		return 0;
+
+	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
+
+	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
+	if (!wait)
+		return -ENOMEM;
+
+	init_waitqueue_head(&wait->wq);
+
+	/* Preallocate a shared id for use under memory pressure. */
+	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
+				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
+	/* Only error if out of memory, not when busy (list full)*/
+	if (err == -ENOMEM) {
+		kfree(wait);
+		return err;
+	}
+
+	return 0;
+}
+
+static void fini_tlb_lookup(struct intel_guc *guc)
+{
+	struct intel_guc_tlb_wait *wait;
+
+	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
+		return;
+
+	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
+	if (wait) {
+		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
+		kfree(wait);
+	}
+
+	xa_destroy(&guc->tlb_lookup);
+}
+
 /*
  * Set up the memory resources to be shared with the GuC (via the GGTT)
  * at firmware loading time.
@@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct intel_guc *guc)
 			return ret;
 	}
 
+	ret = init_tlb_lookup(guc);
+	if (ret)
+		goto destroy_pool;
+
 	guc->submission_state.guc_ids_bitmap =
 		bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
 	if (!guc->submission_state.guc_ids_bitmap) {
 		ret = -ENOMEM;
-		goto destroy_pool;
+		goto destroy_tlb;
 	}
 
 	guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
@@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc *guc)
 
 	return 0;
 
+destroy_tlb:
+	fini_tlb_lookup(guc);
 destroy_pool:
 	guc_lrc_desc_pool_destroy_v69(guc);
-
 	return ret;
 }
 
@@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
 	guc_lrc_desc_pool_destroy_v69(guc);
 	i915_sched_engine_put(guc->sched_engine);
 	bitmap_free(guc->submission_state.guc_ids_bitmap);
+	fini_tlb_lookup(guc);
 	guc->submission_initialized = false;
 }
 
@@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id)
 	return ce;
 }
 
+static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno)
+{
+	struct intel_guc_tlb_wait *wait;
+	unsigned long flags;
+
+	xa_lock_irqsave(&guc->tlb_lookup, flags);
+	wait = xa_load(&guc->tlb_lookup, seqno);
+
+	if (wait)
+		wake_up(&wait->wq);
+	else
+		guc_dbg(guc,
+			"Stale TLB invalidation response with seqno %d\n", seqno);
+
+	xa_unlock_irqrestore(&guc->tlb_lookup, flags);
+}
+
+int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
+{
+	/* Check for underflow */
+	if (unlikely(len < 1 || len > size))
+		return -EPROTO;
+
+	wait_wake_outstanding_tlb_g2h(guc, seqno);
+	return 0;
+}
+
+static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
+{
+	/*
+	 * This is equivalent to wait_woken() with the exception that
+	 * we do not wake up early if the kthread task has been completed.
+	 * As we are called from page reclaim in any task context,
+	 * we may be invoked from stopped kthreads, but we *must*
+	 * complete the wait from the HW .
+	 *
+	 * A second problem is that since we are called under reclaim
+	 * and wait_woken() inspected the thread state, it makes an invalid
+	 * assumption that all PF_KTHREAD tasks have set_kthread_struct()
+	 * called upon them, and will trigger a GPF in is_kthread_should_stop().
+	 */
+	do {
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		if (wq_entry->flags & WQ_FLAG_WOKEN)
+			break;
+
+		timeout = schedule_timeout(timeout);
+	} while (timeout);
+	__set_current_state(TASK_RUNNING);
+
+	/* See wait_woken() and woken_wake_function() */
+	smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
+
+	return timeout;
+}
+
+static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)
+{
+	struct intel_guc_tlb_wait _wq, *wq = &_wq;
+	DEFINE_WAIT_FUNC(wait, woken_wake_function);
+	int err;
+	u32 seqno;
+	u32 action[] = {
+		INTEL_GUC_ACTION_TLB_INVALIDATION,
+		0,
+		REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
+			REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
+				       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
+			INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
+	};
+	u32 size = ARRAY_SIZE(action);
+
+	init_waitqueue_head(&_wq.wq);
+
+	if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
+				xa_limit_32b, &guc->next_seqno,
+				GFP_ATOMIC | __GFP_NOWARN) < 0) {
+		/* Under severe memory pressure? Serialise TLB allocations */
+		xa_lock_irq(&guc->tlb_lookup);
+		wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
+		wait_event_lock_irq(wq->wq,
+				    !READ_ONCE(wq->busy),
+				    guc->tlb_lookup.xa_lock);
+		/*
+		 * Update wq->busy under lock to ensure only one waiter can
+		 * issue the TLB invalidation command using the serial slot at a
+		 * time. The condition is set to true before releasing the lock
+		 * so that other caller continue to wait until woken up again.
+		 */
+		wq->busy = true;
+		xa_unlock_irq(&guc->tlb_lookup);
+
+		seqno = guc->serial_slot;
+	}
+
+	action[1] = seqno;
+
+	add_wait_queue(&wq->wq, &wait);
+
+	/*
+	 * This is a critical reclaim path and thus we must loop here:
+	 * We cannot block for anything that is on the GPU.
+	 */
+	err = intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE_TLB, true);
+	if (err)
+		goto out;
+
+	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
+		guc_err(guc,
+			"TLB invalidation response timed out for seqno %u\n", seqno);
+		err = -ETIME;
+	}
+out:
+	remove_wait_queue(&wq->wq, &wait);
+	if (seqno != guc->serial_slot)
+		xa_erase_irq(&guc->tlb_lookup, seqno);
+
+	return err;
+}
+
+/* Full TLB invalidation */
+int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
+{
+	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
+}
+
+/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
+int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
+{
+	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
+}
+
 int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
 					  const u32 *msg,
 					  u32 len)
-- 
2.25.1

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Tvrtko Ursulin]

On 06/10/2023 19:20, Jonathan Cavitt wrote:
> From: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
> 
> The GuC firmware had defined the interface for Translation Look-Aside
> Buffer (TLB) invalidation.  We should use this interface when
> invalidating the engine and GuC TLBs.
> Add additional functionality to intel_gt_invalidate_tlb, invalidating
> the GuC TLBs and falling back to GT invalidation when the GuC is
> disabled.
> The invalidation is done by sending a request directly to the GuC
> tlb_lookup that invalidates the table.  The invalidation is submitted as
> a wait request and is performed in the CT event handler.  This means we
> cannot perform this TLB invalidation path if the CT is not enabled.
> If the request isn't fulfilled in two seconds, this would constitute
> an error in the invalidation as that would constitute either a lost
> request or a severe GuC overload.
> 
> With this new invalidation routine, we can perform GuC-based GGTT
> invalidations.  GuC-based GGTT invalidation is incompatible with
> MMIO invalidation so we should not perform MMIO invalidation when
> GuC-based GGTT invalidation is expected.
> 
> Purpose of xarray:
> The tlb_lookup table is allocated as an xarray because the set of
> pending TLB invalidations may have no upper bound.  The consequence of
> this is that all actions interfacing with this table need to use the
> xarray functions, such as xa_alloc_cyclic_irq for array insertion.
> 
> Purpose of must_wait_woken:
> Our wait for the G2H ack for the completion of a TLB invalidation is
> mandatory; we must wait for the HW to confirm that the physical
> addresses are no longer accessible before we return those to the system.
> 
> On switching to using the wait_woken() convenience routine, we
> introduced ourselves to an issue where wait_woken() may complete early
> under a kthread that is stopped. Since we send a TLB invalidation when
> we try to release pages from the shrinker, we can be called from any
> process; including kthreads.
> 
> Using wait_woken() from any process context causes another issue. The
> use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
> set was made by kthread_create() and has called set_kthread_struct().
> This is not true for the raw kernel_thread():

This explanation misses the main point of my ask - which is to explain 
why a simpler scheme isn't sufficient. Simpler scheme aka not needed the 
xarray or any flavour of wait_token().

In other words it is obvious we have to wait for the invalidation ack, 
but not obvious why we need a complicated scheme.

> BUG: kernel NULL pointer dereference, address: 0000000000000000
> [ 3089.759660] Call Trace:
> [ 3089.762110]  wait_woken+0x4f/0x80
> [ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
> [ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
> [ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
> [ 3089.779430]  ? __switch_to_asm+0x40/0x70
> [ 3089.783349]  ? __switch_to_asm+0x34/0x70
> [ 3089.787273]  ? __switch_to+0x7a/0x3e0
> [ 3089.790930]  ? __switch_to_asm+0x34/0x70
> [ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
> [ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
> [ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
> [ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
> [ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
> [ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
> [ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
> [ 3089.832230]  do_shrink_slab+0x12c/0x2a0
> [ 3089.836065]  shrink_slab+0xad/0x2b0
> [ 3089.839550]  shrink_node+0xcc/0x410
> [ 3089.843035]  do_try_to_free_pages+0xc6/0x380
> [ 3089.847306]  try_to_free_pages+0xec/0x1c0
> [ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
> [ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
> [ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
> [ 3089.864737]  new_slab+0x265/0xa80
> [ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
> [ 3089.871798]  ? copy_process+0x1e5/0x1a00
> [ 3089.875717]  ? load_balance+0x165/0xb20
> [ 3089.879555]  __slab_alloc+0x1c/0x30
> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> [ 3089.891314]  copy_process+0x1e5/0x1a00
> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> [ 3089.879555]  __slab_alloc+0x1c/0x30
> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> [ 3089.891314]  copy_process+0x1e5/0x1a00
> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> [ 3089.898977]  ? __switch_to_asm+0x34/0x70
> [ 3089.902903]  ? __switch_to_asm+0x40/0x70
> [ 3089.906828]  ? __switch_to_asm+0x34/0x70
> [ 3089.910745]  _do_fork+0x83/0x350
> [ 3089.913969]  ? __switch_to+0x7a/0x3e0
> [ 3089.917626]  ? __switch_to_asm+0x34/0x70
> [ 3089.921545]  kernel_thread+0x58/0x80
> [ 3089.925124]  ? kthread_park+0x80/0x80
> [ 3089.928788]  kthreadd+0x162/0x1b0
> [ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
> [ 3089.936538]  ret_from_fork+0x1f/0x40
> 
> Signed-off-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
> Signed-off-by: Bruce Chang <yu.bruce.chang@intel.com>
> Signed-off-by: Chris Wilson <chris.p.wilson@intel.com>
> Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
> Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@intel.com>
> Signed-off-by: Fei Yang <fei.yang@intel.com>
> CC: Andi Shyti <andi.shyti@linux.intel.com>
> ---
>   drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
>   drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
>   .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
>   drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
>   drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
>   7 files changed, 297 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> index 4d7d88b92632b..a1f7bdc602996 100644
> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> @@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct i915_ggtt *ggtt)
>   	intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
>   }
>   
> +static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
> +{
> +	struct intel_uncore *uncore = gt->uncore;
> +	intel_wakeref_t wakeref;
> +
> +	with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
> +		struct intel_guc *guc = &gt->uc.guc;
> +
> +		intel_guc_invalidate_tlb_guc(guc);
> +	}
> +}
> +
>   static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
>   {
>   	struct drm_i915_private *i915 = ggtt->vm.i915;
> +	struct intel_gt *gt;
>   
> -	gen8_ggtt_invalidate(ggtt);
> -
> -	if (GRAPHICS_VER(i915) >= 12) {
> -		struct intel_gt *gt;
> +	if (!HAS_GUC_TLB_INVALIDATION(i915))
> +		gen8_ggtt_invalidate(ggtt);
>   
> -		list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
> +	list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
> +		if (HAS_GUC_TLB_INVALIDATION(i915) &&
> +		    intel_guc_is_ready(&gt->uc.guc)) {
> +			guc_ggtt_ct_invalidate(gt);
> +		} else if (GRAPHICS_VER(i915) >= 12) {
>   			intel_uncore_write_fw(gt->uncore,
>   					      GEN12_GUC_TLB_INV_CR,
>   					      GEN12_GUC_TLB_INV_CR_INVALIDATE);
> -	} else {
> -		intel_uncore_write_fw(ggtt->vm.gt->uncore,
> -				      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> +		} else {
> +			intel_uncore_write_fw(gt->uncore,
> +					      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> +		}
>   	}
>   }
>   
> @@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
>   		ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
>   	}
>   
> -	if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
> +	if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
>   		ggtt->invalidate = guc_ggtt_invalidate;
>   	else
>   		ggtt->invalidate = gen8_ggtt_invalidate;
> diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c
> index 139608c30d978..1278622b9554d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_tlb.c
> +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
> @@ -12,6 +12,7 @@
>   #include "intel_gt_print.h"
>   #include "intel_gt_regs.h"
>   #include "intel_tlb.h"
> +#include "uc/intel_guc.h"
>   
>   /*
>    * HW architecture suggest typical invalidation time at 40us,
> @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
>   		return;
>   
>   	with_intel_gt_pm_if_awake(gt, wakeref) {
> +		struct intel_guc *guc = &gt->uc.guc;
> +
>   		mutex_lock(&gt->tlb.invalidate_lock);
>   		if (tlb_seqno_passed(gt, seqno))
>   			goto unlock;
>   
> -		mmio_invalidate_full(gt);
> +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
> +			/*
> +			 * Only perform GuC TLB invalidation if GuC is ready.
> +			 * If GuC is not ready, then there are no TLBs to
> +			 * invalidate.  Ergo, skip invalidation.
> +			 */
> +			if (intel_guc_is_ready(guc))
> +				intel_guc_invalidate_tlb_engines(guc);

What was the answer to John's question on why it is okay to just skip 
and not maybe fall back to mmio?

> +		} else {
> +			mmio_invalidate_full(gt);
> +		}
>   
>   		write_seqcount_invalidate(&gt->tlb.seqno);
>   unlock:
> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> index f359bef046e0b..33f253410d0c8 100644
> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> @@ -138,6 +138,8 @@ enum intel_guc_action {
>   	INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
>   	INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
>   	INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
> +	INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
> +	INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
>   	INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
>   	INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
>   	INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
> @@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
>   
>   #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK      0x000000FF
>   
> +#define INTEL_GUC_TLB_INVAL_TYPE_MASK	REG_GENMASK(7, 0)
> +#define INTEL_GUC_TLB_INVAL_MODE_MASK	REG_GENMASK(11, 8)
> +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
> +
> +enum intel_guc_tlb_invalidation_type {
> +	INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
> +	INTEL_GUC_TLB_INVAL_GUC = 0x3,
> +};
> +
> +/*
> + * 0: Heavy mode of Invalidation:
> + * The pipeline of the engine(s) for which the invalidation is targeted to is
> + * blocked, and all the in-flight transactions are guaranteed to be Globally
> + * Observed before completing the TLB invalidation
> + * 1: Lite mode of Invalidation:
> + * TLBs of the targeted engine(s) are immediately invalidated.
> + * In-flight transactions are NOT guaranteed to be Globally Observed before
> + * completing TLB invalidation.
> + * Light Invalidation Mode is to be used only when
> + * it can be guaranteed (by SW) that the address translations remain invariant
> + * for the in-flight transactions across the TLB invalidation. In other words,
> + * this mode can be used when the TLB invalidation is intended to clear out the
> + * stale cached translations that are no longer in use. Light Invalidation Mode
> + * is much faster than the Heavy Invalidation Mode, as it does not wait for the
> + * in-flight transactions to be GOd.
> + */
> +enum intel_guc_tlb_inval_mode {
> +	INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
> +	INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
> +};
> +
>   #endif /* _ABI_GUC_ACTIONS_ABI_H */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> index 6c392bad29c19..88deb43bbdc48 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> @@ -79,6 +79,18 @@ struct intel_guc {
>   	 */
>   	atomic_t outstanding_submission_g2h;
>   
> +	/** @tlb_lookup: xarray to store all pending TLB invalidation requests */
> +	struct xarray tlb_lookup;
> +
> +	/**
> +	 * @serial_slot: id to the initial waiter created in tlb_lookup,
> +	 * which is used only when failed to allocate new waiter.
> +	 */
> +	u32 serial_slot;
> +
> +	/** @next_seqno: the next id (sequence no.) to allocate. */
> +	u32 next_seqno;
> +
>   	/** @interrupts: pointers to GuC interrupt-managing functions. */
>   	struct {
>   		bool enabled;
> @@ -296,6 +308,11 @@ struct intel_guc {
>   #define MAKE_GUC_VER_STRUCT(ver)	MAKE_GUC_VER((ver).major, (ver).minor, (ver).patch)
>   #define GUC_SUBMIT_VER(guc)		MAKE_GUC_VER_STRUCT((guc)->submission_version)
>   
> +struct intel_guc_tlb_wait {
> +	struct wait_queue_head wq;
> +	bool busy;
> +};
> +
>   static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
>   {
>   	return container_of(log, struct intel_guc, log);
> @@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct intel_guc *guc)
>   	return intel_uc_fw_is_supported(&guc->fw);
>   }
>   
> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
> +				    u32 len, u32 seqno);
> +
>   static inline bool intel_guc_is_wanted(struct intel_guc *guc)
>   {
>   	return intel_uc_fw_is_enabled(&guc->fw);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> index 6e22af31513a5..e7ac8402245b7 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> @@ -1115,6 +1115,9 @@ static int ct_process_request(struct intel_guc_ct *ct, struct ct_incoming_msg *r
>   	case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
>   		ret = intel_guc_crash_process_msg(guc, action);
>   		break;
> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
> +		ret = intel_guc_tlb_invalidation_done(guc, request->size, len, payload[0]);
> +		break;
>   	default:
>   		ret = -EOPNOTSUPP;
>   		break;
> @@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct *ct, struct ct_incoming_msg *requ
>   	switch (action) {
>   	case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
>   	case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>   		g2h_release_space(ct, request->size);
>   	}
>   
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> index b4d56eccfb1f0..a7c9874e122a3 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> @@ -22,6 +22,7 @@
>   /* Payload length only i.e. don't include G2H header length */
>   #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET	2
>   #define G2H_LEN_DW_DEREGISTER_CONTEXT		1
> +#define G2H_LEN_DW_INVALIDATE_TLB		1
>   
>   #define GUC_CONTEXT_DISABLE		0
>   #define GUC_CONTEXT_ENABLE		1
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 2cce5ec1ff00d..0c5ccda1b3e87 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
>   
>   void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
>   {
> +	struct intel_guc_tlb_wait *wait;
>   	struct intel_context *ce;
>   	unsigned long index;
>   	unsigned long flags;
> +	unsigned long i;
>   
>   	if (unlikely(!guc_submission_initialized(guc))) {
>   		/* Reset called during driver load? GuC not yet initialised! */
> @@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
>   
>   	/* GuC is blown away, drop all references to contexts */
>   	xa_destroy(&guc->context_lookup);
> +
> +	/*
> +	 * The full GT reset will have cleared the TLB caches and flushed the
> +	 * G2H message queue; we can release all the blocked waiters.
> +	 *
> +	 * This is safe to do unlocked because the xarray is not dependent
> +	 * on the GT reset, and there's a separate execution path for TLB
> +	 * invalidations on GT reset, and there's a large window of time
> +	 * between the GT reset and GuC becoming available.
> +	 */
> +	xa_for_each(&guc->tlb_lookup, i, wait)
> +		wake_up(&wait->wq);

If you are confident there can be no failures to wake up someone, who 
maybe just added themselves to the xarray (via put pages for instance), 
while reset in ongoing. Or even removed themselves after say timing out 
the wait and so freed their entry...

>   }
>   
>   static void guc_cancel_context_requests(struct intel_context *ce)
> @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
>   static void destroyed_worker_func(struct work_struct *w);
>   static void reset_fail_worker_func(struct work_struct *w);
>   
> +static int init_tlb_lookup(struct intel_guc *guc)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +	int err;
> +
> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		return 0;
> +
> +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
> +
> +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
> +	if (!wait)
> +		return -ENOMEM;
> +
> +	init_waitqueue_head(&wait->wq);
> +
> +	/* Preallocate a shared id for use under memory pressure. */
> +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
> +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
> +	/* Only error if out of memory, not when busy (list full)*/
> +	if (err == -ENOMEM) {
> +		kfree(wait);
> +		return err;
> +	}

I agreed with John here that only looking at ENOMEM reads odd and I did 
not see that answered. Did I miss it?

Otherwise, I _know_ it is not likely to get any other error having 
*just* created a new xarray, but still, why not simply catch all? It is 
not like the slot fallback code at runtime would handle guc->serial_slot 
being empty?! It appears it would just explode in 
guc_send_invalidate_tlb if it would hit it..

> +
> +	return 0;
> +}
> +
> +static void fini_tlb_lookup(struct intel_guc *guc)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +
> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		return;
> +
> +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
> +	if (wait) {
> +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");

Hm wait, why is this unexpected when init_tlb_lookup() pre-allocated 
that entry? Who frees it? guc_send_invalidate_tlb() does not appear to.

> +		kfree(wait);
> +	}
> +
> +	xa_destroy(&guc->tlb_lookup);
> +}
> +
>   /*
>    * Set up the memory resources to be shared with the GuC (via the GGTT)
>    * at firmware loading time.
> @@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct intel_guc *guc)
>   			return ret;
>   	}
>   
> +	ret = init_tlb_lookup(guc);
> +	if (ret)
> +		goto destroy_pool;
> +
>   	guc->submission_state.guc_ids_bitmap =
>   		bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
>   	if (!guc->submission_state.guc_ids_bitmap) {
>   		ret = -ENOMEM;
> -		goto destroy_pool;
> +		goto destroy_tlb;
>   	}
>   
>   	guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
> @@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc *guc)
>   
>   	return 0;
>   
> +destroy_tlb:
> +	fini_tlb_lookup(guc);
>   destroy_pool:
>   	guc_lrc_desc_pool_destroy_v69(guc);
> -
>   	return ret;
>   }
>   
> @@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
>   	guc_lrc_desc_pool_destroy_v69(guc);
>   	i915_sched_engine_put(guc->sched_engine);
>   	bitmap_free(guc->submission_state.guc_ids_bitmap);
> +	fini_tlb_lookup(guc);
>   	guc->submission_initialized = false;
>   }
>   
> @@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id)
>   	return ce;
>   }
>   
> +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +	unsigned long flags;
> +
> +	xa_lock_irqsave(&guc->tlb_lookup, flags);
> +	wait = xa_load(&guc->tlb_lookup, seqno);
> +
> +	if (wait)
> +		wake_up(&wait->wq);
> +	else
> +		guc_dbg(guc,
> +			"Stale TLB invalidation response with seqno %d\n", seqno);
> +
> +	xa_unlock_irqrestore(&guc->tlb_lookup, flags);
> +}
> +
> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
> +{
> +	/* Check for underflow */
> +	if (unlikely(len < 1 || len > size))
> +		return -EPROTO;

These check are not valid for any message/action type 
ct_process_request() can receive?

> +
> +	wait_wake_outstanding_tlb_g2h(guc, seqno);
> +	return 0;
> +}
> +
> +static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
> +{
> +	/*
> +	 * This is equivalent to wait_woken() with the exception that
> +	 * we do not wake up early if the kthread task has been completed.
> +	 * As we are called from page reclaim in any task context,
> +	 * we may be invoked from stopped kthreads, but we *must*
> +	 * complete the wait from the HW .
> +	 *
> +	 * A second problem is that since we are called under reclaim
> +	 * and wait_woken() inspected the thread state, it makes an invalid
> +	 * assumption that all PF_KTHREAD tasks have set_kthread_struct()
> +	 * called upon them, and will trigger a GPF in is_kthread_should_stop().
> +	 */
> +	do {
> +		set_current_state(TASK_UNINTERRUPTIBLE);
> +		if (wq_entry->flags & WQ_FLAG_WOKEN)
> +			break;
> +
> +		timeout = schedule_timeout(timeout);
> +	} while (timeout);
> +	__set_current_state(TASK_RUNNING);
> +
> +	/* See wait_woken() and woken_wake_function() */
> +	smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
> +
> +	return timeout;
> +}
> +
> +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)
> +{
> +	struct intel_guc_tlb_wait _wq, *wq = &_wq;
> +	DEFINE_WAIT_FUNC(wait, woken_wake_function);
> +	int err;
> +	u32 seqno;
> +	u32 action[] = {
> +		INTEL_GUC_ACTION_TLB_INVALIDATION,
> +		0,
> +		REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
> +			REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
> +				       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
> +			INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
> +	};
> +	u32 size = ARRAY_SIZE(action);
> +
> +	init_waitqueue_head(&_wq.wq);
> +
> +	if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
> +				xa_limit_32b, &guc->next_seqno,
> +				GFP_ATOMIC | __GFP_NOWARN) < 0) {
> +		/* Under severe memory pressure? Serialise TLB allocations */
> +		xa_lock_irq(&guc->tlb_lookup);
> +		wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
> +		wait_event_lock_irq(wq->wq,
> +				    !READ_ONCE(wq->busy),
> +				    guc->tlb_lookup.xa_lock);
> +		/*
> +		 * Update wq->busy under lock to ensure only one waiter can
> +		 * issue the TLB invalidation command using the serial slot at a
> +		 * time. The condition is set to true before releasing the lock
> +		 * so that other caller continue to wait until woken up again.
> +		 */
> +		wq->busy = true;
> +		xa_unlock_irq(&guc->tlb_lookup);
> +
> +		seqno = guc->serial_slot;
> +	}
> +
> +	action[1] = seqno;
> +
> +	add_wait_queue(&wq->wq, &wait);
> +
> +	/*
> +	 * This is a critical reclaim path and thus we must loop here:
> +	 * We cannot block for anything that is on the GPU.
> +	 */
> +	err = intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE_TLB, true);
> +	if (err)
> +		goto out;
> +
> +	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
> +		guc_err(guc,
> +			"TLB invalidation response timed out for seqno %u\n", seqno);
> +		err = -ETIME;
> +	}
> +out:
> +	remove_wait_queue(&wq->wq, &wait);
> +	if (seqno != guc->serial_slot)
> +		xa_erase_irq(&guc->tlb_lookup, seqno);
> +
> +	return err;
> +}
> +
> +/* Full TLB invalidation */
> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
> +{
> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
> +}
> +
> +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
> +{
> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
> +}
> +
>   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
>   					  const u32 *msg,
>   					  u32 len)

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Andi Shyti]

Hi,

...

> > @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
> >   		return;
> >   	with_intel_gt_pm_if_awake(gt, wakeref) {
> > +		struct intel_guc *guc = &gt->uc.guc;
> > +
> >   		mutex_lock(&gt->tlb.invalidate_lock);
> >   		if (tlb_seqno_passed(gt, seqno))
> >   			goto unlock;
> > -		mmio_invalidate_full(gt);
> > +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
> > +			/*
> > +			 * Only perform GuC TLB invalidation if GuC is ready.
> > +			 * If GuC is not ready, then there are no TLBs to
> > +			 * invalidate.  Ergo, skip invalidation.
> > +			 */
> > +			if (intel_guc_is_ready(guc))
> > +				intel_guc_invalidate_tlb_engines(guc);
> 
> What was the answer to John's question on why it is okay to just skip and
> not maybe fall back to mmio?

this maybe can be written as:

	if (HAS_GUC_TLB_INVALIDATION(gt->i915) &&
	    intel_guc_is_ready(guc))
		intel_guc_invalidate_tlb_engines(guc);
	else
		mmio_invalidate_full(gt);

> > +		} else {
> > +			mmio_invalidate_full(gt);
> > +		}
> >   		write_seqcount_invalidate(&gt->tlb.seqno);
> >   unlock:

...

> > +	/*
> > +	 * The full GT reset will have cleared the TLB caches and flushed the
> > +	 * G2H message queue; we can release all the blocked waiters.
> > +	 *
> > +	 * This is safe to do unlocked because the xarray is not dependent
> > +	 * on the GT reset, and there's a separate execution path for TLB
> > +	 * invalidations on GT reset, and there's a large window of time
> > +	 * between the GT reset and GuC becoming available.
> > +	 */
> > +	xa_for_each(&guc->tlb_lookup, i, wait)
> > +		wake_up(&wait->wq);
> 
> If you are confident there can be no failures to wake up someone, who maybe
> just added themselves to the xarray (via put pages for instance), while
> reset in ongoing. Or even removed themselves after say timing out the wait
> and so freed their entry...

I guess yuo are suggesting here to spinlock around this. The
reset is protected by the uncore->spinlock, but I don't really
see it colliding with reset, to be honest.

> >   }
> >   static void guc_cancel_context_requests(struct intel_context *ce)
> > @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
> >   static void destroyed_worker_func(struct work_struct *w);
> >   static void reset_fail_worker_func(struct work_struct *w);
> > +static int init_tlb_lookup(struct intel_guc *guc)
> > +{
> > +	struct intel_guc_tlb_wait *wait;
> > +	int err;
> > +
> > +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> > +		return 0;
> > +
> > +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
> > +
> > +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
> > +	if (!wait)
> > +		return -ENOMEM;
> > +
> > +	init_waitqueue_head(&wait->wq);
> > +
> > +	/* Preallocate a shared id for use under memory pressure. */
> > +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
> > +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
> > +	/* Only error if out of memory, not when busy (list full)*/
> > +	if (err == -ENOMEM) {
> > +		kfree(wait);
> > +		return err;
> > +	}
> 
> I agreed with John here that only looking at ENOMEM reads odd and I did not
> see that answered. Did I miss it?

xa_alloc_cyclic_irq() can also fail with -EBUSY... so that I
think this is a matter...

> Otherwise, I _know_ it is not likely to get any other error having *just*
> created a new xarray, but still, why not simply catch all? It is not like
> the slot fallback code at runtime would handle guc->serial_slot being
> empty?! It appears it would just explode in guc_send_invalidate_tlb if it
> would hit it..

... of what we want to do for such errors. E.g. Jonathan decided
here not to fail, but ignore the error. Should we fail for every
error?

> > +
> > +	return 0;
> > +}
> > +
> > +static void fini_tlb_lookup(struct intel_guc *guc)
> > +{
> > +	struct intel_guc_tlb_wait *wait;
> > +
> > +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> > +		return;
> > +
> > +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
> > +	if (wait) {
> > +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
> 
> Hm wait, why is this unexpected when init_tlb_lookup() pre-allocated that
> entry? Who frees it? guc_send_invalidate_tlb() does not appear to.

I think this links with my answer above, right? With th
refactoring of the if's for not skipping tlb invalidation.

> > +		kfree(wait);
> > +	}
> > +
> > +	xa_destroy(&guc->tlb_lookup);
> > +}
> > +
> >   /*
> >    * Set up the memory resources to be shared with the GuC (via the GGTT)
> >    * at firmware loading time.

...

> > +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
> > +{
> > +	/* Check for underflow */
> > +	if (unlikely(len < 1 || len > size))
> > +		return -EPROTO;
> 
> These check are not valid for any message/action type ct_process_request()
> can receive?

You mean discriminating by payload? Jonathan... you konw the
details here?

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Tvrtko Ursulin]

On 09/10/2023 12:40, Andi Shyti wrote:
> Hi,
> 
> ...
> 
>>> @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
>>>    		return;
>>>    	with_intel_gt_pm_if_awake(gt, wakeref) {
>>> +		struct intel_guc *guc = &gt->uc.guc;
>>> +
>>>    		mutex_lock(&gt->tlb.invalidate_lock);
>>>    		if (tlb_seqno_passed(gt, seqno))
>>>    			goto unlock;
>>> -		mmio_invalidate_full(gt);
>>> +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
>>> +			/*
>>> +			 * Only perform GuC TLB invalidation if GuC is ready.
>>> +			 * If GuC is not ready, then there are no TLBs to
>>> +			 * invalidate.  Ergo, skip invalidation.
>>> +			 */
>>> +			if (intel_guc_is_ready(guc))
>>> +				intel_guc_invalidate_tlb_engines(guc);
>>
>> What was the answer to John's question on why it is okay to just skip and
>> not maybe fall back to mmio?
> 
> this maybe can be written as:
> 
> 	if (HAS_GUC_TLB_INVALIDATION(gt->i915) &&
> 	    intel_guc_is_ready(guc))
> 		intel_guc_invalidate_tlb_engines(guc);
> 	else
> 		mmio_invalidate_full(gt);
> 
>>> +		} else {
>>> +			mmio_invalidate_full(gt);
>>> +		}
>>>    		write_seqcount_invalidate(&gt->tlb.seqno);
>>>    unlock:
> 
> ...
> 
>>> +	/*
>>> +	 * The full GT reset will have cleared the TLB caches and flushed the
>>> +	 * G2H message queue; we can release all the blocked waiters.
>>> +	 *
>>> +	 * This is safe to do unlocked because the xarray is not dependent
>>> +	 * on the GT reset, and there's a separate execution path for TLB
>>> +	 * invalidations on GT reset, and there's a large window of time
>>> +	 * between the GT reset and GuC becoming available.
>>> +	 */
>>> +	xa_for_each(&guc->tlb_lookup, i, wait)
>>> +		wake_up(&wait->wq);
>>
>> If you are confident there can be no failures to wake up someone, who maybe
>> just added themselves to the xarray (via put pages for instance), while
>> reset in ongoing. Or even removed themselves after say timing out the wait
>> and so freed their entry...
> 
> I guess yuo are suggesting here to spinlock around this. The
> reset is protected by the uncore->spinlock, but I don't really
> see it colliding with reset, to be honest.

I am not suggesting since I don't know the flows well enough, just asking. If you are confident what I wrote is impossible then okay.

>>>    }
>>>    static void guc_cancel_context_requests(struct intel_context *ce)
>>> @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
>>>    static void destroyed_worker_func(struct work_struct *w);
>>>    static void reset_fail_worker_func(struct work_struct *w);
>>> +static int init_tlb_lookup(struct intel_guc *guc)
>>> +{
>>> +	struct intel_guc_tlb_wait *wait;
>>> +	int err;
>>> +
>>> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>>> +		return 0;
>>> +
>>> +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
>>> +
>>> +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
>>> +	if (!wait)
>>> +		return -ENOMEM;
>>> +
>>> +	init_waitqueue_head(&wait->wq);
>>> +
>>> +	/* Preallocate a shared id for use under memory pressure. */
>>> +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
>>> +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
>>> +	/* Only error if out of memory, not when busy (list full)*/
>>> +	if (err == -ENOMEM) {
>>> +		kfree(wait);
>>> +		return err;
>>> +	}
>>
>> I agreed with John here that only looking at ENOMEM reads odd and I did not
>> see that answered. Did I miss it?
> 
> xa_alloc_cyclic_irq() can also fail with -EBUSY... so that I
> think this is a matter...
> 
>> Otherwise, I _know_ it is not likely to get any other error having *just*
>> created a new xarray, but still, why not simply catch all? It is not like
>> the slot fallback code at runtime would handle guc->serial_slot being
>> empty?! It appears it would just explode in guc_send_invalidate_tlb if it
>> would hit it..
> 
> ... of what we want to do for such errors. E.g. Jonathan decided
> here not to fail, but ignore the error. Should we fail for every
> error?

Why not fail on *any* error? This is a fresh and empty xarray. If there is any error at this point how can the driver continue operating?

As I wrote, AFAICT it would explode in guc_send_invalidate_tlb() where it assumes xa_load on the guc_serial slot always returns a valid pointer.

> 
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void fini_tlb_lookup(struct intel_guc *guc)
>>> +{
>>> +	struct intel_guc_tlb_wait *wait;
>>> +
>>> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>>> +		return;
>>> +
>>> +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
>>> +	if (wait) {
>>> +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
>>
>> Hm wait, why is this unexpected when init_tlb_lookup() pre-allocated that
>> entry? Who frees it? guc_send_invalidate_tlb() does not appear to.
> 
> I think this links with my answer above, right? With th
> refactoring of the if's for not skipping tlb invalidation.

I don't think so, init/fini is driver load/unload, no?

I went and looked at CI results and they agree with me:

<7> [156.380021] i915 0000:00:02.0: [drm:fini_tlb_lookup.part.0 [i915]] GT0: GUC: fini_tlb_lookup: Unexpected item in tlb_lookup
<7> [156.395370] i915 0000:00:02.0: [drm:fini_tlb_lookup.part.0 [i915]] GT1: GUC: fini_tlb_lookup: Unexpected item in tlb_lookup

So yeah.. code is a bit confused here.

>>> +		kfree(wait);
>>> +	}
>>> +
>>> +	xa_destroy(&guc->tlb_lookup);
>>> +}
>>> +
>>>    /*
>>>     * Set up the memory resources to be shared with the GuC (via the GGTT)
>>>     * at firmware loading time.
> 
> ...
> 
>>> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
>>> +{
>>> +	/* Check for underflow */
>>> +	if (unlikely(len < 1 || len > size))
>>> +		return -EPROTO;
>>
>> These check are not valid for any message/action type ct_process_request()
>> can receive?
> 
> You mean discriminating by payload? Jonathan... you konw the
> details here?

I mean that ct_process_request() already does:

	hxg_len = request->size - GUC_CTB_MSG_MIN_LEN;
	payload = &hxg[GUC_HXG_MSG_MIN_LEN];
	action = FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, hxg[0]);
	len = hxg_len - GUC_HXG_MSG_MIN_LEN;

So I guess at least the len > size check looks dubious since it appears it is only doubting what ct_process_request() should already sanitize.

len < 1 after a look at other handlers looks justified since it is message specific.

Also all other handlers in there have the signature of (guc, payload, len). So maybe this one should do the same for consistency?

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Cavitt, Jonathan]

-----Original Message-----
From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> 
Sent: Monday, October 9, 2023 1:57 AM
To: Cavitt, Jonathan <jonathan.cavitt@intel.com>; intel-gfx@lists.freedesktop.org
Cc: Dutt, Sudeep <sudeep.dutt@intel.com>; Chang, Yu bruce <yu.bruce.chang@intel.com>; chris.p.wilson@linux.intel.com; Iddamsetty, Aravind <aravind.iddamsetty@intel.com>; Yang, Fei <fei.yang@intel.com>; Shyti, Andi <andi.shyti@intel.com>; Harrison, John C <john.c.harrison@intel.com>; Das, Nirmoy <nirmoy.das@intel.com>; Krzysztofik, Janusz <janusz.krzysztofik@intel.com>; Roper, Matthew D <matthew.d.roper@intel.com>; jani.nikula@linux.intel.com
Subject: Re: [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines
> 
> 
> On 06/10/2023 19:20, Jonathan Cavitt wrote:
> > From: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
> > 
> > The GuC firmware had defined the interface for Translation Look-Aside
> > Buffer (TLB) invalidation.  We should use this interface when
> > invalidating the engine and GuC TLBs.
> > Add additional functionality to intel_gt_invalidate_tlb, invalidating
> > the GuC TLBs and falling back to GT invalidation when the GuC is
> > disabled.
> > The invalidation is done by sending a request directly to the GuC
> > tlb_lookup that invalidates the table.  The invalidation is submitted as
> > a wait request and is performed in the CT event handler.  This means we
> > cannot perform this TLB invalidation path if the CT is not enabled.
> > If the request isn't fulfilled in two seconds, this would constitute
> > an error in the invalidation as that would constitute either a lost
> > request or a severe GuC overload.
> > 
> > With this new invalidation routine, we can perform GuC-based GGTT
> > invalidations.  GuC-based GGTT invalidation is incompatible with
> > MMIO invalidation so we should not perform MMIO invalidation when
> > GuC-based GGTT invalidation is expected.
> > 
> > Purpose of xarray:
> > The tlb_lookup table is allocated as an xarray because the set of
> > pending TLB invalidations may have no upper bound.  The consequence of
> > this is that all actions interfacing with this table need to use the
> > xarray functions, such as xa_alloc_cyclic_irq for array insertion.
> > 
> > Purpose of must_wait_woken:
> > Our wait for the G2H ack for the completion of a TLB invalidation is
> > mandatory; we must wait for the HW to confirm that the physical
> > addresses are no longer accessible before we return those to the system.
> > 
> > On switching to using the wait_woken() convenience routine, we
> > introduced ourselves to an issue where wait_woken() may complete early
> > under a kthread that is stopped. Since we send a TLB invalidation when
> > we try to release pages from the shrinker, we can be called from any
> > process; including kthreads.
> > 
> > Using wait_woken() from any process context causes another issue. The
> > use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
> > set was made by kthread_create() and has called set_kthread_struct().
> > This is not true for the raw kernel_thread():
> 
> This explanation misses the main point of my ask - which is to explain 
> why a simpler scheme isn't sufficient. Simpler scheme aka not needed the 
> xarray or any flavour of wait_token().
> 
> In other words it is obvious we have to wait for the invalidation ack, 
> but not obvious why we need a complicated scheme.


Okay.  I'll remove these chunks and explain that the complexity is required
for range-based TLB invalidaitons, which will land upstream eventually.


> 
> > BUG: kernel NULL pointer dereference, address: 0000000000000000
> > [ 3089.759660] Call Trace:
> > [ 3089.762110]  wait_woken+0x4f/0x80
> > [ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
> > [ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
> > [ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
> > [ 3089.779430]  ? __switch_to_asm+0x40/0x70
> > [ 3089.783349]  ? __switch_to_asm+0x34/0x70
> > [ 3089.787273]  ? __switch_to+0x7a/0x3e0
> > [ 3089.790930]  ? __switch_to_asm+0x34/0x70
> > [ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
> > [ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
> > [ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
> > [ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
> > [ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
> > [ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
> > [ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
> > [ 3089.832230]  do_shrink_slab+0x12c/0x2a0
> > [ 3089.836065]  shrink_slab+0xad/0x2b0
> > [ 3089.839550]  shrink_node+0xcc/0x410
> > [ 3089.843035]  do_try_to_free_pages+0xc6/0x380
> > [ 3089.847306]  try_to_free_pages+0xec/0x1c0
> > [ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
> > [ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
> > [ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
> > [ 3089.864737]  new_slab+0x265/0xa80
> > [ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
> > [ 3089.871798]  ? copy_process+0x1e5/0x1a00
> > [ 3089.875717]  ? load_balance+0x165/0xb20
> > [ 3089.879555]  __slab_alloc+0x1c/0x30
> > [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> > [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> > [ 3089.891314]  copy_process+0x1e5/0x1a00
> > [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> > [ 3089.879555]  __slab_alloc+0x1c/0x30
> > [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> > [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> > [ 3089.891314]  copy_process+0x1e5/0x1a00
> > [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> > [ 3089.898977]  ? __switch_to_asm+0x34/0x70
> > [ 3089.902903]  ? __switch_to_asm+0x40/0x70
> > [ 3089.906828]  ? __switch_to_asm+0x34/0x70
> > [ 3089.910745]  _do_fork+0x83/0x350
> > [ 3089.913969]  ? __switch_to+0x7a/0x3e0
> > [ 3089.917626]  ? __switch_to_asm+0x34/0x70
> > [ 3089.921545]  kernel_thread+0x58/0x80
> > [ 3089.925124]  ? kthread_park+0x80/0x80
> > [ 3089.928788]  kthreadd+0x162/0x1b0
> > [ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
> > [ 3089.936538]  ret_from_fork+0x1f/0x40
> > 
> > Signed-off-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
> > Signed-off-by: Bruce Chang <yu.bruce.chang@intel.com>
> > Signed-off-by: Chris Wilson <chris.p.wilson@intel.com>
> > Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
> > Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
> > Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@intel.com>
> > Signed-off-by: Fei Yang <fei.yang@intel.com>
> > CC: Andi Shyti <andi.shyti@linux.intel.com>
> > ---
> >   drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
> >   drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
> >   .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
> >   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
> >   drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
> >   drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
> >   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
> >   7 files changed, 297 insertions(+), 12 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> > index 4d7d88b92632b..a1f7bdc602996 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> > @@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct i915_ggtt *ggtt)
> >   	intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
> >   }
> >   
> > +static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
> > +{
> > +	struct intel_uncore *uncore = gt->uncore;
> > +	intel_wakeref_t wakeref;
> > +
> > +	with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
> > +		struct intel_guc *guc = &gt->uc.guc;
> > +
> > +		intel_guc_invalidate_tlb_guc(guc);
> > +	}
> > +}
> > +
> >   static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
> >   {
> >   	struct drm_i915_private *i915 = ggtt->vm.i915;
> > +	struct intel_gt *gt;
> >   
> > -	gen8_ggtt_invalidate(ggtt);
> > -
> > -	if (GRAPHICS_VER(i915) >= 12) {
> > -		struct intel_gt *gt;
> > +	if (!HAS_GUC_TLB_INVALIDATION(i915))
> > +		gen8_ggtt_invalidate(ggtt);
> >   
> > -		list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
> > +	list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
> > +		if (HAS_GUC_TLB_INVALIDATION(i915) &&
> > +		    intel_guc_is_ready(&gt->uc.guc)) {
> > +			guc_ggtt_ct_invalidate(gt);
> > +		} else if (GRAPHICS_VER(i915) >= 12) {
> >   			intel_uncore_write_fw(gt->uncore,
> >   					      GEN12_GUC_TLB_INV_CR,
> >   					      GEN12_GUC_TLB_INV_CR_INVALIDATE);
> > -	} else {
> > -		intel_uncore_write_fw(ggtt->vm.gt->uncore,
> > -				      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> > +		} else {
> > +			intel_uncore_write_fw(gt->uncore,
> > +					      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> > +		}
> >   	}
> >   }
> >   
> > @@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
> >   		ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
> >   	}
> >   
> > -	if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
> > +	if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
> >   		ggtt->invalidate = guc_ggtt_invalidate;
> >   	else
> >   		ggtt->invalidate = gen8_ggtt_invalidate;
> > diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c
> > index 139608c30d978..1278622b9554d 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_tlb.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
> > @@ -12,6 +12,7 @@
> >   #include "intel_gt_print.h"
> >   #include "intel_gt_regs.h"
> >   #include "intel_tlb.h"
> > +#include "uc/intel_guc.h"
> >   
> >   /*
> >    * HW architecture suggest typical invalidation time at 40us,
> > @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
> >   		return;
> >   
> >   	with_intel_gt_pm_if_awake(gt, wakeref) {
> > +		struct intel_guc *guc = &gt->uc.guc;
> > +
> >   		mutex_lock(&gt->tlb.invalidate_lock);
> >   		if (tlb_seqno_passed(gt, seqno))
> >   			goto unlock;
> >   
> > -		mmio_invalidate_full(gt);
> > +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
> > +			/*
> > +			 * Only perform GuC TLB invalidation if GuC is ready.
> > +			 * If GuC is not ready, then there are no TLBs to
> > +			 * invalidate.  Ergo, skip invalidation.
> > +			 */
> > +			if (intel_guc_is_ready(guc))
> > +				intel_guc_invalidate_tlb_engines(guc);
> 
> What was the answer to John's question on why it is okay to just skip 
> and not maybe fall back to mmio?


Intel_guc_is_ready is only disabled on a GT reset, which will clobber all the TLBs anyways,
so it is safe to skip invalidations in this case.  I'll add this as a comment.


> 
> > +		} else {
> > +			mmio_invalidate_full(gt);
> > +		}
> >   
> >   		write_seqcount_invalidate(&gt->tlb.seqno);
> >   unlock:
> > diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> > index f359bef046e0b..33f253410d0c8 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> > +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> > @@ -138,6 +138,8 @@ enum intel_guc_action {
> >   	INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
> >   	INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
> >   	INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
> > +	INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
> > +	INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
> >   	INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
> >   	INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
> >   	INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
> > @@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
> >   
> >   #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK      0x000000FF
> >   
> > +#define INTEL_GUC_TLB_INVAL_TYPE_MASK	REG_GENMASK(7, 0)
> > +#define INTEL_GUC_TLB_INVAL_MODE_MASK	REG_GENMASK(11, 8)
> > +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
> > +
> > +enum intel_guc_tlb_invalidation_type {
> > +	INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
> > +	INTEL_GUC_TLB_INVAL_GUC = 0x3,
> > +};
> > +
> > +/*
> > + * 0: Heavy mode of Invalidation:
> > + * The pipeline of the engine(s) for which the invalidation is targeted to is
> > + * blocked, and all the in-flight transactions are guaranteed to be Globally
> > + * Observed before completing the TLB invalidation
> > + * 1: Lite mode of Invalidation:
> > + * TLBs of the targeted engine(s) are immediately invalidated.
> > + * In-flight transactions are NOT guaranteed to be Globally Observed before
> > + * completing TLB invalidation.
> > + * Light Invalidation Mode is to be used only when
> > + * it can be guaranteed (by SW) that the address translations remain invariant
> > + * for the in-flight transactions across the TLB invalidation. In other words,
> > + * this mode can be used when the TLB invalidation is intended to clear out the
> > + * stale cached translations that are no longer in use. Light Invalidation Mode
> > + * is much faster than the Heavy Invalidation Mode, as it does not wait for the
> > + * in-flight transactions to be GOd.
> > + */
> > +enum intel_guc_tlb_inval_mode {
> > +	INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
> > +	INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
> > +};
> > +
> >   #endif /* _ABI_GUC_ACTIONS_ABI_H */
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> > index 6c392bad29c19..88deb43bbdc48 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> > @@ -79,6 +79,18 @@ struct intel_guc {
> >   	 */
> >   	atomic_t outstanding_submission_g2h;
> >   
> > +	/** @tlb_lookup: xarray to store all pending TLB invalidation requests */
> > +	struct xarray tlb_lookup;
> > +
> > +	/**
> > +	 * @serial_slot: id to the initial waiter created in tlb_lookup,
> > +	 * which is used only when failed to allocate new waiter.
> > +	 */
> > +	u32 serial_slot;
> > +
> > +	/** @next_seqno: the next id (sequence no.) to allocate. */
> > +	u32 next_seqno;
> > +
> >   	/** @interrupts: pointers to GuC interrupt-managing functions. */
> >   	struct {
> >   		bool enabled;
> > @@ -296,6 +308,11 @@ struct intel_guc {
> >   #define MAKE_GUC_VER_STRUCT(ver)	MAKE_GUC_VER((ver).major, (ver).minor, (ver).patch)
> >   #define GUC_SUBMIT_VER(guc)		MAKE_GUC_VER_STRUCT((guc)->submission_version)
> >   
> > +struct intel_guc_tlb_wait {
> > +	struct wait_queue_head wq;
> > +	bool busy;
> > +};
> > +
> >   static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
> >   {
> >   	return container_of(log, struct intel_guc, log);
> > @@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct intel_guc *guc)
> >   	return intel_uc_fw_is_supported(&guc->fw);
> >   }
> >   
> > +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
> > +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
> > +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
> > +				    u32 len, u32 seqno);
> > +
> >   static inline bool intel_guc_is_wanted(struct intel_guc *guc)
> >   {
> >   	return intel_uc_fw_is_enabled(&guc->fw);
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > index 6e22af31513a5..e7ac8402245b7 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > @@ -1115,6 +1115,9 @@ static int ct_process_request(struct intel_guc_ct *ct, struct ct_incoming_msg *r
> >   	case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
> >   		ret = intel_guc_crash_process_msg(guc, action);
> >   		break;
> > +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
> > +		ret = intel_guc_tlb_invalidation_done(guc, request->size, len, payload[0]);
> > +		break;
> >   	default:
> >   		ret = -EOPNOTSUPP;
> >   		break;
> > @@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct *ct, struct ct_incoming_msg *requ
> >   	switch (action) {
> >   	case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
> >   	case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
> > +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
> >   		g2h_release_space(ct, request->size);
> >   	}
> >   
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> > index b4d56eccfb1f0..a7c9874e122a3 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> > @@ -22,6 +22,7 @@
> >   /* Payload length only i.e. don't include G2H header length */
> >   #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET	2
> >   #define G2H_LEN_DW_DEREGISTER_CONTEXT		1
> > +#define G2H_LEN_DW_INVALIDATE_TLB		1
> >   
> >   #define GUC_CONTEXT_DISABLE		0
> >   #define GUC_CONTEXT_ENABLE		1
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 2cce5ec1ff00d..0c5ccda1b3e87 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
> >   
> >   void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
> >   {
> > +	struct intel_guc_tlb_wait *wait;
> >   	struct intel_context *ce;
> >   	unsigned long index;
> >   	unsigned long flags;
> > +	unsigned long i;
> >   
> >   	if (unlikely(!guc_submission_initialized(guc))) {
> >   		/* Reset called during driver load? GuC not yet initialised! */
> > @@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
> >   
> >   	/* GuC is blown away, drop all references to contexts */
> >   	xa_destroy(&guc->context_lookup);
> > +
> > +	/*
> > +	 * The full GT reset will have cleared the TLB caches and flushed the
> > +	 * G2H message queue; we can release all the blocked waiters.
> > +	 *
> > +	 * This is safe to do unlocked because the xarray is not dependent
> > +	 * on the GT reset, and there's a separate execution path for TLB
> > +	 * invalidations on GT reset, and there's a large window of time
> > +	 * between the GT reset and GuC becoming available.
> > +	 */
> > +	xa_for_each(&guc->tlb_lookup, i, wait)
> > +		wake_up(&wait->wq);
> 
> If you are confident there can be no failures to wake up someone, who 
> maybe just added themselves to the xarray (via put pages for instance), 
> while reset in ongoing. Or even removed themselves after say timing out 
> the wait and so freed their entry...


I'll reduce the comment complexity to just
"This is safe to do unlocked because there are no races with waiters entering the array."


> 
> >   }
> >   
> >   static void guc_cancel_context_requests(struct intel_context *ce)
> > @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
> >   static void destroyed_worker_func(struct work_struct *w);
> >   static void reset_fail_worker_func(struct work_struct *w);
> >   
> > +static int init_tlb_lookup(struct intel_guc *guc)
> > +{
> > +	struct intel_guc_tlb_wait *wait;
> > +	int err;
> > +
> > +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> > +		return 0;
> > +
> > +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
> > +
> > +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
> > +	if (!wait)
> > +		return -ENOMEM;
> > +
> > +	init_waitqueue_head(&wait->wq);
> > +
> > +	/* Preallocate a shared id for use under memory pressure. */
> > +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
> > +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
> > +	/* Only error if out of memory, not when busy (list full)*/
> > +	if (err == -ENOMEM) {
> > +		kfree(wait);
> > +		return err;
> > +	}
> 
> I agreed with John here that only looking at ENOMEM reads odd and I did 
> not see that answered. Did I miss it?
> 
> Otherwise, I _know_ it is not likely to get any other error having 
> *just* created a new xarray, but still, why not simply catch all? It is 
> not like the slot fallback code at runtime would handle guc->serial_slot 
> being empty?! It appears it would just explode in 
> guc_send_invalidate_tlb if it would hit it..


I'll error on all err < 0, then.


> 
> > +
> > +	return 0;
> > +}
> > +
> > +static void fini_tlb_lookup(struct intel_guc *guc)
> > +{
> > +	struct intel_guc_tlb_wait *wait;
> > +
> > +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> > +		return;
> > +
> > +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
> > +	if (wait) {
> > +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
> 
> Hm wait, why is this unexpected when init_tlb_lookup() pre-allocated 
> that entry? Who frees it? guc_send_invalidate_tlb() does not appear to.
> 


It seems I accidentally dropped the if (wait->busy) check, though it seems a debug message
here is not needed anyways.
I'll drop it.


> > +		kfree(wait);
> > +	}
> > +
> > +	xa_destroy(&guc->tlb_lookup);
> > +}
> > +
> >   /*
> >    * Set up the memory resources to be shared with the GuC (via the GGTT)
> >    * at firmware loading time.
> > @@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct intel_guc *guc)
> >   			return ret;
> >   	}
> >   
> > +	ret = init_tlb_lookup(guc);
> > +	if (ret)
> > +		goto destroy_pool;
> > +
> >   	guc->submission_state.guc_ids_bitmap =
> >   		bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
> >   	if (!guc->submission_state.guc_ids_bitmap) {
> >   		ret = -ENOMEM;
> > -		goto destroy_pool;
> > +		goto destroy_tlb;
> >   	}
> >   
> >   	guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
> > @@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc *guc)
> >   
> >   	return 0;
> >   
> > +destroy_tlb:
> > +	fini_tlb_lookup(guc);
> >   destroy_pool:
> >   	guc_lrc_desc_pool_destroy_v69(guc);
> > -
> >   	return ret;
> >   }
> >   
> > @@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
> >   	guc_lrc_desc_pool_destroy_v69(guc);
> >   	i915_sched_engine_put(guc->sched_engine);
> >   	bitmap_free(guc->submission_state.guc_ids_bitmap);
> > +	fini_tlb_lookup(guc);
> >   	guc->submission_initialized = false;
> >   }
> >   
> > @@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id)
> >   	return ce;
> >   }
> >   
> > +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno)
> > +{
> > +	struct intel_guc_tlb_wait *wait;
> > +	unsigned long flags;
> > +
> > +	xa_lock_irqsave(&guc->tlb_lookup, flags);
> > +	wait = xa_load(&guc->tlb_lookup, seqno);
> > +
> > +	if (wait)
> > +		wake_up(&wait->wq);
> > +	else
> > +		guc_dbg(guc,
> > +			"Stale TLB invalidation response with seqno %d\n", seqno);
> > +
> > +	xa_unlock_irqrestore(&guc->tlb_lookup, flags);
> > +}
> > +
> > +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
> > +{
> > +	/* Check for underflow */
> > +	if (unlikely(len < 1 || len > size))
> > +		return -EPROTO;
> 
> These check are not valid for any message/action type 
> ct_process_request() can receive?
> 


Okay, I'll drop this check and reorient intel_guc_tlb_invalidation_done to use
(guc, payload, len) format.


> > +
> > +	wait_wake_outstanding_tlb_g2h(guc, seqno);
> > +	return 0;
> > +}
> > +
> > +static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
> > +{
> > +	/*
> > +	 * This is equivalent to wait_woken() with the exception that
> > +	 * we do not wake up early if the kthread task has been completed.
> > +	 * As we are called from page reclaim in any task context,
> > +	 * we may be invoked from stopped kthreads, but we *must*
> > +	 * complete the wait from the HW .
> > +	 *
> > +	 * A second problem is that since we are called under reclaim
> > +	 * and wait_woken() inspected the thread state, it makes an invalid
> > +	 * assumption that all PF_KTHREAD tasks have set_kthread_struct()
> > +	 * called upon them, and will trigger a GPF in is_kthread_should_stop().
> > +	 */


It seems this GPF issue will be resolved soon.  We still need the additional
complexity because we need an uninterruptible wait, which is not currently
provided by the kernel.


> > +	do {
> > +		set_current_state(TASK_UNINTERRUPTIBLE);
> > +		if (wq_entry->flags & WQ_FLAG_WOKEN)
> > +			break;
> > +
> > +		timeout = schedule_timeout(timeout);
> > +	} while (timeout);
> > +	__set_current_state(TASK_RUNNING);
> > +
> > +	/* See wait_woken() and woken_wake_function() */
> > +	smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
> > +
> > +	return timeout;
> > +}
> > +
> > +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)


This turned out to be the wrong enum.  I'll fix it.
-Jonathan Cavitt


> > +{
> > +	struct intel_guc_tlb_wait _wq, *wq = &_wq;
> > +	DEFINE_WAIT_FUNC(wait, woken_wake_function);
> > +	int err;
> > +	u32 seqno;
> > +	u32 action[] = {
> > +		INTEL_GUC_ACTION_TLB_INVALIDATION,
> > +		0,
> > +		REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
> > +			REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
> > +				       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
> > +			INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
> > +	};
> > +	u32 size = ARRAY_SIZE(action);
> > +
> > +	init_waitqueue_head(&_wq.wq);
> > +
> > +	if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
> > +				xa_limit_32b, &guc->next_seqno,
> > +				GFP_ATOMIC | __GFP_NOWARN) < 0) {
> > +		/* Under severe memory pressure? Serialise TLB allocations */
> > +		xa_lock_irq(&guc->tlb_lookup);
> > +		wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
> > +		wait_event_lock_irq(wq->wq,
> > +				    !READ_ONCE(wq->busy),
> > +				    guc->tlb_lookup.xa_lock);
> > +		/*
> > +		 * Update wq->busy under lock to ensure only one waiter can
> > +		 * issue the TLB invalidation command using the serial slot at a
> > +		 * time. The condition is set to true before releasing the lock
> > +		 * so that other caller continue to wait until woken up again.
> > +		 */
> > +		wq->busy = true;
> > +		xa_unlock_irq(&guc->tlb_lookup);
> > +
> > +		seqno = guc->serial_slot;
> > +	}
> > +
> > +	action[1] = seqno;
> > +
> > +	add_wait_queue(&wq->wq, &wait);
> > +
> > +	/*
> > +	 * This is a critical reclaim path and thus we must loop here:
> > +	 * We cannot block for anything that is on the GPU.
> > +	 */
> > +	err = intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE_TLB, true);
> > +	if (err)
> > +		goto out;
> > +
> > +	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
> > +		guc_err(guc,
> > +			"TLB invalidation response timed out for seqno %u\n", seqno);
> > +		err = -ETIME;
> > +	}
> > +out:
> > +	remove_wait_queue(&wq->wq, &wait);
> > +	if (seqno != guc->serial_slot)
> > +		xa_erase_irq(&guc->tlb_lookup, seqno);
> > +
> > +	return err;
> > +}
> > +
> > +/* Full TLB invalidation */
> > +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
> > +{
> > +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
> > +}
> > +
> > +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
> > +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
> > +{
> > +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
> > +}
> > +
> >   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
> >   					  const u32 *msg,
> >   					  u32 len)
> 
> Regards,
> 
> Tvrtko
> 

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[John Harrison]

On 10/9/2023 01:56, Tvrtko Ursulin wrote:
> On 06/10/2023 19:20, Jonathan Cavitt wrote:
>> From: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
>>
>> The GuC firmware had defined the interface for Translation Look-Aside
>> Buffer (TLB) invalidation.  We should use this interface when
>> invalidating the engine and GuC TLBs.
>> Add additional functionality to intel_gt_invalidate_tlb, invalidating
>> the GuC TLBs and falling back to GT invalidation when the GuC is
>> disabled.
>> The invalidation is done by sending a request directly to the GuC
>> tlb_lookup that invalidates the table.  The invalidation is submitted as
>> a wait request and is performed in the CT event handler.  This means we
>> cannot perform this TLB invalidation path if the CT is not enabled.
>> If the request isn't fulfilled in two seconds, this would constitute
>> an error in the invalidation as that would constitute either a lost
>> request or a severe GuC overload.
>>
>> With this new invalidation routine, we can perform GuC-based GGTT
>> invalidations.  GuC-based GGTT invalidation is incompatible with
>> MMIO invalidation so we should not perform MMIO invalidation when
>> GuC-based GGTT invalidation is expected.
>>
>> Purpose of xarray:
>> The tlb_lookup table is allocated as an xarray because the set of
>> pending TLB invalidations may have no upper bound.  The consequence of
>> this is that all actions interfacing with this table need to use the
>> xarray functions, such as xa_alloc_cyclic_irq for array insertion.
>>
>> Purpose of must_wait_woken:
>> Our wait for the G2H ack for the completion of a TLB invalidation is
>> mandatory; we must wait for the HW to confirm that the physical
>> addresses are no longer accessible before we return those to the system.
>>
>> On switching to using the wait_woken() convenience routine, we
>> introduced ourselves to an issue where wait_woken() may complete early
>> under a kthread that is stopped. Since we send a TLB invalidation when
>> we try to release pages from the shrinker, we can be called from any
>> process; including kthreads.
>>
>> Using wait_woken() from any process context causes another issue. The
>> use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
>> set was made by kthread_create() and has called set_kthread_struct().
>> This is not true for the raw kernel_thread():
>
> This explanation misses the main point of my ask - which is to explain 
> why a simpler scheme isn't sufficient. Simpler scheme aka not needed 
> the xarray or any flavour of wait_token().
>
> In other words it is obvious we have to wait for the invalidation ack, 
> but not obvious why we need a complicated scheme.
The alternative being to simply serialise all TLB invalidation requests? 
Thus, no complex tracking required as there is only one in flight at a 
time? That seems inefficient and a potential performance impact if a 
bunch of invalidations are required back to back. But given that the 
current scheme is global invalidation only (no support for ranges / per 
page invalidation yet), is it possible to get multiple back-to-back 
requests?

>
>> BUG: kernel NULL pointer dereference, address: 0000000000000000
>> [ 3089.759660] Call Trace:
>> [ 3089.762110]  wait_woken+0x4f/0x80
>> [ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
>> [ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
>> [ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
>> [ 3089.779430]  ? __switch_to_asm+0x40/0x70
>> [ 3089.783349]  ? __switch_to_asm+0x34/0x70
>> [ 3089.787273]  ? __switch_to+0x7a/0x3e0
>> [ 3089.790930]  ? __switch_to_asm+0x34/0x70
>> [ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
>> [ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
>> [ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
>> [ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
>> [ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
>> [ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
>> [ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
>> [ 3089.832230]  do_shrink_slab+0x12c/0x2a0
>> [ 3089.836065]  shrink_slab+0xad/0x2b0
>> [ 3089.839550]  shrink_node+0xcc/0x410
>> [ 3089.843035]  do_try_to_free_pages+0xc6/0x380
>> [ 3089.847306]  try_to_free_pages+0xec/0x1c0
>> [ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
>> [ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
>> [ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
>> [ 3089.864737]  new_slab+0x265/0xa80
>> [ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
>> [ 3089.871798]  ? copy_process+0x1e5/0x1a00
>> [ 3089.875717]  ? load_balance+0x165/0xb20
>> [ 3089.879555]  __slab_alloc+0x1c/0x30
>> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
>> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
>> [ 3089.891314]  copy_process+0x1e5/0x1a00
>> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
>> [ 3089.879555]  __slab_alloc+0x1c/0x30
>> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
>> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
>> [ 3089.891314]  copy_process+0x1e5/0x1a00
>> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
>> [ 3089.898977]  ? __switch_to_asm+0x34/0x70
>> [ 3089.902903]  ? __switch_to_asm+0x40/0x70
>> [ 3089.906828]  ? __switch_to_asm+0x34/0x70
>> [ 3089.910745]  _do_fork+0x83/0x350
>> [ 3089.913969]  ? __switch_to+0x7a/0x3e0
>> [ 3089.917626]  ? __switch_to_asm+0x34/0x70
>> [ 3089.921545]  kernel_thread+0x58/0x80
>> [ 3089.925124]  ? kthread_park+0x80/0x80
>> [ 3089.928788]  kthreadd+0x162/0x1b0
>> [ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
>> [ 3089.936538]  ret_from_fork+0x1f/0x40
>>
>> Signed-off-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
>> Signed-off-by: Bruce Chang <yu.bruce.chang@intel.com>
>> Signed-off-by: Chris Wilson <chris.p.wilson@intel.com>
>> Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>> Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
>> Signed-off-by: Aravind Iddamsetty <aravind.iddamsetty@intel.com>
>> Signed-off-by: Fei Yang <fei.yang@intel.com>
>> CC: Andi Shyti <andi.shyti@linux.intel.com>
>> ---
>>   drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
>>   drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
>>   .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
>>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
>>   drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
>>   drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
>>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
>>   7 files changed, 297 insertions(+), 12 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c 
>> b/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> index 4d7d88b92632b..a1f7bdc602996 100644
>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> @@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct 
>> i915_ggtt *ggtt)
>>       intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, 
>> GFX_FLSH_CNTL_EN);
>>   }
>>   +static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
>> +{
>> +    struct intel_uncore *uncore = gt->uncore;
>> +    intel_wakeref_t wakeref;
>> +
>> +    with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
>> +        struct intel_guc *guc = &gt->uc.guc;
>> +
>> +        intel_guc_invalidate_tlb_guc(guc);
>> +    }
>> +}
>> +
>>   static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
>>   {
>>       struct drm_i915_private *i915 = ggtt->vm.i915;
>> +    struct intel_gt *gt;
>>   -    gen8_ggtt_invalidate(ggtt);
>> -
>> -    if (GRAPHICS_VER(i915) >= 12) {
>> -        struct intel_gt *gt;
>> +    if (!HAS_GUC_TLB_INVALIDATION(i915))
>> +        gen8_ggtt_invalidate(ggtt);
>>   -        list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
>> +    list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
>> +        if (HAS_GUC_TLB_INVALIDATION(i915) &&
>> +            intel_guc_is_ready(&gt->uc.guc)) {
>> +            guc_ggtt_ct_invalidate(gt);
>> +        } else if (GRAPHICS_VER(i915) >= 12) {
>>               intel_uncore_write_fw(gt->uncore,
>>                             GEN12_GUC_TLB_INV_CR,
>>                             GEN12_GUC_TLB_INV_CR_INVALIDATE);
>> -    } else {
>> -        intel_uncore_write_fw(ggtt->vm.gt->uncore,
>> -                      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
>> +        } else {
>> +            intel_uncore_write_fw(gt->uncore,
>> +                          GEN8_GTCR, GEN8_GTCR_INVALIDATE);
>> +        }
>>       }
>>   }
>>   @@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt 
>> *ggtt)
>>           ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
>>       }
>>   -    if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
>> +    if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
>>           ggtt->invalidate = guc_ggtt_invalidate;
>>       else
>>           ggtt->invalidate = gen8_ggtt_invalidate;
>> diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c 
>> b/drivers/gpu/drm/i915/gt/intel_tlb.c
>> index 139608c30d978..1278622b9554d 100644
>> --- a/drivers/gpu/drm/i915/gt/intel_tlb.c
>> +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
>> @@ -12,6 +12,7 @@
>>   #include "intel_gt_print.h"
>>   #include "intel_gt_regs.h"
>>   #include "intel_tlb.h"
>> +#include "uc/intel_guc.h"
>>     /*
>>    * HW architecture suggest typical invalidation time at 40us,
>> @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct 
>> intel_gt *gt, u32 seqno)
>>           return;
>>         with_intel_gt_pm_if_awake(gt, wakeref) {
>> +        struct intel_guc *guc = &gt->uc.guc;
>> +
>>           mutex_lock(&gt->tlb.invalidate_lock);
>>           if (tlb_seqno_passed(gt, seqno))
>>               goto unlock;
>>   -        mmio_invalidate_full(gt);
>> +        if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
>> +            /*
>> +             * Only perform GuC TLB invalidation if GuC is ready.
>> +             * If GuC is not ready, then there are no TLBs to
>> +             * invalidate.  Ergo, skip invalidation.
>> +             */
>> +            if (intel_guc_is_ready(guc))
>> +                intel_guc_invalidate_tlb_engines(guc);
>
> What was the answer to John's question on why it is okay to just skip 
> and not maybe fall back to mmio?
>
>> +        } else {
>> +            mmio_invalidate_full(gt);
>> +        }
>>             write_seqcount_invalidate(&gt->tlb.seqno);
>>   unlock:
>> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h 
>> b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> index f359bef046e0b..33f253410d0c8 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> @@ -138,6 +138,8 @@ enum intel_guc_action {
>>       INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
>>       INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
>>       INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
>> +    INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
>> +    INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
>>       INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
>>       INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
>>       INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
>> @@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
>>     #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK 0x000000FF
>>   +#define INTEL_GUC_TLB_INVAL_TYPE_MASK    REG_GENMASK(7, 0)
>> +#define INTEL_GUC_TLB_INVAL_MODE_MASK    REG_GENMASK(11, 8)
>> +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
>> +
>> +enum intel_guc_tlb_invalidation_type {
>> +    INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
>> +    INTEL_GUC_TLB_INVAL_GUC = 0x3,
>> +};
>> +
>> +/*
>> + * 0: Heavy mode of Invalidation:
>> + * The pipeline of the engine(s) for which the invalidation is 
>> targeted to is
>> + * blocked, and all the in-flight transactions are guaranteed to be 
>> Globally
>> + * Observed before completing the TLB invalidation
>> + * 1: Lite mode of Invalidation:
>> + * TLBs of the targeted engine(s) are immediately invalidated.
>> + * In-flight transactions are NOT guaranteed to be Globally Observed 
>> before
>> + * completing TLB invalidation.
>> + * Light Invalidation Mode is to be used only when
>> + * it can be guaranteed (by SW) that the address translations remain 
>> invariant
>> + * for the in-flight transactions across the TLB invalidation. In 
>> other words,
>> + * this mode can be used when the TLB invalidation is intended to 
>> clear out the
>> + * stale cached translations that are no longer in use. Light 
>> Invalidation Mode
>> + * is much faster than the Heavy Invalidation Mode, as it does not 
>> wait for the
>> + * in-flight transactions to be GOd.
>> + */
>> +enum intel_guc_tlb_inval_mode {
>> +    INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
>> +    INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
>> +};
>> +
>>   #endif /* _ABI_GUC_ACTIONS_ABI_H */
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h 
>> b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> index 6c392bad29c19..88deb43bbdc48 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> @@ -79,6 +79,18 @@ struct intel_guc {
>>        */
>>       atomic_t outstanding_submission_g2h;
>>   +    /** @tlb_lookup: xarray to store all pending TLB invalidation 
>> requests */
>> +    struct xarray tlb_lookup;
>> +
>> +    /**
>> +     * @serial_slot: id to the initial waiter created in tlb_lookup,
>> +     * which is used only when failed to allocate new waiter.
>> +     */
>> +    u32 serial_slot;
>> +
>> +    /** @next_seqno: the next id (sequence no.) to allocate. */
>> +    u32 next_seqno;
>> +
>>       /** @interrupts: pointers to GuC interrupt-managing functions. */
>>       struct {
>>           bool enabled;
>> @@ -296,6 +308,11 @@ struct intel_guc {
>>   #define MAKE_GUC_VER_STRUCT(ver)    MAKE_GUC_VER((ver).major, 
>> (ver).minor, (ver).patch)
>>   #define GUC_SUBMIT_VER(guc) 
>> MAKE_GUC_VER_STRUCT((guc)->submission_version)
>>   +struct intel_guc_tlb_wait {
>> +    struct wait_queue_head wq;
>> +    bool busy;
>> +};
>> +
>>   static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
>>   {
>>       return container_of(log, struct intel_guc, log);
>> @@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct 
>> intel_guc *guc)
>>       return intel_uc_fw_is_supported(&guc->fw);
>>   }
>>   +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
>> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
>> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
>> +                    u32 len, u32 seqno);
>> +
>>   static inline bool intel_guc_is_wanted(struct intel_guc *guc)
>>   {
>>       return intel_uc_fw_is_enabled(&guc->fw);
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c 
>> b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> index 6e22af31513a5..e7ac8402245b7 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> @@ -1115,6 +1115,9 @@ static int ct_process_request(struct 
>> intel_guc_ct *ct, struct ct_incoming_msg *r
>>       case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
>>           ret = intel_guc_crash_process_msg(guc, action);
>>           break;
>> +    case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>> +        ret = intel_guc_tlb_invalidation_done(guc, request->size, 
>> len, payload[0]);
This is still non-standard with respect to all other notification handlers.

The convention is to pass '(guc, payload, len)'. The only exception is 
the two exception notifications above which have no payload and share a 
common handler for two different actions. This is just a normal single 
action, with payload handler. So should match the convention of the others.

>> +        break;
>>       default:
>>           ret = -EOPNOTSUPP;
>>           break;
>> @@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct 
>> *ct, struct ct_incoming_msg *requ
>>       switch (action) {
>>       case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
>>       case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
>> +    case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>>           g2h_release_space(ct, request->size);
>>       }
>>   diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h 
>> b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> index b4d56eccfb1f0..a7c9874e122a3 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> @@ -22,6 +22,7 @@
>>   /* Payload length only i.e. don't include G2H header length */
>>   #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET    2
>>   #define G2H_LEN_DW_DEREGISTER_CONTEXT        1
>> +#define G2H_LEN_DW_INVALIDATE_TLB        1
>>     #define GUC_CONTEXT_DISABLE        0
>>   #define GUC_CONTEXT_ENABLE        1
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c 
>> b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> index 2cce5ec1ff00d..0c5ccda1b3e87 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> @@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct 
>> intel_context *ce, intel_engine_mask_t st
>>     void intel_guc_submission_reset(struct intel_guc *guc, 
>> intel_engine_mask_t stalled)
>>   {
>> +    struct intel_guc_tlb_wait *wait;
>>       struct intel_context *ce;
>>       unsigned long index;
>>       unsigned long flags;
>> +    unsigned long i;
>>         if (unlikely(!guc_submission_initialized(guc))) {
>>           /* Reset called during driver load? GuC not yet 
>> initialised! */
>> @@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct 
>> intel_guc *guc, intel_engine_mask_t stall
>>         /* GuC is blown away, drop all references to contexts */
>>       xa_destroy(&guc->context_lookup);
>> +
>> +    /*
>> +     * The full GT reset will have cleared the TLB caches and 
>> flushed the
>> +     * G2H message queue; we can release all the blocked waiters.
>> +     *
>> +     * This is safe to do unlocked because the xarray is not dependent
>> +     * on the GT reset, and there's a separate execution path for TLB
>> +     * invalidations on GT reset, and there's a large window of time
>> +     * between the GT reset and GuC becoming available.
>> +     */
>> +    xa_for_each(&guc->tlb_lookup, i, wait)
>> +        wake_up(&wait->wq);
>
> If you are confident there can be no failures to wake up someone, who 
> maybe just added themselves to the xarray (via put pages for 
> instance), while reset in ongoing. Or even removed themselves after 
> say timing out the wait and so freed their entry...
>
>>   }
>>     static void guc_cancel_context_requests(struct intel_context *ce)
>> @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct 
>> intel_guc *guc)
>>   static void destroyed_worker_func(struct work_struct *w);
>>   static void reset_fail_worker_func(struct work_struct *w);
>>   +static int init_tlb_lookup(struct intel_guc *guc)
>> +{
>> +    struct intel_guc_tlb_wait *wait;
>> +    int err;
>> +
>> +    if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +        return 0;
>> +
>> +    xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
>> +
>> +    wait = kzalloc(sizeof(*wait), GFP_KERNEL);
>> +    if (!wait)
>> +        return -ENOMEM;
>> +
>> +    init_waitqueue_head(&wait->wq);
>> +
>> +    /* Preallocate a shared id for use under memory pressure. */
>> +    err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, 
>> wait,
>> +                  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
>> +    /* Only error if out of memory, not when busy (list full)*/
>> +    if (err == -ENOMEM) {
>> +        kfree(wait);
>> +        return err;
>> +    }
>
> I agreed with John here that only looking at ENOMEM reads odd and I 
> did not see that answered. Did I miss it?
>
> Otherwise, I _know_ it is not likely to get any other error having 
> *just* created a new xarray, but still, why not simply catch all? It 
> is not like the slot fallback code at runtime would handle 
> guc->serial_slot being empty?! It appears it would just explode in 
> guc_send_invalidate_tlb if it would hit it..
To be clear, if you are going to write an error handler that checks for 
one specific error and only that one error, i.e. it completely ignores 
any other error, then it needs a big clear comment as to exactly why it 
is safe and valid to ignore any and all other errors. Saying 'do not 
error out when busy' makes no sense for an an allocation function. This 
is not acquiring a resource. This is allocating a tracking structure. 
How can it be busy? How can it not be a fatal problem if a tracking 
structure failed to allocate due to being 'busy'?

And even if 'busy' is safe to ignore, the code is not explicitly 
checking for EBUSY and letting that through. What if the call were to 
return EINVAL, EINTR, EANYTHINGELSEATALL? If you have a known safe error 
case then you must explicitly check for that and fail on anything else. 
Not fail on one specific case and pass through anything else.

>
>> +
>> +    return 0;
>> +}
>> +
>> +static void fini_tlb_lookup(struct intel_guc *guc)
>> +{
>> +    struct intel_guc_tlb_wait *wait;
>> +
>> +    if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +        return;
>> +
>> +    wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
>> +    if (wait) {
>> +        guc_dbg(guc, "fini_tlb_lookup: Unexpected item in 
>> tlb_lookup\n");
>
> Hm wait, why is this unexpected when init_tlb_lookup() pre-allocated 
> that entry? Who frees it? guc_send_invalidate_tlb() does not appear to.
The original version of this code was 'BUG_ON(wait->busy)';. Should the 
dbg print be conditional on wait->busy being set? And should it be an 
error not a debug print?

John.

>
>> +        kfree(wait);
>> +    }
>> +
>> +    xa_destroy(&guc->tlb_lookup);
>> +}
>> +
>>   /*
>>    * Set up the memory resources to be shared with the GuC (via the 
>> GGTT)
>>    * at firmware loading time.
>> @@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct 
>> intel_guc *guc)
>>               return ret;
>>       }
>>   +    ret = init_tlb_lookup(guc);
>> +    if (ret)
>> +        goto destroy_pool;
>> +
>>       guc->submission_state.guc_ids_bitmap =
>>           bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
>>       if (!guc->submission_state.guc_ids_bitmap) {
>>           ret = -ENOMEM;
>> -        goto destroy_pool;
>> +        goto destroy_tlb;
>>       }
>>         guc->timestamp.ping_delay = (POLL_TIME_CLKS / 
>> gt->clock_frequency + 1) * HZ;
>> @@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc 
>> *guc)
>>         return 0;
>>   +destroy_tlb:
>> +    fini_tlb_lookup(guc);
>>   destroy_pool:
>>       guc_lrc_desc_pool_destroy_v69(guc);
>> -
>>       return ret;
>>   }
>>   @@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct 
>> intel_guc *guc)
>>       guc_lrc_desc_pool_destroy_v69(guc);
>>       i915_sched_engine_put(guc->sched_engine);
>>       bitmap_free(guc->submission_state.guc_ids_bitmap);
>> +    fini_tlb_lookup(guc);
>>       guc->submission_initialized = false;
>>   }
>>   @@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, 
>> u32 ctx_id)
>>       return ce;
>>   }
>>   +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, 
>> u32 seqno)
>> +{
>> +    struct intel_guc_tlb_wait *wait;
>> +    unsigned long flags;
>> +
>> +    xa_lock_irqsave(&guc->tlb_lookup, flags);
>> +    wait = xa_load(&guc->tlb_lookup, seqno);
>> +
>> +    if (wait)
>> +        wake_up(&wait->wq);
>> +    else
>> +        guc_dbg(guc,
>> +            "Stale TLB invalidation response with seqno %d\n", seqno);
>> +
>> +    xa_unlock_irqrestore(&guc->tlb_lookup, flags);
>> +}
>> +
>> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, 
>> u32 len, u32 seqno)
>> +{
>> +    /* Check for underflow */
>> +    if (unlikely(len < 1 || len > size))
>> +        return -EPROTO;
>
> These check are not valid for any message/action type 
> ct_process_request() can receive?
>
>> +
>> +    wait_wake_outstanding_tlb_g2h(guc, seqno);
>> +    return 0;
>> +}
>> +
>> +static long must_wait_woken(struct wait_queue_entry *wq_entry, long 
>> timeout)
>> +{
>> +    /*
>> +     * This is equivalent to wait_woken() with the exception that
>> +     * we do not wake up early if the kthread task has been completed.
>> +     * As we are called from page reclaim in any task context,
>> +     * we may be invoked from stopped kthreads, but we *must*
>> +     * complete the wait from the HW .
>> +     *
>> +     * A second problem is that since we are called under reclaim
>> +     * and wait_woken() inspected the thread state, it makes an invalid
>> +     * assumption that all PF_KTHREAD tasks have set_kthread_struct()
>> +     * called upon them, and will trigger a GPF in 
>> is_kthread_should_stop().
>> +     */
>> +    do {
>> +        set_current_state(TASK_UNINTERRUPTIBLE);
>> +        if (wq_entry->flags & WQ_FLAG_WOKEN)
>> +            break;
>> +
>> +        timeout = schedule_timeout(timeout);
>> +    } while (timeout);
>> +    __set_current_state(TASK_RUNNING);
>> +
>> +    /* See wait_woken() and woken_wake_function() */
>> +    smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
>> +
>> +    return timeout;
>> +}
>> +
>> +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum 
>> intel_guc_tlb_inval_mode type)
>> +{
>> +    struct intel_guc_tlb_wait _wq, *wq = &_wq;
>> +    DEFINE_WAIT_FUNC(wait, woken_wake_function);
>> +    int err;
>> +    u32 seqno;
>> +    u32 action[] = {
>> +        INTEL_GUC_ACTION_TLB_INVALIDATION,
>> +        0,
>> +        REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
>> +            REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
>> +                       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
>> +            INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
>> +    };
>> +    u32 size = ARRAY_SIZE(action);
>> +
>> +    init_waitqueue_head(&_wq.wq);
>> +
>> +    if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
>> +                xa_limit_32b, &guc->next_seqno,
>> +                GFP_ATOMIC | __GFP_NOWARN) < 0) {
>> +        /* Under severe memory pressure? Serialise TLB allocations */
>> +        xa_lock_irq(&guc->tlb_lookup);
>> +        wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
>> +        wait_event_lock_irq(wq->wq,
>> +                    !READ_ONCE(wq->busy),
>> +                    guc->tlb_lookup.xa_lock);
>> +        /*
>> +         * Update wq->busy under lock to ensure only one waiter can
>> +         * issue the TLB invalidation command using the serial slot 
>> at a
>> +         * time. The condition is set to true before releasing the lock
>> +         * so that other caller continue to wait until woken up again.
>> +         */
>> +        wq->busy = true;
>> +        xa_unlock_irq(&guc->tlb_lookup);
>> +
>> +        seqno = guc->serial_slot;
>> +    }
>> +
>> +    action[1] = seqno;
>> +
>> +    add_wait_queue(&wq->wq, &wait);
>> +
>> +    /*
>> +     * This is a critical reclaim path and thus we must loop here:
>> +     * We cannot block for anything that is on the GPU.
>> +     */
>> +    err = intel_guc_send_busy_loop(guc, action, size, 
>> G2H_LEN_DW_INVALIDATE_TLB, true);
>> +    if (err)
>> +        goto out;
>> +
>> +    if (!must_wait_woken(&wait, 
>> intel_guc_ct_expected_delay(&guc->ct))) {
>> +        guc_err(guc,
>> +            "TLB invalidation response timed out for seqno %u\n", 
>> seqno);
>> +        err = -ETIME;
>> +    }
>> +out:
>> +    remove_wait_queue(&wq->wq, &wait);
>> +    if (seqno != guc->serial_slot)
>> +        xa_erase_irq(&guc->tlb_lookup, seqno);
>> +
>> +    return err;
>> +}
>> +
>> +/* Full TLB invalidation */
>> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
>> +{
>> +    return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
>> +}
>> +
>> +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
>> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
>> +{
>> +    return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
>> +}
>> +
>>   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
>>                         const u32 *msg,
>>                         u32 len)
>
> Regards,
>
> Tvrtko

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Tvrtko Ursulin]

On 09/10/2023 20:14, John Harrison wrote:
> On 10/9/2023 01:56, Tvrtko Ursulin wrote:
>> On 06/10/2023 19:20, Jonathan Cavitt wrote:
>>> From: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
>>>
>>> The GuC firmware had defined the interface for Translation Look-Aside
>>> Buffer (TLB) invalidation.  We should use this interface when
>>> invalidating the engine and GuC TLBs.
>>> Add additional functionality to intel_gt_invalidate_tlb, invalidating
>>> the GuC TLBs and falling back to GT invalidation when the GuC is
>>> disabled.
>>> The invalidation is done by sending a request directly to the GuC
>>> tlb_lookup that invalidates the table.  The invalidation is submitted as
>>> a wait request and is performed in the CT event handler.  This means we
>>> cannot perform this TLB invalidation path if the CT is not enabled.
>>> If the request isn't fulfilled in two seconds, this would constitute
>>> an error in the invalidation as that would constitute either a lost
>>> request or a severe GuC overload.
>>>
>>> With this new invalidation routine, we can perform GuC-based GGTT
>>> invalidations.  GuC-based GGTT invalidation is incompatible with
>>> MMIO invalidation so we should not perform MMIO invalidation when
>>> GuC-based GGTT invalidation is expected.
>>>
>>> Purpose of xarray:
>>> The tlb_lookup table is allocated as an xarray because the set of
>>> pending TLB invalidations may have no upper bound.  The consequence of
>>> this is that all actions interfacing with this table need to use the
>>> xarray functions, such as xa_alloc_cyclic_irq for array insertion.
>>>
>>> Purpose of must_wait_woken:
>>> Our wait for the G2H ack for the completion of a TLB invalidation is
>>> mandatory; we must wait for the HW to confirm that the physical
>>> addresses are no longer accessible before we return those to the system.
>>>
>>> On switching to using the wait_woken() convenience routine, we
>>> introduced ourselves to an issue where wait_woken() may complete early
>>> under a kthread that is stopped. Since we send a TLB invalidation when
>>> we try to release pages from the shrinker, we can be called from any
>>> process; including kthreads.
>>>
>>> Using wait_woken() from any process context causes another issue. The
>>> use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
>>> set was made by kthread_create() and has called set_kthread_struct().
>>> This is not true for the raw kernel_thread():
>>
>> This explanation misses the main point of my ask - which is to explain 
>> why a simpler scheme isn't sufficient. Simpler scheme aka not needed 
>> the xarray or any flavour of wait_token().
>>
>> In other words it is obvious we have to wait for the invalidation ack, 
>> but not obvious why we need a complicated scheme.
> The alternative being to simply serialise all TLB invalidation requests? 
> Thus, no complex tracking required as there is only one in flight at a 
> time? That seems inefficient and a potential performance impact if a 
> bunch of invalidations are required back to back. But given that the 
> current scheme is global invalidation only (no support for ranges / per 
> page invalidation yet), is it possible to get multiple back-to-back 
> requests?

It is possible to get a stream of invalidation requests but all that 
come with put_pages() are serialized on the gt->tlb.invalidate_lock 
anyway. So for them only benefit with the complicated approach versus 
the dumb single wait queue is avoiding wake up storms.

Then the second source of TLB invalidations is ggtt->invalidate(). I am 
not sure if those are frequent enough to warrant parallelism. Definitely 
shouldn't be for things like context images and ringbuffers. So I was 
asking if maybe framebuffers but don't know.

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Nirmoy Das]

[-- Attachment #1: Type: text/plain, Size: 23321 bytes --]


On 10/6/2023 8:20 PM, Jonathan Cavitt wrote:
> From: Prathap Kumar Valsan<prathap.kumar.valsan@intel.com>
>
> The GuC firmware had defined the interface for Translation Look-Aside
> Buffer (TLB) invalidation.  We should use this interface when
> invalidating the engine and GuC TLBs.
> Add additional functionality to intel_gt_invalidate_tlb, invalidating
> the GuC TLBs and falling back to GT invalidation when the GuC is
> disabled.
> The invalidation is done by sending a request directly to the GuC
> tlb_lookup that invalidates the table.  The invalidation is submitted as
> a wait request and is performed in the CT event handler.  This means we
> cannot perform this TLB invalidation path if the CT is not enabled.
> If the request isn't fulfilled in two seconds, this would constitute
> an error in the invalidation as that would constitute either a lost
> request or a severe GuC overload.
>
> With this new invalidation routine, we can perform GuC-based GGTT
> invalidations.  GuC-based GGTT invalidation is incompatible with
> MMIO invalidation so we should not perform MMIO invalidation when
> GuC-based GGTT invalidation is expected.
>
> Purpose of xarray:
> The tlb_lookup table is allocated as an xarray because the set of
> pending TLB invalidations may have no upper bound.  The consequence of
> this is that all actions interfacing with this table need to use the
> xarray functions, such as xa_alloc_cyclic_irq for array insertion.
>
> Purpose of must_wait_woken:
> Our wait for the G2H ack for the completion of a TLB invalidation is
> mandatory; we must wait for the HW to confirm that the physical
> addresses are no longer accessible before we return those to the system.
>
> On switching to using the wait_woken() convenience routine, we
> introduced ourselves to an issue where wait_woken() may complete early
> under a kthread that is stopped. Since we send a TLB invalidation when
> we try to release pages from the shrinker, we can be called from any
> process; including kthreads.
>
> Using wait_woken() from any process context causes another issue. The
> use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
> set was made by kthread_create() and has called set_kthread_struct().
> This is not true for the raw kernel_thread():
>
> BUG: kernel NULL pointer dereference, address: 0000000000000000
> [ 3089.759660] Call Trace:
> [ 3089.762110]  wait_woken+0x4f/0x80
> [ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
> [ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
> [ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
> [ 3089.779430]  ? __switch_to_asm+0x40/0x70
> [ 3089.783349]  ? __switch_to_asm+0x34/0x70
> [ 3089.787273]  ? __switch_to+0x7a/0x3e0
> [ 3089.790930]  ? __switch_to_asm+0x34/0x70
> [ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
> [ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
> [ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
> [ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
> [ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
> [ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
> [ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
> [ 3089.832230]  do_shrink_slab+0x12c/0x2a0
> [ 3089.836065]  shrink_slab+0xad/0x2b0
> [ 3089.839550]  shrink_node+0xcc/0x410
> [ 3089.843035]  do_try_to_free_pages+0xc6/0x380
> [ 3089.847306]  try_to_free_pages+0xec/0x1c0
> [ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
> [ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
> [ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
> [ 3089.864737]  new_slab+0x265/0xa80
> [ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
> [ 3089.871798]  ? copy_process+0x1e5/0x1a00
> [ 3089.875717]  ? load_balance+0x165/0xb20
> [ 3089.879555]  __slab_alloc+0x1c/0x30
> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> [ 3089.891314]  copy_process+0x1e5/0x1a00
> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> [ 3089.879555]  __slab_alloc+0x1c/0x30
> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
> [ 3089.891314]  copy_process+0x1e5/0x1a00
> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
> [ 3089.898977]  ? __switch_to_asm+0x34/0x70
> [ 3089.902903]  ? __switch_to_asm+0x40/0x70
> [ 3089.906828]  ? __switch_to_asm+0x34/0x70
> [ 3089.910745]  _do_fork+0x83/0x350
> [ 3089.913969]  ? __switch_to+0x7a/0x3e0
> [ 3089.917626]  ? __switch_to_asm+0x34/0x70
> [ 3089.921545]  kernel_thread+0x58/0x80
> [ 3089.925124]  ? kthread_park+0x80/0x80
> [ 3089.928788]  kthreadd+0x162/0x1b0
> [ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
> [ 3089.936538]  ret_from_fork+0x1f/0x40
>
> Signed-off-by: Prathap Kumar Valsan<prathap.kumar.valsan@intel.com>
> Signed-off-by: Bruce Chang<yu.bruce.chang@intel.com>
> Signed-off-by: Chris Wilson<chris.p.wilson@intel.com>
> Signed-off-by: Umesh Nerlige Ramappa<umesh.nerlige.ramappa@intel.com>
> Signed-off-by: Jonathan Cavitt<jonathan.cavitt@intel.com>
> Signed-off-by: Aravind Iddamsetty<aravind.iddamsetty@intel.com>
> Signed-off-by: Fei Yang<fei.yang@intel.com>
> CC: Andi Shyti<andi.shyti@linux.intel.com>
> ---
>   drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
>   drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
>   .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
>   drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
>   drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
>   7 files changed, 297 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> index 4d7d88b92632b..a1f7bdc602996 100644
> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> @@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct i915_ggtt *ggtt)
>   	intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
>   }
>   
> +static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
> +{
> +	struct intel_uncore *uncore = gt->uncore;
> +	intel_wakeref_t wakeref;
> +
> +	with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
> +		struct intel_guc *guc = &gt->uc.guc;
> +
> +		intel_guc_invalidate_tlb_guc(guc);
> +	}
> +}
> +
>   static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
>   {
>   	struct drm_i915_private *i915 = ggtt->vm.i915;
> +	struct intel_gt *gt;
>   
> -	gen8_ggtt_invalidate(ggtt);
> -
> -	if (GRAPHICS_VER(i915) >= 12) {
> -		struct intel_gt *gt;
> +	if (!HAS_GUC_TLB_INVALIDATION(i915))
> +		gen8_ggtt_invalidate(ggtt);
>   
> -		list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
> +	list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
> +		if (HAS_GUC_TLB_INVALIDATION(i915) &&
> +		    intel_guc_is_ready(&gt->uc.guc)) {
> +			guc_ggtt_ct_invalidate(gt);
> +		} else if (GRAPHICS_VER(i915) >= 12) {
>   			intel_uncore_write_fw(gt->uncore,
>   					      GEN12_GUC_TLB_INV_CR,
>   					      GEN12_GUC_TLB_INV_CR_INVALIDATE);
> -	} else {
> -		intel_uncore_write_fw(ggtt->vm.gt->uncore,
> -				      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> +		} else {
> +			intel_uncore_write_fw(gt->uncore,
> +					      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
> +		}
>   	}
>   }
>   
> @@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
>   		ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
>   	}
>   
> -	if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
> +	if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
>   		ggtt->invalidate = guc_ggtt_invalidate;
>   	else
>   		ggtt->invalidate = gen8_ggtt_invalidate;
> diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c
> index 139608c30d978..1278622b9554d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_tlb.c
> +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
> @@ -12,6 +12,7 @@
>   #include "intel_gt_print.h"
>   #include "intel_gt_regs.h"
>   #include "intel_tlb.h"
> +#include "uc/intel_guc.h"
>   
>   /*
>    * HW architecture suggest typical invalidation time at 40us,
> @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
>   		return;
>   
>   	with_intel_gt_pm_if_awake(gt, wakeref) {
> +		struct intel_guc *guc = &gt->uc.guc;
> +
>   		mutex_lock(&gt->tlb.invalidate_lock);
>   		if (tlb_seqno_passed(gt, seqno))
>   			goto unlock;
>   
> -		mmio_invalidate_full(gt);
> +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
> +			/*
> +			 * Only perform GuC TLB invalidation if GuC is ready.
> +			 * If GuC is not ready, then there are no TLBs to
> +			 * invalidate.  Ergo, skip invalidation.
> +			 */
> +			if (intel_guc_is_ready(guc))
> +				intel_guc_invalidate_tlb_engines(guc);
> +		} else {
> +			mmio_invalidate_full(gt);
> +		}
>   
>   		write_seqcount_invalidate(&gt->tlb.seqno);
>   unlock:
> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> index f359bef046e0b..33f253410d0c8 100644
> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> @@ -138,6 +138,8 @@ enum intel_guc_action {
>   	INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
>   	INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
>   	INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
> +	INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
> +	INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
>   	INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
>   	INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
>   	INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
> @@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
>   
>   #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK      0x000000FF
>   
> +#define INTEL_GUC_TLB_INVAL_TYPE_MASK	REG_GENMASK(7, 0)
> +#define INTEL_GUC_TLB_INVAL_MODE_MASK	REG_GENMASK(11, 8)
> +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
> +
> +enum intel_guc_tlb_invalidation_type {
> +	INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
> +	INTEL_GUC_TLB_INVAL_GUC = 0x3,
> +};
> +
> +/*
> + * 0: Heavy mode of Invalidation:
> + * The pipeline of the engine(s) for which the invalidation is targeted to is
> + * blocked, and all the in-flight transactions are guaranteed to be Globally
> + * Observed before completing the TLB invalidation
> + * 1: Lite mode of Invalidation:
> + * TLBs of the targeted engine(s) are immediately invalidated.
> + * In-flight transactions are NOT guaranteed to be Globally Observed before
> + * completing TLB invalidation.
> + * Light Invalidation Mode is to be used only when
> + * it can be guaranteed (by SW) that the address translations remain invariant
> + * for the in-flight transactions across the TLB invalidation. In other words,
> + * this mode can be used when the TLB invalidation is intended to clear out the
> + * stale cached translations that are no longer in use. Light Invalidation Mode
> + * is much faster than the Heavy Invalidation Mode, as it does not wait for the
> + * in-flight transactions to be GOd.
> + */
> +enum intel_guc_tlb_inval_mode {
> +	INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
> +	INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
> +};
> +
>   #endif /* _ABI_GUC_ACTIONS_ABI_H */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> index 6c392bad29c19..88deb43bbdc48 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> @@ -79,6 +79,18 @@ struct intel_guc {
>   	 */
>   	atomic_t outstanding_submission_g2h;
>   
> +	/** @tlb_lookup: xarray to store all pending TLB invalidation requests */
> +	struct xarray tlb_lookup;
> +
> +	/**
> +	 * @serial_slot: id to the initial waiter created in tlb_lookup,
> +	 * which is used only when failed to allocate new waiter.
> +	 */
> +	u32 serial_slot;
> +
> +	/** @next_seqno: the next id (sequence no.) to allocate. */
> +	u32 next_seqno;
> +
>   	/** @interrupts: pointers to GuC interrupt-managing functions. */
>   	struct {
>   		bool enabled;
> @@ -296,6 +308,11 @@ struct intel_guc {
>   #define MAKE_GUC_VER_STRUCT(ver)	MAKE_GUC_VER((ver).major, (ver).minor, (ver).patch)
>   #define GUC_SUBMIT_VER(guc)		MAKE_GUC_VER_STRUCT((guc)->submission_version)
>   
> +struct intel_guc_tlb_wait {
> +	struct wait_queue_head wq;
> +	bool busy;
> +};
> +
>   static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
>   {
>   	return container_of(log, struct intel_guc, log);
> @@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct intel_guc *guc)
>   	return intel_uc_fw_is_supported(&guc->fw);
>   }
>   
> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
> +				    u32 len, u32 seqno);
> +
>   static inline bool intel_guc_is_wanted(struct intel_guc *guc)
>   {
>   	return intel_uc_fw_is_enabled(&guc->fw);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> index 6e22af31513a5..e7ac8402245b7 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> @@ -1115,6 +1115,9 @@ static int ct_process_request(struct intel_guc_ct *ct, struct ct_incoming_msg *r
>   	case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
>   		ret = intel_guc_crash_process_msg(guc, action);
>   		break;
> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
> +		ret = intel_guc_tlb_invalidation_done(guc, request->size, len, payload[0]);
> +		break;
>   	default:
>   		ret = -EOPNOTSUPP;
>   		break;
> @@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct *ct, struct ct_incoming_msg *requ
>   	switch (action) {
>   	case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
>   	case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>   		g2h_release_space(ct, request->size);
>   	}
>   
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> index b4d56eccfb1f0..a7c9874e122a3 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> @@ -22,6 +22,7 @@
>   /* Payload length only i.e. don't include G2H header length */
>   #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET	2
>   #define G2H_LEN_DW_DEREGISTER_CONTEXT		1
> +#define G2H_LEN_DW_INVALIDATE_TLB		1
>   
>   #define GUC_CONTEXT_DISABLE		0
>   #define GUC_CONTEXT_ENABLE		1
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 2cce5ec1ff00d..0c5ccda1b3e87 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
>   
>   void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
>   {
> +	struct intel_guc_tlb_wait *wait;
>   	struct intel_context *ce;
>   	unsigned long index;
>   	unsigned long flags;
> +	unsigned long i;
>   
>   	if (unlikely(!guc_submission_initialized(guc))) {
>   		/* Reset called during driver load? GuC not yet initialised! */
> @@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
>   
>   	/* GuC is blown away, drop all references to contexts */
>   	xa_destroy(&guc->context_lookup);
> +
> +	/*
> +	 * The full GT reset will have cleared the TLB caches and flushed the
> +	 * G2H message queue; we can release all the blocked waiters.
> +	 *
> +	 * This is safe to do unlocked because the xarray is not dependent
> +	 * on the GT reset, and there's a separate execution path for TLB
> +	 * invalidations on GT reset, and there's a large window of time
> +	 * between the GT reset and GuC becoming available.
> +	 */
> +	xa_for_each(&guc->tlb_lookup, i, wait)
> +		wake_up(&wait->wq);
>   }
>   
>   static void guc_cancel_context_requests(struct intel_context *ce)
> @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
>   static void destroyed_worker_func(struct work_struct *w);
>   static void reset_fail_worker_func(struct work_struct *w);
>   
> +static int init_tlb_lookup(struct intel_guc *guc)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +	int err;
> +
> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		return 0;
> +
> +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
> +
> +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
> +	if (!wait)
> +		return -ENOMEM;
> +
> +	init_waitqueue_head(&wait->wq);
> +
> +	/* Preallocate a shared id for use under memory pressure. */
> +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
> +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
> +	/* Only error if out of memory, not when busy (list full)*/
> +	if (err == -ENOMEM) {
> +		kfree(wait);
> +		return err;
> +	}
> +
> +	return 0;
> +}
> +
> +static void fini_tlb_lookup(struct intel_guc *guc)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +
> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		return;
> +
> +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
> +	if (wait) {
> +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
> +		kfree(wait);
> +	}
> +
> +	xa_destroy(&guc->tlb_lookup);
> +}
> +
>   /*
>    * Set up the memory resources to be shared with the GuC (via the GGTT)
>    * at firmware loading time.
> @@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct intel_guc *guc)
>   			return ret;
>   	}
>   
> +	ret = init_tlb_lookup(guc);
> +	if (ret)
> +		goto destroy_pool;
> +
>   	guc->submission_state.guc_ids_bitmap =
>   		bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
>   	if (!guc->submission_state.guc_ids_bitmap) {
>   		ret = -ENOMEM;
> -		goto destroy_pool;
> +		goto destroy_tlb;
>   	}
>   
>   	guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
> @@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc *guc)
>   
>   	return 0;
>   
> +destroy_tlb:
> +	fini_tlb_lookup(guc);
>   destroy_pool:
>   	guc_lrc_desc_pool_destroy_v69(guc);
> -
>   	return ret;
>   }
>   
> @@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
>   	guc_lrc_desc_pool_destroy_v69(guc);
>   	i915_sched_engine_put(guc->sched_engine);
>   	bitmap_free(guc->submission_state.guc_ids_bitmap);
> +	fini_tlb_lookup(guc);
>   	guc->submission_initialized = false;
>   }
>   
> @@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id)
>   	return ce;
>   }
>   
> +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno)
> +{
> +	struct intel_guc_tlb_wait *wait;
> +	unsigned long flags;
> +
> +	xa_lock_irqsave(&guc->tlb_lookup, flags);
> +	wait = xa_load(&guc->tlb_lookup, seqno);
> +
> +	if (wait)
> +		wake_up(&wait->wq);
> +	else
> +		guc_dbg(guc,
> +			"Stale TLB invalidation response with seqno %d\n", seqno);
> +
> +	xa_unlock_irqrestore(&guc->tlb_lookup, flags);
> +}
> +
> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
> +{
> +	/* Check for underflow */
> +	if (unlikely(len < 1 || len > size))
> +		return -EPROTO;
> +
> +	wait_wake_outstanding_tlb_g2h(guc, seqno);
> +	return 0;
> +}
> +
> +static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
> +{
> +	/*
> +	 * This is equivalent to wait_woken() with the exception that
> +	 * we do not wake up early if the kthread task has been completed.
> +	 * As we are called from page reclaim in any task context,
> +	 * we may be invoked from stopped kthreads, but we *must*
> +	 * complete the wait from the HW .
> +	 *
> +	 * A second problem is that since we are called under reclaim
> +	 * and wait_woken() inspected the thread state, it makes an invalid
> +	 * assumption that all PF_KTHREAD tasks have set_kthread_struct()
> +	 * called upon them, and will trigger a GPF

As discussed internally, the GPF issue is resolved with 
https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/ 
<https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/>


>   in is_kthread_should_stop().
> +	 */
> +	do {
> +		set_current_state(TASK_UNINTERRUPTIBLE);
> +		if (wq_entry->flags & WQ_FLAG_WOKEN)
> +			break;
> +
> +		timeout = schedule_timeout(timeout);
> +	} while (timeout);
> +	__set_current_state(TASK_RUNNING);
> +
> +	/* See wait_woken() and woken_wake_function() */
> +	smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
> +
> +	return timeout;
> +}
> +
> +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)


2nd param should be intel_guc_tlb_invalidation_type not 
intel_guc_tlb_inval_mod.

Not sure why didn't CI complained.


Regards,

Nirmoy

> +{
> +	struct intel_guc_tlb_wait _wq, *wq = &_wq;
> +	DEFINE_WAIT_FUNC(wait, woken_wake_function);
> +	int err;
> +	u32 seqno;
> +	u32 action[] = {
> +		INTEL_GUC_ACTION_TLB_INVALIDATION,
> +		0,
> +		REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
> +			REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
> +				       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
> +			INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
> +	};
> +	u32 size = ARRAY_SIZE(action);
> +
> +	init_waitqueue_head(&_wq.wq);
> +
> +	if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
> +				xa_limit_32b, &guc->next_seqno,
> +				GFP_ATOMIC | __GFP_NOWARN) < 0) {
> +		/* Under severe memory pressure? Serialise TLB allocations */
> +		xa_lock_irq(&guc->tlb_lookup);
> +		wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
> +		wait_event_lock_irq(wq->wq,
> +				    !READ_ONCE(wq->busy),
> +				    guc->tlb_lookup.xa_lock);
> +		/*
> +		 * Update wq->busy under lock to ensure only one waiter can
> +		 * issue the TLB invalidation command using the serial slot at a
> +		 * time. The condition is set to true before releasing the lock
> +		 * so that other caller continue to wait until woken up again.
> +		 */
> +		wq->busy = true;
> +		xa_unlock_irq(&guc->tlb_lookup);
> +
> +		seqno = guc->serial_slot;
> +	}
> +
> +	action[1] = seqno;
> +
> +	add_wait_queue(&wq->wq, &wait);
> +
> +	/*
> +	 * This is a critical reclaim path and thus we must loop here:
> +	 * We cannot block for anything that is on the GPU.
> +	 */
> +	err = intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE_TLB, true);
> +	if (err)
> +		goto out;
> +
> +	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
> +		guc_err(guc,
> +			"TLB invalidation response timed out for seqno %u\n", seqno);
> +		err = -ETIME;
> +	}
> +out:
> +	remove_wait_queue(&wq->wq, &wait);
> +	if (seqno != guc->serial_slot)
> +		xa_erase_irq(&guc->tlb_lookup, seqno);
> +
> +	return err;
> +}
> +
> +/* Full TLB invalidation */
> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
> +{
> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
> +}
> +
> +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
> +{
> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
> +}
> +
>   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
>   					  const u32 *msg,
>   					  u32 len)

[-- Attachment #2: Type: text/html, Size: 25432 bytes --]

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Tvrtko Ursulin]

On 09/10/2023 13:12, Nirmoy Das wrote:
> 
> On 10/6/2023 8:20 PM, Jonathan Cavitt wrote:
>> From: Prathap Kumar Valsan<prathap.kumar.valsan@intel.com>
>>
>> The GuC firmware had defined the interface for Translation Look-Aside
>> Buffer (TLB) invalidation.  We should use this interface when
>> invalidating the engine and GuC TLBs.
>> Add additional functionality to intel_gt_invalidate_tlb, invalidating
>> the GuC TLBs and falling back to GT invalidation when the GuC is
>> disabled.
>> The invalidation is done by sending a request directly to the GuC
>> tlb_lookup that invalidates the table.  The invalidation is submitted as
>> a wait request and is performed in the CT event handler.  This means we
>> cannot perform this TLB invalidation path if the CT is not enabled.
>> If the request isn't fulfilled in two seconds, this would constitute
>> an error in the invalidation as that would constitute either a lost
>> request or a severe GuC overload.
>>
>> With this new invalidation routine, we can perform GuC-based GGTT
>> invalidations.  GuC-based GGTT invalidation is incompatible with
>> MMIO invalidation so we should not perform MMIO invalidation when
>> GuC-based GGTT invalidation is expected.
>>
>> Purpose of xarray:
>> The tlb_lookup table is allocated as an xarray because the set of
>> pending TLB invalidations may have no upper bound.  The consequence of
>> this is that all actions interfacing with this table need to use the
>> xarray functions, such as xa_alloc_cyclic_irq for array insertion.
>>
>> Purpose of must_wait_woken:
>> Our wait for the G2H ack for the completion of a TLB invalidation is
>> mandatory; we must wait for the HW to confirm that the physical
>> addresses are no longer accessible before we return those to the system.
>>
>> On switching to using the wait_woken() convenience routine, we
>> introduced ourselves to an issue where wait_woken() may complete early
>> under a kthread that is stopped. Since we send a TLB invalidation when
>> we try to release pages from the shrinker, we can be called from any
>> process; including kthreads.
>>
>> Using wait_woken() from any process context causes another issue. The
>> use of is_kthread_should_stop() assumes that any task with PF_KTHREAD
>> set was made by kthread_create() and has called set_kthread_struct().
>> This is not true for the raw kernel_thread():
>>
>> BUG: kernel NULL pointer dereference, address: 0000000000000000
>> [ 3089.759660] Call Trace:
>> [ 3089.762110]  wait_woken+0x4f/0x80
>> [ 3089.765496]  guc_send_invalidate_tlb+0x1fe/0x310 [i915]
>> [ 3089.770725]  ? syscall_return_via_sysret+0xf/0x7f
>> [ 3089.775426]  ? do_wait_intr_irq+0xb0/0xb0
>> [ 3089.779430]  ? __switch_to_asm+0x40/0x70
>> [ 3089.783349]  ? __switch_to_asm+0x34/0x70
>> [ 3089.787273]  ? __switch_to+0x7a/0x3e0
>> [ 3089.790930]  ? __switch_to_asm+0x34/0x70
>> [ 3089.794883]  intel_guc_invalidate_tlb_full+0x92/0xa0 [i915]
>> [ 3089.800487]  intel_invalidate_tlb_full+0x94/0x190 [i915]
>> [ 3089.805824]  intel_invalidate_tlb_full_sync+0x1b/0x30 [i915]
>> [ 3089.811508]  __i915_gem_object_unset_pages+0x138/0x150 [i915]
>> [ 3089.817279]  __i915_gem_object_put_pages+0x25/0x90 [i915]
>> [ 3089.822706]  i915_gem_shrink+0x532/0x7e0 [i915]
>> [ 3089.827264]  i915_gem_shrinker_scan+0x3d/0xd0 [i915]
>> [ 3089.832230]  do_shrink_slab+0x12c/0x2a0
>> [ 3089.836065]  shrink_slab+0xad/0x2b0
>> [ 3089.839550]  shrink_node+0xcc/0x410
>> [ 3089.843035]  do_try_to_free_pages+0xc6/0x380
>> [ 3089.847306]  try_to_free_pages+0xec/0x1c0
>> [ 3089.851312]  __alloc_pages_slowpath+0x3ad/0xd10
>> [ 3089.855845]  ? update_sd_lb_stats+0x636/0x710
>> [ 3089.860204]  __alloc_pages_nodemask+0x2d5/0x310
>> [ 3089.864737]  new_slab+0x265/0xa80
>> [ 3089.868053]  ___slab_alloc+0y_to_free_pages+0xec/0x1c0
>> [ 3089.871798]  ? copy_process+0x1e5/0x1a00
>> [ 3089.875717]  ? load_balance+0x165/0xb20
>> [ 3089.879555]  __slab_alloc+0x1c/0x30
>> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
>> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
>> [ 3089.891314]  copy_process+0x1e5/0x1a00
>> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
>> [ 3089.879555]  __slab_alloc+0x1c/0x30
>> [ 3089.883047]  kmem_cache_alloc_node+0x9f/0x240
>> [ 3089.887397]  ? copy_process+0x1e5/0x1a00
>> [ 3089.891314]  copy_process+0x1e5/0x1a00
>> [ 3089.895058]  ? __switch_to_asm+0x40/0x70
>> [ 3089.898977]  ? __switch_to_asm+0x34/0x70
>> [ 3089.902903]  ? __switch_to_asm+0x40/0x70
>> [ 3089.906828]  ? __switch_to_asm+0x34/0x70
>> [ 3089.910745]  _do_fork+0x83/0x350
>> [ 3089.913969]  ? __switch_to+0x7a/0x3e0
>> [ 3089.917626]  ? __switch_to_asm+0x34/0x70
>> [ 3089.921545]  kernel_thread+0x58/0x80
>> [ 3089.925124]  ? kthread_park+0x80/0x80
>> [ 3089.928788]  kthreadd+0x162/0x1b0
>> [ 3089.932098]  ? kthread_create_on_cpu+0xa0/0xa0
>> [ 3089.936538]  ret_from_fork+0x1f/0x40
>>
>> Signed-off-by: Prathap Kumar Valsan<prathap.kumar.valsan@intel.com>
>> Signed-off-by: Bruce Chang<yu.bruce.chang@intel.com>
>> Signed-off-by: Chris Wilson<chris.p.wilson@intel.com>
>> Signed-off-by: Umesh Nerlige Ramappa<umesh.nerlige.ramappa@intel.com>
>> Signed-off-by: Jonathan Cavitt<jonathan.cavitt@intel.com>
>> Signed-off-by: Aravind Iddamsetty<aravind.iddamsetty@intel.com>
>> Signed-off-by: Fei Yang<fei.yang@intel.com>
>> CC: Andi Shyti<andi.shyti@linux.intel.com>
>> ---
>>   drivers/gpu/drm/i915/gt/intel_ggtt.c          |  34 ++-
>>   drivers/gpu/drm/i915/gt/intel_tlb.c           |  15 +-
>>   .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h  |  33 +++
>>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  22 ++
>>   drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c     |   4 +
>>   drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h   |   1 +
>>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 200 +++++++++++++++++-
>>   7 files changed, 297 insertions(+), 12 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> index 4d7d88b92632b..a1f7bdc602996 100644
>> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
>> @@ -206,22 +206,38 @@ static void gen8_ggtt_invalidate(struct i915_ggtt *ggtt)
>>   	intel_uncore_write_fw(uncore, GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
>>   }
>>   
>> +static void guc_ggtt_ct_invalidate(struct intel_gt *gt)
>> +{
>> +	struct intel_uncore *uncore = gt->uncore;
>> +	intel_wakeref_t wakeref;
>> +
>> +	with_intel_runtime_pm_if_active(uncore->rpm, wakeref) {
>> +		struct intel_guc *guc = &gt->uc.guc;
>> +
>> +		intel_guc_invalidate_tlb_guc(guc);
>> +	}
>> +}
>> +
>>   static void guc_ggtt_invalidate(struct i915_ggtt *ggtt)
>>   {
>>   	struct drm_i915_private *i915 = ggtt->vm.i915;
>> +	struct intel_gt *gt;
>>   
>> -	gen8_ggtt_invalidate(ggtt);
>> -
>> -	if (GRAPHICS_VER(i915) >= 12) {
>> -		struct intel_gt *gt;
>> +	if (!HAS_GUC_TLB_INVALIDATION(i915))
>> +		gen8_ggtt_invalidate(ggtt);
>>   
>> -		list_for_each_entry(gt, &ggtt->gt_list, ggtt_link)
>> +	list_for_each_entry(gt, &ggtt->gt_list, ggtt_link) {
>> +		if (HAS_GUC_TLB_INVALIDATION(i915) &&
>> +		    intel_guc_is_ready(&gt->uc.guc)) {
>> +			guc_ggtt_ct_invalidate(gt);
>> +		} else if (GRAPHICS_VER(i915) >= 12) {
>>   			intel_uncore_write_fw(gt->uncore,
>>   					      GEN12_GUC_TLB_INV_CR,
>>   					      GEN12_GUC_TLB_INV_CR_INVALIDATE);
>> -	} else {
>> -		intel_uncore_write_fw(ggtt->vm.gt->uncore,
>> -				      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
>> +		} else {
>> +			intel_uncore_write_fw(gt->uncore,
>> +					      GEN8_GTCR, GEN8_GTCR_INVALIDATE);
>> +		}
>>   	}
>>   }
>>   
>> @@ -1243,7 +1259,7 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
>>   		ggtt->vm.raw_insert_page = gen8_ggtt_insert_page;
>>   	}
>>   
>> -	if (intel_uc_wants_guc(&ggtt->vm.gt->uc))
>> +	if (intel_uc_wants_guc_submission(&ggtt->vm.gt->uc))
>>   		ggtt->invalidate = guc_ggtt_invalidate;
>>   	else
>>   		ggtt->invalidate = gen8_ggtt_invalidate;
>> diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c
>> index 139608c30d978..1278622b9554d 100644
>> --- a/drivers/gpu/drm/i915/gt/intel_tlb.c
>> +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c
>> @@ -12,6 +12,7 @@
>>   #include "intel_gt_print.h"
>>   #include "intel_gt_regs.h"
>>   #include "intel_tlb.h"
>> +#include "uc/intel_guc.h"
>>   
>>   /*
>>    * HW architecture suggest typical invalidation time at 40us,
>> @@ -131,11 +132,23 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno)
>>   		return;
>>   
>>   	with_intel_gt_pm_if_awake(gt, wakeref) {
>> +		struct intel_guc *guc = &gt->uc.guc;
>> +
>>   		mutex_lock(&gt->tlb.invalidate_lock);
>>   		if (tlb_seqno_passed(gt, seqno))
>>   			goto unlock;
>>   
>> -		mmio_invalidate_full(gt);
>> +		if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
>> +			/*
>> +			 * Only perform GuC TLB invalidation if GuC is ready.
>> +			 * If GuC is not ready, then there are no TLBs to
>> +			 * invalidate.  Ergo, skip invalidation.
>> +			 */
>> +			if (intel_guc_is_ready(guc))
>> +				intel_guc_invalidate_tlb_engines(guc);
>> +		} else {
>> +			mmio_invalidate_full(gt);
>> +		}
>>   
>>   		write_seqcount_invalidate(&gt->tlb.seqno);
>>   unlock:
>> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> index f359bef046e0b..33f253410d0c8 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
>> @@ -138,6 +138,8 @@ enum intel_guc_action {
>>   	INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
>>   	INTEL_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
>>   	INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
>> +	INTEL_GUC_ACTION_TLB_INVALIDATION = 0x7000,
>> +	INTEL_GUC_ACTION_TLB_INVALIDATION_DONE = 0x7001,
>>   	INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION = 0x8002,
>>   	INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
>>   	INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
>> @@ -181,4 +183,35 @@ enum intel_guc_state_capture_event_status {
>>   
>>   #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK      0x000000FF
>>   
>> +#define INTEL_GUC_TLB_INVAL_TYPE_MASK	REG_GENMASK(7, 0)
>> +#define INTEL_GUC_TLB_INVAL_MODE_MASK	REG_GENMASK(11, 8)
>> +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE REG_BIT(31)
>> +
>> +enum intel_guc_tlb_invalidation_type {
>> +	INTEL_GUC_TLB_INVAL_ENGINES = 0x0,
>> +	INTEL_GUC_TLB_INVAL_GUC = 0x3,
>> +};
>> +
>> +/*
>> + * 0: Heavy mode of Invalidation:
>> + * The pipeline of the engine(s) for which the invalidation is targeted to is
>> + * blocked, and all the in-flight transactions are guaranteed to be Globally
>> + * Observed before completing the TLB invalidation
>> + * 1: Lite mode of Invalidation:
>> + * TLBs of the targeted engine(s) are immediately invalidated.
>> + * In-flight transactions are NOT guaranteed to be Globally Observed before
>> + * completing TLB invalidation.
>> + * Light Invalidation Mode is to be used only when
>> + * it can be guaranteed (by SW) that the address translations remain invariant
>> + * for the in-flight transactions across the TLB invalidation. In other words,
>> + * this mode can be used when the TLB invalidation is intended to clear out the
>> + * stale cached translations that are no longer in use. Light Invalidation Mode
>> + * is much faster than the Heavy Invalidation Mode, as it does not wait for the
>> + * in-flight transactions to be GOd.
>> + */
>> +enum intel_guc_tlb_inval_mode {
>> +	INTEL_GUC_TLB_INVAL_MODE_HEAVY = 0x0,
>> +	INTEL_GUC_TLB_INVAL_MODE_LITE = 0x1,
>> +};
>> +
>>   #endif /* _ABI_GUC_ACTIONS_ABI_H */
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> index 6c392bad29c19..88deb43bbdc48 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> @@ -79,6 +79,18 @@ struct intel_guc {
>>   	 */
>>   	atomic_t outstanding_submission_g2h;
>>   
>> +	/** @tlb_lookup: xarray to store all pending TLB invalidation requests */
>> +	struct xarray tlb_lookup;
>> +
>> +	/**
>> +	 * @serial_slot: id to the initial waiter created in tlb_lookup,
>> +	 * which is used only when failed to allocate new waiter.
>> +	 */
>> +	u32 serial_slot;
>> +
>> +	/** @next_seqno: the next id (sequence no.) to allocate. */
>> +	u32 next_seqno;
>> +
>>   	/** @interrupts: pointers to GuC interrupt-managing functions. */
>>   	struct {
>>   		bool enabled;
>> @@ -296,6 +308,11 @@ struct intel_guc {
>>   #define MAKE_GUC_VER_STRUCT(ver)	MAKE_GUC_VER((ver).major, (ver).minor, (ver).patch)
>>   #define GUC_SUBMIT_VER(guc)		MAKE_GUC_VER_STRUCT((guc)->submission_version)
>>   
>> +struct intel_guc_tlb_wait {
>> +	struct wait_queue_head wq;
>> +	bool busy;
>> +};
>> +
>>   static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
>>   {
>>   	return container_of(log, struct intel_guc, log);
>> @@ -418,6 +435,11 @@ static inline bool intel_guc_is_supported(struct intel_guc *guc)
>>   	return intel_uc_fw_is_supported(&guc->fw);
>>   }
>>   
>> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc);
>> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc);
>> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size,
>> +				    u32 len, u32 seqno);
>> +
>>   static inline bool intel_guc_is_wanted(struct intel_guc *guc)
>>   {
>>   	return intel_uc_fw_is_enabled(&guc->fw);
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> index 6e22af31513a5..e7ac8402245b7 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
>> @@ -1115,6 +1115,9 @@ static int ct_process_request(struct intel_guc_ct *ct, struct ct_incoming_msg *r
>>   	case INTEL_GUC_ACTION_NOTIFY_EXCEPTION:
>>   		ret = intel_guc_crash_process_msg(guc, action);
>>   		break;
>> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>> +		ret = intel_guc_tlb_invalidation_done(guc, request->size, len, payload[0]);
>> +		break;
>>   	default:
>>   		ret = -EOPNOTSUPP;
>>   		break;
>> @@ -1186,6 +1189,7 @@ static int ct_handle_event(struct intel_guc_ct *ct, struct ct_incoming_msg *requ
>>   	switch (action) {
>>   	case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE:
>>   	case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE:
>> +	case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE:
>>   		g2h_release_space(ct, request->size);
>>   	}
>>   
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> index b4d56eccfb1f0..a7c9874e122a3 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
>> @@ -22,6 +22,7 @@
>>   /* Payload length only i.e. don't include G2H header length */
>>   #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET	2
>>   #define G2H_LEN_DW_DEREGISTER_CONTEXT		1
>> +#define G2H_LEN_DW_INVALIDATE_TLB		1
>>   
>>   #define GUC_CONTEXT_DISABLE		0
>>   #define GUC_CONTEXT_ENABLE		1
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> index 2cce5ec1ff00d..0c5ccda1b3e87 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> @@ -1798,9 +1798,11 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
>>   
>>   void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
>>   {
>> +	struct intel_guc_tlb_wait *wait;
>>   	struct intel_context *ce;
>>   	unsigned long index;
>>   	unsigned long flags;
>> +	unsigned long i;
>>   
>>   	if (unlikely(!guc_submission_initialized(guc))) {
>>   		/* Reset called during driver load? GuC not yet initialised! */
>> @@ -1826,6 +1828,18 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
>>   
>>   	/* GuC is blown away, drop all references to contexts */
>>   	xa_destroy(&guc->context_lookup);
>> +
>> +	/*
>> +	 * The full GT reset will have cleared the TLB caches and flushed the
>> +	 * G2H message queue; we can release all the blocked waiters.
>> +	 *
>> +	 * This is safe to do unlocked because the xarray is not dependent
>> +	 * on the GT reset, and there's a separate execution path for TLB
>> +	 * invalidations on GT reset, and there's a large window of time
>> +	 * between the GT reset and GuC becoming available.
>> +	 */
>> +	xa_for_each(&guc->tlb_lookup, i, wait)
>> +		wake_up(&wait->wq);
>>   }
>>   
>>   static void guc_cancel_context_requests(struct intel_context *ce)
>> @@ -1948,6 +1962,50 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
>>   static void destroyed_worker_func(struct work_struct *w);
>>   static void reset_fail_worker_func(struct work_struct *w);
>>   
>> +static int init_tlb_lookup(struct intel_guc *guc)
>> +{
>> +	struct intel_guc_tlb_wait *wait;
>> +	int err;
>> +
>> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +		return 0;
>> +
>> +	xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC);
>> +
>> +	wait = kzalloc(sizeof(*wait), GFP_KERNEL);
>> +	if (!wait)
>> +		return -ENOMEM;
>> +
>> +	init_waitqueue_head(&wait->wq);
>> +
>> +	/* Preallocate a shared id for use under memory pressure. */
>> +	err = xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait,
>> +				  xa_limit_32b, &guc->next_seqno, GFP_KERNEL);
>> +	/* Only error if out of memory, not when busy (list full)*/
>> +	if (err == -ENOMEM) {
>> +		kfree(wait);
>> +		return err;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static void fini_tlb_lookup(struct intel_guc *guc)
>> +{
>> +	struct intel_guc_tlb_wait *wait;
>> +
>> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +		return;
>> +
>> +	wait = xa_load(&guc->tlb_lookup, guc->serial_slot);
>> +	if (wait) {
>> +		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");
>> +		kfree(wait);
>> +	}
>> +
>> +	xa_destroy(&guc->tlb_lookup);
>> +}
>> +
>>   /*
>>    * Set up the memory resources to be shared with the GuC (via the GGTT)
>>    * at firmware loading time.
>> @@ -1966,11 +2024,15 @@ int intel_guc_submission_init(struct intel_guc *guc)
>>   			return ret;
>>   	}
>>   
>> +	ret = init_tlb_lookup(guc);
>> +	if (ret)
>> +		goto destroy_pool;
>> +
>>   	guc->submission_state.guc_ids_bitmap =
>>   		bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL);
>>   	if (!guc->submission_state.guc_ids_bitmap) {
>>   		ret = -ENOMEM;
>> -		goto destroy_pool;
>> +		goto destroy_tlb;
>>   	}
>>   
>>   	guc->timestamp.ping_delay = (POLL_TIME_CLKS / gt->clock_frequency + 1) * HZ;
>> @@ -1979,9 +2041,10 @@ int intel_guc_submission_init(struct intel_guc *guc)
>>   
>>   	return 0;
>>   
>> +destroy_tlb:
>> +	fini_tlb_lookup(guc);
>>   destroy_pool:
>>   	guc_lrc_desc_pool_destroy_v69(guc);
>> -
>>   	return ret;
>>   }
>>   
>> @@ -1994,6 +2057,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
>>   	guc_lrc_desc_pool_destroy_v69(guc);
>>   	i915_sched_engine_put(guc->sched_engine);
>>   	bitmap_free(guc->submission_state.guc_ids_bitmap);
>> +	fini_tlb_lookup(guc);
>>   	guc->submission_initialized = false;
>>   }
>>   
>> @@ -4624,6 +4688,138 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id)
>>   	return ce;
>>   }
>>   
>> +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno)
>> +{
>> +	struct intel_guc_tlb_wait *wait;
>> +	unsigned long flags;
>> +
>> +	xa_lock_irqsave(&guc->tlb_lookup, flags);
>> +	wait = xa_load(&guc->tlb_lookup, seqno);
>> +
>> +	if (wait)
>> +		wake_up(&wait->wq);
>> +	else
>> +		guc_dbg(guc,
>> +			"Stale TLB invalidation response with seqno %d\n", seqno);
>> +
>> +	xa_unlock_irqrestore(&guc->tlb_lookup, flags);
>> +}
>> +
>> +int intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 size, u32 len, u32 seqno)
>> +{
>> +	/* Check for underflow */
>> +	if (unlikely(len < 1 || len > size))
>> +		return -EPROTO;
>> +
>> +	wait_wake_outstanding_tlb_g2h(guc, seqno);
>> +	return 0;
>> +}
>> +
>> +static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
>> +{
>> +	/*
>> +	 * This is equivalent to wait_woken() with the exception that
>> +	 * we do not wake up early if the kthread task has been completed.
>> +	 * As we are called from page reclaim in any task context,
>> +	 * we may be invoked from stopped kthreads, but we *must*
>> +	 * complete the wait from the HW .
>> +	 *
>> +	 * A second problem is that since we are called under reclaim
>> +	 * and wait_woken() inspected the thread state, it makes an invalid
>> +	 * assumption that all PF_KTHREAD tasks have set_kthread_struct()
>> +	 * called upon them, and will trigger a GPF
> 
> As discussed internally, the GPF issue is resolved with 
> https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/ 
> <https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/>

If it means no open coding a slighlt different wait_token() that would 
be a lot better ineed.

Although the question of why a single wait queue is not good enough 
still remains. As I wrote before - give each invalidation a seqno, upon 
receiving the done h2g store the latest seqno in the driver and wake up 
all sleepers. They check the seqno and if their has passed exit, 
otherwise go back to sleep. No xarray needed. Put_pages() invalidations 
are already serialized so no ill-effect and GGTT invalidations are, or 
are not, performance sensitive for some reason?

Regards,

Tvrtko

> 
> 
>>   in is_kthread_should_stop().
>> +	 */
>> +	do {
>> +		set_current_state(TASK_UNINTERRUPTIBLE);
>> +		if (wq_entry->flags & WQ_FLAG_WOKEN)
>> +			break;
>> +
>> +		timeout = schedule_timeout(timeout);
>> +	} while (timeout);
>> +	__set_current_state(TASK_RUNNING);
>> +
>> +	/* See wait_woken() and woken_wake_function() */
>> +	smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
>> +
>> +	return timeout;
>> +}
>> +
>> +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)
> 
> 
> 2nd param should be intel_guc_tlb_invalidation_type not 
> intel_guc_tlb_inval_mod.
> 
> Not sure why didn't CI complained.
> 
> 
> Regards,
> 
> Nirmoy
> 
>> +{
>> +	struct intel_guc_tlb_wait _wq, *wq = &_wq;
>> +	DEFINE_WAIT_FUNC(wait, woken_wake_function);
>> +	int err;
>> +	u32 seqno;
>> +	u32 action[] = {
>> +		INTEL_GUC_ACTION_TLB_INVALIDATION,
>> +		0,
>> +		REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
>> +			REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
>> +				       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
>> +			INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
>> +	};
>> +	u32 size = ARRAY_SIZE(action);
>> +
>> +	init_waitqueue_head(&_wq.wq);
>> +
>> +	if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
>> +				xa_limit_32b, &guc->next_seqno,
>> +				GFP_ATOMIC | __GFP_NOWARN) < 0) {
>> +		/* Under severe memory pressure? Serialise TLB allocations */
>> +		xa_lock_irq(&guc->tlb_lookup);
>> +		wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
>> +		wait_event_lock_irq(wq->wq,
>> +				    !READ_ONCE(wq->busy),
>> +				    guc->tlb_lookup.xa_lock);
>> +		/*
>> +		 * Update wq->busy under lock to ensure only one waiter can
>> +		 * issue the TLB invalidation command using the serial slot at a
>> +		 * time. The condition is set to true before releasing the lock
>> +		 * so that other caller continue to wait until woken up again.
>> +		 */
>> +		wq->busy = true;
>> +		xa_unlock_irq(&guc->tlb_lookup);
>> +
>> +		seqno = guc->serial_slot;
>> +	}
>> +
>> +	action[1] = seqno;
>> +
>> +	add_wait_queue(&wq->wq, &wait);
>> +
>> +	/*
>> +	 * This is a critical reclaim path and thus we must loop here:
>> +	 * We cannot block for anything that is on the GPU.
>> +	 */
>> +	err = intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE_TLB, true);
>> +	if (err)
>> +		goto out;
>> +
>> +	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
>> +		guc_err(guc,
>> +			"TLB invalidation response timed out for seqno %u\n", seqno);
>> +		err = -ETIME;
>> +	}
>> +out:
>> +	remove_wait_queue(&wq->wq, &wait);
>> +	if (seqno != guc->serial_slot)
>> +		xa_erase_irq(&guc->tlb_lookup, seqno);
>> +
>> +	return err;
>> +}
>> +
>> +/* Full TLB invalidation */
>> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
>> +{
>> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
>> +}
>> +
>> +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
>> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
>> +{
>> +	return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
>> +}
>> +
>>   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
>>   					  const u32 *msg,
>>   					  u32 len)

Re: [Intel-gfx] [PATCH v8 3/7] drm/i915: Define and use GuC and CTB TLB invalidation routines

[Nirmoy Das]

[-- Attachment #1: Type: text/plain, Size: 6260 bytes --]

Hi Tvrtko,

On 10/9/2023 2:54 PM, Tvrtko Ursulin wrote:
<snip>
>
>>> +static long must_wait_woken(struct wait_queue_entry *wq_entry, long 
>>> timeout)
>>> +{
>>> +    /*
>>> +     * This is equivalent to wait_woken() with the exception that
>>> +     * we do not wake up early if the kthread task has been completed.
>>> +     * As we are called from page reclaim in any task context,
>>> +     * we may be invoked from stopped kthreads, but we *must*
>>> +     * complete the wait from the HW .
>>> +     *
>>> +     * A second problem is that since we are called under reclaim
>>> +     * and wait_woken() inspected the thread state, it makes an 
>>> invalid
>>> +     * assumption that all PF_KTHREAD tasks have set_kthread_struct()
>>> +     * called upon them, and will trigger a GPF
>>
>> As discussed internally, the GPF issue is resolved with 
>> https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/ 
>> <https://lore.kernel.org/all/20230602212350.535358-1-jstultz@google.com/> 
>>
>
> If it means no open coding a slighlt different wait_token() that would 
> be a lot better ineed.
>
> Although the question of why a single wait queue is not good enough 
> still remains. As I wrote before - give each invalidation a seqno, 
> upon receiving the done h2g store the latest seqno in the driver and 
> wake up all sleepers. They check the seqno and if their has passed 
> exit, otherwise go back to sleep. No xarray needed. Put_pages() 
> invalidations are already serialized so no ill-effect and GGTT 
> invalidations are, or are not, performance sensitive for some reason?


I think your proposed solution should work as per my understanding 
because we are doing TLB invalidation of all engines and it is 
serialized with gt->tlb.invalidate_lock.

We might need this when we want to make it more fine grain and do vma 
based ranged invalidation for better performance. So I think we should 
try with a single wait queue now.


I don't have an concrete answer yet, may be someone else?


Regards,

Nirmoy

>
> Regards,
>
> Tvrtko
>
>>
>>
>>>   in is_kthread_should_stop().
>>> +     */
>>> +    do {
>>> +        set_current_state(TASK_UNINTERRUPTIBLE);
>>> +        if (wq_entry->flags & WQ_FLAG_WOKEN)
>>> +            break;
>>> +
>>> +        timeout = schedule_timeout(timeout);
>>> +    } while (timeout);
>>> +    __set_current_state(TASK_RUNNING);
>>> +
>>> +    /* See wait_woken() and woken_wake_function() */
>>> +    smp_store_mb(wq_entry->flags, wq_entry->flags & ~WQ_FLAG_WOKEN);
>>> +
>>> +    return timeout;
>>> +}
>>> +
>>> +static int guc_send_invalidate_tlb(struct intel_guc *guc, enum 
>>> intel_guc_tlb_inval_mode type)
>>
>>
>> 2nd param should be intel_guc_tlb_invalidation_type not 
>> intel_guc_tlb_inval_mod.
>>
>> Not sure why didn't CI complained.
>>
>>
>> Regards,
>>
>> Nirmoy
>>
>>> +{
>>> +    struct intel_guc_tlb_wait _wq, *wq = &_wq;
>>> +    DEFINE_WAIT_FUNC(wait, woken_wake_function);
>>> +    int err;
>>> +    u32 seqno;
>>> +    u32 action[] = {
>>> +        INTEL_GUC_ACTION_TLB_INVALIDATION,
>>> +        0,
>>> +        REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_TYPE_MASK, type) |
>>> +            REG_FIELD_PREP(INTEL_GUC_TLB_INVAL_MODE_MASK,
>>> +                       INTEL_GUC_TLB_INVAL_MODE_HEAVY) |
>>> +            INTEL_GUC_TLB_INVAL_FLUSH_CACHE,
>>> +    };
>>> +    u32 size = ARRAY_SIZE(action);
>>> +
>>> +    init_waitqueue_head(&_wq.wq);
>>> +
>>> +    if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq,
>>> +                xa_limit_32b, &guc->next_seqno,
>>> +                GFP_ATOMIC | __GFP_NOWARN) < 0) {
>>> +        /* Under severe memory pressure? Serialise TLB allocations */
>>> +        xa_lock_irq(&guc->tlb_lookup);
>>> +        wq = xa_load(&guc->tlb_lookup, guc->serial_slot);
>>> +        wait_event_lock_irq(wq->wq,
>>> +                    !READ_ONCE(wq->busy),
>>> +                    guc->tlb_lookup.xa_lock);
>>> +        /*
>>> +         * Update wq->busy under lock to ensure only one waiter can
>>> +         * issue the TLB invalidation command using the serial slot 
>>> at a
>>> +         * time. The condition is set to true before releasing the 
>>> lock
>>> +         * so that other caller continue to wait until woken up again.
>>> +         */
>>> +        wq->busy = true;
>>> +        xa_unlock_irq(&guc->tlb_lookup);
>>> +
>>> +        seqno = guc->serial_slot;
>>> +    }
>>> +
>>> +    action[1] = seqno;
>>> +
>>> +    add_wait_queue(&wq->wq, &wait);
>>> +
>>> +    /*
>>> +     * This is a critical reclaim path and thus we must loop here:
>>> +     * We cannot block for anything that is on the GPU.
>>> +     */
>>> +    err = intel_guc_send_busy_loop(guc, action, size, 
>>> G2H_LEN_DW_INVALIDATE_TLB, true);
>>> +    if (err)
>>> +        goto out;
>>> +
>>> +    if (!must_wait_woken(&wait, 
>>> intel_guc_ct_expected_delay(&guc->ct))) {
>>> +        guc_err(guc,
>>> +            "TLB invalidation response timed out for seqno %u\n", 
>>> seqno);
>>> +        err = -ETIME;
>>> +    }
>>> +out:
>>> +    remove_wait_queue(&wq->wq, &wait);
>>> +    if (seqno != guc->serial_slot)
>>> +        xa_erase_irq(&guc->tlb_lookup, seqno);
>>> +
>>> +    return err;
>>> +}
>>> +
>>> +/* Full TLB invalidation */
>>> +int intel_guc_invalidate_tlb_engines(struct intel_guc *guc)
>>> +{
>>> +    return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_ENGINES);
>>> +}
>>> +
>>> +/* GuC TLB Invalidation: Invalidate the TLB's of GuC itself. */
>>> +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc)
>>> +{
>>> +    return guc_send_invalidate_tlb(guc, INTEL_GUC_TLB_INVAL_GUC);
>>> +}
>>> +
>>>   int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
>>>                         const u32 *msg,
>>>                         u32 len)

[-- Attachment #2: Type: text/html, Size: 12470 bytes --]

[Intel-gfx] [PATCH v8 4/7] drm/i915: No TLB invalidation on suspended GT

[Jonathan Cavitt]

In case of GT is suspended, don't allow submission of new TLB invalidation
request and cancel all pending requests. The TLB entries will be
invalidated either during GuC reload or on system resume.

Signed-off-by: Fei Yang <fei.yang@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
CC: John Harrison <john.c.harrison@intel.com>
---
 drivers/gpu/drm/i915/gt/intel_gt.h            |  1 +
 drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  1 +
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 23 +++++++++++++++----
 drivers/gpu/drm/i915/gt/uc/intel_uc.c         |  8 +++++++
 drivers/gpu/drm/i915/i915_driver.c            |  2 ++
 5 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/intel_gt.h
index 970bedf6b78a7..4e3bb221d2f4d 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt.h
@@ -179,4 +179,5 @@ enum i915_map_type intel_gt_coherent_map_type(struct intel_gt *gt,
 void intel_gt_bind_context_set_ready(struct intel_gt *gt);
 void intel_gt_bind_context_set_unready(struct intel_gt *gt);
 bool intel_gt_is_bind_context_ready(struct intel_gt *gt);
+
 #endif /* __INTEL_GT_H__ */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 88deb43bbdc48..add74000e621e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -536,4 +536,5 @@ void intel_guc_dump_time_info(struct intel_guc *guc, struct drm_printer *p);
 
 int intel_guc_sched_disable_gucid_threshold_max(struct intel_guc *guc);
 
+void wake_up_all_tlb_invalidate(struct intel_guc *guc);
 #endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 0c5ccda1b3e87..4ed6e9e759007 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1796,13 +1796,23 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
 	intel_context_put(parent);
 }
 
-void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
+void wake_up_all_tlb_invalidate(struct intel_guc *guc)
 {
 	struct intel_guc_tlb_wait *wait;
+	unsigned long i;
+
+	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
+		return;
+	xa_for_each(&guc->tlb_lookup, i, wait) {
+		wake_up(&wait->wq);
+	}
+}
+
+void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
+{
 	struct intel_context *ce;
 	unsigned long index;
 	unsigned long flags;
-	unsigned long i;
 
 	if (unlikely(!guc_submission_initialized(guc))) {
 		/* Reset called during driver load? GuC not yet initialised! */
@@ -1838,8 +1848,7 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
 	 * invalidations on GT reset, and there's a large window of time
 	 * between the GT reset and GuC becoming available.
 	 */
-	xa_for_each(&guc->tlb_lookup, i, wait)
-		wake_up(&wait->wq);
+	wake_up_all_tlb_invalidate(guc);
 }
 
 static void guc_cancel_context_requests(struct intel_context *ce)
@@ -1935,6 +1944,12 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc)
 
 	/* GuC is blown away, drop all references to contexts */
 	xa_destroy(&guc->context_lookup);
+
+	/*
+	 * Wedged GT won't respond to any TLB invalidation request. Simply
+	 * release all the blocked waiters.
+	 */
+	wake_up_all_tlb_invalidate(guc);
 }
 
 void intel_guc_submission_reset_finish(struct intel_guc *guc)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc.c b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
index 98b103375b7ab..8d6a4d8ce61bb 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_uc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
@@ -688,6 +688,9 @@ void intel_uc_suspend(struct intel_uc *uc)
 	/* flush the GSC worker */
 	intel_gsc_uc_flush_work(&uc->gsc);
 
+	if (HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
+		wake_up_all_tlb_invalidate(guc);
+
 	if (!intel_guc_is_ready(guc)) {
 		guc->interrupts.enabled = false;
 		return;
@@ -736,6 +739,11 @@ static int __uc_resume(struct intel_uc *uc, bool enable_communication)
 
 	intel_gsc_uc_resume(&uc->gsc);
 
+	if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
+		intel_guc_invalidate_tlb_engines(guc);
+		intel_guc_invalidate_tlb_guc(guc);
+	}
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
index ccbb2834cde07..85ac9400c2dc7 100644
--- a/drivers/gpu/drm/i915/i915_driver.c
+++ b/drivers/gpu/drm/i915/i915_driver.c
@@ -72,6 +72,7 @@
 #include "gt/intel_gt.h"
 #include "gt/intel_gt_pm.h"
 #include "gt/intel_rc6.h"
+#include "gt/intel_tlb.h"
 
 #include "pxp/intel_pxp.h"
 #include "pxp/intel_pxp_debugfs.h"
@@ -1093,6 +1094,7 @@ static int i915_drm_suspend(struct drm_device *dev)
 	intel_dp_mst_suspend(dev_priv);
 
 	intel_runtime_pm_disable_interrupts(dev_priv);
+
 	intel_hpd_cancel_work(dev_priv);
 
 	intel_suspend_encoders(dev_priv);
-- 
2.25.1

Re: [Intel-gfx] [PATCH v8 4/7] drm/i915: No TLB invalidation on suspended GT

[Tvrtko Ursulin]

On 06/10/2023 19:20, Jonathan Cavitt wrote:
> In case of GT is suspended, don't allow submission of new TLB invalidation
> request and cancel all pending requests. The TLB entries will be
> invalidated either during GuC reload or on system resume.
> 
> Signed-off-by: Fei Yang <fei.yang@intel.com>
> Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
> CC: John Harrison <john.c.harrison@intel.com>
> ---
>   drivers/gpu/drm/i915/gt/intel_gt.h            |  1 +
>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  1 +
>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 23 +++++++++++++++----
>   drivers/gpu/drm/i915/gt/uc/intel_uc.c         |  8 +++++++
>   drivers/gpu/drm/i915/i915_driver.c            |  2 ++
>   5 files changed, 31 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/intel_gt.h
> index 970bedf6b78a7..4e3bb221d2f4d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_gt.h
> +++ b/drivers/gpu/drm/i915/gt/intel_gt.h
> @@ -179,4 +179,5 @@ enum i915_map_type intel_gt_coherent_map_type(struct intel_gt *gt,
>   void intel_gt_bind_context_set_ready(struct intel_gt *gt);
>   void intel_gt_bind_context_set_unready(struct intel_gt *gt);
>   bool intel_gt_is_bind_context_ready(struct intel_gt *gt);
> +

Zap this.

>   #endif /* __INTEL_GT_H__ */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> index 88deb43bbdc48..add74000e621e 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
> @@ -536,4 +536,5 @@ void intel_guc_dump_time_info(struct intel_guc *guc, struct drm_printer *p);
>   
>   int intel_guc_sched_disable_gucid_threshold_max(struct intel_guc *guc);
>   
> +void wake_up_all_tlb_invalidate(struct intel_guc *guc);
>   #endif
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 0c5ccda1b3e87..4ed6e9e759007 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -1796,13 +1796,23 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
>   	intel_context_put(parent);
>   }
>   
> -void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
> +void wake_up_all_tlb_invalidate(struct intel_guc *guc)
>   {
>   	struct intel_guc_tlb_wait *wait;
> +	unsigned long i;
> +
> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		return;
> +	xa_for_each(&guc->tlb_lookup, i, wait) {
> +		wake_up(&wait->wq);
> +	}

Drop the curlies?

> +}
> +
> +void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
> +{
>   	struct intel_context *ce;
>   	unsigned long index;
>   	unsigned long flags;
> -	unsigned long i;
>   
>   	if (unlikely(!guc_submission_initialized(guc))) {
>   		/* Reset called during driver load? GuC not yet initialised! */
> @@ -1838,8 +1848,7 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
>   	 * invalidations on GT reset, and there's a large window of time
>   	 * between the GT reset and GuC becoming available.
>   	 */
> -	xa_for_each(&guc->tlb_lookup, i, wait)
> -		wake_up(&wait->wq);
> +	wake_up_all_tlb_invalidate(guc);
>   }
>   
>   static void guc_cancel_context_requests(struct intel_context *ce)
> @@ -1935,6 +1944,12 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc)
>   
>   	/* GuC is blown away, drop all references to contexts */
>   	xa_destroy(&guc->context_lookup);
> +
> +	/*
> +	 * Wedged GT won't respond to any TLB invalidation request. Simply
> +	 * release all the blocked waiters.
> +	 */
> +	wake_up_all_tlb_invalidate(guc);

This goes to the following patch I think.

>   }
>   
>   void intel_guc_submission_reset_finish(struct intel_guc *guc)
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc.c b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
> index 98b103375b7ab..8d6a4d8ce61bb 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_uc.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
> @@ -688,6 +688,9 @@ void intel_uc_suspend(struct intel_uc *uc)
>   	/* flush the GSC worker */
>   	intel_gsc_uc_flush_work(&uc->gsc);
>   
> +	if (HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
> +		wake_up_all_tlb_invalidate(guc);
> +
>   	if (!intel_guc_is_ready(guc)) {
>   		guc->interrupts.enabled = false;
>   		return;
> @@ -736,6 +739,11 @@ static int __uc_resume(struct intel_uc *uc, bool enable_communication)
>   
>   	intel_gsc_uc_resume(&uc->gsc);
>   
> +	if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
> +		intel_guc_invalidate_tlb_engines(guc);
> +		intel_guc_invalidate_tlb_guc(guc);
> +	}
> +
>   	return 0;
>   }
>   
> diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
> index ccbb2834cde07..85ac9400c2dc7 100644
> --- a/drivers/gpu/drm/i915/i915_driver.c
> +++ b/drivers/gpu/drm/i915/i915_driver.c
> @@ -72,6 +72,7 @@
>   #include "gt/intel_gt.h"
>   #include "gt/intel_gt_pm.h"
>   #include "gt/intel_rc6.h"
> +#include "gt/intel_tlb.h"
>   
>   #include "pxp/intel_pxp.h"
>   #include "pxp/intel_pxp_debugfs.h"
> @@ -1093,6 +1094,7 @@ static int i915_drm_suspend(struct drm_device *dev)
>   	intel_dp_mst_suspend(dev_priv);
>   
>   	intel_runtime_pm_disable_interrupts(dev_priv);
> +

Zap this too please.

>   	intel_hpd_cancel_work(dev_priv);
>   
>   	intel_suspend_encoders(dev_priv);

Patch got smaller and will be smaller still which is neat! If it is 
still functionally correct it is a win-win.

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v8 4/7] drm/i915: No TLB invalidation on suspended GT

[Jani Nikula]

On Mon, 09 Oct 2023, Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> wrote:
> On 06/10/2023 19:20, Jonathan Cavitt wrote:
>> In case of GT is suspended, don't allow submission of new TLB invalidation
>> request and cancel all pending requests. The TLB entries will be
>> invalidated either during GuC reload or on system resume.
>> 
>> Signed-off-by: Fei Yang <fei.yang@intel.com>
>> Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
>> CC: John Harrison <john.c.harrison@intel.com>
>> ---
>>   drivers/gpu/drm/i915/gt/intel_gt.h            |  1 +
>>   drivers/gpu/drm/i915/gt/uc/intel_guc.h        |  1 +
>>   .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 23 +++++++++++++++----
>>   drivers/gpu/drm/i915/gt/uc/intel_uc.c         |  8 +++++++
>>   drivers/gpu/drm/i915/i915_driver.c            |  2 ++
>>   5 files changed, 31 insertions(+), 4 deletions(-)
>> 
>> diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/intel_gt.h
>> index 970bedf6b78a7..4e3bb221d2f4d 100644
>> --- a/drivers/gpu/drm/i915/gt/intel_gt.h
>> +++ b/drivers/gpu/drm/i915/gt/intel_gt.h
>> @@ -179,4 +179,5 @@ enum i915_map_type intel_gt_coherent_map_type(struct intel_gt *gt,
>>   void intel_gt_bind_context_set_ready(struct intel_gt *gt);
>>   void intel_gt_bind_context_set_unready(struct intel_gt *gt);
>>   bool intel_gt_is_bind_context_ready(struct intel_gt *gt);
>> +
>
> Zap this.
>
>>   #endif /* __INTEL_GT_H__ */
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> index 88deb43bbdc48..add74000e621e 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
>> @@ -536,4 +536,5 @@ void intel_guc_dump_time_info(struct intel_guc *guc, struct drm_printer *p);
>>   
>>   int intel_guc_sched_disable_gucid_threshold_max(struct intel_guc *guc);
>>   
>> +void wake_up_all_tlb_invalidate(struct intel_guc *guc);
>>   #endif
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> index 0c5ccda1b3e87..4ed6e9e759007 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
>> @@ -1796,13 +1796,23 @@ static void __guc_reset_context(struct intel_context *ce, intel_engine_mask_t st
>>   	intel_context_put(parent);
>>   }
>>   
>> -void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
>> +void wake_up_all_tlb_invalidate(struct intel_guc *guc)
>>   {
>>   	struct intel_guc_tlb_wait *wait;
>> +	unsigned long i;
>> +
>> +	if (!HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +		return;
>> +	xa_for_each(&guc->tlb_lookup, i, wait) {
>> +		wake_up(&wait->wq);
>> +	}
>
> Drop the curlies?
>
>> +}
>> +
>> +void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stalled)
>> +{
>>   	struct intel_context *ce;
>>   	unsigned long index;
>>   	unsigned long flags;
>> -	unsigned long i;
>>   
>>   	if (unlikely(!guc_submission_initialized(guc))) {
>>   		/* Reset called during driver load? GuC not yet initialised! */
>> @@ -1838,8 +1848,7 @@ void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t stall
>>   	 * invalidations on GT reset, and there's a large window of time
>>   	 * between the GT reset and GuC becoming available.
>>   	 */
>> -	xa_for_each(&guc->tlb_lookup, i, wait)
>> -		wake_up(&wait->wq);
>> +	wake_up_all_tlb_invalidate(guc);
>>   }
>>   
>>   static void guc_cancel_context_requests(struct intel_context *ce)
>> @@ -1935,6 +1944,12 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc)
>>   
>>   	/* GuC is blown away, drop all references to contexts */
>>   	xa_destroy(&guc->context_lookup);
>> +
>> +	/*
>> +	 * Wedged GT won't respond to any TLB invalidation request. Simply
>> +	 * release all the blocked waiters.
>> +	 */
>> +	wake_up_all_tlb_invalidate(guc);
>
> This goes to the following patch I think.
>
>>   }
>>   
>>   void intel_guc_submission_reset_finish(struct intel_guc *guc)
>> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc.c b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
>> index 98b103375b7ab..8d6a4d8ce61bb 100644
>> --- a/drivers/gpu/drm/i915/gt/uc/intel_uc.c
>> +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc.c
>> @@ -688,6 +688,9 @@ void intel_uc_suspend(struct intel_uc *uc)
>>   	/* flush the GSC worker */
>>   	intel_gsc_uc_flush_work(&uc->gsc);
>>   
>> +	if (HAS_GUC_TLB_INVALIDATION(guc_to_gt(guc)->i915))
>> +		wake_up_all_tlb_invalidate(guc);
>> +
>>   	if (!intel_guc_is_ready(guc)) {
>>   		guc->interrupts.enabled = false;
>>   		return;
>> @@ -736,6 +739,11 @@ static int __uc_resume(struct intel_uc *uc, bool enable_communication)
>>   
>>   	intel_gsc_uc_resume(&uc->gsc);
>>   
>> +	if (HAS_GUC_TLB_INVALIDATION(gt->i915)) {
>> +		intel_guc_invalidate_tlb_engines(guc);
>> +		intel_guc_invalidate_tlb_guc(guc);
>> +	}
>> +
>>   	return 0;
>>   }
>>   
>> diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
>> index ccbb2834cde07..85ac9400c2dc7 100644
>> --- a/drivers/gpu/drm/i915/i915_driver.c
>> +++ b/drivers/gpu/drm/i915/i915_driver.c
>> @@ -72,6 +72,7 @@
>>   #include "gt/intel_gt.h"
>>   #include "gt/intel_gt_pm.h"
>>   #include "gt/intel_rc6.h"
>> +#include "gt/intel_tlb.h"

Does not belong in this patch.

>>   
>>   #include "pxp/intel_pxp.h"
>>   #include "pxp/intel_pxp_debugfs.h"
>> @@ -1093,6 +1094,7 @@ static int i915_drm_suspend(struct drm_device *dev)
>>   	intel_dp_mst_suspend(dev_priv);
>>   
>>   	intel_runtime_pm_disable_interrupts(dev_priv);
>> +
>
> Zap this too please.
>
>>   	intel_hpd_cancel_work(dev_priv);
>>   
>>   	intel_suspend_encoders(dev_priv);
>
> Patch got smaller and will be smaller still which is neat! If it is 
> still functionally correct it is a win-win.
>
> Regards,
>
> Tvrtko

-- 
Jani Nikula, Intel

[Intel-gfx] [PATCH v8 5/7] drm/i915: No TLB invalidation on wedged GT

[Jonathan Cavitt]

It is not an error for GuC TLB invalidations to fail when the GT is
wedged or disabled, so do not process a wait failure as one in
guc_send_invalidate_tlb.

Signed-off-by: Fei Yang <fei.yang@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
CC: John Harrison <john.c.harrison@intel.com>
---
 drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 4ed6e9e759007..ddb3115715e41 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -32,6 +32,7 @@
 
 #include "i915_drv.h"
 #include "i915_reg.h"
+#include "i915_irq.h"
 #include "i915_trace.h"
 
 /**
@@ -4759,6 +4760,14 @@ static long must_wait_woken(struct wait_queue_entry *wq_entry, long timeout)
 	return timeout;
 }
 
+static bool intel_gt_is_enabled(const struct intel_gt *gt)
+{
+	/* Check if GT is wedged or suspended */
+	if (intel_gt_is_wedged(gt) || !intel_irqs_enabled(gt->i915))
+		return false;
+	return true;
+}
+
 static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inval_mode type)
 {
 	struct intel_guc_tlb_wait _wq, *wq = &_wq;
@@ -4810,7 +4819,8 @@ static int guc_send_invalidate_tlb(struct intel_guc *guc, enum intel_guc_tlb_inv
 	if (err)
 		goto out;
 
-	if (!must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
+	if (intel_gt_is_enabled(guc_to_gt(guc)) &&
+	    !must_wait_woken(&wait, intel_guc_ct_expected_delay(&guc->ct))) {
 		guc_err(guc,
 			"TLB invalidation response timed out for seqno %u\n", seqno);
 		err = -ETIME;
-- 
2.25.1

[Intel-gfx] [PATCH v8 6/7] drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck

[Jonathan Cavitt]

For the gt_tlb live selftest, when operating on the GSC engine,
increase the timeout from 10 ms to 200 ms because the GSC
engine is a bit slower than the rest.

Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
---
 drivers/gpu/drm/i915/gt/selftest_tlb.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/gt/selftest_tlb.c b/drivers/gpu/drm/i915/gt/selftest_tlb.c
index 7e41f69fc818f..8c3bb9893dae8 100644
--- a/drivers/gpu/drm/i915/gt/selftest_tlb.c
+++ b/drivers/gpu/drm/i915/gt/selftest_tlb.c
@@ -136,8 +136,15 @@ pte_tlbinv(struct intel_context *ce,
 	i915_request_get(rq);
 	i915_request_add(rq);
 
-	/* Short sleep to sanitycheck the batch is spinning before we begin */
-	msleep(10);
+	/*
+	 * Short sleep to sanitycheck the batch is spinning before we begin 
+	 * FIXME: Why is GSC so slow?
+	 */
+	if (ce->engine->class == OTHER_CLASS)
+		msleep(200);
+	else
+		msleep(10);
+
 	if (va == vb) {
 		if (!i915_request_completed(rq)) {
 			pr_err("%s(%s): Semaphore sanitycheck failed %llx, with alignment %llx, using PTE size %x (phys %x, sg %x)\n",
-- 
2.25.1

[Intel-gfx] [PATCH v8 7/7] drm/i915: Enable GuC TLB invalidations for MTL

[Jonathan Cavitt]

Enable GuC TLB invalidations for MTL.  Though more platforms than just
MTL support GuC TLB invalidations, MTL is presently the only platform
that requires it for any purpose, so only enable it there for now to
minimize cross-platform impact.

Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
---
 drivers/gpu/drm/i915/i915_pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
index df7c261410f79..d4b51ececbb12 100644
--- a/drivers/gpu/drm/i915/i915_pci.c
+++ b/drivers/gpu/drm/i915/i915_pci.c
@@ -829,6 +829,7 @@ static const struct intel_device_info mtl_info = {
 	.has_flat_ccs = 0,
 	.has_gmd_id = 1,
 	.has_guc_deprivilege = 1,
+	.has_guc_tlb_invalidation = 1,
 	.has_llc = 0,
 	.has_mslice_steering = 0,
 	.has_snoop = 1,
-- 
2.25.1

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim checkpatch failed
a989e19c9fa8 drm/i915: Add GuC TLB Invalidation device info flags
b8ea885e3069 drm/i915/guc: Add CT size delay helper
ec7ad50f3bc3 drm/i915: Define and use GuC and CTB TLB invalidation routines
-:411: WARNING:EMBEDDED_FUNCTION_NAME: Prefer using '"%s...", __func__' to using 'fini_tlb_lookup', this function's name, in a string
#411: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c:2002:
+		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");

total: 0 errors, 1 warnings, 0 checks, 442 lines checked
a71d6627453a drm/i915: No TLB invalidation on suspended GT
9a28bcd71d02 drm/i915: No TLB invalidation on wedged GT
6810984ab8f8 drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
-:23: ERROR:TRAILING_WHITESPACE: trailing whitespace
#23: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:140:
+^I * Short sleep to sanitycheck the batch is spinning before we begin $

-:29: WARNING:MSLEEP: msleep < 20ms can sleep for up to 20ms; see Documentation/timers/timers-howto.rst
#29: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:146:
+		msleep(10);

total: 1 errors, 1 warnings, 0 checks, 17 lines checked
7ffe5af8a6bd drm/i915: Enable GuC TLB invalidations for MTL

[Intel-gfx] ✗ Fi.CI.SPARSE: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.

[Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915: Define and use GuC and CTB TLB invalidation routines

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 8255 bytes --]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines
URL   : https://patchwork.freedesktop.org/series/124744/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_13722 -> Patchwork_124744v1
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with Patchwork_124744v1 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in Patchwork_124744v1, please notify your bug team (lgci.bug.filing@intel.com) to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/index.html

Participating hosts (39 -> 37)
------------------------------

  Additional (1): bat-dg2-9 
  Missing    (3): bat-adlp-11 fi-snb-2520m fi-pnv-d510 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in Patchwork_124744v1:

### IGT changes ###

#### Possible regressions ####

  * igt@i915_selftest@live@guc_multi_lrc:
    - bat-dg2-11:         [PASS][1] -> [INCOMPLETE][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13722/bat-dg2-11/igt@i915_selftest@live@guc_multi_lrc.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-11/igt@i915_selftest@live@guc_multi_lrc.html

  
Known issues
------------

  Here are the changes found in Patchwork_124744v1 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_mmap@basic:
    - bat-dg2-9:          NOTRUN -> [SKIP][3] ([i915#4083])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@gem_mmap@basic.html

  * igt@gem_mmap_gtt@basic:
    - bat-dg2-9:          NOTRUN -> [SKIP][4] ([i915#4077]) +2 other tests skip
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@gem_mmap_gtt@basic.html

  * igt@gem_render_tiled_blits@basic:
    - bat-dg2-9:          NOTRUN -> [SKIP][5] ([i915#4079]) +1 other test skip
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@gem_render_tiled_blits@basic.html

  * igt@i915_pm_rps@basic-api:
    - bat-dg2-9:          NOTRUN -> [SKIP][6] ([i915#6621])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@i915_pm_rps@basic-api.html

  * igt@i915_selftest@live@gt_heartbeat:
    - fi-apl-guc:         [PASS][7] -> [DMESG-FAIL][8] ([i915#5334])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13722/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html

  * igt@kms_addfb_basic@addfb25-y-tiled-small-legacy:
    - bat-dg2-9:          NOTRUN -> [SKIP][9] ([i915#5190])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_addfb_basic@addfb25-y-tiled-small-legacy.html

  * igt@kms_addfb_basic@basic-y-tiled-legacy:
    - bat-dg2-9:          NOTRUN -> [SKIP][10] ([i915#4215] / [i915#5190])
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_addfb_basic@basic-y-tiled-legacy.html

  * igt@kms_addfb_basic@framebuffer-vs-set-tiling:
    - bat-dg2-9:          NOTRUN -> [SKIP][11] ([i915#4212]) +6 other tests skip
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_addfb_basic@framebuffer-vs-set-tiling.html

  * igt@kms_addfb_basic@tile-pitch-mismatch:
    - bat-dg2-9:          NOTRUN -> [SKIP][12] ([i915#4212] / [i915#5608])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_addfb_basic@tile-pitch-mismatch.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy:
    - bat-dg2-9:          NOTRUN -> [SKIP][13] ([i915#4103] / [i915#4213] / [i915#5608]) +1 other test skip
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html

  * igt@kms_force_connector_basic@force-load-detect:
    - bat-dg2-9:          NOTRUN -> [SKIP][14] ([fdo#109285])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_force_connector_basic@force-load-detect.html

  * igt@kms_force_connector_basic@prune-stale-modes:
    - bat-dg2-9:          NOTRUN -> [SKIP][15] ([i915#5274])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_force_connector_basic@prune-stale-modes.html

  * igt@kms_psr@sprite_plane_onoff:
    - bat-dg2-9:          NOTRUN -> [SKIP][16] ([i915#1072]) +3 other tests skip
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_psr@sprite_plane_onoff.html

  * igt@kms_setmode@basic-clone-single-crtc:
    - bat-dg2-9:          NOTRUN -> [SKIP][17] ([i915#3555])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@kms_setmode@basic-clone-single-crtc.html

  * igt@prime_vgem@basic-fence-flip:
    - bat-dg2-9:          NOTRUN -> [SKIP][18] ([i915#3708])
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@prime_vgem@basic-fence-flip.html

  * igt@prime_vgem@basic-fence-mmap:
    - bat-dg2-9:          NOTRUN -> [SKIP][19] ([i915#3708] / [i915#4077]) +1 other test skip
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@prime_vgem@basic-fence-mmap.html

  * igt@prime_vgem@basic-write:
    - bat-dg2-9:          NOTRUN -> [SKIP][20] ([i915#3291] / [i915#3708]) +2 other tests skip
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/bat-dg2-9/igt@prime_vgem@basic-write.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
  [i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213
  [i915#4215]: https://gitlab.freedesktop.org/drm/intel/issues/4215
  [i915#5190]: https://gitlab.freedesktop.org/drm/intel/issues/5190
  [i915#5274]: https://gitlab.freedesktop.org/drm/intel/issues/5274
  [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#5608]: https://gitlab.freedesktop.org/drm/intel/issues/5608
  [i915#6621]: https://gitlab.freedesktop.org/drm/intel/issues/6621
  [i915#7359]: https://gitlab.freedesktop.org/drm/intel/issues/7359
  [i915#8981]: https://gitlab.freedesktop.org/drm/intel/issues/8981


Build changes
-------------

  * Linux: CI_DRM_13722 -> Patchwork_124744v1

  CI-20190529: 20190529
  CI_DRM_13722: ead693ec2d2fddb30bff4718845d42f9adabcce6 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7518: 2d4a57e28db0c2ccbf8b2e763074c9aa74a1ae52 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_124744v1: ead693ec2d2fddb30bff4718845d42f9adabcce6 @ git://anongit.freedesktop.org/gfx-ci/linux


### Linux commits

bb0824a21b81 drm/i915: Enable GuC TLB invalidations for MTL
8f0035ef7f20 drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
fae01bfde72e drm/i915: No TLB invalidation on wedged GT
193b66db8988 drm/i915: No TLB invalidation on suspended GT
84340d71aad6 drm/i915: Define and use GuC and CTB TLB invalidation routines
6e568b610cfe drm/i915/guc: Add CT size delay helper
bc6fdf5f3e19 drm/i915: Add GuC TLB Invalidation device info flags

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v1/index.html

[-- Attachment #2: Type: text/html, Size: 9563 bytes --]

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim checkpatch failed
e44ffdf14c0b drm/i915: Add GuC TLB Invalidation device info flags
d57c779f7d68 drm/i915/guc: Add CT size delay helper
de4bef329bd4 drm/i915: Define and use GuC and CTB TLB invalidation routines
-:411: WARNING:EMBEDDED_FUNCTION_NAME: Prefer using '"%s...", __func__' to using 'fini_tlb_lookup', this function's name, in a string
#411: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c:2002:
+		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");

total: 0 errors, 1 warnings, 0 checks, 442 lines checked
05b6153f0d8e drm/i915: No TLB invalidation on suspended GT
774631eb2723 drm/i915: No TLB invalidation on wedged GT
a81ca53dd7f1 drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
-:23: ERROR:TRAILING_WHITESPACE: trailing whitespace
#23: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:140:
+^I * Short sleep to sanitycheck the batch is spinning before we begin $

-:29: WARNING:MSLEEP: msleep < 20ms can sleep for up to 20ms; see Documentation/timers/timers-howto.rst
#29: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:146:
+		msleep(10);

total: 1 errors, 1 warnings, 0 checks, 17 lines checked
b387a13104b1 drm/i915: Enable GuC TLB invalidations for MTL

[Intel-gfx] ✗ Fi.CI.SPARSE: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.

[Intel-gfx] ✓ Fi.CI.BAT: success for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)
URL   : https://patchwork.freedesktop.org/series/124744/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_13723 -> Patchwork_124744v2
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/index.html

Participating hosts (40 -> 38)
------------------------------

  Missing    (2): fi-kbl-soraka fi-snb-2520m 

Known issues
------------

  Here are the changes found in Patchwork_124744v2 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1:
    - bat-rplp-1:         [PASS][1] -> [ABORT][2] ([i915#8668])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
  [i915#8913]: https://gitlab.freedesktop.org/drm/intel/issues/8913


Build changes
-------------

  * Linux: CI_DRM_13723 -> Patchwork_124744v2

  CI-20190529: 20190529
  CI_DRM_13723: 2f2f3a5735816736995263c4de4f221480183a99 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7519: d1db7333d9c5fbbb05e50b0804123950d9dc1c46 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_124744v2: 2f2f3a5735816736995263c4de4f221480183a99 @ git://anongit.freedesktop.org/gfx-ci/linux


### Linux commits

d6083b4e9d85 drm/i915: Enable GuC TLB invalidations for MTL
6536fdbf6df0 drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
08fb23f5dbf8 drm/i915: No TLB invalidation on wedged GT
cc1ccfe706c7 drm/i915: No TLB invalidation on suspended GT
ff8680ace9bf drm/i915: Define and use GuC and CTB TLB invalidation routines
f512934e96d1 drm/i915/guc: Add CT size delay helper
5262a952d056 drm/i915: Add GuC TLB Invalidation device info flags

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/index.html

[-- Attachment #2: Type: text/html, Size: 2972 bytes --]

[Intel-gfx] ✗ Fi.CI.IGT: failure for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 53565 bytes --]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev2)
URL   : https://patchwork.freedesktop.org/series/124744/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_13723_full -> Patchwork_124744v2_full
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with Patchwork_124744v2_full absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in Patchwork_124744v2_full, please notify your bug team (lgci.bug.filing@intel.com) to allow them
  to document this new failure mode, which will reduce false positives in CI.

  

Participating hosts (9 -> 10)
------------------------------

  Additional (1): shard-rkl0 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in Patchwork_124744v2_full:

### IGT changes ###

#### Possible regressions ####

  * igt@gem_exec_capture@pi@ccs0:
    - shard-mtlp:         [PASS][1] -> [DMESG-WARN][2] +5 other tests dmesg-warn
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-1/igt@gem_exec_capture@pi@ccs0.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-3/igt@gem_exec_capture@pi@ccs0.html

  
Known issues
------------

  Here are the changes found in Patchwork_124744v2_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@device_reset@unbind-cold-reset-rebind:
    - shard-dg2:          NOTRUN -> [SKIP][3] ([i915#7701])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@device_reset@unbind-cold-reset-rebind.html

  * igt@drm_fdinfo@all-busy-idle-check-all:
    - shard-dg2:          NOTRUN -> [SKIP][4] ([i915#8414])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@drm_fdinfo@all-busy-idle-check-all.html

  * igt@drm_fdinfo@busy-hang@rcs0:
    - shard-mtlp:         NOTRUN -> [SKIP][5] ([i915#8414]) +5 other tests skip
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@drm_fdinfo@busy-hang@rcs0.html

  * igt@drm_fdinfo@most-busy-check-all@rcs0:
    - shard-rkl:          [PASS][6] -> [FAIL][7] ([i915#7742])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-1/igt@drm_fdinfo@most-busy-check-all@rcs0.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-2/igt@drm_fdinfo@most-busy-check-all@rcs0.html

  * igt@gem_close_race@multigpu-basic-threads:
    - shard-dg2:          NOTRUN -> [SKIP][8] ([i915#7697])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@gem_close_race@multigpu-basic-threads.html

  * igt@gem_ctx_persistence@saturated-hostile-nopreempt@vcs1:
    - shard-mtlp:         NOTRUN -> [SKIP][9] ([i915#5882]) +5 other tests skip
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_ctx_persistence@saturated-hostile-nopreempt@vcs1.html

  * igt@gem_ctx_sseu@mmap-args:
    - shard-dg2:          NOTRUN -> [SKIP][10] ([i915#280]) +1 other test skip
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@gem_ctx_sseu@mmap-args.html

  * igt@gem_exec_balancer@bonded-false-hang:
    - shard-dg2:          NOTRUN -> [SKIP][11] ([i915#4812]) +1 other test skip
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@gem_exec_balancer@bonded-false-hang.html

  * igt@gem_exec_balancer@bonded-pair:
    - shard-dg2:          NOTRUN -> [SKIP][12] ([i915#4771])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_exec_balancer@bonded-pair.html

  * igt@gem_exec_balancer@sliced:
    - shard-mtlp:         NOTRUN -> [SKIP][13] ([i915#4812])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_exec_balancer@sliced.html

  * igt@gem_exec_fair@basic-deadline:
    - shard-glk:          NOTRUN -> [FAIL][14] ([i915#2846])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk5/igt@gem_exec_fair@basic-deadline.html
    - shard-dg2:          NOTRUN -> [SKIP][15] ([i915#3539] / [i915#4852])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@gem_exec_fair@basic-deadline.html

  * igt@gem_exec_fair@basic-none-solo:
    - shard-mtlp:         NOTRUN -> [SKIP][16] ([i915#4473])
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_exec_fair@basic-none-solo.html

  * igt@gem_exec_fair@basic-none-vip:
    - shard-mtlp:         NOTRUN -> [SKIP][17] ([i915#4473] / [i915#4771])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_exec_fair@basic-none-vip.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-glk:          [PASS][18] -> [FAIL][19] ([i915#2842])
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-glk2/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk8/igt@gem_exec_fair@basic-pace-share@rcs0.html
    - shard-rkl:          [PASS][20] -> [FAIL][21] ([i915#2842])
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-7/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-1/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_exec_flush@basic-uc-set-default:
    - shard-dg2:          NOTRUN -> [SKIP][22] ([i915#3539])
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@gem_exec_flush@basic-uc-set-default.html

  * igt@gem_exec_reloc@basic-concurrent0:
    - shard-dg2:          NOTRUN -> [SKIP][23] ([i915#3281]) +5 other tests skip
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_exec_reloc@basic-concurrent0.html

  * igt@gem_exec_reloc@basic-wc:
    - shard-mtlp:         NOTRUN -> [SKIP][24] ([i915#3281])
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_exec_reloc@basic-wc.html

  * igt@gem_exec_schedule@reorder-wide:
    - shard-dg2:          NOTRUN -> [SKIP][25] ([i915#4537] / [i915#4812]) +1 other test skip
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_exec_schedule@reorder-wide.html

  * igt@gem_fence_thrash@bo-write-verify-threaded-none:
    - shard-mtlp:         NOTRUN -> [SKIP][26] ([i915#4860]) +1 other test skip
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_fence_thrash@bo-write-verify-threaded-none.html

  * igt@gem_fenced_exec_thrash@no-spare-fences-busy-interruptible:
    - shard-dg2:          NOTRUN -> [SKIP][27] ([i915#4860])
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@gem_fenced_exec_thrash@no-spare-fences-busy-interruptible.html

  * igt@gem_lmem_swapping@smem-oom:
    - shard-mtlp:         NOTRUN -> [SKIP][28] ([i915#4613]) +1 other test skip
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_lmem_swapping@smem-oom.html

  * igt@gem_media_vme:
    - shard-dg2:          NOTRUN -> [SKIP][29] ([i915#284])
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_media_vme.html

  * igt@gem_mmap_gtt@bad-object:
    - shard-dg2:          NOTRUN -> [SKIP][30] ([i915#4077]) +5 other tests skip
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_mmap_gtt@bad-object.html

  * igt@gem_mmap_gtt@basic-short:
    - shard-mtlp:         NOTRUN -> [SKIP][31] ([i915#4077]) +3 other tests skip
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_mmap_gtt@basic-short.html

  * igt@gem_mmap_wc@bad-object:
    - shard-mtlp:         NOTRUN -> [SKIP][32] ([i915#4083]) +2 other tests skip
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_mmap_wc@bad-object.html

  * igt@gem_mmap_wc@coherency:
    - shard-dg2:          NOTRUN -> [SKIP][33] ([i915#4083]) +2 other tests skip
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@gem_mmap_wc@coherency.html

  * igt@gem_partial_pwrite_pread@reads-display:
    - shard-mtlp:         NOTRUN -> [SKIP][34] ([i915#3282]) +2 other tests skip
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_partial_pwrite_pread@reads-display.html

  * igt@gem_pxp@fail-invalid-protected-context:
    - shard-mtlp:         NOTRUN -> [SKIP][35] ([i915#4270])
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_pxp@fail-invalid-protected-context.html

  * igt@gem_pxp@verify-pxp-execution-after-suspend-resume:
    - shard-dg2:          NOTRUN -> [SKIP][36] ([i915#4270]) +2 other tests skip
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@gem_pxp@verify-pxp-execution-after-suspend-resume.html

  * igt@gem_readwrite@read-write:
    - shard-dg2:          NOTRUN -> [SKIP][37] ([i915#3282]) +2 other tests skip
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@gem_readwrite@read-write.html

  * igt@gem_render_copy@yf-tiled-ccs-to-y-tiled:
    - shard-mtlp:         NOTRUN -> [SKIP][38] ([i915#8428])
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_render_copy@yf-tiled-ccs-to-y-tiled.html

  * igt@gem_set_tiling_vs_blt@untiled-to-tiled:
    - shard-dg2:          NOTRUN -> [SKIP][39] ([i915#4079])
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gem_set_tiling_vs_blt@untiled-to-tiled.html

  * igt@gem_tiled_pread_pwrite:
    - shard-mtlp:         NOTRUN -> [SKIP][40] ([i915#4079])
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_tiled_pread_pwrite.html

  * igt@gem_userptr_blits@map-fixed-invalidate-overlap-busy:
    - shard-dg2:          NOTRUN -> [SKIP][41] ([i915#3297] / [i915#4880])
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@gem_userptr_blits@map-fixed-invalidate-overlap-busy.html

  * igt@gem_userptr_blits@mmap-offset-banned@gtt:
    - shard-mtlp:         NOTRUN -> [SKIP][42] ([i915#3297])
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gem_userptr_blits@mmap-offset-banned@gtt.html

  * igt@gen7_exec_parse@basic-offset:
    - shard-dg2:          NOTRUN -> [SKIP][43] ([fdo#109289]) +2 other tests skip
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gen7_exec_parse@basic-offset.html

  * igt@gen9_exec_parse@bb-large:
    - shard-mtlp:         NOTRUN -> [SKIP][44] ([i915#2856]) +2 other tests skip
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@gen9_exec_parse@bb-large.html

  * igt@gen9_exec_parse@shadow-peek:
    - shard-dg2:          NOTRUN -> [SKIP][45] ([i915#2856])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@gen9_exec_parse@shadow-peek.html

  * igt@i915_pm_rc6_residency@media-rc6-accuracy:
    - shard-mtlp:         NOTRUN -> [SKIP][46] ([fdo#109289]) +1 other test skip
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@i915_pm_rc6_residency@media-rc6-accuracy.html

  * igt@i915_pm_rpm@modeset-lpsp:
    - shard-dg2:          [PASS][47] -> [SKIP][48] ([i915#1397]) +2 other tests skip
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg2-10/igt@i915_pm_rpm@modeset-lpsp.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-2/igt@i915_pm_rpm@modeset-lpsp.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress:
    - shard-rkl:          [PASS][49] -> [SKIP][50] ([i915#1397]) +2 other tests skip
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-1/igt@i915_pm_rpm@modeset-non-lpsp-stress.html
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-7/igt@i915_pm_rpm@modeset-non-lpsp-stress.html

  * igt@i915_pm_rpm@pc8-residency:
    - shard-dg2:          NOTRUN -> [SKIP][51] ([fdo#109506]) +1 other test skip
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@i915_pm_rpm@pc8-residency.html

  * igt@i915_pm_rpm@pm-caching:
    - shard-glk:          [PASS][52] -> [DMESG-WARN][53] ([i915#1982])
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-glk7/igt@i915_pm_rpm@pm-caching.html
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk9/igt@i915_pm_rpm@pm-caching.html

  * igt@i915_selftest@mock@memory_region:
    - shard-dg2:          NOTRUN -> [DMESG-WARN][54] ([i915#9311])
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@i915_selftest@mock@memory_region.html

  * igt@i915_suspend@forcewake:
    - shard-dg2:          [PASS][55] -> [TIMEOUT][56] ([fdo#103375])
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg2-2/igt@i915_suspend@forcewake.html
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-5/igt@i915_suspend@forcewake.html

  * igt@i915_suspend@sysfs-reader:
    - shard-mtlp:         [PASS][57] -> [ABORT][58] ([i915#9414]) +1 other test abort
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-1/igt@i915_suspend@sysfs-reader.html
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-3/igt@i915_suspend@sysfs-reader.html

  * igt@kms_addfb_basic@addfb25-x-tiled-legacy:
    - shard-mtlp:         NOTRUN -> [SKIP][59] ([i915#4212]) +1 other test skip
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_addfb_basic@addfb25-x-tiled-legacy.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-hdmi-a-3-4-mc_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][60] ([i915#8502] / [i915#8709]) +11 other tests skip
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-7/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-hdmi-a-3-4-mc_ccs.html

  * igt@kms_async_flips@crc@pipe-a-hdmi-a-2:
    - shard-dg2:          NOTRUN -> [FAIL][61] ([i915#8247]) +3 other tests fail
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-2/igt@kms_async_flips@crc@pipe-a-hdmi-a-2.html

  * igt@kms_async_flips@crc@pipe-c-hdmi-a-1:
    - shard-dg1:          NOTRUN -> [FAIL][62] ([i915#8247]) +3 other tests fail
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-19/igt@kms_async_flips@crc@pipe-c-hdmi-a-1.html

  * igt@kms_async_flips@test-cursor:
    - shard-mtlp:         NOTRUN -> [SKIP][63] ([i915#6229])
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_async_flips@test-cursor.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip:
    - shard-mtlp:         [PASS][64] -> [FAIL][65] ([i915#5138]) +2 other tests fail
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-8/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html

  * igt@kms_big_fb@linear-32bpp-rotate-90:
    - shard-mtlp:         NOTRUN -> [SKIP][66] ([fdo#111614]) +3 other tests skip
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_big_fb@linear-32bpp-rotate-90.html

  * igt@kms_big_fb@x-tiled-16bpp-rotate-90:
    - shard-dg2:          NOTRUN -> [SKIP][67] ([fdo#111614]) +1 other test skip
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_big_fb@x-tiled-16bpp-rotate-90.html

  * igt@kms_big_fb@y-tiled-64bpp-rotate-0:
    - shard-dg2:          NOTRUN -> [SKIP][68] ([i915#5190]) +6 other tests skip
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_big_fb@y-tiled-64bpp-rotate-0.html

  * igt@kms_big_fb@y-tiled-addfb-size-overflow:
    - shard-mtlp:         NOTRUN -> [SKIP][69] ([i915#6187])
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_big_fb@y-tiled-addfb-size-overflow.html

  * igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-async-flip:
    - shard-mtlp:         NOTRUN -> [SKIP][70] ([fdo#111615]) +2 other tests skip
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-async-flip.html

  * igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-async-flip:
    - shard-tglu:         [PASS][71] -> [FAIL][72] ([i915#3743])
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-tglu-4/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-async-flip.html
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-tglu-3/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-async-flip.html

  * igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0:
    - shard-dg2:          NOTRUN -> [SKIP][73] ([i915#4538] / [i915#5190]) +2 other tests skip
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html

  * igt@kms_ccs@pipe-a-crc-primary-basic-4_tiled_dg2_rc_ccs_cc:
    - shard-mtlp:         NOTRUN -> [SKIP][74] ([i915#5354] / [i915#6095]) +10 other tests skip
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_ccs@pipe-a-crc-primary-basic-4_tiled_dg2_rc_ccs_cc.html

  * igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_gen12_rc_ccs_cc:
    - shard-glk:          NOTRUN -> [SKIP][75] ([fdo#109271] / [i915#3886])
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk5/igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][76] ([i915#3689] / [i915#5354]) +11 other tests skip
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_ccs.html

  * igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_mc_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][77] ([i915#3689] / [i915#3886] / [i915#5354]) +5 other tests skip
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][78] ([i915#3886] / [i915#5354] / [i915#6095]) +3 other tests skip
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-d-crc-sprite-planes-basic-4_tiled_mtl_rc_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][79] ([i915#5354]) +36 other tests skip
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_ccs@pipe-d-crc-sprite-planes-basic-4_tiled_mtl_rc_ccs.html

  * igt@kms_chamelium_color@ctm-red-to-blue:
    - shard-dg2:          NOTRUN -> [SKIP][80] ([fdo#111827])
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_chamelium_color@ctm-red-to-blue.html

  * igt@kms_chamelium_color@gamma:
    - shard-mtlp:         NOTRUN -> [SKIP][81] ([fdo#111827])
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_chamelium_color@gamma.html

  * igt@kms_chamelium_frames@hdmi-crc-nonplanar-formats:
    - shard-glk:          NOTRUN -> [SKIP][82] ([fdo#109271]) +18 other tests skip
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk5/igt@kms_chamelium_frames@hdmi-crc-nonplanar-formats.html
    - shard-dg2:          NOTRUN -> [SKIP][83] ([i915#7828]) +6 other tests skip
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_chamelium_frames@hdmi-crc-nonplanar-formats.html

  * igt@kms_chamelium_hpd@hdmi-hpd:
    - shard-mtlp:         NOTRUN -> [SKIP][84] ([i915#7828]) +4 other tests skip
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_chamelium_hpd@hdmi-hpd.html

  * igt@kms_content_protection@legacy@pipe-a-dp-4:
    - shard-dg2:          NOTRUN -> [TIMEOUT][85] ([i915#7173])
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_content_protection@legacy@pipe-a-dp-4.html

  * igt@kms_cursor_crc@cursor-onscreen-max-size:
    - shard-dg2:          NOTRUN -> [SKIP][86] ([i915#3555]) +3 other tests skip
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@kms_cursor_crc@cursor-onscreen-max-size.html

  * igt@kms_cursor_crc@cursor-random-512x512:
    - shard-dg2:          NOTRUN -> [SKIP][87] ([i915#3359]) +1 other test skip
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@kms_cursor_crc@cursor-random-512x512.html

  * igt@kms_cursor_crc@cursor-rapid-movement-512x512:
    - shard-mtlp:         NOTRUN -> [SKIP][88] ([i915#3359])
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_cursor_crc@cursor-rapid-movement-512x512.html

  * igt@kms_cursor_crc@cursor-sliding-32x32:
    - shard-mtlp:         NOTRUN -> [SKIP][89] ([i915#3555] / [i915#8814]) +1 other test skip
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_cursor_crc@cursor-sliding-32x32.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy:
    - shard-dg2:          NOTRUN -> [SKIP][90] ([i915#4103] / [i915#4213] / [i915#5608])
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-varying-size:
    - shard-mtlp:         NOTRUN -> [SKIP][91] ([i915#4213])
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-varying-size.html

  * igt@kms_cursor_legacy@cursora-vs-flipb-toggle:
    - shard-dg2:          NOTRUN -> [SKIP][92] ([fdo#109274] / [i915#5354]) +1 other test skip
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_cursor_legacy@cursora-vs-flipb-toggle.html

  * igt@kms_cursor_legacy@cursorb-vs-flipb-atomic-transitions:
    - shard-dg2:          NOTRUN -> [SKIP][93] ([fdo#109274] / [fdo#111767] / [i915#5354])
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_cursor_legacy@cursorb-vs-flipb-atomic-transitions.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
    - shard-glk:          [PASS][94] -> [FAIL][95] ([i915#2346])
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-glk2/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk8/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

  * igt@kms_cursor_legacy@single-bo@all-pipes:
    - shard-mtlp:         [PASS][96] -> [DMESG-WARN][97] ([i915#2017])
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-2/igt@kms_cursor_legacy@single-bo@all-pipes.html
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-4/igt@kms_cursor_legacy@single-bo@all-pipes.html

  * igt@kms_dsc@dsc-with-bpc-formats:
    - shard-mtlp:         NOTRUN -> [SKIP][98] ([i915#3555] / [i915#3840]) +1 other test skip
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_dsc@dsc-with-bpc-formats.html

  * igt@kms_flip@2x-flip-vs-modeset:
    - shard-mtlp:         NOTRUN -> [SKIP][99] ([i915#3637]) +2 other tests skip
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_flip@2x-flip-vs-modeset.html

  * igt@kms_flip@2x-flip-vs-wf_vblank-interruptible:
    - shard-dg2:          NOTRUN -> [SKIP][100] ([fdo#109274]) +5 other tests skip
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@kms_flip@2x-flip-vs-wf_vblank-interruptible.html

  * igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-32bpp-yftileccs-upscaling@pipe-a-valid-mode:
    - shard-dg2:          NOTRUN -> [SKIP][101] ([i915#2672]) +2 other tests skip
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-32bpp-yftileccs-upscaling@pipe-a-valid-mode.html

  * igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling@pipe-a-default-mode:
    - shard-mtlp:         NOTRUN -> [SKIP][102] ([i915#2672])
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-linear-to-16bpp-linear-downscaling@pipe-a-default-mode:
    - shard-mtlp:         NOTRUN -> [SKIP][103] ([i915#3555] / [i915#8810])
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_flip_scaled_crc@flip-64bpp-linear-to-16bpp-linear-downscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling@pipe-a-default-mode:
    - shard-mtlp:         NOTRUN -> [SKIP][104] ([i915#2672] / [i915#3555])
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling@pipe-a-default-mode.html

  * igt@kms_force_connector_basic@prune-stale-modes:
    - shard-mtlp:         NOTRUN -> [SKIP][105] ([i915#5274])
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_force_connector_basic@prune-stale-modes.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-blt:
    - shard-dg2:          NOTRUN -> [FAIL][106] ([i915#6880])
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-mmap-cpu:
    - shard-dg2:          [PASS][107] -> [FAIL][108] ([i915#6880])
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg2-10/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-mmap-cpu.html
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-7/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-mmap-cpu.html

  * igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-blt:
    - shard-mtlp:         NOTRUN -> [SKIP][109] ([i915#1825]) +15 other tests skip
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-mmap-wc:
    - shard-dg2:          NOTRUN -> [SKIP][110] ([i915#8708]) +16 other tests skip
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@psr-1p-primscrn-cur-indfb-move:
    - shard-dg2:          NOTRUN -> [SKIP][111] ([i915#3458]) +10 other tests skip
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_frontbuffer_tracking@psr-1p-primscrn-cur-indfb-move.html

  * igt@kms_frontbuffer_tracking@psr-1p-primscrn-pri-indfb-draw-mmap-gtt:
    - shard-mtlp:         NOTRUN -> [SKIP][112] ([i915#8708]) +3 other tests skip
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_frontbuffer_tracking@psr-1p-primscrn-pri-indfb-draw-mmap-gtt.html

  * igt@kms_hdr@invalid-metadata-sizes:
    - shard-mtlp:         NOTRUN -> [SKIP][113] ([i915#3555] / [i915#8228]) +1 other test skip
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_hdr@invalid-metadata-sizes.html

  * igt@kms_plane_scaling@intel-max-src-size@pipe-a-hdmi-a-1:
    - shard-dg1:          NOTRUN -> [FAIL][114] ([i915#8292])
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-19/igt@kms_plane_scaling@intel-max-src-size@pipe-a-hdmi-a-1.html

  * igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-a-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][115] ([i915#5176]) +1 other test skip
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-7/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-a-hdmi-a-1.html

  * igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-c-hdmi-a-4:
    - shard-dg1:          NOTRUN -> [SKIP][116] ([i915#5176]) +3 other tests skip
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-16/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-c-hdmi-a-4.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-b-edp-1:
    - shard-mtlp:         NOTRUN -> [SKIP][117] ([i915#5235]) +3 other tests skip
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-b-edp-1.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-20x20@pipe-b-hdmi-a-2:
    - shard-rkl:          NOTRUN -> [SKIP][118] ([i915#5235]) +3 other tests skip
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-4/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-20x20@pipe-b-hdmi-a-2.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25@pipe-d-hdmi-a-2:
    - shard-dg2:          NOTRUN -> [SKIP][119] ([i915#5235]) +15 other tests skip
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-2/igt@kms_plane_scaling@planes-downscale-factor-0-25@pipe-d-hdmi-a-2.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-75-unity-scaling@pipe-a-vga-1:
    - shard-snb:          NOTRUN -> [SKIP][120] ([fdo#109271]) +15 other tests skip
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-snb2/igt@kms_plane_scaling@planes-downscale-factor-0-75-unity-scaling@pipe-a-vga-1.html

  * igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-25@pipe-c-hdmi-a-1:
    - shard-dg1:          NOTRUN -> [SKIP][121] ([i915#5235]) +11 other tests skip
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-19/igt@kms_plane_scaling@planes-unity-scaling-downscale-factor-0-25@pipe-c-hdmi-a-1.html

  * igt@kms_prime@d3hot:
    - shard-mtlp:         NOTRUN -> [SKIP][122] ([i915#6524])
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_prime@d3hot.html

  * igt@kms_psr2_su@page_flip-xrgb8888:
    - shard-dg2:          NOTRUN -> [SKIP][123] ([i915#658])
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_psr2_su@page_flip-xrgb8888.html

  * igt@kms_psr@sprite_blt:
    - shard-dg2:          NOTRUN -> [SKIP][124] ([i915#1072]) +3 other tests skip
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@kms_psr@sprite_blt.html

  * igt@kms_psr_stress_test@flip-primary-invalidate-overlay:
    - shard-dg2:          NOTRUN -> [SKIP][125] ([i915#5461] / [i915#658])
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html

  * igt@kms_rotation_crc@bad-tiling:
    - shard-mtlp:         NOTRUN -> [SKIP][126] ([i915#4235])
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@kms_rotation_crc@bad-tiling.html

  * igt@kms_rotation_crc@primary-rotation-270:
    - shard-rkl:          [PASS][127] -> [INCOMPLETE][128] ([i915#8875])
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-3/igt@kms_rotation_crc@primary-rotation-270.html
   [128]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-2/igt@kms_rotation_crc@primary-rotation-270.html

  * igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90:
    - shard-dg2:          NOTRUN -> [SKIP][129] ([i915#4235] / [i915#5190]) +1 other test skip
   [129]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90.html

  * igt@kms_universal_plane@cursor-fb-leak-pipe-c:
    - shard-dg1:          [PASS][130] -> [FAIL][131] ([i915#9196])
   [130]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg1-18/igt@kms_universal_plane@cursor-fb-leak-pipe-c.html
   [131]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-12/igt@kms_universal_plane@cursor-fb-leak-pipe-c.html
    - shard-tglu:         [PASS][132] -> [FAIL][133] ([i915#9196])
   [132]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-tglu-5/igt@kms_universal_plane@cursor-fb-leak-pipe-c.html
   [133]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-tglu-10/igt@kms_universal_plane@cursor-fb-leak-pipe-c.html

  * igt@kms_vrr@flipline:
    - shard-mtlp:         NOTRUN -> [SKIP][134] ([i915#3555] / [i915#8808])
   [134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@kms_vrr@flipline.html

  * igt@kms_writeback@writeback-check-output:
    - shard-dg2:          NOTRUN -> [SKIP][135] ([i915#2437])
   [135]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@kms_writeback@writeback-check-output.html

  * igt@perf@gen8-unprivileged-single-ctx-counters:
    - shard-dg2:          NOTRUN -> [SKIP][136] ([i915#2436])
   [136]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@perf@gen8-unprivileged-single-ctx-counters.html

  * igt@perf@mi-rpc:
    - shard-mtlp:         NOTRUN -> [SKIP][137] ([i915#2434])
   [137]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@perf@mi-rpc.html

  * igt@perf_pmu@cpu-hotplug:
    - shard-dg2:          NOTRUN -> [SKIP][138] ([i915#8850])
   [138]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@perf_pmu@cpu-hotplug.html

  * igt@perf_pmu@faulting-read@gtt:
    - shard-mtlp:         NOTRUN -> [SKIP][139] ([i915#8440])
   [139]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@perf_pmu@faulting-read@gtt.html

  * igt@perf_pmu@rc6-all-gts:
    - shard-dg2:          NOTRUN -> [SKIP][140] ([i915#5608] / [i915#8516])
   [140]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-11/igt@perf_pmu@rc6-all-gts.html

  * igt@prime_vgem@basic-blt:
    - shard-mtlp:         NOTRUN -> [FAIL][141] ([i915#8445])
   [141]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@prime_vgem@basic-blt.html

  * igt@tools_test@sysfs_l3_parity:
    - shard-dg2:          NOTRUN -> [SKIP][142] ([i915#4818])
   [142]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@tools_test@sysfs_l3_parity.html

  * igt@v3d/v3d_get_param@get-bad-param:
    - shard-mtlp:         NOTRUN -> [SKIP][143] ([i915#2575]) +5 other tests skip
   [143]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@v3d/v3d_get_param@get-bad-param.html

  * igt@v3d/v3d_submit_cl@valid-multisync-submission:
    - shard-dg2:          NOTRUN -> [SKIP][144] ([i915#2575]) +8 other tests skip
   [144]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-6/igt@v3d/v3d_submit_cl@valid-multisync-submission.html

  * igt@vc4/vc4_tiling@get-after-free:
    - shard-mtlp:         NOTRUN -> [SKIP][145] ([i915#7711]) +3 other tests skip
   [145]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@vc4/vc4_tiling@get-after-free.html

  * igt@vc4/vc4_tiling@get-bad-handle:
    - shard-dg2:          NOTRUN -> [SKIP][146] ([i915#7711]) +5 other tests skip
   [146]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@vc4/vc4_tiling@get-bad-handle.html

  
#### Possible fixes ####

  * igt@drm_fdinfo@most-busy-idle-check-all@rcs0:
    - shard-rkl:          [FAIL][147] ([i915#7742]) -> [PASS][148]
   [147]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-2/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html
   [148]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-4/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html

  * igt@gem_barrier_race@remote-request@rcs0:
    - shard-glk:          [ABORT][149] ([i915#8190]) -> [PASS][150]
   [149]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-glk8/igt@gem_barrier_race@remote-request@rcs0.html
   [150]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-glk5/igt@gem_barrier_race@remote-request@rcs0.html

  * igt@gem_ctx_exec@basic-nohangcheck:
    - shard-tglu:         [FAIL][151] ([i915#6268]) -> [PASS][152]
   [151]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-tglu-2/igt@gem_ctx_exec@basic-nohangcheck.html
   [152]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-tglu-4/igt@gem_ctx_exec@basic-nohangcheck.html

  * igt@gem_eio@in-flight-suspend:
    - shard-mtlp:         [ABORT][153] ([i915#7892] / [i915#9262]) -> [PASS][154]
   [153]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-4/igt@gem_eio@in-flight-suspend.html
   [154]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-7/igt@gem_eio@in-flight-suspend.html

  * igt@gem_eio@unwedge-stress:
    - shard-dg1:          [FAIL][155] ([i915#5784]) -> [PASS][156]
   [155]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg1-16/igt@gem_eio@unwedge-stress.html
   [156]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-19/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_schedule@noreorder-corked@ccs0:
    - shard-mtlp:         [DMESG-WARN][157] ([i915#8962]) -> [PASS][158] +1 other test pass
   [157]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-4/igt@gem_exec_schedule@noreorder-corked@ccs0.html
   [158]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-5/igt@gem_exec_schedule@noreorder-corked@ccs0.html

  * igt@gem_mmap_gtt@fault-concurrent-x:
    - shard-snb:          [INCOMPLETE][159] ([i915#5161]) -> [PASS][160]
   [159]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-snb7/igt@gem_mmap_gtt@fault-concurrent-x.html
   [160]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-snb2/igt@gem_mmap_gtt@fault-concurrent-x.html

  * igt@gem_spin_batch@user-each:
    - shard-mtlp:         [DMESG-FAIL][161] ([i915#8962]) -> [PASS][162] +1 other test pass
   [161]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-4/igt@gem_spin_batch@user-each.html
   [162]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-5/igt@gem_spin_batch@user-each.html

  * igt@i915_module_load@reload-with-fault-injection:
    - shard-mtlp:         [ABORT][163] ([i915#8489] / [i915#8668]) -> [PASS][164]
   [163]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-3/igt@i915_module_load@reload-with-fault-injection.html
   [164]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@i915_module_load@reload-with-fault-injection.html

  * igt@i915_pm_rc6_residency@rc6-idle@rcs0:
    - shard-dg1:          [FAIL][165] ([i915#3591]) -> [PASS][166] +1 other test pass
   [165]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg1-17/igt@i915_pm_rc6_residency@rc6-idle@rcs0.html
   [166]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-19/igt@i915_pm_rc6_residency@rc6-idle@rcs0.html

  * igt@i915_pm_rpm@dpms-lpsp:
    - shard-rkl:          [SKIP][167] ([i915#1397]) -> [PASS][168]
   [167]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-1/igt@i915_pm_rpm@dpms-lpsp.html
   [168]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-7/igt@i915_pm_rpm@dpms-lpsp.html

  * igt@i915_pm_rpm@modeset-lpsp-stress-no-wait:
    - shard-dg2:          [SKIP][169] ([i915#1397]) -> [PASS][170] +1 other test pass
   [169]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg2-3/igt@i915_pm_rpm@modeset-lpsp-stress-no-wait.html
   [170]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-10/igt@i915_pm_rpm@modeset-lpsp-stress-no-wait.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress:
    - shard-dg1:          [SKIP][171] ([i915#1397]) -> [PASS][172] +1 other test pass
   [171]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg1-19/igt@i915_pm_rpm@modeset-non-lpsp-stress.html
   [172]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg1-12/igt@i915_pm_rpm@modeset-non-lpsp-stress.html

  * igt@i915_selftest@live@gt_heartbeat:
    - shard-apl:          [DMESG-FAIL][173] ([i915#5334]) -> [PASS][174]
   [173]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-apl3/igt@i915_selftest@live@gt_heartbeat.html
   [174]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-apl2/igt@i915_selftest@live@gt_heartbeat.html

  * igt@i915_suspend@basic-s3-without-i915:
    - shard-rkl:          [FAIL][175] ([fdo#103375]) -> [PASS][176]
   [175]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-6/igt@i915_suspend@basic-s3-without-i915.html
   [176]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-2/igt@i915_suspend@basic-s3-without-i915.html

  * igt@kms_cursor_crc@cursor-sliding-256x256@pipe-a-edp-1:
    - shard-mtlp:         [FAIL][177] -> [PASS][178] +1 other test pass
   [177]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-2/igt@kms_cursor_crc@cursor-sliding-256x256@pipe-a-edp-1.html
   [178]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-4/igt@kms_cursor_crc@cursor-sliding-256x256@pipe-a-edp-1.html

  * igt@perf_pmu@busy-double-start@rcs0:
    - shard-mtlp:         [FAIL][179] ([i915#4349]) -> [PASS][180]
   [179]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-3/igt@perf_pmu@busy-double-start@rcs0.html
   [180]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-6/igt@perf_pmu@busy-double-start@rcs0.html

  
#### Warnings ####

  * igt@i915_pm_rc6_residency@rc6-idle@vecs0:
    - shard-tglu:         [WARN][181] ([i915#2681]) -> [FAIL][182] ([i915#2681] / [i915#3591])
   [181]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-tglu-5/igt@i915_pm_rc6_residency@rc6-idle@vecs0.html
   [182]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-tglu-6/igt@i915_pm_rc6_residency@rc6-idle@vecs0.html

  * igt@kms_fbcon_fbt@psr:
    - shard-rkl:          [SKIP][183] ([i915#3955]) -> [SKIP][184] ([fdo#110189] / [i915#3955])
   [183]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-rkl-7/igt@kms_fbcon_fbt@psr.html
   [184]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-rkl-1/igt@kms_fbcon_fbt@psr.html

  * igt@prime_mmap@test_aperture_limit@test_aperture_limit-smem:
    - shard-dg2:          [CRASH][185] ([i915#9351]) -> [INCOMPLETE][186] ([i915#5493])
   [185]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-dg2-7/igt@prime_mmap@test_aperture_limit@test_aperture_limit-smem.html
   [186]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-dg2-1/igt@prime_mmap@test_aperture_limit@test_aperture_limit-smem.html

  * igt@sysfs_timeslice_duration@timeout@vecs0:
    - shard-mtlp:         [TIMEOUT][187] -> [ABORT][188] ([i915#8521] / [i915#8865])
   [187]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13723/shard-mtlp-4/igt@sysfs_timeslice_duration@timeout@vecs0.html
   [188]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/shard-mtlp-1/igt@sysfs_timeslice_duration@timeout@vecs0.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109506]: https://bugs.freedesktop.org/show_bug.cgi?id=109506
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111767]: https://bugs.freedesktop.org/show_bug.cgi?id=111767
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
  [i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#2017]: https://gitlab.freedesktop.org/drm/intel/issues/2017
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2434]: https://gitlab.freedesktop.org/drm/intel/issues/2434
  [i915#2436]: https://gitlab.freedesktop.org/drm/intel/issues/2436
  [i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
  [i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2681]: https://gitlab.freedesktop.org/drm/intel/issues/2681
  [i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
  [i915#284]: https://gitlab.freedesktop.org/drm/intel/issues/284
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
  [i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3591]: https://gitlab.freedesktop.org/drm/intel/issues/3591
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3743]: https://gitlab.freedesktop.org/drm/intel/issues/3743
  [i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
  [i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213
  [i915#4235]: https://gitlab.freedesktop.org/drm/intel/issues/4235
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4349]: https://gitlab.freedesktop.org/drm/intel/issues/4349
  [i915#4473]: https://gitlab.freedesktop.org/drm/intel/issues/4473
  [i915#4537]: https://gitlab.freedesktop.org/drm/intel/issues/4537
  [i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
  [i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
  [i915#4818]: https://gitlab.freedesktop.org/drm/intel/issues/4818
  [i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
  [i915#4854]: https://gitlab.freedesktop.org/drm/intel/issues/4854
  [i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
  [i915#4880]: https://gitlab.freedesktop.org/drm/intel/issues/4880
  [i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
  [i915#5161]: https://gitlab.freedesktop.org/drm/intel/issues/5161
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5190]: https://gitlab.freedesktop.org/drm/intel/issues/5190
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5274]: https://gitlab.freedesktop.org/drm/intel/issues/5274
  [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5493]: https://gitlab.freedesktop.org/drm/intel/issues/5493
  [i915#5608]: https://gitlab.freedesktop.org/drm/intel/issues/5608
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#5882]: https://gitlab.freedesktop.org/drm/intel/issues/5882
  [i915#5978]: https://gitlab.freedesktop.org/drm/intel/issues/5978
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6187]: https://gitlab.freedesktop.org/drm/intel/issues/6187
  [i915#6229]: https://gitlab.freedesktop.org/drm/intel/issues/6229
  [i915#6268]: https://gitlab.freedesktop.org/drm/intel/issues/6268
  [i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6880]: https://gitlab.freedesktop.org/drm/intel/issues/6880
  [i915#7173]: https://gitlab.freedesktop.org/drm/intel/issues/7173
  [i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
  [i915#7701]: https://gitlab.freedesktop.org/drm/intel/issues/7701
  [i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
  [i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#7892]: https://gitlab.freedesktop.org/drm/intel/issues/7892
  [i915#8063]: https://gitlab.freedesktop.org/drm/intel/issues/8063
  [i915#8190]: https://gitlab.freedesktop.org/drm/intel/issues/8190
  [i915#8228]: https://gitlab.freedesktop.org/drm/intel/issues/8228
  [i915#8247]: https://gitlab.freedesktop.org/drm/intel/issues/8247
  [i915#8292]: https://gitlab.freedesktop.org/drm/intel/issues/8292
  [i915#8414]: https://gitlab.freedesktop.org/drm/intel/issues/8414
  [i915#8428]: https://gitlab.freedesktop.org/drm/intel/issues/8428
  [i915#8440]: https://gitlab.freedesktop.org/drm/intel/issues/8440
  [i915#8445]: https://gitlab.freedesktop.org/drm/intel/issues/8445
  [i915#8489]: https://gitlab.freedesktop.org/drm/intel/issues/8489
  [i915#8502]: https://gitlab.freedesktop.org/drm/intel/issues/8502
  [i915#8516]: https://gitlab.freedesktop.org/drm/intel/issues/8516
  [i915#8521]: https://gitlab.freedesktop.org/drm/intel/issues/8521
  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
  [i915#8708]: https://gitlab.freedesktop.org/drm/intel/issues/8708
  [i915#8709]: https://gitlab.freedesktop.org/drm/intel/issues/8709
  [i915#8808]: https://gitlab.freedesktop.org/drm/intel/issues/8808
  [i915#8810]: https://gitlab.freedesktop.org/drm/intel/issues/8810
  [i915#8814]: https://gitlab.freedesktop.org/drm/intel/issues/8814
  [i915#8850]: https://gitlab.freedesktop.org/drm/intel/issues/8850
  [i915#8865]: https://gitlab.freedesktop.org/drm/intel/issues/8865
  [i915#8875]: https://gitlab.freedesktop.org/drm/intel/issues/8875
  [i915#8962]: https://gitlab.freedesktop.org/drm/intel/issues/8962
  [i915#9196]: https://gitlab.freedesktop.org/drm/intel/issues/9196
  [i915#9226]: https://gitlab.freedesktop.org/drm/intel/issues/9226
  [i915#9227]: https://gitlab.freedesktop.org/drm/intel/issues/9227
  [i915#9261]: https://gitlab.freedesktop.org/drm/intel/issues/9261
  [i915#9262]: https://gitlab.freedesktop.org/drm/intel/issues/9262
  [i915#9310]: https://gitlab.freedesktop.org/drm/intel/issues/9310
  [i915#9311]: https://gitlab.freedesktop.org/drm/intel/issues/9311
  [i915#9351]: https://gitlab.freedesktop.org/drm/intel/issues/9351
  [i915#9412]: https://gitlab.freedesktop.org/drm/intel/issues/9412
  [i915#9414]: https://gitlab.freedesktop.org/drm/intel/issues/9414
  [i915#9423]: https://gitlab.freedesktop.org/drm/intel/issues/9423
  [i915#9433]: https://gitlab.freedesktop.org/drm/intel/issues/9433


Build changes
-------------

  * Linux: CI_DRM_13723 -> Patchwork_124744v2

  CI-20190529: 20190529
  CI_DRM_13723: 2f2f3a5735816736995263c4de4f221480183a99 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7519: d1db7333d9c5fbbb05e50b0804123950d9dc1c46 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_124744v2: 2f2f3a5735816736995263c4de4f221480183a99 @ git://anongit.freedesktop.org/gfx-ci/linux
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v2/index.html

[-- Attachment #2: Type: text/html, Size: 61667 bytes --]

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim checkpatch failed
5214d0d5d9ab drm/i915: Add GuC TLB Invalidation device info flags
2ee0cfbd271a drm/i915/guc: Add CT size delay helper
d86b647b81a9 drm/i915: Define and use GuC and CTB TLB invalidation routines
-:411: WARNING:EMBEDDED_FUNCTION_NAME: Prefer using '"%s...", __func__' to using 'fini_tlb_lookup', this function's name, in a string
#411: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c:2002:
+		guc_dbg(guc, "fini_tlb_lookup: Unexpected item in tlb_lookup\n");

total: 0 errors, 1 warnings, 0 checks, 442 lines checked
d094564be8e6 drm/i915: No TLB invalidation on suspended GT
863e68385952 drm/i915: No TLB invalidation on wedged GT
1bcbf4c6c3b2 drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
-:23: ERROR:TRAILING_WHITESPACE: trailing whitespace
#23: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:140:
+^I * Short sleep to sanitycheck the batch is spinning before we begin $

-:29: WARNING:MSLEEP: msleep < 20ms can sleep for up to 20ms; see Documentation/timers/timers-howto.rst
#29: FILE: drivers/gpu/drm/i915/gt/selftest_tlb.c:146:
+		msleep(10);

total: 1 errors, 1 warnings, 0 checks, 17 lines checked
f84657534d2a drm/i915: Enable GuC TLB invalidations for MTL

[Intel-gfx] ✗ Fi.CI.SPARSE: warning for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)

[Patchwork]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)
URL   : https://patchwork.freedesktop.org/series/124744/
State : warning

== Summary ==

Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.

[Intel-gfx] ✓ Fi.CI.BAT: success for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 3296 bytes --]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)
URL   : https://patchwork.freedesktop.org/series/124744/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_13728 -> Patchwork_124744v3
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/index.html

Participating hosts (37 -> 34)
------------------------------

  Missing    (3): fi-bsw-nick fi-snb-2520m fi-pnv-d510 

Known issues
------------

  Here are the changes found in Patchwork_124744v3 that come from known issues:

### IGT changes ###

#### Possible fixes ####

  * igt@gem_exec_suspend@basic-s3@smem:
    - bat-mtlp-8:         [ABORT][1] ([i915#8213] / [i915#9262]) -> [PASS][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/bat-mtlp-8/igt@gem_exec_suspend@basic-s3@smem.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/bat-mtlp-8/igt@gem_exec_suspend@basic-s3@smem.html
    - fi-blb-e6850:       [ABORT][3] -> [PASS][4]
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/fi-blb-e6850/igt@gem_exec_suspend@basic-s3@smem.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/fi-blb-e6850/igt@gem_exec_suspend@basic-s3@smem.html

  * igt@i915_selftest@live@gt_heartbeat:
    - fi-apl-guc:         [DMESG-FAIL][5] ([i915#5334]) -> [PASS][6]
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html

  * igt@kms_hdmi_inject@inject-audio:
    - fi-kbl-guc:         [FAIL][7] ([IGT#3]) -> [PASS][8]
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/fi-kbl-guc/igt@kms_hdmi_inject@inject-audio.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/fi-kbl-guc/igt@kms_hdmi_inject@inject-audio.html

  
  [IGT#3]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/3
  [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
  [i915#8213]: https://gitlab.freedesktop.org/drm/intel/issues/8213
  [i915#9262]: https://gitlab.freedesktop.org/drm/intel/issues/9262


Build changes
-------------

  * Linux: CI_DRM_13728 -> Patchwork_124744v3

  CI-20190529: 20190529
  CI_DRM_13728: e642685fe494c1893ab2fd94e6dc7b6827303d90 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7520: 6435c8825e9269bdac515ca96cba4502b5b770f5 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_124744v3: e642685fe494c1893ab2fd94e6dc7b6827303d90 @ git://anongit.freedesktop.org/gfx-ci/linux


### Linux commits

91e7ad1a7eb3 drm/i915: Enable GuC TLB invalidations for MTL
2c10f277e70f drm/i915/gt: Increase sleep in gt_tlb selftest sanitycheck
d003a77528e1 drm/i915: No TLB invalidation on wedged GT
619f08570c19 drm/i915: No TLB invalidation on suspended GT
180934d654ff drm/i915: Define and use GuC and CTB TLB invalidation routines
72a1d206a610 drm/i915/guc: Add CT size delay helper
f7cec4fecc6d drm/i915: Add GuC TLB Invalidation device info flags

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/index.html

[-- Attachment #2: Type: text/html, Size: 4021 bytes --]

[Intel-gfx] ✗ Fi.CI.IGT: failure for drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 67575 bytes --]

== Series Details ==

Series: drm/i915: Define and use GuC and CTB TLB invalidation routines (rev3)
URL   : https://patchwork.freedesktop.org/series/124744/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_13728_full -> Patchwork_124744v3_full
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with Patchwork_124744v3_full absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in Patchwork_124744v3_full, please notify your bug team (lgci.bug.filing@intel.com) to allow them
  to document this new failure mode, which will reduce false positives in CI.

  

Participating hosts (9 -> 9)
------------------------------

  No changes in participating hosts

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in Patchwork_124744v3_full:

### IGT changes ###

#### Possible regressions ####

  * igt@gem_exec_capture@pi@ccs0:
    - shard-mtlp:         [PASS][1] -> [DMESG-WARN][2] +5 other tests dmesg-warn
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-5/igt@gem_exec_capture@pi@ccs0.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-1/igt@gem_exec_capture@pi@ccs0.html

  * igt@kms_flip@dpms-vs-vblank-race@c-hdmi-a2:
    - shard-glk:          [PASS][3] -> [INCOMPLETE][4]
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-glk3/igt@kms_flip@dpms-vs-vblank-race@c-hdmi-a2.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-glk5/igt@kms_flip@dpms-vs-vblank-race@c-hdmi-a2.html

  * igt@kms_frontbuffer_tracking@fbc-rgb101010-draw-render:
    - shard-mtlp:         [PASS][5] -> [FAIL][6]
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-3/igt@kms_frontbuffer_tracking@fbc-rgb101010-draw-render.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-2/igt@kms_frontbuffer_tracking@fbc-rgb101010-draw-render.html

  
Known issues
------------

  Here are the changes found in Patchwork_124744v3_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@drm_fdinfo@busy-idle@bcs0:
    - shard-dg1:          NOTRUN -> [SKIP][7] ([i915#8414]) +4 other tests skip
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@drm_fdinfo@busy-idle@bcs0.html

  * igt@drm_fdinfo@most-busy-check-all@rcs0:
    - shard-rkl:          [PASS][8] -> [FAIL][9] ([i915#7742])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-6/igt@drm_fdinfo@most-busy-check-all@rcs0.html
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-1/igt@drm_fdinfo@most-busy-check-all@rcs0.html

  * igt@drm_fdinfo@most-busy-idle-check-all@vecs1:
    - shard-dg2:          NOTRUN -> [SKIP][10] ([i915#8414]) +21 other tests skip
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@drm_fdinfo@most-busy-idle-check-all@vecs1.html

  * igt@drm_fdinfo@virtual-busy-idle-all:
    - shard-mtlp:         NOTRUN -> [SKIP][11] ([i915#8414])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@drm_fdinfo@virtual-busy-idle-all.html

  * igt@gem_ccs@suspend-resume@linear-compressed-compfmt0-smem-lmem0:
    - shard-dg2:          NOTRUN -> [INCOMPLETE][12] ([i915#7297])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@gem_ccs@suspend-resume@linear-compressed-compfmt0-smem-lmem0.html

  * igt@gem_close_race@multigpu-basic-threads:
    - shard-dg2:          NOTRUN -> [SKIP][13] ([i915#7697]) +1 other test skip
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_close_race@multigpu-basic-threads.html

  * igt@gem_create@create-ext-set-pat:
    - shard-dg1:          NOTRUN -> [SKIP][14] ([i915#8562])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_create@create-ext-set-pat.html

  * igt@gem_ctx_isolation@preservation-s3@vecs1:
    - shard-dg2:          [PASS][15] -> [FAIL][16] ([fdo#103375]) +1 other test fail
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-11/igt@gem_ctx_isolation@preservation-s3@vecs1.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gem_ctx_isolation@preservation-s3@vecs1.html

  * igt@gem_ctx_persistence@engines-persistence:
    - shard-snb:          NOTRUN -> [SKIP][17] ([fdo#109271] / [i915#1099])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-snb2/igt@gem_ctx_persistence@engines-persistence.html

  * igt@gem_ctx_persistence@heartbeat-hang:
    - shard-dg2:          NOTRUN -> [SKIP][18] ([i915#8555]) +1 other test skip
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_ctx_persistence@heartbeat-hang.html

  * igt@gem_ctx_persistence@heartbeat-hostile:
    - shard-mtlp:         NOTRUN -> [SKIP][19] ([i915#8555])
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_ctx_persistence@heartbeat-hostile.html

  * igt@gem_ctx_sseu@invalid-args:
    - shard-dg2:          NOTRUN -> [SKIP][20] ([i915#280])
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gem_ctx_sseu@invalid-args.html

  * igt@gem_exec_balancer@invalid-bonds:
    - shard-dg2:          NOTRUN -> [SKIP][21] ([i915#4036])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_exec_balancer@invalid-bonds.html

  * igt@gem_exec_fair@basic-none-rrul:
    - shard-mtlp:         NOTRUN -> [SKIP][22] ([i915#4473] / [i915#4771])
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_exec_fair@basic-none-rrul.html

  * igt@gem_exec_fair@basic-throttle:
    - shard-dg1:          NOTRUN -> [SKIP][23] ([i915#3539]) +1 other test skip
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_exec_fair@basic-throttle.html

  * igt@gem_exec_fence@submit67:
    - shard-mtlp:         NOTRUN -> [SKIP][24] ([i915#4812])
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_exec_fence@submit67.html

  * igt@gem_exec_flush@basic-batch-kernel-default-wb:
    - shard-dg1:          NOTRUN -> [SKIP][25] ([i915#3539] / [i915#4852]) +2 other tests skip
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_exec_flush@basic-batch-kernel-default-wb.html

  * igt@gem_exec_flush@basic-wb-ro-before-default:
    - shard-dg2:          NOTRUN -> [SKIP][26] ([i915#3539] / [i915#4852]) +1 other test skip
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@gem_exec_flush@basic-wb-ro-before-default.html

  * igt@gem_exec_params@secure-non-root:
    - shard-dg2:          NOTRUN -> [SKIP][27] ([fdo#112283])
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gem_exec_params@secure-non-root.html

  * igt@gem_exec_reloc@basic-wc-read-active:
    - shard-dg1:          NOTRUN -> [SKIP][28] ([i915#3281]) +7 other tests skip
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gem_exec_reloc@basic-wc-read-active.html

  * igt@gem_exec_reloc@basic-write-read-active:
    - shard-dg2:          NOTRUN -> [SKIP][29] ([i915#3281]) +10 other tests skip
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gem_exec_reloc@basic-write-read-active.html

  * igt@gem_exec_schedule@preempt-queue-contexts-chain:
    - shard-mtlp:         NOTRUN -> [SKIP][30] ([i915#4537] / [i915#4812])
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_exec_schedule@preempt-queue-contexts-chain.html

  * igt@gem_exec_schedule@reorder-wide:
    - shard-dg1:          NOTRUN -> [SKIP][31] ([i915#4812]) +1 other test skip
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_exec_schedule@reorder-wide.html

  * igt@gem_exec_suspend@basic-s4-devices@lmem0:
    - shard-dg1:          [PASS][32] -> [ABORT][33] ([i915#7975] / [i915#8213])
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg1-12/igt@gem_exec_suspend@basic-s4-devices@lmem0.html
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-14/igt@gem_exec_suspend@basic-s4-devices@lmem0.html

  * igt@gem_fence_thrash@bo-write-verify-none:
    - shard-dg2:          NOTRUN -> [SKIP][34] ([i915#4860])
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_fence_thrash@bo-write-verify-none.html

  * igt@gem_mmap_gtt@cpuset-medium-copy-xy:
    - shard-dg1:          NOTRUN -> [SKIP][35] ([i915#4077]) +1 other test skip
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_mmap_gtt@cpuset-medium-copy-xy.html

  * igt@gem_mmap_gtt@pf-nonblock:
    - shard-mtlp:         NOTRUN -> [SKIP][36] ([i915#4077])
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_mmap_gtt@pf-nonblock.html

  * igt@gem_mmap_wc@bad-object:
    - shard-dg2:          NOTRUN -> [SKIP][37] ([i915#4083]) +3 other tests skip
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@gem_mmap_wc@bad-object.html

  * igt@gem_mmap_wc@bad-size:
    - shard-dg1:          NOTRUN -> [SKIP][38] ([i915#4083])
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gem_mmap_wc@bad-size.html

  * igt@gem_mmap_wc@close:
    - shard-mtlp:         NOTRUN -> [SKIP][39] ([i915#4083]) +1 other test skip
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_mmap_wc@close.html

  * igt@gem_partial_pwrite_pread@reads-uncached:
    - shard-dg1:          NOTRUN -> [SKIP][40] ([i915#3282]) +2 other tests skip
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_partial_pwrite_pread@reads-uncached.html

  * igt@gem_partial_pwrite_pread@writes-after-reads-display:
    - shard-dg2:          NOTRUN -> [SKIP][41] ([i915#3282]) +5 other tests skip
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@gem_partial_pwrite_pread@writes-after-reads-display.html

  * igt@gem_pxp@protected-raw-src-copy-not-readible:
    - shard-dg1:          NOTRUN -> [SKIP][42] ([i915#4270])
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gem_pxp@protected-raw-src-copy-not-readible.html

  * igt@gem_pxp@regular-baseline-src-copy-readible:
    - shard-dg2:          NOTRUN -> [SKIP][43] ([i915#4270]) +5 other tests skip
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gem_pxp@regular-baseline-src-copy-readible.html

  * igt@gem_set_tiling_vs_blt@tiled-to-tiled:
    - shard-dg1:          NOTRUN -> [SKIP][44] ([i915#4079])
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_set_tiling_vs_blt@tiled-to-tiled.html

  * igt@gem_spin_batch@spin-all-new:
    - shard-dg2:          NOTRUN -> [FAIL][45] ([i915#5889])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_spin_batch@spin-all-new.html

  * igt@gem_userptr_blits@invalid-mmap-offset-unsync:
    - shard-dg1:          NOTRUN -> [SKIP][46] ([i915#3297]) +1 other test skip
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gem_userptr_blits@invalid-mmap-offset-unsync.html

  * igt@gem_userptr_blits@map-fixed-invalidate-overlap:
    - shard-dg2:          NOTRUN -> [SKIP][47] ([i915#3297] / [i915#4880])
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@gem_userptr_blits@map-fixed-invalidate-overlap.html

  * igt@gem_userptr_blits@unsync-overlap:
    - shard-dg2:          NOTRUN -> [SKIP][48] ([i915#3297]) +2 other tests skip
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_userptr_blits@unsync-overlap.html

  * igt@gem_userptr_blits@unsync-unmap:
    - shard-mtlp:         NOTRUN -> [SKIP][49] ([i915#3297])
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@gem_userptr_blits@unsync-unmap.html

  * igt@gen7_exec_parse@basic-rejected:
    - shard-dg2:          NOTRUN -> [SKIP][50] ([fdo#109289]) +3 other tests skip
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@gen7_exec_parse@basic-rejected.html

  * igt@gen7_exec_parse@oacontrol-tracking:
    - shard-dg1:          NOTRUN -> [SKIP][51] ([fdo#109289])
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gen7_exec_parse@oacontrol-tracking.html

  * igt@gen9_exec_parse@bb-oversize:
    - shard-dg1:          NOTRUN -> [SKIP][52] ([i915#2527]) +1 other test skip
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@gen9_exec_parse@bb-oversize.html

  * igt@gen9_exec_parse@valid-registers:
    - shard-dg2:          NOTRUN -> [SKIP][53] ([i915#2856]) +2 other tests skip
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@gen9_exec_parse@valid-registers.html

  * igt@i915_pm_rpm@dpms-lpsp:
    - shard-dg1:          NOTRUN -> [SKIP][54] ([i915#1397])
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@i915_pm_rpm@dpms-lpsp.html

  * igt@i915_pm_rpm@gem-evict-pwrite:
    - shard-dg2:          NOTRUN -> [SKIP][55] ([i915#4077]) +7 other tests skip
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@i915_pm_rpm@gem-evict-pwrite.html

  * igt@i915_pm_rpm@gem-mmap-type@gtt-smem0:
    - shard-mtlp:         NOTRUN -> [SKIP][56] ([i915#8431])
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@i915_pm_rpm@gem-mmap-type@gtt-smem0.html

  * igt@i915_pm_rpm@modeset-non-lpsp:
    - shard-dg2:          NOTRUN -> [SKIP][57] ([i915#1397]) +1 other test skip
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@i915_pm_rpm@modeset-non-lpsp.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait:
    - shard-rkl:          [PASS][58] -> [SKIP][59] ([i915#1397]) +1 other test skip
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-6/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html

  * igt@i915_pm_rps@basic-api:
    - shard-dg1:          NOTRUN -> [SKIP][60] ([i915#6621])
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@i915_pm_rps@basic-api.html

  * igt@i915_query@query-topology-known-pci-ids:
    - shard-dg1:          NOTRUN -> [SKIP][61] ([fdo#109303])
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@i915_query@query-topology-known-pci-ids.html

  * igt@i915_suspend@debugfs-reader:
    - shard-mtlp:         NOTRUN -> [ABORT][62] ([i915#8213] / [i915#9414])
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-6/igt@i915_suspend@debugfs-reader.html

  * igt@kms_addfb_basic@addfb25-x-tiled-mismatch-legacy:
    - shard-dg2:          NOTRUN -> [SKIP][63] ([i915#4212])
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_addfb_basic@addfb25-x-tiled-mismatch-legacy.html

  * igt@kms_addfb_basic@basic-y-tiled-legacy:
    - shard-dg1:          NOTRUN -> [SKIP][64] ([i915#4215])
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_addfb_basic@basic-y-tiled-legacy.html

  * igt@kms_addfb_basic@bo-too-small-due-to-tiling:
    - shard-dg1:          NOTRUN -> [SKIP][65] ([i915#4212]) +1 other test skip
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_addfb_basic@bo-too-small-due-to-tiling.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-hdmi-a-3-4-mc_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][66] ([i915#8502] / [i915#8709]) +11 other tests skip
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-5/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-hdmi-a-3-4-mc_ccs.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-b-hdmi-a-1-y-rc_ccs:
    - shard-rkl:          NOTRUN -> [SKIP][67] ([i915#8502]) +3 other tests skip
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-b-hdmi-a-1-y-rc_ccs.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-c-hdmi-a-4-y-rc_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][68] ([i915#8502]) +7 other tests skip
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-15/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-c-hdmi-a-4-y-rc_ccs.html

  * igt@kms_async_flips@crc@pipe-a-hdmi-a-3:
    - shard-dg2:          NOTRUN -> [FAIL][69] ([i915#8247]) +3 other tests fail
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_async_flips@crc@pipe-a-hdmi-a-3.html

  * igt@kms_async_flips@crc@pipe-b-hdmi-a-3:
    - shard-dg1:          NOTRUN -> [FAIL][70] ([i915#8247]) +3 other tests fail
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-12/igt@kms_async_flips@crc@pipe-b-hdmi-a-3.html

  * igt@kms_atomic@plane-primary-overlay-mutable-zpos:
    - shard-dg2:          NOTRUN -> [SKIP][71] ([i915#404])
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_atomic@plane-primary-overlay-mutable-zpos.html

  * igt@kms_atomic_transition@plane-all-modeset-transition-fencing-internal-panels:
    - shard-dg1:          NOTRUN -> [SKIP][72] ([i915#1769] / [i915#3555])
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_atomic_transition@plane-all-modeset-transition-fencing-internal-panels.html

  * igt@kms_atomic_transition@plane-all-modeset-transition-internal-panels:
    - shard-dg2:          NOTRUN -> [SKIP][73] ([i915#1769] / [i915#3555])
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_atomic_transition@plane-all-modeset-transition-internal-panels.html

  * igt@kms_big_fb@4-tiled-64bpp-rotate-180:
    - shard-dg1:          NOTRUN -> [SKIP][74] ([i915#4538] / [i915#5286]) +2 other tests skip
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_big_fb@4-tiled-64bpp-rotate-180.html

  * igt@kms_big_fb@4-tiled-64bpp-rotate-90:
    - shard-mtlp:         NOTRUN -> [SKIP][75] ([fdo#111614]) +1 other test skip
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_big_fb@4-tiled-64bpp-rotate-90.html

  * igt@kms_big_fb@4-tiled-addfb-size-overflow:
    - shard-dg1:          NOTRUN -> [SKIP][76] ([i915#5286])
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_big_fb@4-tiled-addfb-size-overflow.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0:
    - shard-mtlp:         [PASS][77] -> [FAIL][78] ([i915#5138])
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-2/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0.html
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-4/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0.html

  * igt@kms_big_fb@x-tiled-16bpp-rotate-90:
    - shard-dg2:          NOTRUN -> [SKIP][79] ([fdo#111614]) +4 other tests skip
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_big_fb@x-tiled-16bpp-rotate-90.html

  * igt@kms_big_fb@x-tiled-64bpp-rotate-90:
    - shard-dg1:          NOTRUN -> [SKIP][80] ([i915#3638])
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_big_fb@x-tiled-64bpp-rotate-90.html

  * igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip:
    - shard-tglu:         [PASS][81] -> [FAIL][82] ([i915#3743])
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-tglu-10/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-tglu-6/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html

  * igt@kms_big_fb@y-tiled-64bpp-rotate-0:
    - shard-dg2:          NOTRUN -> [SKIP][83] ([i915#5190]) +10 other tests skip
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_big_fb@y-tiled-64bpp-rotate-0.html

  * igt@kms_big_fb@yf-tiled-16bpp-rotate-0:
    - shard-mtlp:         NOTRUN -> [SKIP][84] ([fdo#111615]) +1 other test skip
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_big_fb@yf-tiled-16bpp-rotate-0.html

  * igt@kms_big_fb@yf-tiled-8bpp-rotate-90:
    - shard-dg2:          NOTRUN -> [SKIP][85] ([i915#4538] / [i915#5190]) +4 other tests skip
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_big_fb@yf-tiled-8bpp-rotate-90.html

  * igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0:
    - shard-dg1:          NOTRUN -> [SKIP][86] ([i915#4538]) +2 other tests skip
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html

  * igt@kms_big_joiner@basic:
    - shard-dg2:          NOTRUN -> [SKIP][87] ([i915#2705])
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_big_joiner@basic.html

  * igt@kms_big_joiner@invalid-modeset:
    - shard-mtlp:         NOTRUN -> [SKIP][88] ([i915#2705])
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_big_joiner@invalid-modeset.html

  * igt@kms_ccs@pipe-a-bad-rotation-90-yf_tiled_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][89] ([i915#5354] / [i915#6095]) +2 other tests skip
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_ccs@pipe-a-bad-rotation-90-yf_tiled_ccs.html

  * igt@kms_ccs@pipe-b-bad-rotation-90-4_tiled_dg2_rc_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][90] ([i915#3689] / [i915#5354] / [i915#6095]) +5 other tests skip
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_ccs@pipe-b-bad-rotation-90-4_tiled_dg2_rc_ccs.html

  * igt@kms_ccs@pipe-c-ccs-on-another-bo-y_tiled_gen12_rc_ccs_cc:
    - shard-dg2:          NOTRUN -> [SKIP][91] ([i915#3689] / [i915#3886] / [i915#5354]) +7 other tests skip
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_ccs@pipe-c-ccs-on-another-bo-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-c-missing-ccs-buffer-y_tiled_gen12_mc_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][92] ([i915#3689] / [i915#3886] / [i915#5354] / [i915#6095]) +1 other test skip
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_ccs@pipe-c-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-c-random-ccs-data-y_tiled_gen12_mc_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][93] ([i915#3886] / [i915#5354] / [i915#6095]) +1 other test skip
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_ccs@pipe-c-random-ccs-data-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-d-bad-pixel-format-4_tiled_mtl_mc_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][94] ([i915#5354] / [i915#6095]) +13 other tests skip
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_ccs@pipe-d-bad-pixel-format-4_tiled_mtl_mc_ccs.html

  * igt@kms_ccs@pipe-d-crc-primary-basic-yf_tiled_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][95] ([i915#3689] / [i915#5354]) +23 other tests skip
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_ccs@pipe-d-crc-primary-basic-yf_tiled_ccs.html

  * igt@kms_cdclk@plane-scaling@pipe-c-hdmi-a-3:
    - shard-dg2:          NOTRUN -> [SKIP][96] ([i915#4087]) +3 other tests skip
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_cdclk@plane-scaling@pipe-c-hdmi-a-3.html

  * igt@kms_chamelium_color@ctm-0-25:
    - shard-dg1:          NOTRUN -> [SKIP][97] ([fdo#111827]) +1 other test skip
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_chamelium_color@ctm-0-25.html

  * igt@kms_chamelium_color@ctm-0-50:
    - shard-mtlp:         NOTRUN -> [SKIP][98] ([fdo#111827])
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_chamelium_color@ctm-0-50.html

  * igt@kms_chamelium_color@ctm-green-to-red:
    - shard-dg2:          NOTRUN -> [SKIP][99] ([fdo#111827]) +1 other test skip
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_chamelium_color@ctm-green-to-red.html

  * igt@kms_chamelium_edid@dp-mode-timings:
    - shard-dg2:          NOTRUN -> [SKIP][100] ([i915#7828]) +8 other tests skip
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_chamelium_edid@dp-mode-timings.html

  * igt@kms_chamelium_frames@dp-crc-single:
    - shard-dg1:          NOTRUN -> [SKIP][101] ([i915#7828]) +4 other tests skip
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_chamelium_frames@dp-crc-single.html

  * igt@kms_content_protection@atomic:
    - shard-dg2:          NOTRUN -> [SKIP][102] ([i915#7118])
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_content_protection@atomic.html

  * igt@kms_content_protection@dp-mst-type-1:
    - shard-dg2:          NOTRUN -> [SKIP][103] ([i915#3299]) +1 other test skip
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_content_protection@dp-mst-type-1.html

  * igt@kms_content_protection@legacy:
    - shard-dg1:          NOTRUN -> [SKIP][104] ([i915#7116])
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_content_protection@legacy.html

  * igt@kms_cursor_crc@cursor-onscreen-32x32:
    - shard-dg1:          NOTRUN -> [SKIP][105] ([i915#3555]) +2 other tests skip
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_cursor_crc@cursor-onscreen-32x32.html

  * igt@kms_cursor_crc@cursor-onscreen-512x512:
    - shard-dg2:          NOTRUN -> [SKIP][106] ([i915#3359]) +1 other test skip
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_cursor_crc@cursor-onscreen-512x512.html

  * igt@kms_cursor_crc@cursor-rapid-movement-32x32:
    - shard-dg2:          NOTRUN -> [SKIP][107] ([i915#3555]) +5 other tests skip
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_cursor_crc@cursor-rapid-movement-32x32.html

  * igt@kms_cursor_crc@cursor-rapid-movement-512x170:
    - shard-dg1:          NOTRUN -> [SKIP][108] ([i915#3359])
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_cursor_crc@cursor-rapid-movement-512x170.html

  * igt@kms_cursor_legacy@2x-flip-vs-cursor-atomic:
    - shard-dg2:          NOTRUN -> [SKIP][109] ([fdo#109274] / [fdo#111767] / [i915#5354])
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_cursor_legacy@2x-flip-vs-cursor-atomic.html

  * igt@kms_cursor_legacy@2x-long-cursor-vs-flip-atomic:
    - shard-dg2:          NOTRUN -> [SKIP][110] ([fdo#109274] / [i915#5354]) +2 other tests skip
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_cursor_legacy@2x-long-cursor-vs-flip-atomic.html

  * igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic:
    - shard-glk:          [PASS][111] -> [FAIL][112] ([i915#72])
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-glk2/igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic.html
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-glk2/igt@kms_cursor_legacy@2x-long-flip-vs-cursor-atomic.html

  * igt@kms_cursor_legacy@cursora-vs-flipb-atomic-transitions-varying-size:
    - shard-mtlp:         NOTRUN -> [SKIP][113] ([i915#3546])
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_cursor_legacy@cursora-vs-flipb-atomic-transitions-varying-size.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
    - shard-glk:          [PASS][114] -> [FAIL][115] ([i915#2346])
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-glk3/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-glk9/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
    - shard-apl:          [PASS][116] -> [FAIL][117] ([i915#2346])
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl1/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl2/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html

  * igt@kms_dirtyfb@dirtyfb-ioctl@drrs-hdmi-a-2:
    - shard-dg2:          NOTRUN -> [SKIP][118] ([i915#9226] / [i915#9261]) +1 other test skip
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-2/igt@kms_dirtyfb@dirtyfb-ioctl@drrs-hdmi-a-2.html

  * igt@kms_dirtyfb@dirtyfb-ioctl@fbc-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][119] ([i915#9227])
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_dirtyfb@dirtyfb-ioctl@fbc-hdmi-a-1.html
    - shard-dg1:          NOTRUN -> [SKIP][120] ([i915#9227])
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_dirtyfb@dirtyfb-ioctl@fbc-hdmi-a-1.html

  * igt@kms_dirtyfb@dirtyfb-ioctl@fbc-hdmi-a-2:
    - shard-dg2:          NOTRUN -> [SKIP][121] ([i915#9227])
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-2/igt@kms_dirtyfb@dirtyfb-ioctl@fbc-hdmi-a-2.html

  * igt@kms_dirtyfb@dirtyfb-ioctl@psr-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][122] ([i915#9226] / [i915#9261]) +1 other test skip
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_dirtyfb@dirtyfb-ioctl@psr-hdmi-a-1.html
    - shard-dg1:          NOTRUN -> [SKIP][123] ([i915#9226] / [i915#9261]) +1 other test skip
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_dirtyfb@dirtyfb-ioctl@psr-hdmi-a-1.html

  * igt@kms_dsc@dsc-with-bpc:
    - shard-mtlp:         NOTRUN -> [SKIP][124] ([i915#3555] / [i915#3840])
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_dsc@dsc-with-bpc.html

  * igt@kms_dsc@dsc-with-output-formats:
    - shard-dg1:          NOTRUN -> [SKIP][125] ([i915#3555] / [i915#3840])
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_dsc@dsc-with-output-formats.html

  * igt@kms_flip@2x-flip-vs-fences:
    - shard-dg1:          NOTRUN -> [SKIP][126] ([i915#8381])
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_flip@2x-flip-vs-fences.html

  * igt@kms_flip@2x-flip-vs-panning-vs-hang:
    - shard-dg2:          NOTRUN -> [SKIP][127] ([fdo#109274]) +8 other tests skip
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_flip@2x-flip-vs-panning-vs-hang.html

  * igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-upscaling@pipe-a-valid-mode:
    - shard-dg1:          NOTRUN -> [SKIP][128] ([i915#2587] / [i915#2672]) +1 other test skip
   [128]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-upscaling@pipe-a-valid-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-yftile-to-16bpp-yftile-upscaling@pipe-a-valid-mode:
    - shard-dg2:          NOTRUN -> [SKIP][129] ([i915#2672]) +4 other tests skip
   [129]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_flip_scaled_crc@flip-64bpp-yftile-to-16bpp-yftile-upscaling@pipe-a-valid-mode.html

  * igt@kms_force_connector_basic@prune-stale-modes:
    - shard-dg2:          NOTRUN -> [SKIP][130] ([i915#5274])
   [130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_force_connector_basic@prune-stale-modes.html

  * igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-blt:
    - shard-dg2:          NOTRUN -> [SKIP][131] ([i915#5354]) +38 other tests skip
   [131]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-pwrite:
    - shard-mtlp:         NOTRUN -> [SKIP][132] ([i915#1825]) +4 other tests skip
   [132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-pwrite.html
    - shard-snb:          [PASS][133] -> [SKIP][134] ([fdo#109271])
   [133]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-snb1/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-pwrite.html
   [134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-snb4/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-pwrite.html

  * igt@kms_frontbuffer_tracking@fbc-tiling-y:
    - shard-dg2:          NOTRUN -> [SKIP][135] ([i915#5460])
   [135]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_frontbuffer_tracking@fbc-tiling-y.html

  * igt@kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-indfb-draw-render:
    - shard-dg2:          NOTRUN -> [SKIP][136] ([i915#3458]) +17 other tests skip
   [136]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_frontbuffer_tracking@fbcpsr-1p-offscren-pri-indfb-draw-render.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-blt:
    - shard-dg1:          NOTRUN -> [SKIP][137] ([fdo#111825]) +15 other tests skip
   [137]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-mmap-wc:
    - shard-dg2:          NOTRUN -> [SKIP][138] ([i915#8708]) +17 other tests skip
   [138]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-spr-indfb-onoff:
    - shard-snb:          NOTRUN -> [SKIP][139] ([fdo#109271]) +22 other tests skip
   [139]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-snb2/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-spr-indfb-onoff.html

  * igt@kms_frontbuffer_tracking@fbcpsr-rgb101010-draw-render:
    - shard-dg1:          NOTRUN -> [SKIP][140] ([i915#3458]) +7 other tests skip
   [140]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_frontbuffer_tracking@fbcpsr-rgb101010-draw-render.html

  * igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-mmap-gtt:
    - shard-mtlp:         NOTRUN -> [SKIP][141] ([i915#8708])
   [141]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-mmap-gtt.html

  * igt@kms_frontbuffer_tracking@psr-rgb101010-draw-mmap-wc:
    - shard-dg1:          NOTRUN -> [SKIP][142] ([i915#8708]) +9 other tests skip
   [142]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_frontbuffer_tracking@psr-rgb101010-draw-mmap-wc.html

  * igt@kms_hdr@invalid-hdr:
    - shard-dg1:          NOTRUN -> [SKIP][143] ([i915#3555] / [i915#8228])
   [143]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_hdr@invalid-hdr.html

  * igt@kms_pipe_crc_basic@suspend-read-crc@pipe-c-dp-1:
    - shard-apl:          [PASS][144] -> [INCOMPLETE][145] ([i915#180] / [i915#9392])
   [144]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl2/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-c-dp-1.html
   [145]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl2/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-c-dp-1.html

  * igt@kms_plane_multiple@tiling-yf:
    - shard-mtlp:         NOTRUN -> [SKIP][146] ([i915#3555] / [i915#8806])
   [146]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_plane_multiple@tiling-yf.html

  * igt@kms_plane_scaling@intel-max-src-size@pipe-a-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [FAIL][147] ([i915#8292])
   [147]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_plane_scaling@intel-max-src-size@pipe-a-hdmi-a-1.html
    - shard-dg1:          NOTRUN -> [FAIL][148] ([i915#8292])
   [148]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_plane_scaling@intel-max-src-size@pipe-a-hdmi-a-1.html

  * igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-a-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][149] ([i915#5176] / [i915#9423]) +1 other test skip
   [149]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-rotation@pipe-a-hdmi-a-1.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-c-hdmi-a-1:
    - shard-dg1:          NOTRUN -> [SKIP][150] ([i915#5235]) +11 other tests skip
   [150]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-c-hdmi-a-1.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-20x20@pipe-b-hdmi-a-2:
    - shard-rkl:          NOTRUN -> [SKIP][151] ([i915#5235]) +3 other tests skip
   [151]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-1/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-20x20@pipe-b-hdmi-a-2.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-d-hdmi-a-3:
    - shard-dg2:          NOTRUN -> [SKIP][152] ([i915#5235]) +15 other tests skip
   [152]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-5/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-d-hdmi-a-3.html

  * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-25@pipe-b-edp-1:
    - shard-mtlp:         NOTRUN -> [SKIP][153] ([i915#5235]) +3 other tests skip
   [153]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-25@pipe-b-edp-1.html

  * igt@kms_prime@basic-crc-vgem:
    - shard-dg1:          NOTRUN -> [SKIP][154] ([i915#6524])
   [154]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_prime@basic-crc-vgem.html

  * igt@kms_psr2_sf@overlay-plane-move-continuous-sf:
    - shard-dg2:          NOTRUN -> [SKIP][155] ([i915#658]) +2 other tests skip
   [155]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@kms_psr2_sf@overlay-plane-move-continuous-sf.html

  * igt@kms_psr2_sf@plane-move-sf-dmg-area:
    - shard-dg1:          NOTRUN -> [SKIP][156] ([fdo#111068] / [i915#658])
   [156]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_psr2_sf@plane-move-sf-dmg-area.html

  * igt@kms_psr@psr2_sprite_plane_move:
    - shard-dg2:          NOTRUN -> [SKIP][157] ([i915#1072]) +7 other tests skip
   [157]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_psr@psr2_sprite_plane_move.html

  * igt@kms_psr@psr2_sprite_render:
    - shard-dg1:          NOTRUN -> [SKIP][158] ([i915#1072] / [i915#4078]) +1 other test skip
   [158]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-16/igt@kms_psr@psr2_sprite_render.html

  * igt@kms_psr@suspend:
    - shard-mtlp:         [PASS][159] -> [ABORT][160] ([i915#9414])
   [159]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-2/igt@kms_psr@suspend.html
   [160]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-4/igt@kms_psr@suspend.html

  * igt@kms_rotation_crc@primary-rotation-90:
    - shard-dg2:          NOTRUN -> [SKIP][161] ([i915#4235])
   [161]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_rotation_crc@primary-rotation-90.html

  * igt@kms_rotation_crc@primary-y-tiled-reflect-x-270:
    - shard-mtlp:         NOTRUN -> [SKIP][162] ([i915#4235])
   [162]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_rotation_crc@primary-y-tiled-reflect-x-270.html

  * igt@kms_rotation_crc@primary-yf-tiled-reflect-x-270:
    - shard-dg1:          NOTRUN -> [SKIP][163] ([fdo#111615] / [i915#5289])
   [163]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-270.html

  * igt@kms_setmode@invalid-clone-single-crtc-stealing:
    - shard-mtlp:         NOTRUN -> [SKIP][164] ([i915#3555] / [i915#8809])
   [164]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@kms_setmode@invalid-clone-single-crtc-stealing.html

  * igt@kms_tiled_display@basic-test-pattern:
    - shard-dg2:          NOTRUN -> [SKIP][165] ([i915#8623])
   [165]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-10/igt@kms_tiled_display@basic-test-pattern.html

  * igt@kms_writeback@writeback-check-output:
    - shard-dg1:          NOTRUN -> [SKIP][166] ([i915#2437])
   [166]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@kms_writeback@writeback-check-output.html

  * igt@kms_writeback@writeback-invalid-parameters:
    - shard-dg2:          NOTRUN -> [SKIP][167] ([i915#2437])
   [167]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@kms_writeback@writeback-invalid-parameters.html

  * igt@perf_pmu@busy-double-start@rcs0:
    - shard-mtlp:         [PASS][168] -> [FAIL][169] ([i915#4349])
   [168]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-7/igt@perf_pmu@busy-double-start@rcs0.html
   [169]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@perf_pmu@busy-double-start@rcs0.html

  * igt@prime_vgem@basic-fence-mmap:
    - shard-dg2:          NOTRUN -> [SKIP][170] ([i915#3708] / [i915#4077])
   [170]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@prime_vgem@basic-fence-mmap.html

  * igt@prime_vgem@basic-gtt:
    - shard-dg1:          NOTRUN -> [SKIP][171] ([i915#3708] / [i915#4077])
   [171]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@prime_vgem@basic-gtt.html

  * igt@prime_vgem@basic-read:
    - shard-dg2:          NOTRUN -> [SKIP][172] ([i915#3291] / [i915#3708])
   [172]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-11/igt@prime_vgem@basic-read.html

  * igt@prime_vgem@fence-read-hang:
    - shard-dg2:          NOTRUN -> [SKIP][173] ([i915#3708])
   [173]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@prime_vgem@fence-read-hang.html

  * igt@v3d/v3d_job_submission@array-job-submission:
    - shard-dg2:          NOTRUN -> [SKIP][174] ([i915#2575]) +13 other tests skip
   [174]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@v3d/v3d_job_submission@array-job-submission.html

  * igt@v3d/v3d_submit_csd@bad-bo:
    - shard-mtlp:         NOTRUN -> [SKIP][175] ([i915#2575]) +1 other test skip
   [175]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@v3d/v3d_submit_csd@bad-bo.html

  * igt@v3d/v3d_submit_csd@bad-perfmon:
    - shard-dg1:          NOTRUN -> [SKIP][176] ([i915#2575]) +3 other tests skip
   [176]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@v3d/v3d_submit_csd@bad-perfmon.html

  * igt@vc4/vc4_tiling@get-bad-modifier:
    - shard-mtlp:         NOTRUN -> [SKIP][177] ([i915#7711]) +1 other test skip
   [177]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-5/igt@vc4/vc4_tiling@get-bad-modifier.html

  * igt@vc4/vc4_wait_bo@used-bo-1ns:
    - shard-dg2:          NOTRUN -> [SKIP][178] ([i915#7711]) +6 other tests skip
   [178]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@vc4/vc4_wait_bo@used-bo-1ns.html

  * igt@vc4/vc4_wait_seqno@bad-seqno-1ns:
    - shard-dg1:          NOTRUN -> [SKIP][179] ([i915#7711]) +4 other tests skip
   [179]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@vc4/vc4_wait_seqno@bad-seqno-1ns.html

  
#### Possible fixes ####

  * igt@drm_fdinfo@most-busy-idle-check-all@rcs0:
    - shard-rkl:          [FAIL][180] ([i915#7742]) -> [PASS][181]
   [180]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-7/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html
   [181]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-6/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html

  * igt@gem_ctx_persistence@engines-hang@ccs0:
    - shard-mtlp:         [ABORT][182] ([i915#9414]) -> [PASS][183] +1 other test pass
   [182]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-4/igt@gem_ctx_persistence@engines-hang@ccs0.html
   [183]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-6/igt@gem_ctx_persistence@engines-hang@ccs0.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-rkl:          [FAIL][184] ([i915#2842]) -> [PASS][185]
   [184]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-6/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [185]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-1/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_exec_schedule@preempt-engines@ccs0:
    - shard-mtlp:         [FAIL][186] ([i915#9119]) -> [PASS][187] +4 other tests pass
   [186]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-4/igt@gem_exec_schedule@preempt-engines@ccs0.html
   [187]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-8/igt@gem_exec_schedule@preempt-engines@ccs0.html

  * igt@gem_exec_schedule@preempt-engines@rcs0:
    - shard-mtlp:         [DMESG-FAIL][188] ([i915#8962]) -> [PASS][189]
   [188]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-4/igt@gem_exec_schedule@preempt-engines@rcs0.html
   [189]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-8/igt@gem_exec_schedule@preempt-engines@rcs0.html

  * igt@i915_pm_rpm@dpms-mode-unset-non-lpsp:
    - shard-rkl:          [SKIP][190] ([i915#1397]) -> [PASS][191] +2 other tests pass
   [190]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-7/igt@i915_pm_rpm@dpms-mode-unset-non-lpsp.html
   [191]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-2/igt@i915_pm_rpm@dpms-mode-unset-non-lpsp.html

  * igt@i915_pm_rpm@modeset-lpsp:
    - shard-dg1:          [SKIP][192] ([i915#1397]) -> [PASS][193]
   [192]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg1-17/igt@i915_pm_rpm@modeset-lpsp.html
   [193]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@i915_pm_rpm@modeset-lpsp.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait:
    - shard-dg2:          [SKIP][194] ([i915#1397]) -> [PASS][195]
   [194]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-10/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html
   [195]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-5/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html

  * igt@i915_pm_rpm@system-suspend-execbuf:
    - shard-dg2:          [INCOMPLETE][196] ([i915#9142]) -> [PASS][197]
   [196]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-5/igt@i915_pm_rpm@system-suspend-execbuf.html
   [197]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@i915_pm_rpm@system-suspend-execbuf.html

  * igt@i915_selftest@live@gt_heartbeat:
    - shard-apl:          [DMESG-FAIL][198] ([i915#5334]) -> [PASS][199]
   [198]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl4/igt@i915_selftest@live@gt_heartbeat.html
   [199]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl7/igt@i915_selftest@live@gt_heartbeat.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip:
    - shard-mtlp:         [FAIL][200] ([i915#5138]) -> [PASS][201]
   [200]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-4/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
   [201]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-8/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html

  * igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip:
    - shard-tglu:         [FAIL][202] ([i915#3743]) -> [PASS][203] +1 other test pass
   [202]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-tglu-10/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip.html
   [203]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-tglu-6/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip.html

  * igt@kms_big_fb@yf-tiled-32bpp-rotate-0:
    - shard-apl:          [DMESG-WARN][204] ([i915#1982] / [i915#7634]) -> [PASS][205]
   [204]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl2/igt@kms_big_fb@yf-tiled-32bpp-rotate-0.html
   [205]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl6/igt@kms_big_fb@yf-tiled-32bpp-rotate-0.html

  * igt@kms_cursor_crc@cursor-rapid-movement-256x85@pipe-c-dp-1:
    - shard-apl:          [DMESG-WARN][206] ([i915#180] / [i915#7634]) -> [PASS][207] +4 other tests pass
   [206]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl2/igt@kms_cursor_crc@cursor-rapid-movement-256x85@pipe-c-dp-1.html
   [207]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl6/igt@kms_cursor_crc@cursor-rapid-movement-256x85@pipe-c-dp-1.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
    - shard-apl:          [FAIL][208] ([i915#2346]) -> [PASS][209]
   [208]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-apl3/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
   [209]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-apl4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

  * igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a3:
    - shard-dg2:          [FAIL][210] ([fdo#103375]) -> [PASS][211]
   [210]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-5/igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a3.html
   [211]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-6/igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a3.html

  * igt@kms_frontbuffer_tracking@fbc-1p-indfb-fliptrack-mmap-gtt:
    - shard-snb:          [INCOMPLETE][212] -> [PASS][213]
   [212]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-snb1/igt@kms_frontbuffer_tracking@fbc-1p-indfb-fliptrack-mmap-gtt.html
   [213]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-snb2/igt@kms_frontbuffer_tracking@fbc-1p-indfb-fliptrack-mmap-gtt.html

  * igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-blt:
    - shard-dg2:          [FAIL][214] ([i915#6880]) -> [PASS][215]
   [214]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-6/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-blt.html
   [215]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-3/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-shrfb-draw-blt.html

  * igt@kms_plane_cursor@overlay@pipe-c-edp-1-size-128:
    - shard-mtlp:         [DMESG-WARN][216] ([i915#1982]) -> [PASS][217]
   [216]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-2/igt@kms_plane_cursor@overlay@pipe-c-edp-1-size-128.html
   [217]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-4/igt@kms_plane_cursor@overlay@pipe-c-edp-1-size-128.html

  * {igt@kms_pm_dc@dc9-dpms}:
    - shard-tglu:         [SKIP][218] ([i915#4281]) -> [PASS][219]
   [218]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-tglu-5/igt@kms_pm_dc@dc9-dpms.html
   [219]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-tglu-4/igt@kms_pm_dc@dc9-dpms.html

  * igt@kms_universal_plane@cursor-fb-leak@pipe-a-hdmi-a-1:
    - shard-tglu:         [FAIL][220] -> [PASS][221] +1 other test pass
   [220]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-tglu-5/igt@kms_universal_plane@cursor-fb-leak@pipe-a-hdmi-a-1.html
   [221]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-tglu-4/igt@kms_universal_plane@cursor-fb-leak@pipe-a-hdmi-a-1.html

  * igt@kms_universal_plane@cursor-fb-leak@pipe-a-vga-1:
    - shard-snb:          [FAIL][222] -> [PASS][223]
   [222]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-snb2/igt@kms_universal_plane@cursor-fb-leak@pipe-a-vga-1.html
   [223]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-snb4/igt@kms_universal_plane@cursor-fb-leak@pipe-a-vga-1.html

  
#### Warnings ####

  * igt@gem_create@create-ext-cpu-access-big:
    - shard-dg2:          [ABORT][224] ([i915#7461]) -> [INCOMPLETE][225] ([i915#9364])
   [224]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg2-7/igt@gem_create@create-ext-cpu-access-big.html
   [225]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg2-1/igt@gem_create@create-ext-cpu-access-big.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-glk:          [FAIL][226] ([i915#2842]) -> [FAIL][227] ([i915#2876])
   [226]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-glk3/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [227]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-glk9/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_lmem_swapping@smem-oom@lmem0:
    - shard-dg1:          [TIMEOUT][228] ([i915#5493]) -> [DMESG-WARN][229] ([i915#4936] / [i915#5493])
   [228]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-dg1-19/igt@gem_lmem_swapping@smem-oom@lmem0.html
   [229]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-dg1-19/igt@gem_lmem_swapping@smem-oom@lmem0.html

  * igt@i915_pm_rc6_residency@rc6-idle@vecs0:
    - shard-tglu:         [WARN][230] ([i915#2681]) -> [FAIL][231] ([i915#2681] / [i915#3591])
   [230]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-tglu-2/igt@i915_pm_rc6_residency@rc6-idle@vecs0.html
   [231]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-tglu-8/igt@i915_pm_rc6_residency@rc6-idle@vecs0.html

  * igt@kms_async_flips@crc@pipe-b-edp-1:
    - shard-mtlp:         [FAIL][232] ([i915#8247]) -> [DMESG-FAIL][233] ([i915#8561]) +1 other test dmesg-fail
   [232]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-8/igt@kms_async_flips@crc@pipe-b-edp-1.html
   [233]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-3/igt@kms_async_flips@crc@pipe-b-edp-1.html

  * igt@kms_cursor_crc@cursor-dpms@pipe-a-edp-1:
    - shard-mtlp:         [DMESG-WARN][234] ([i915#9157]) -> [DMESG-WARN][235] ([i915#2017] / [i915#9157])
   [234]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-mtlp-2/igt@kms_cursor_crc@cursor-dpms@pipe-a-edp-1.html
   [235]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-mtlp-4/igt@kms_cursor_crc@cursor-dpms@pipe-a-edp-1.html

  * igt@kms_fbcon_fbt@psr:
    - shard-rkl:          [SKIP][236] ([i915#3955]) -> [SKIP][237] ([fdo#110189] / [i915#3955])
   [236]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-7/igt@kms_fbcon_fbt@psr.html
   [237]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-2/igt@kms_fbcon_fbt@psr.html

  * igt@kms_force_connector_basic@force-load-detect:
    - shard-rkl:          [SKIP][238] ([fdo#109285] / [i915#4098]) -> [SKIP][239] ([fdo#109285])
   [238]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-2/igt@kms_force_connector_basic@force-load-detect.html
   [239]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_force_connector_basic@force-load-detect.html

  * igt@kms_multipipe_modeset@basic-max-pipe-crc-check:
    - shard-rkl:          [SKIP][240] ([i915#4070] / [i915#4816]) -> [SKIP][241] ([i915#4816])
   [240]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13728/shard-rkl-6/igt@kms_multipipe_modeset@basic-max-pipe-crc-check.html
   [241]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/shard-rkl-7/igt@kms_multipipe_modeset@basic-max-pipe-crc-check.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109303]: https://bugs.freedesktop.org/show_bug.cgi?id=109303
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111767]: https://bugs.freedesktop.org/show_bug.cgi?id=111767
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [fdo#112283]: https://bugs.freedesktop.org/show_bug.cgi?id=112283
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1769]: https://gitlab.freedesktop.org/drm/intel/issues/1769
  [i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
  [i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
  [i915#1937]: https://gitlab.freedesktop.org/drm/intel/issues/1937
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#2017]: https://gitlab.freedesktop.org/drm/intel/issues/2017
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2681]: https://gitlab.freedesktop.org/drm/intel/issues/2681
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#2876]: https://gitlab.freedesktop.org/drm/intel/issues/2876
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
  [i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
  [i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3591]: https://gitlab.freedesktop.org/drm/intel/issues/3591
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#3743]: https://gitlab.freedesktop.org/drm/intel/issues/3743
  [i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#4036]: https://gitlab.freedesktop.org/drm/intel/issues/4036
  [i915#404]: https://gitlab.freedesktop.org/drm/intel/issues/404
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4087]: https://gitlab.freedesktop.org/drm/intel/issues/4087
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
  [i915#4215]: https://gitlab.freedesktop.org/drm/intel/issues/4215
  [i915#4235]: https://gitlab.freedesktop.org/drm/intel/issues/4235
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4281]: https://gitlab.freedesktop.org/drm/intel/issues/4281
  [i915#4349]: https://gitlab.freedesktop.org/drm/intel/issues/4349
  [i915#4473]: https://gitlab.freedesktop.org/drm/intel/issues/4473
  [i915#4537]: https://gitlab.freedesktop.org/drm/intel/issues/4537
  [i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
  [i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
  [i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
  [i915#4816]: https://gitlab.freedesktop.org/drm/intel/issues/4816
  [i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
  [i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
  [i915#4880]: https://gitlab.freedesktop.org/drm/intel/issues/4880
  [i915#4936]: https://gitlab.freedesktop.org/drm/intel/issues/4936
  [i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5190]: https://gitlab.freedesktop.org/drm/intel/issues/5190
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5274]: https://gitlab.freedesktop.org/drm/intel/issues/5274
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#5460]: https://gitlab.freedesktop.org/drm/intel/issues/5460
  [i915#5493]: https://gitlab.freedesktop.org/drm/intel/issues/5493
  [i915#5889]: https://gitlab.freedesktop.org/drm/intel/issues/5889
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6621]: https://gitlab.freedesktop.org/drm/intel/issues/6621
  [i915#6880]: https://gitlab.freedesktop.org/drm/intel/issues/6880
  [i915#7116]: https://gitlab.freedesktop.org/drm/intel/issues/7116
  [i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
  [i915#72]: https://gitlab.freedesktop.org/drm/intel/issues/72
  [i915#7297]: https://gitlab.freedesktop.org/drm/intel/issues/7297
  [i915#7461]: https://gitlab.freedesktop.org/drm/intel/issues/7461
  [i915#7634]: https://gitlab.freedesktop.org/drm/intel/issues/7634
  [i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
  [i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
  [i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#7975]: https://gitlab.freedesktop.org/drm/intel/issues/7975
  [i915#8213]: https://gitlab.freedesktop.org/drm/intel/issues/8213
  [i915#8228]: https://gitlab.freedesktop.org/drm/intel/issues/8228
  [i915#8247]: https://gitlab.freedesktop.org/drm/intel/issues/8247
  [i915#8292]: https://gitlab.freedesktop.org/drm/intel/issues/8292
  [i915#8381]: https://gitlab.freedesktop.org/drm/intel/issues/8381
  [i915#8414]: https://gitlab.freedesktop.org/drm/intel/issues/8414
  [i915#8430]: https://gitlab.freedesktop.org/drm/intel/issues/8430
  [i915#8431]: https://gitlab.freedesktop.org/drm/intel/issues/8431
  [i915#8502]: https://gitlab.freedesktop.org/drm/intel/issues/8502
  [i915#8555]: https://gitlab.freedesktop.org/drm/intel/issues/8555
  [i915#8561]: https://gitlab.freedesktop.org/drm/intel/issues/8561
  [i915#8562]: https://gitlab.freedesktop.org/drm/intel/issues/8562
  [i915#8623]: https://gitlab.freedesktop.org/drm/intel/issues/8623
  [i915#8708]: https://gitlab.freedesktop.org/drm/intel/issues/8708
  [i915#8709]: https://gitlab.freedesktop.org/drm/intel/issues/8709
  [i915#8806]: https://gitlab.freedesktop.org/drm/intel/issues/8806
  [i915#8809]: https://gitlab.freedesktop.org/drm/intel/issues/8809
  [i915#8962]: https://gitlab.freedesktop.org/drm/intel/issues/8962
  [i915#9053]: https://gitlab.freedesktop.org/drm/intel/issues/9053
  [i915#9067]: https://gitlab.freedesktop.org/drm/intel/issues/9067
  [i915#9119]: https://gitlab.freedesktop.org/drm/intel/issues/9119
  [i915#9142]: https://gitlab.freedesktop.org/drm/intel/issues/9142
  [i915#9157]: https://gitlab.freedesktop.org/drm/intel/issues/9157
  [i915#9226]: https://gitlab.freedesktop.org/drm/intel/issues/9226
  [i915#9227]: https://gitlab.freedesktop.org/drm/intel/issues/9227
  [i915#9261]: https://gitlab.freedesktop.org/drm/intel/issues/9261
  [i915#9364]: https://gitlab.freedesktop.org/drm/intel/issues/9364
  [i915#9392]: https://gitlab.freedesktop.org/drm/intel/issues/9392
  [i915#9414]: https://gitlab.freedesktop.org/drm/intel/issues/9414
  [i915#9423]: https://gitlab.freedesktop.org/drm/intel/issues/9423


Build changes
-------------

  * Linux: CI_DRM_13728 -> Patchwork_124744v3
  * Piglit: None -> piglit_4509

  CI-20190529: 20190529
  CI_DRM_13728: e642685fe494c1893ab2fd94e6dc7b6827303d90 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7520: 6435c8825e9269bdac515ca96cba4502b5b770f5 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_124744v3: e642685fe494c1893ab2fd94e6dc7b6827303d90 @ git://anongit.freedesktop.org/gfx-ci/linux
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_124744v3/index.html

[-- Attachment #2: Type: text/html, Size: 80451 bytes --]