Intel-XE Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-01-28 22:21 [PATCH 1/3] drm/xe: Add a function to zap page table by address range Oak Zeng
@ 2025-01-28 22:21 ` Oak Zeng
  2025-01-28 23:05   ` Cavitt, Jonathan
  2025-01-29  8:52   ` Thomas Hellström
  0 siblings, 2 replies; 38+ messages in thread
From: Oak Zeng @ 2025-01-28 22:21 UTC (permalink / raw)
  To: intel-xe; +Cc: joonas.lahtinen, Thomas.Hellstrom

Normally scratch page is not allowed when a vm is operate under page
fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
is fault mode relies on recoverable page to work, while scratch page
can mute recoverable page fault.

On xe2 and xe3, HW prefetching can cause page fault interrupt. Due to
page fault interrupt overhead (i.e., need Guc and KMD involved to fix
the page fault), HW prefetching can be slowed by many orders of magnitude.

Fix this problem by allowing scratch page under fault mode for xe2 and xe3.
With scratch page in place, HW prefetching could always hit scratch page
instead of causing interrupt.

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 196d347c6ac0..3346f88f284a 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1732,6 +1732,11 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->extensions))
 		return -EINVAL;
 
+	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
+			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
+			 !(IS_LUNARLAKE(xe) || IS_BATTLEMAGE(xe) || IS_PANTHERLAKE(xe))))
+		return -EINVAL;
+
 	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
 		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
 
@@ -1745,10 +1750,6 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->flags & ~ALL_DRM_XE_VM_CREATE_FLAGS))
 		return -EINVAL;
 
-	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
-			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
-		return -EINVAL;
-
 	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
 			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
 		return -EINVAL;
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* RE: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-01-28 22:21 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
@ 2025-01-28 23:05   ` Cavitt, Jonathan
  2025-01-29  8:52   ` Thomas Hellström
  1 sibling, 0 replies; 38+ messages in thread
From: Cavitt, Jonathan @ 2025-01-28 23:05 UTC (permalink / raw)
  To: Zeng, Oak, intel-xe@lists.freedesktop.org
  Cc: joonas.lahtinen@linux.intel.com, Thomas.Hellstrom@linux.intel.com,
	Cavitt, Jonathan

-----Original Message-----
From: Intel-xe <intel-xe-bounces@lists.freedesktop.org> On Behalf Of Oak Zeng
Sent: Tuesday, January 28, 2025 2:22 PM
To: intel-xe@lists.freedesktop.org
Cc: joonas.lahtinen@linux.intel.com; Thomas.Hellstrom@linux.intel.com
Subject: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
> 
> Normally scratch page is not allowed when a vm is operate under page
> fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> is fault mode relies on recoverable page to work, while scratch page
> can mute recoverable page fault.
> 
> On xe2 and xe3, HW prefetching can cause page fault interrupt. Due to
> page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> the page fault), HW prefetching can be slowed by many orders of magnitude.
> 
> Fix this problem by allowing scratch page under fault mode for xe2 and xe3.
> With scratch page in place, HW prefetching could always hit scratch page
> instead of causing interrupt.
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>

LGTM, though see my comment for patch 2 regarding the added platform checks.
Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
-Jonathan Cavitt

> ---
>  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 196d347c6ac0..3346f88f284a 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1732,6 +1732,11 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>  	if (XE_IOCTL_DBG(xe, args->extensions))
>  		return -EINVAL;
>  
> +	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> +			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> +			 !(IS_LUNARLAKE(xe) || IS_BATTLEMAGE(xe) || IS_PANTHERLAKE(xe))))
> +		return -EINVAL;
> +
>  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
>  		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
>  
> @@ -1745,10 +1750,6 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>  	if (XE_IOCTL_DBG(xe, args->flags & ~ALL_DRM_XE_VM_CREATE_FLAGS))
>  		return -EINVAL;
>  
> -	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> -			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> -		return -EINVAL;
> -
>  	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
>  			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>  		return -EINVAL;
> -- 
> 2.26.3
> 
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-01-28 22:21 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
  2025-01-28 23:05   ` Cavitt, Jonathan
@ 2025-01-29  8:52   ` Thomas Hellström
  2025-01-29 16:41     ` Matthew Brost
  1 sibling, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-01-29  8:52 UTC (permalink / raw)
  To: Oak Zeng, intel-xe; +Cc: joonas.lahtinen

On Tue, 2025-01-28 at 17:21 -0500, Oak Zeng wrote:
> Normally scratch page is not allowed when a vm is operate under page
> fault mode, i.e., in the existing codes,
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> is fault mode relies on recoverable page to work, while scratch page
> can mute recoverable page fault.
> 
> On xe2 and xe3, HW prefetching can cause page fault interrupt. Due to
> page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> the page fault), HW prefetching can be slowed by many orders of
> magnitude.

The problem on xe2 and xe3 as I understand it is not the overhead but
the fact that OOB prefetching causes pagefaults and we can't resolve
those.

> 
> Fix this problem by allowing scratch page under fault mode for xe2
> and xe3.
> With scratch page in place, HW prefetching could always hit scratch
> page
> instead of causing interrupt.
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 196d347c6ac0..3346f88f284a 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1732,6 +1732,11 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> void *data,
>  	if (XE_IOCTL_DBG(xe, args->extensions))
>  		return -EINVAL;
>  
> +	if (XE_IOCTL_DBG(xe, args->flags &
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> +			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> +			 !(IS_LUNARLAKE(xe) || IS_BATTLEMAGE(xe) ||
> IS_PANTHERLAKE(xe))))
> +		return -EINVAL;
> +

Could we perhaps add a bit in the xe->info structure?

/Thomas


>  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
>  		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
>  
> @@ -1745,10 +1750,6 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> void *data,
>  	if (XE_IOCTL_DBG(xe, args->flags &
> ~ALL_DRM_XE_VM_CREATE_FLAGS))
>  		return -EINVAL;
>  
> -	if (XE_IOCTL_DBG(xe, args->flags &
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> -			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> -		return -EINVAL;
> -
>  	if (XE_IOCTL_DBG(xe, !(args->flags &
> DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
>  			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>  		return -EINVAL;


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-01-29  8:52   ` Thomas Hellström
@ 2025-01-29 16:41     ` Matthew Brost
  0 siblings, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-01-29 16:41 UTC (permalink / raw)
  To: Thomas Hellström; +Cc: Oak Zeng, intel-xe, joonas.lahtinen

On Wed, Jan 29, 2025 at 09:52:54AM +0100, Thomas Hellström wrote:
> On Tue, 2025-01-28 at 17:21 -0500, Oak Zeng wrote:
> > Normally scratch page is not allowed when a vm is operate under page
> > fault mode, i.e., in the existing codes,
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> > is fault mode relies on recoverable page to work, while scratch page
> > can mute recoverable page fault.
> > 
> > On xe2 and xe3, HW prefetching can cause page fault interrupt. Due to
> > page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> > the page fault), HW prefetching can be slowed by many orders of
> > magnitude.
> 
> The problem on xe2 and xe3 as I understand it is not the overhead but
> the fact that OOB prefetching causes pagefaults and we can't resolve
> those.
> 
> > 
> > Fix this problem by allowing scratch page under fault mode for xe2
> > and xe3.
> > With scratch page in place, HW prefetching could always hit scratch
> > page
> > instead of causing interrupt.
> > 
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> > index 196d347c6ac0..3346f88f284a 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1732,6 +1732,11 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> > void *data,
> >  	if (XE_IOCTL_DBG(xe, args->extensions))
> >  		return -EINVAL;
> >  
> > +	if (XE_IOCTL_DBG(xe, args->flags &
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > +			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > +			 !(IS_LUNARLAKE(xe) || IS_BATTLEMAGE(xe) ||
> > IS_PANTHERLAKE(xe))))
> > +		return -EINVAL;
> > +
> 
> Could we perhaps add a bit in the xe->info structure?
> 

+1. This seems like a good idea.

Matt

> /Thomas
> 
> 
> >  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
> >  		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
> >  
> > @@ -1745,10 +1750,6 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> > void *data,
> >  	if (XE_IOCTL_DBG(xe, args->flags &
> > ~ALL_DRM_XE_VM_CREATE_FLAGS))
> >  		return -EINVAL;
> >  
> > -	if (XE_IOCTL_DBG(xe, args->flags &
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > -			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > -		return -EINVAL;
> > -
> >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> > DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> >  			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> >  		return -EINVAL;
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor
@ 2025-02-04 18:45 Oak Zeng
  2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
                   ` (10 more replies)
  0 siblings, 11 replies; 38+ messages in thread
From: Oak Zeng @ 2025-02-04 18:45 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

On some platform, scratch page is needed for out of bound prefetch
to work. Introduce a bit in device descriptor to specify whether
this device needs scratch page to work.

v2: introduce a needs_scratch bit in device info (Thomas, Jonathan)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_device_types.h | 3 +++
 drivers/gpu/drm/xe/xe_pci.c          | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index c0e886bac183..1197ab31e528 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -45,6 +45,7 @@ struct xe_pxp;
 #define GRAPHICS_VERx100(xe) ((xe)->info.graphics_verx100)
 #define MEDIA_VERx100(xe) ((xe)->info.media_verx100)
 #define IS_DGFX(xe) ((xe)->info.is_dgfx)
+#define NEEDS_SCRATCH(xe) ((xe)->info.needs_scratch)
 
 #define XE_VRAM_FLAGS_NEED64K		BIT(0)
 
@@ -318,6 +319,8 @@ struct xe_device {
 		u8 has_usm:1;
 		/** @info.is_dgfx: is discrete device */
 		u8 is_dgfx:1;
+		/** @info.needs_scratch: needs scratch page for oob prefetch to work */
+		u8 needs_scratch:1;
 		/**
 		 * @info.probe_display: Probe display hardware.  If set to
 		 * false, the driver will behave as if there is no display
diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
index e8ef7d6b4db8..09736674a77e 100644
--- a/drivers/gpu/drm/xe/xe_pci.c
+++ b/drivers/gpu/drm/xe/xe_pci.c
@@ -67,6 +67,7 @@ struct xe_device_desc {
 	u8 has_llc:1;
 	u8 has_pxp:1;
 	u8 has_sriov:1;
+	u8 needs_scratch:1;
 	u8 skip_guc_pc:1;
 	u8 skip_mtcfg:1;
 	u8 skip_pcode:1;
@@ -353,6 +354,7 @@ static const struct xe_device_desc lnl_desc = {
 	.dma_mask_size = 46,
 	.has_display = true,
 	.has_pxp = true,
+	.needs_scratch = true,
 };
 
 static const struct xe_device_desc bmg_desc = {
@@ -361,6 +363,7 @@ static const struct xe_device_desc bmg_desc = {
 	.dma_mask_size = 46,
 	.has_display = true,
 	.has_heci_cscfi = 1,
+	.needs_scratch = true,
 };
 
 static const struct xe_device_desc ptl_desc = {
@@ -368,6 +371,7 @@ static const struct xe_device_desc ptl_desc = {
 	.dma_mask_size = 46,
 	.has_display = true,
 	.require_force_probe = true,
+	.needs_scratch = true,
 };
 
 #undef PLATFORM
@@ -643,6 +647,7 @@ static int xe_info_init_early(struct xe_device *xe,
 	xe->info.skip_guc_pc = desc->skip_guc_pc;
 	xe->info.skip_mtcfg = desc->skip_mtcfg;
 	xe->info.skip_pcode = desc->skip_pcode;
+	xe->info.needs_scratch = desc->needs_scratch;
 
 	xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) &&
 				 xe_modparam.probe_display &&
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
@ 2025-02-04 18:45 ` Oak Zeng
  2025-02-05 14:35   ` Thomas Hellström
  2025-02-06 10:34   ` Matthew Brost
  2025-02-04 18:45 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
                   ` (9 subsequent siblings)
  10 siblings, 2 replies; 38+ messages in thread
From: Oak Zeng @ 2025-02-04 18:45 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

When a vm runs under fault mode, if scratch page is enabled, we need
to clear the scratch page mapping before vm_bind for the vm_bind
address range. Under fault mode, we depend on recoverable page fault
to establish mapping in page table. If scratch page is not cleared,
GPU access of address won't cause page fault because it always hits
the existing scratch page mapping.

When vm_bind with IMMEDIATE flag, there is no need of clearing as
immediate bind can overwrite the scratch page mapping.

So far only is xe2 and xe3 products are allowed to enable scratch page
under fault mode. On other platform we don't allow scratch page under
fault mode, so no need of such clearing.

v2: Rework vm_bind pipeline to clear scratch page mapping. This is similar
to a map operation, with the exception that PTEs are cleared instead of
pointing to valid physical pages. (Matt, Thomas)

TLB invalidation is needed after clear scratch page mapping as larger
scratch page mapping could be backed by physical page and cached in
TLB. (Matt, Thomas)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++----------
 drivers/gpu/drm/xe/xe_pt_types.h |  2 +
 drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
 drivers/gpu/drm/xe/xe_vm_types.h |  2 +
 4 files changed, 75 insertions(+), 24 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
index 1ddcc7e79a93..3fd0ae2dbe7d 100644
--- a/drivers/gpu/drm/xe/xe_pt.c
+++ b/drivers/gpu/drm/xe/xe_pt.c
@@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
 	 * granularity.
 	 */
 	bool needs_64K;
+	/* @clear_pt: clear page table entries during the bind walk */
+	bool clear_pt;
 	/**
 	 * @vma: VMA being mapped
 	 */
@@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,
 
 		XE_WARN_ON(xe_walk->va_curs_start != addr);
 
-		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
-						 xe_res_dma(curs) + xe_walk->dma_offset,
-						 xe_walk->vma, pat_index, level);
-		pte |= xe_walk->default_pte;
+		if (xe_walk->clear_pt) {
+			pte = 0;
+		} else {
+			pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
+					xe_res_dma(curs) + xe_walk->dma_offset,
+					xe_walk->vma, pat_index, level);
+			pte |= xe_walk->default_pte;
 
-		/*
-		 * Set the XE_PTE_PS64 hint if possible, otherwise if
-		 * this device *requires* 64K PTE size for VRAM, fail.
-		 */
-		if (level == 0 && !xe_parent->is_compact) {
-			if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
-				xe_walk->vma->gpuva.flags |= XE_VMA_PTE_64K;
-				pte |= XE_PTE_PS64;
-			} else if (XE_WARN_ON(xe_walk->needs_64K)) {
-				return -EINVAL;
+			/*
+			 * Set the XE_PTE_PS64 hint if possible, otherwise if
+			 * this device *requires* 64K PTE size for VRAM, fail.
+			 */
+			if (level == 0 && !xe_parent->is_compact) {
+				if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
+					xe_walk->vma->gpuva.flags |= XE_VMA_PTE_64K;
+					pte |= XE_PTE_PS64;
+				} else if (XE_WARN_ON(xe_walk->needs_64K)) {
+					return -EINVAL;
+				}
 			}
 		}
 
@@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,
 		if (unlikely(ret))
 			return ret;
 
-		if (!is_null)
+		if (!is_null && !xe_walk->clear_pt)
 			xe_res_next(curs, next - addr);
 		xe_walk->va_curs_start = next;
 		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);
@@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = {
  * @vma: The vma indicating the address range.
  * @entries: Storage for the update entries used for connecting the tree to
  * the main tree at commit time.
+ * @clear_pt: Clear the page table entries.
  * @num_entries: On output contains the number of @entries used.
  *
  * This function builds a disconnected page-table tree for a given address
@@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = {
  */
 static int
 xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
-		 struct xe_vm_pgtable_update *entries, u32 *num_entries)
+		 struct xe_vm_pgtable_update *entries,
+		 bool clear_pt, u32 *num_entries)
 {
 	struct xe_device *xe = tile_to_xe(tile);
 	struct xe_bo *bo = xe_vma_bo(vma);
@@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
 		.vma = vma,
 		.wupd.entries = entries,
 		.needs_64K = (xe_vma_vm(vma)->flags & XE_VM_FLAG_64K) && is_devmem,
+		.clear_pt = clear_pt,
 	};
 	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
 	int ret;
 
+	if (clear_pt) {
+		ret = xe_pt_walk_range(&pt->base, pt->level, xe_vma_start(vma),
+				       xe_vma_end(vma), &xe_walk.base);
+
+		*num_entries = xe_walk.wupd.num_used_entries;
+		return ret;
+	}
+
 	/**
 	 * Default atomic expectations for different allocation scenarios are as follows:
 	 *
@@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct xe_vm_pgtable_update *entries,
 
 static int
 xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
-		   struct xe_vm_pgtable_update *entries, u32 *num_entries)
+		   struct xe_vm_pgtable_update *entries,
+		   bool invalidate_on_bind, u32 *num_entries)
 {
 	int err;
 
 	*num_entries = 0;
-	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
+	err = xe_pt_stage_bind(tile, vma, entries, invalidate_on_bind,
+			       num_entries);
 	if (!err)
 		xe_tile_assert(tile, *num_entries);
 
@@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
 		return err;
 
 	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
+				 pt_update_ops->invalidate_on_bind,
 				 &pt_op->num_entries);
 	if (!err) {
 		xe_tile_assert(tile, pt_op->num_entries <=
@@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
 		 * it needs to be done here.
 		 */
 		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
-		     xe_vm_in_preempt_fence_mode(vm)))
+		     xe_vm_in_preempt_fence_mode(vm)) || pt_update_ops->invalidate_on_bind)
 			pt_update_ops->needs_invalidation = true;
 		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
 			/* We bump also if batch_invalidate_tlb is true */
@@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
 
 	switch (op->base.op) {
 	case DRM_GPUVA_OP_MAP:
-		if (!op->map.immediate && xe_vm_in_fault_mode(vm))
+		if (!op->map.immediate && xe_vm_in_fault_mode(vm) &&
+		    !op->map.invalidate_on_bind)
 			break;
 
+		if (op->map.invalidate_on_bind)
+			pt_update_ops->invalidate_on_bind = true;
+
 		err = bind_op_prepare(vm, tile, pt_update_ops, op->map.vma);
 		pt_update_ops->wait_vm_kernel = true;
 		break;
@@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm *vm, struct xe_tile *tile,
 	}
 	vma->tile_present |= BIT(tile->id);
 	vma->tile_staged &= ~BIT(tile->id);
+	if (pt_update_ops->invalidate_on_bind)
+		vma->tile_invalidated |= BIT(tile->id);
 	if (xe_vma_is_userptr(vma)) {
 		lockdep_assert_held_read(&vm->userptr.notifier_lock);
 		to_userptr_vma(vma)->userptr.initial_bind = true;
diff --git a/drivers/gpu/drm/xe/xe_pt_types.h b/drivers/gpu/drm/xe/xe_pt_types.h
index 384cc04de719..3d0aa2a5102e 100644
--- a/drivers/gpu/drm/xe/xe_pt_types.h
+++ b/drivers/gpu/drm/xe/xe_pt_types.h
@@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
 	bool needs_userptr_lock;
 	/** @needs_invalidation: Needs invalidation */
 	bool needs_invalidation;
+	/** @invalidate_on_bind: Invalidate the range before bind */
+	bool invalidate_on_bind;
 	/**
 	 * @wait_vm_bookkeep: PT operations need to wait until VM is idle
 	 * (bookkeep dma-resv slots are idle) and stage all future VM activity
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index d664f2e418b2..813d893d9b63 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op)
 }
 #endif
 
+static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm, u32 bind_flags)
+{
+	if (!xe_vm_in_fault_mode(vm))
+		return false;
+
+	if (!NEEDS_SCRATCH(vm->xe))
+		return false;
+
+	if (!xe_vm_has_scratch(vm))
+		return false;
+
+	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
+		return false;
+
+	return true;
+}
+
 /*
  * Create operations list from IOCTL arguments, setup operations fields so parse
  * and commit steps are decoupled from IOCTL arguments. This step can fail.
@@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
 			op->map.is_null = flags & DRM_XE_VM_BIND_FLAG_NULL;
 			op->map.dumpable = flags & DRM_XE_VM_BIND_FLAG_DUMPABLE;
 			op->map.pat_index = pat_index;
+			op->map.invalidate_on_bind =
+				__xe_vm_needs_clear_scratch_pages(vm, flags);
 		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
 			op->prefetch.region = prefetch_region;
 		}
@@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct drm_gpuva_ops *ops,
 				return PTR_ERR(vma);
 
 			op->map.vma = vma;
-			if (op->map.immediate || !xe_vm_in_fault_mode(vm))
+			if (op->map.immediate || !xe_vm_in_fault_mode(vm) ||
+			    op->map.invalidate_on_bind)
 				xe_vma_ops_incr_pt_update_ops(vops,
 							      op->tile_mask);
 			break;
@@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
 
 	switch (op->base.op) {
 	case DRM_GPUVA_OP_MAP:
-		err = vma_lock_and_validate(exec, op->map.vma,
-					    !xe_vm_in_fault_mode(vm) ||
-					    op->map.immediate);
+		if (!op->map.invalidate_on_bind)
+			err = vma_lock_and_validate(exec, op->map.vma,
+						    !xe_vm_in_fault_mode(vm) ||
+						    op->map.immediate);
 		break;
 	case DRM_GPUVA_OP_REMAP:
 		err = check_ufence(gpuva_to_vma(op->base.remap.unmap->va));
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index 52467b9b5348..dace04f4ea5e 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -297,6 +297,8 @@ struct xe_vma_op_map {
 	bool is_null;
 	/** @dumpable: whether BO is dumped on GPU hang */
 	bool dumpable;
+	/** @invalidate: invalidate the VMA before bind */
+	bool invalidate_on_bind;
 	/** @pat_index: The pat index to use for this operation. */
 	u16 pat_index;
 };
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
  2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
@ 2025-02-04 18:45 ` Oak Zeng
  2025-02-05 13:14   ` Thomas Hellström
  2025-02-25 22:05   ` Matthew Brost
  2025-02-04 23:27 ` ✓ CI.Patch_applied: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor Patchwork
                   ` (8 subsequent siblings)
  10 siblings, 2 replies; 38+ messages in thread
From: Oak Zeng @ 2025-02-04 18:45 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

Normally scratch page is not allowed when a vm is operate under page
fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
is fault mode relies on recoverable page to work, while scratch page
can mute recoverable page fault.

On xe2 and xe3, out of bound prefetch can cause page fault and further
system hang because xekmd can't resolve such page fault. SYCL and OCL
language runtime requires out of bound prefetch to be silently dropped
without causing any functional problem, thus the existing behavior
doesn't meet language runtime requirement.

At the same time, HW prefetching can cause page fault interrupt. Due to
page fault interrupt overhead (i.e., need Guc and KMD involved to fix
the page fault), HW prefetching can be slowed by many orders of magnitude.

Fix those problems by allowing scratch page under fault mode for xe2 and
xe3. With scratch page in place, HW prefetching could always hit scratch
page instead of causing interrupt.

A side effect is, scratch page could hide application program error.
Application out of bound accesses are hided by scratch page mapping,
instead of get reported to user.

igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
BMG:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738684805 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
Subtest scratch-fault: SUCCESS (0.080s)

Without this series, the test result is:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738686046 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
(xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
(xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
(xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
(xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
Stack trace:
  #0 ../lib/igt_core.c:2266 __igt_fail_assert()
  #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
  #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
  #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
  #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
  #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
  #6 [_start+0x2e]
Subtest scratch-fault failed.

v2: Refine commit message (Thomas)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 813d893d9b63..c0372f083d42 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->extensions))
 		return -EINVAL;
 
+	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
+			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
+			 !(NEEDS_SCRATCH(xe))))
+		return -EINVAL;
+
 	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
 		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
 
@@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->flags & ~ALL_DRM_XE_VM_CREATE_FLAGS))
 		return -EINVAL;
 
-	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
-			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
-		return -EINVAL;
-
 	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
 			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
 		return -EINVAL;
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* ✓ CI.Patch_applied: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
  2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
  2025-02-04 18:45 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
@ 2025-02-04 23:27 ` Patchwork
  2025-02-04 23:27 ` ✗ CI.checkpatch: warning " Patchwork
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:27 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: a19d8731db07 drm-tip: 2025y-02m-04d-23h-11m-08s UTC integration manifest
=== git am output follows ===
Applying: drm/xe: Introduced needs_scratch bit in device descriptor
Applying: drm/xe: Clear scratch page before vm_bind
Applying: drm/xe: Allow scratch page under fault mode for certain platform



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✗ CI.checkpatch: warning for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (2 preceding siblings ...)
  2025-02-04 23:27 ` ✓ CI.Patch_applied: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor Patchwork
@ 2025-02-04 23:27 ` Patchwork
  2025-02-04 23:28 ` ✓ CI.KUnit: success " Patchwork
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:27 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : warning

== Summary ==

+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt
Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master
30ab6715fc09baee6cc14cb3c89ad8858688d474
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit c4ffa0a79ac0cd4bc35a75faf42024ca54fed088
Author: Oak Zeng <oak.zeng@intel.com>
Date:   Tue Feb 4 13:45:58 2025 -0500

    drm/xe: Allow scratch page under fault mode for certain platform
    
    Normally scratch page is not allowed when a vm is operate under page
    fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
    and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
    is fault mode relies on recoverable page to work, while scratch page
    can mute recoverable page fault.
    
    On xe2 and xe3, out of bound prefetch can cause page fault and further
    system hang because xekmd can't resolve such page fault. SYCL and OCL
    language runtime requires out of bound prefetch to be silently dropped
    without causing any functional problem, thus the existing behavior
    doesn't meet language runtime requirement.
    
    At the same time, HW prefetching can cause page fault interrupt. Due to
    page fault interrupt overhead (i.e., need Guc and KMD involved to fix
    the page fault), HW prefetching can be slowed by many orders of magnitude.
    
    Fix those problems by allowing scratch page under fault mode for xe2 and
    xe3. With scratch page in place, HW prefetching could always hit scratch
    page instead of causing interrupt.
    
    A side effect is, scratch page could hide application program error.
    Application out of bound accesses are hided by scratch page mapping,
    instead of get reported to user.
    
    igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
    BMG:
    
    root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
    IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
    Using IGT_SRANDOM=1738684805 for randomisation
    Opened device: /dev/dri/card0
    Starting subtest: scratch-fault
    Subtest scratch-fault: SUCCESS (0.080s)
    
    Without this series, the test result is:
    
    root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
    IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
    Using IGT_SRANDOM=1738686046 for randomisation
    Opened device: /dev/dri/card0
    Starting subtest: scratch-fault
    (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
    (xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
    (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
    (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
    Stack trace:
      #0 ../lib/igt_core.c:2266 __igt_fail_assert()
      #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
      #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
      #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
      #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
      #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
      #6 [_start+0x2e]
    Subtest scratch-fault failed.
    
    v2: Refine commit message (Thomas)
    
    Signed-off-by: Oak Zeng <oak.zeng@intel.com>
+ /mt/dim checkpatch a19d8731db07e41101ed00b9d86ac8868df2a763 drm-intel
33af94ea466f drm/xe: Introduced needs_scratch bit in device descriptor
9a6167d1da96 drm/xe: Clear scratch page before vm_bind
c4ffa0a79ac0 drm/xe: Allow scratch page under fault mode for certain platform
-:34: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?)
#34: 
root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault

total: 0 errors, 1 warnings, 0 checks, 21 lines checked



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✓ CI.KUnit: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (3 preceding siblings ...)
  2025-02-04 23:27 ` ✗ CI.checkpatch: warning " Patchwork
@ 2025-02-04 23:28 ` Patchwork
  2025-02-04 23:45 ` ✓ CI.Build: " Patchwork
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:28 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
[23:27:36] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[23:27:40] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make all compile_commands.json ARCH=um O=.kunit --jobs=48
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
  156 | u64 ioread64_lo_hi(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
  163 | u64 ioread64_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
  170 | u64 ioread64be_lo_hi(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
  178 | u64 ioread64be_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
  264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
  272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
  280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
  288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~

[23:28:06] Starting KUnit Kernel (1/1)...
[23:28:06] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[23:28:07] ================== guc_buf (11 subtests) ===================
[23:28:07] [PASSED] test_smallest
[23:28:07] [PASSED] test_largest
[23:28:07] [PASSED] test_granular
[23:28:07] [PASSED] test_unique
[23:28:07] [PASSED] test_overlap
[23:28:07] [PASSED] test_reusable
[23:28:07] [PASSED] test_too_big
[23:28:07] [PASSED] test_flush
[23:28:07] [PASSED] test_lookup
[23:28:07] [PASSED] test_data
[23:28:07] [PASSED] test_class
[23:28:07] ===================== [PASSED] guc_buf =====================
[23:28:07] =================== guc_dbm (7 subtests) ===================
[23:28:07] [PASSED] test_empty
[23:28:07] [PASSED] test_default
[23:28:07] ======================== test_size  ========================
[23:28:07] [PASSED] 4
[23:28:07] [PASSED] 8
[23:28:07] [PASSED] 32
[23:28:07] [PASSED] 256
[23:28:07] ==================== [PASSED] test_size ====================
[23:28:07] ======================= test_reuse  ========================
[23:28:07] [PASSED] 4
[23:28:07] [PASSED] 8
[23:28:07] [PASSED] 32
[23:28:07] [PASSED] 256
[23:28:07] =================== [PASSED] test_reuse ====================
[23:28:07] =================== test_range_overlap  ====================
[23:28:07] [PASSED] 4
[23:28:07] [PASSED] 8
[23:28:07] [PASSED] 32
[23:28:07] [PASSED] 256
[23:28:07] =============== [PASSED] test_range_overlap ================
[23:28:07] =================== test_range_compact  ====================
[23:28:07] [PASSED] 4
[23:28:07] [PASSED] 8
[23:28:07] [PASSED] 32
[23:28:07] [PASSED] 256
[23:28:07] =============== [PASSED] test_range_compact ================
[23:28:07] ==================== test_range_spare  =====================
[23:28:07] [PASSED] 4
[23:28:07] [PASSED] 8
[23:28:07] [PASSED] 32
[23:28:07] [PASSED] 256
[23:28:07] ================ [PASSED] test_range_spare =================
[23:28:07] ===================== [PASSED] guc_dbm =====================
[23:28:07] =================== guc_idm (6 subtests) ===================
[23:28:07] [PASSED] bad_init
[23:28:07] [PASSED] no_init
[23:28:07] [PASSED] init_fini
[23:28:07] [PASSED] check_used
[23:28:07] [PASSED] check_quota
[23:28:07] [PASSED] check_all
[23:28:07] ===================== [PASSED] guc_idm =====================
[23:28:07] ================== no_relay (3 subtests) ===================
[23:28:07] [PASSED] xe_drops_guc2pf_if_not_ready
[23:28:07] [PASSED] xe_drops_guc2vf_if_not_ready
[23:28:07] [PASSED] xe_rejects_send_if_not_ready
[23:28:07] ==================== [PASSED] no_relay =====================
[23:28:07] ================== pf_relay (14 subtests) ==================
[23:28:07] [PASSED] pf_rejects_guc2pf_too_short
[23:28:07] [PASSED] pf_rejects_guc2pf_too_long
[23:28:07] [PASSED] pf_rejects_guc2pf_no_payload
[23:28:07] [PASSED] pf_fails_no_payload
[23:28:07] [PASSED] pf_fails_bad_origin
[23:28:07] [PASSED] pf_fails_bad_type
[23:28:07] [PASSED] pf_txn_reports_error
[23:28:07] [PASSED] pf_txn_sends_pf2guc
[23:28:07] [PASSED] pf_sends_pf2guc
[23:28:07] [SKIPPED] pf_loopback_nop
[23:28:07] [SKIPPED] pf_loopback_echo
[23:28:07] [SKIPPED] pf_loopback_fail
[23:28:07] [SKIPPED] pf_loopback_busy
[23:28:07] [SKIPPED] pf_loopback_retry
[23:28:07] ==================== [PASSED] pf_relay =====================
[23:28:07] ================== vf_relay (3 subtests) ===================
[23:28:07] [PASSED] vf_rejects_guc2vf_too_short
[23:28:07] [PASSED] vf_rejects_guc2vf_too_long
[23:28:07] [PASSED] vf_rejects_guc2vf_no_payload
[23:28:07] ==================== [PASSED] vf_relay =====================
[23:28:07] ================= pf_service (11 subtests) =================
[23:28:07] [PASSED] pf_negotiate_any
[23:28:07] [PASSED] pf_negotiate_base_match
[23:28:07] [PASSED] pf_negotiate_base_newer
[23:28:07] [PASSED] pf_negotiate_base_next
[23:28:07] [SKIPPED] pf_negotiate_base_older
[23:28:07] [PASSED] pf_negotiate_base_prev
[23:28:07] [PASSED] pf_negotiate_latest_match
[23:28:07] [PASSED] pf_negotiate_latest_newer
[23:28:07] [PASSED] pf_negotiate_latest_next
[23:28:07] [SKIPPED] pf_negotiate_latest_older
[23:28:07] [SKIPPED] pf_negotiate_latest_prev
[23:28:07] =================== [PASSED] pf_service ====================
[23:28:07] ===================== lmtt (1 subtest) =====================
[23:28:07] ======================== test_ops  =========================
[23:28:07] [PASSED] 2-level
[23:28:07] [PASSED] multi-level
[23:28:07] ==================== [PASSED] test_ops =====================
[23:28:07] ====================== [PASSED] lmtt =======================
[23:28:07] =================== xe_mocs (2 subtests) ===================
[23:28:07] ================ xe_live_mocs_kernel_kunit  ================
[23:28:07] =========== [SKIPPED] xe_live_mocs_kernel_kunit ============
[23:28:07] ================ xe_live_mocs_reset_kunit  =================
[23:28:07] ============ [SKIPPED] xe_live_mocs_reset_kunit ============
[23:28:07] ==================== [SKIPPED] xe_mocs =====================
[23:28:07] ================= xe_migrate (2 subtests) ==================
[23:28:07] ================= xe_migrate_sanity_kunit  =================
[23:28:07] ============ [SKIPPED] xe_migrate_sanity_kunit =============
[23:28:07] ================== xe_validate_ccs_kunit  ==================
[23:28:07] ============= [SKIPPED] xe_validate_ccs_kunit ==============
[23:28:07] =================== [SKIPPED] xe_migrate ===================
[23:28:07] ================== xe_dma_buf (1 subtest) ==================
[23:28:07] ==================== xe_dma_buf_kunit  =====================
[23:28:07] ================ [SKIPPED] xe_dma_buf_kunit ================
[23:28:07] =================== [SKIPPED] xe_dma_buf ===================
[23:28:07] ================= xe_bo_shrink (1 subtest) =================
[23:28:07] =================== xe_bo_shrink_kunit  ====================
[23:28:07] =============== [SKIPPED] xe_bo_shrink_kunit ===============
[23:28:07] ================== [SKIPPED] xe_bo_shrink ==================
[23:28:07] ==================== xe_bo (2 subtests) ====================
[23:28:07] ================== xe_ccs_migrate_kunit  ===================
[23:28:07] ============== [SKIPPED] xe_ccs_migrate_kunit ==============
stty: 'standard input': Inappropriate ioctl for device
[23:28:07] ==================== xe_bo_evict_kunit  ====================
[23:28:07] =============== [SKIPPED] xe_bo_evict_kunit ================
[23:28:07] ===================== [SKIPPED] xe_bo ======================
[23:28:07] ==================== args (11 subtests) ====================
[23:28:07] [PASSED] count_args_test
[23:28:07] [PASSED] call_args_example
[23:28:07] [PASSED] call_args_test
[23:28:07] [PASSED] drop_first_arg_example
[23:28:07] [PASSED] drop_first_arg_test
[23:28:07] [PASSED] first_arg_example
[23:28:07] [PASSED] first_arg_test
[23:28:07] [PASSED] last_arg_example
[23:28:07] [PASSED] last_arg_test
[23:28:07] [PASSED] pick_arg_example
[23:28:07] [PASSED] sep_comma_example
[23:28:07] ====================== [PASSED] args =======================
[23:28:07] =================== xe_pci (2 subtests) ====================
[23:28:07] [PASSED] xe_gmdid_graphics_ip
[23:28:07] [PASSED] xe_gmdid_media_ip
[23:28:07] ===================== [PASSED] xe_pci ======================
[23:28:07] =================== xe_rtp (2 subtests) ====================
[23:28:07] =============== xe_rtp_process_to_sr_tests  ================
[23:28:07] [PASSED] coalesce-same-reg
[23:28:07] [PASSED] no-match-no-add
[23:28:07] [PASSED] match-or
[23:28:07] [PASSED] match-or-xfail
[23:28:07] [PASSED] no-match-no-add-multiple-rules
[23:28:07] [PASSED] two-regs-two-entries
[23:28:07] [PASSED] clr-one-set-other
[23:28:07] [PASSED] set-field
[23:28:07] [PASSED] conflict-duplicate
[23:28:07] [PASSED] conflict-not-disjoint
[23:28:07] [PASSED] conflict-reg-type
[23:28:07] =========== [PASSED] xe_rtp_process_to_sr_tests ============
[23:28:07] ================== xe_rtp_process_tests  ===================
[23:28:07] [PASSED] active1
[23:28:07] [PASSED] active2
[23:28:07] [PASSED] active-inactive
[23:28:07] [PASSED] inactive-active
[23:28:07] [PASSED] inactive-1st_or_active-inactive
[23:28:07] [PASSED] inactive-2nd_or_active-inactive
[23:28:07] [PASSED] inactive-last_or_active-inactive
[23:28:07] [PASSED] inactive-no_or_active-inactive
[23:28:07] ============== [PASSED] xe_rtp_process_tests ===============
[23:28:07] ===================== [PASSED] xe_rtp ======================
[23:28:07] ==================== xe_wa (1 subtest) =====================
[23:28:07] ======================== xe_wa_gt  =========================
[23:28:07] [PASSED] TIGERLAKE (B0)
[23:28:07] [PASSED] DG1 (A0)
[23:28:07] [PASSED] DG1 (B0)
[23:28:07] [PASSED] ALDERLAKE_S (A0)
[23:28:07] [PASSED] ALDERLAKE_S (B0)
[23:28:07] [PASSED] ALDERLAKE_S (C0)
[23:28:07] [PASSED] ALDERLAKE_S (D0)
[23:28:07] [PASSED] ALDERLAKE_P (A0)
[23:28:07] [PASSED] ALDERLAKE_P (B0)
[23:28:07] [PASSED] ALDERLAKE_P (C0)
[23:28:07] [PASSED] ALDERLAKE_S_RPLS (D0)
[23:28:07] [PASSED] ALDERLAKE_P_RPLU (E0)
[23:28:07] [PASSED] DG2_G10 (C0)
[23:28:07] [PASSED] DG2_G11 (B1)
[23:28:07] [PASSED] DG2_G12 (A1)
[23:28:07] [PASSED] METEORLAKE (g:A0, m:A0)
[23:28:07] [PASSED] METEORLAKE (g:A0, m:A0)
[23:28:07] [PASSED] METEORLAKE (g:A0, m:A0)
[23:28:07] [PASSED] LUNARLAKE (g:A0, m:A0)
[23:28:07] [PASSED] LUNARLAKE (g:B0, m:A0)
[23:28:07] [PASSED] BATTLEMAGE (g:A0, m:A1)
[23:28:07] ==================== [PASSED] xe_wa_gt =====================
[23:28:07] ====================== [PASSED] xe_wa ======================
[23:28:07] ============================================================
[23:28:07] Testing complete. Ran 133 tests: passed: 117, skipped: 16
[23:28:07] Elapsed time: 30.355s total, 4.140s configuring, 25.950s building, 0.260s running

+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig
[23:28:07] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[23:28:08] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make all compile_commands.json ARCH=um O=.kunit --jobs=48
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
  156 | u64 ioread64_lo_hi(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
  163 | u64 ioread64_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
  170 | u64 ioread64be_lo_hi(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
  178 | u64 ioread64be_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
  264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
  272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
  280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
  288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~

[23:28:29] Starting KUnit Kernel (1/1)...
[23:28:29] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[23:28:29] =========== drm_validate_clone_mode (2 subtests) ===========
[23:28:29] ============== drm_test_check_in_clone_mode  ===============
[23:28:29] [PASSED] in_clone_mode
[23:28:29] [PASSED] not_in_clone_mode
[23:28:29] ========== [PASSED] drm_test_check_in_clone_mode ===========
[23:28:29] =============== drm_test_check_valid_clones  ===============
[23:28:29] [PASSED] not_in_clone_mode
[23:28:29] [PASSED] valid_clone
[23:28:29] [PASSED] invalid_clone
[23:28:29] =========== [PASSED] drm_test_check_valid_clones ===========
[23:28:29] ============= [PASSED] drm_validate_clone_mode =============
[23:28:29] ============= drm_validate_modeset (1 subtest) =============
[23:28:29] [PASSED] drm_test_check_connector_changed_modeset
[23:28:29] ============== [PASSED] drm_validate_modeset ===============
[23:28:29] ================== drm_buddy (7 subtests) ==================
[23:28:29] [PASSED] drm_test_buddy_alloc_limit
[23:28:29] [PASSED] drm_test_buddy_alloc_optimistic
[23:28:29] [PASSED] drm_test_buddy_alloc_pessimistic
[23:28:29] [PASSED] drm_test_buddy_alloc_pathological
[23:28:30] [PASSED] drm_test_buddy_alloc_contiguous
[23:28:30] [PASSED] drm_test_buddy_alloc_clear
[23:28:30] [PASSED] drm_test_buddy_alloc_range_bias
[23:28:30] ==================== [PASSED] drm_buddy ====================
[23:28:30] ============= drm_cmdline_parser (40 subtests) =============
[23:28:30] [PASSED] drm_test_cmdline_force_d_only
[23:28:30] [PASSED] drm_test_cmdline_force_D_only_dvi
[23:28:30] [PASSED] drm_test_cmdline_force_D_only_hdmi
[23:28:30] [PASSED] drm_test_cmdline_force_D_only_not_digital
[23:28:30] [PASSED] drm_test_cmdline_force_e_only
[23:28:30] [PASSED] drm_test_cmdline_res
[23:28:30] [PASSED] drm_test_cmdline_res_vesa
[23:28:30] [PASSED] drm_test_cmdline_res_vesa_rblank
[23:28:30] [PASSED] drm_test_cmdline_res_rblank
[23:28:30] [PASSED] drm_test_cmdline_res_bpp
[23:28:30] [PASSED] drm_test_cmdline_res_refresh
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_margins
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_force_off
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_analog
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_digital
[23:28:30] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced_margins_force_on
[23:28:30] [PASSED] drm_test_cmdline_res_margins_force_on
[23:28:30] [PASSED] drm_test_cmdline_res_vesa_margins
[23:28:30] [PASSED] drm_test_cmdline_name
[23:28:30] [PASSED] drm_test_cmdline_name_bpp
[23:28:30] [PASSED] drm_test_cmdline_name_option
[23:28:30] [PASSED] drm_test_cmdline_name_bpp_option
[23:28:30] [PASSED] drm_test_cmdline_rotate_0
[23:28:30] [PASSED] drm_test_cmdline_rotate_90
[23:28:30] [PASSED] drm_test_cmdline_rotate_180
[23:28:30] [PASSED] drm_test_cmdline_rotate_270
[23:28:30] [PASSED] drm_test_cmdline_hmirror
[23:28:30] [PASSED] drm_test_cmdline_vmirror
[23:28:30] [PASSED] drm_test_cmdline_margin_options
[23:28:30] [PASSED] drm_test_cmdline_multiple_options
[23:28:30] [PASSED] drm_test_cmdline_bpp_extra_and_option
[23:28:30] [PASSED] drm_test_cmdline_extra_and_option
[23:28:30] [PASSED] drm_test_cmdline_freestanding_options
[23:28:30] [PASSED] drm_test_cmdline_freestanding_force_e_and_options
[23:28:30] [PASSED] drm_test_cmdline_panel_orientation
[23:28:30] ================ drm_test_cmdline_invalid  =================
[23:28:30] [PASSED] margin_only
[23:28:30] [PASSED] interlace_only
[23:28:30] [PASSED] res_missing_x
[23:28:30] [PASSED] res_missing_y
[23:28:30] [PASSED] res_bad_y
[23:28:30] [PASSED] res_missing_y_bpp
[23:28:30] [PASSED] res_bad_bpp
[23:28:30] [PASSED] res_bad_refresh
[23:28:30] [PASSED] res_bpp_refresh_force_on_off
[23:28:30] [PASSED] res_invalid_mode
[23:28:30] [PASSED] res_bpp_wrong_place_mode
[23:28:30] [PASSED] name_bpp_refresh
[23:28:30] [PASSED] name_refresh
[23:28:30] [PASSED] name_refresh_wrong_mode
[23:28:30] [PASSED] name_refresh_invalid_mode
[23:28:30] [PASSED] rotate_multiple
[23:28:30] [PASSED] rotate_invalid_val
[23:28:30] [PASSED] rotate_truncated
[23:28:30] [PASSED] invalid_option
[23:28:30] [PASSED] invalid_tv_option
[23:28:30] [PASSED] truncated_tv_option
[23:28:30] ============ [PASSED] drm_test_cmdline_invalid =============
[23:28:30] =============== drm_test_cmdline_tv_options  ===============
[23:28:30] [PASSED] NTSC
[23:28:30] [PASSED] NTSC_443
[23:28:30] [PASSED] NTSC_J
[23:28:30] [PASSED] PAL
[23:28:30] [PASSED] PAL_M
[23:28:30] [PASSED] PAL_N
[23:28:30] [PASSED] SECAM
[23:28:30] [PASSED] MONO_525
[23:28:30] [PASSED] MONO_625
[23:28:30] =========== [PASSED] drm_test_cmdline_tv_options ===========
[23:28:30] =============== [PASSED] drm_cmdline_parser ================
[23:28:30] ========== drmm_connector_hdmi_init (20 subtests) ==========
[23:28:30] [PASSED] drm_test_connector_hdmi_init_valid
[23:28:30] [PASSED] drm_test_connector_hdmi_init_bpc_8
[23:28:30] [PASSED] drm_test_connector_hdmi_init_bpc_10
[23:28:30] [PASSED] drm_test_connector_hdmi_init_bpc_12
[23:28:30] [PASSED] drm_test_connector_hdmi_init_bpc_invalid
[23:28:30] [PASSED] drm_test_connector_hdmi_init_bpc_null
[23:28:30] [PASSED] drm_test_connector_hdmi_init_formats_empty
[23:28:30] [PASSED] drm_test_connector_hdmi_init_formats_no_rgb
[23:28:30] === drm_test_connector_hdmi_init_formats_yuv420_allowed  ===
[23:28:30] [PASSED] supported_formats=0x9 yuv420_allowed=1
[23:28:30] [PASSED] supported_formats=0x9 yuv420_allowed=0
[23:28:30] [PASSED] supported_formats=0x3 yuv420_allowed=1
[23:28:30] [PASSED] supported_formats=0x3 yuv420_allowed=0
[23:28:30] === [PASSED] drm_test_connector_hdmi_init_formats_yuv420_allowed ===
[23:28:30] [PASSED] drm_test_connector_hdmi_init_null_ddc
[23:28:30] [PASSED] drm_test_connector_hdmi_init_null_product
[23:28:30] [PASSED] drm_test_connector_hdmi_init_null_vendor
[23:28:30] [PASSED] drm_test_connector_hdmi_init_product_length_exact
[23:28:30] [PASSED] drm_test_connector_hdmi_init_product_length_too_long
[23:28:30] [PASSED] drm_test_connector_hdmi_init_product_valid
[23:28:30] [PASSED] drm_test_connector_hdmi_init_vendor_length_exact
[23:28:30] [PASSED] drm_test_connector_hdmi_init_vendor_length_too_long
[23:28:30] [PASSED] drm_test_connector_hdmi_init_vendor_valid
[23:28:30] ========= drm_test_connector_hdmi_init_type_valid  =========
[23:28:30] [PASSED] HDMI-A
[23:28:30] [PASSED] HDMI-B
[23:28:30] ===== [PASSED] drm_test_connector_hdmi_init_type_valid =====
[23:28:30] ======== drm_test_connector_hdmi_init_type_invalid  ========
[23:28:30] [PASSED] Unknown
[23:28:30] [PASSED] VGA
[23:28:30] [PASSED] DVI-I
[23:28:30] [PASSED] DVI-D
[23:28:30] [PASSED] DVI-A
[23:28:30] [PASSED] Composite
[23:28:30] [PASSED] SVIDEO
[23:28:30] [PASSED] LVDS
[23:28:30] [PASSED] Component
[23:28:30] [PASSED] DIN
[23:28:30] [PASSED] DP
[23:28:30] [PASSED] TV
[23:28:30] [PASSED] eDP
[23:28:30] [PASSED] Virtual
[23:28:30] [PASSED] DSI
[23:28:30] [PASSED] DPI
[23:28:30] [PASSED] Writeback
[23:28:30] [PASSED] SPI
[23:28:30] [PASSED] USB
[23:28:30] ==== [PASSED] drm_test_connector_hdmi_init_type_invalid ====
[23:28:30] ============ [PASSED] drmm_connector_hdmi_init =============
[23:28:30] ============= drmm_connector_init (3 subtests) =============
[23:28:30] [PASSED] drm_test_drmm_connector_init
[23:28:30] [PASSED] drm_test_drmm_connector_init_null_ddc
[23:28:30] ========= drm_test_drmm_connector_init_type_valid  =========
[23:28:30] [PASSED] Unknown
[23:28:30] [PASSED] VGA
[23:28:30] [PASSED] DVI-I
[23:28:30] [PASSED] DVI-D
[23:28:30] [PASSED] DVI-A
[23:28:30] [PASSED] Composite
[23:28:30] [PASSED] SVIDEO
[23:28:30] [PASSED] LVDS
[23:28:30] [PASSED] Component
[23:28:30] [PASSED] DIN
[23:28:30] [PASSED] DP
[23:28:30] [PASSED] HDMI-A
[23:28:30] [PASSED] HDMI-B
[23:28:30] [PASSED] TV
[23:28:30] [PASSED] eDP
[23:28:30] [PASSED] Virtual
[23:28:30] [PASSED] DSI
[23:28:30] [PASSED] DPI
[23:28:30] [PASSED] Writeback
[23:28:30] [PASSED] SPI
[23:28:30] [PASSED] USB
[23:28:30] ===== [PASSED] drm_test_drmm_connector_init_type_valid =====
[23:28:30] =============== [PASSED] drmm_connector_init ===============
[23:28:30] ========= drm_connector_dynamic_init (6 subtests) ==========
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_init
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_init_null_ddc
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_init_not_added
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_init_properties
[23:28:30] ===== drm_test_drm_connector_dynamic_init_type_valid  ======
[23:28:30] [PASSED] Unknown
[23:28:30] [PASSED] VGA
[23:28:30] [PASSED] DVI-I
[23:28:30] [PASSED] DVI-D
[23:28:30] [PASSED] DVI-A
[23:28:30] [PASSED] Composite
[23:28:30] [PASSED] SVIDEO
[23:28:30] [PASSED] LVDS
[23:28:30] [PASSED] Component
[23:28:30] [PASSED] DIN
[23:28:30] [PASSED] DP
[23:28:30] [PASSED] HDMI-A
[23:28:30] [PASSED] HDMI-B
[23:28:30] [PASSED] TV
[23:28:30] [PASSED] eDP
[23:28:30] [PASSED] Virtual
[23:28:30] [PASSED] DSI
[23:28:30] [PASSED] DPI
[23:28:30] [PASSED] Writeback
[23:28:30] [PASSED] SPI
[23:28:30] [PASSED] USB
[23:28:30] = [PASSED] drm_test_drm_connector_dynamic_init_type_valid ==
[23:28:30] ======== drm_test_drm_connector_dynamic_init_name  =========
[23:28:30] [PASSED] Unknown
[23:28:30] [PASSED] VGA
[23:28:30] [PASSED] DVI-I
[23:28:30] [PASSED] DVI-D
[23:28:30] [PASSED] DVI-A
[23:28:30] [PASSED] Composite
[23:28:30] [PASSED] SVIDEO
[23:28:30] [PASSED] LVDS
[23:28:30] [PASSED] Component
[23:28:30] [PASSED] DIN
[23:28:30] [PASSED] DP
[23:28:30] [PASSED] HDMI-A
[23:28:30] [PASSED] HDMI-B
[23:28:30] [PASSED] TV
[23:28:30] [PASSED] eDP
[23:28:30] [PASSED] Virtual
[23:28:30] [PASSED] DSI
[23:28:30] [PASSED] DPI
[23:28:30] [PASSED] Writeback
[23:28:30] [PASSED] SPI
[23:28:30] [PASSED] USB
[23:28:30] ==== [PASSED] drm_test_drm_connector_dynamic_init_name =====
[23:28:30] =========== [PASSED] drm_connector_dynamic_init ============
[23:28:30] ==== drm_connector_dynamic_register_early (4 subtests) =====
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_early_on_list
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_early_defer
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_early_no_init
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_early_no_mode_object
[23:28:30] ====== [PASSED] drm_connector_dynamic_register_early =======
[23:28:30] ======= drm_connector_dynamic_register (7 subtests) ========
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_on_list
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_no_defer
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_no_init
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_mode_object
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_sysfs
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_sysfs_name
[23:28:30] [PASSED] drm_test_drm_connector_dynamic_register_debugfs
[23:28:30] ========= [PASSED] drm_connector_dynamic_register ==========
[23:28:30] = drm_connector_attach_broadcast_rgb_property (2 subtests) =
[23:28:30] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property
[23:28:30] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property_hdmi_connector
[23:28:30] === [PASSED] drm_connector_attach_broadcast_rgb_property ===
[23:28:30] ========== drm_get_tv_mode_from_name (2 subtests) ==========
[23:28:30] ========== drm_test_get_tv_mode_from_name_valid  ===========
[23:28:30] [PASSED] NTSC
[23:28:30] [PASSED] NTSC-443
[23:28:30] [PASSED] NTSC-J
[23:28:30] [PASSED] PAL
[23:28:30] [PASSED] PAL-M
[23:28:30] [PASSED] PAL-N
[23:28:30] [PASSED] SECAM
[23:28:30] [PASSED] Mono
[23:28:30] ====== [PASSED] drm_test_get_tv_mode_from_name_valid =======
[23:28:30] [PASSED] drm_test_get_tv_mode_from_name_truncated
[23:28:30] ============ [PASSED] drm_get_tv_mode_from_name ============
[23:28:30] = drm_test_connector_hdmi_compute_mode_clock (12 subtests) =
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc_vic_1
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc_vic_1
[23:28:30] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_double
[23:28:30] = drm_test_connector_hdmi_compute_mode_clock_yuv420_valid  =
[23:28:30] [PASSED] VIC 96
[23:28:30] [PASSED] VIC 97
[23:28:30] [PASSED] VIC 101
[23:28:30] [PASSED] VIC 102
[23:28:30] [PASSED] VIC 106
[23:28:30] [PASSED] VIC 107
[23:28:30] === [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_valid ===
[23:28:30] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_10_bpc
[23:28:30] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_12_bpc
[23:28:30] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_8_bpc
[23:28:30] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_10_bpc
[23:28:30] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_12_bpc
[23:28:30] === [PASSED] drm_test_connector_hdmi_compute_mode_clock ====
[23:28:30] == drm_hdmi_connector_get_broadcast_rgb_name (2 subtests) ==
[23:28:30] === drm_test_drm_hdmi_connector_get_broadcast_rgb_name  ====
[23:28:30] [PASSED] Automatic
[23:28:30] [PASSED] Full
[23:28:30] [PASSED] Limited 16:235
[23:28:30] === [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name ===
[23:28:30] [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name_invalid
[23:28:30] ==== [PASSED] drm_hdmi_connector_get_broadcast_rgb_name ====
[23:28:30] == drm_hdmi_connector_get_output_format_name (2 subtests) ==
[23:28:30] === drm_test_drm_hdmi_connector_get_output_format_name  ====
[23:28:30] [PASSED] RGB
[23:28:30] [PASSED] YUV 4:2:0
[23:28:30] [PASSED] YUV 4:2:2
[23:28:30] [PASSED] YUV 4:4:4
[23:28:30] === [PASSED] drm_test_drm_hdmi_connector_get_output_format_name ===
[23:28:30] [PASSED] drm_test_drm_hdmi_connector_get_output_format_name_invalid
[23:28:30] ==== [PASSED] drm_hdmi_connector_get_output_format_name ====
[23:28:30] ============= drm_damage_helper (21 subtests) ==============
[23:28:30] [PASSED] drm_test_damage_iter_no_damage
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_fractional_src
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_src_moved
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_fractional_src_moved
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_not_visible
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_no_crtc
[23:28:30] [PASSED] drm_test_damage_iter_no_damage_no_fb
[23:28:30] [PASSED] drm_test_damage_iter_simple_damage
[23:28:30] [PASSED] drm_test_damage_iter_single_damage
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_intersect_src
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_outside_src
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_fractional_src
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_intersect_fractional_src
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_outside_fractional_src
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_src_moved
[23:28:30] [PASSED] drm_test_damage_iter_single_damage_fractional_src_moved
[23:28:30] [PASSED] drm_test_damage_iter_damage
[23:28:30] [PASSED] drm_test_damage_iter_damage_one_intersect
[23:28:30] [PASSED] drm_test_damage_iter_damage_one_outside
[23:28:30] [PASSED] drm_test_damage_iter_damage_src_moved
[23:28:30] [PASSED] drm_test_damage_iter_damage_not_visible
[23:28:30] ================ [PASSED] drm_damage_helper ================
[23:28:30] ============== drm_dp_mst_helper (3 subtests) ==============
[23:28:30] ============== drm_test_dp_mst_calc_pbn_mode  ==============
[23:28:30] [PASSED] Clock 154000 BPP 30 DSC disabled
[23:28:30] [PASSED] Clock 234000 BPP 30 DSC disabled
[23:28:30] [PASSED] Clock 297000 BPP 24 DSC disabled
[23:28:30] [PASSED] Clock 332880 BPP 24 DSC enabled
[23:28:30] [PASSED] Clock 324540 BPP 24 DSC enabled
[23:28:30] ========== [PASSED] drm_test_dp_mst_calc_pbn_mode ==========
[23:28:30] ============== drm_test_dp_mst_calc_pbn_div  ===============
[23:28:30] [PASSED] Link rate 2000000 lane count 4
[23:28:30] [PASSED] Link rate 2000000 lane count 2
[23:28:30] [PASSED] Link rate 2000000 lane count 1
[23:28:30] [PASSED] Link rate 1350000 lane count 4
[23:28:30] [PASSED] Link rate 1350000 lane count 2
[23:28:30] [PASSED] Link rate 1350000 lane count 1
[23:28:30] [PASSED] Link rate 1000000 lane count 4
[23:28:30] [PASSED] Link rate 1000000 lane count 2
[23:28:30] [PASSED] Link rate 1000000 lane count 1
[23:28:30] [PASSED] Link rate 810000 lane count 4
[23:28:30] [PASSED] Link rate 810000 lane count 2
[23:28:30] [PASSED] Link rate 810000 lane count 1
[23:28:30] [PASSED] Link rate 540000 lane count 4
[23:28:30] [PASSED] Link rate 540000 lane count 2
[23:28:30] [PASSED] Link rate 540000 lane count 1
[23:28:30] [PASSED] Link rate 270000 lane count 4
[23:28:30] [PASSED] Link rate 270000 lane count 2
[23:28:30] [PASSED] Link rate 270000 lane count 1
[23:28:30] [PASSED] Link rate 162000 lane count 4
[23:28:30] [PASSED] Link rate 162000 lane count 2
[23:28:30] [PASSED] Link rate 162000 lane count 1
[23:28:30] ========== [PASSED] drm_test_dp_mst_calc_pbn_div ===========
[23:28:30] ========= drm_test_dp_mst_sideband_msg_req_decode  =========
[23:28:30] [PASSED] DP_ENUM_PATH_RESOURCES with port number
[23:28:30] [PASSED] DP_POWER_UP_PHY with port number
[23:28:30] [PASSED] DP_POWER_DOWN_PHY with port number
[23:28:30] [PASSED] DP_ALLOCATE_PAYLOAD with SDP stream sinks
[23:28:30] [PASSED] DP_ALLOCATE_PAYLOAD with port number
[23:28:30] [PASSED] DP_ALLOCATE_PAYLOAD with VCPI
[23:28:30] [PASSED] DP_ALLOCATE_PAYLOAD with PBN
[23:28:30] [PASSED] DP_QUERY_PAYLOAD with port number
[23:28:30] [PASSED] DP_QUERY_PAYLOAD with VCPI
[23:28:30] [PASSED] DP_REMOTE_DPCD_READ with port number
[23:28:30] [PASSED] DP_REMOTE_DPCD_READ with DPCD address
[23:28:30] [PASSED] DP_REMOTE_DPCD_READ with max number of bytes
[23:28:30] [PASSED] DP_REMOTE_DPCD_WRITE with port number
[23:28:30] [PASSED] DP_REMOTE_DPCD_WRITE with DPCD address
[23:28:30] [PASSED] DP_REMOTE_DPCD_WRITE with data array
[23:28:30] [PASSED] DP_REMOTE_I2C_READ with port number
[23:28:30] [PASSED] DP_REMOTE_I2C_READ with I2C device ID
[23:28:30] [PASSED] DP_REMOTE_I2C_READ with transactions array
[23:28:30] [PASSED] DP_REMOTE_I2C_WRITE with port number
[23:28:30] [PASSED] DP_REMOTE_I2C_WRITE with I2C device ID
[23:28:30] [PASSED] DP_REMOTE_I2C_WRITE with data array
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream ID
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with client ID
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream event
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with valid stream event
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream behavior
[23:28:30] [PASSED] DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior
[23:28:30] ===== [PASSED] drm_test_dp_mst_sideband_msg_req_decode =====
[23:28:30] ================ [PASSED] drm_dp_mst_helper ================
[23:28:30] ================== drm_exec (7 subtests) ===================
[23:28:30] [PASSED] sanitycheck
[23:28:30] [PASSED] test_lock
[23:28:30] [PASSED] test_lock_unlock
[23:28:30] [PASSED] test_duplicates
[23:28:30] [PASSED] test_prepare
[23:28:30] [PASSED] test_prepare_array
[23:28:30] [PASSED] test_multiple_loops
[23:28:30] ==================== [PASSED] drm_exec =====================
[23:28:30] =========== drm_format_helper_test (17 subtests) ===========
[23:28:30] ============== drm_test_fb_xrgb8888_to_gray8  ==============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ========== [PASSED] drm_test_fb_xrgb8888_to_gray8 ==========
[23:28:30] ============= drm_test_fb_xrgb8888_to_rgb332  ==============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb332 ==========
[23:28:30] ============= drm_test_fb_xrgb8888_to_rgb565  ==============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb565 ==========
[23:28:30] ============ drm_test_fb_xrgb8888_to_xrgb1555  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_xrgb1555 =========
[23:28:30] ============ drm_test_fb_xrgb8888_to_argb1555  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_argb1555 =========
[23:28:30] ============ drm_test_fb_xrgb8888_to_rgba5551  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_rgba5551 =========
[23:28:30] ============= drm_test_fb_xrgb8888_to_rgb888  ==============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb888 ==========
[23:28:30] ============ drm_test_fb_xrgb8888_to_argb8888  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_argb8888 =========
[23:28:30] =========== drm_test_fb_xrgb8888_to_xrgb2101010  ===========
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======= [PASSED] drm_test_fb_xrgb8888_to_xrgb2101010 =======
[23:28:30] =========== drm_test_fb_xrgb8888_to_argb2101010  ===========
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======= [PASSED] drm_test_fb_xrgb8888_to_argb2101010 =======
[23:28:30] ============== drm_test_fb_xrgb8888_to_mono  ===============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ========== [PASSED] drm_test_fb_xrgb8888_to_mono ===========
[23:28:30] ==================== drm_test_fb_swab  =====================
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ================ [PASSED] drm_test_fb_swab =================
[23:28:30] ============ drm_test_fb_xrgb8888_to_xbgr8888  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_xbgr8888 =========
[23:28:30] ============ drm_test_fb_xrgb8888_to_abgr8888  =============
[23:28:30] [PASSED] single_pixel_source_buffer
[23:28:30] [PASSED] single_pixel_clip_rectangle
[23:28:30] [PASSED] well_known_colors
[23:28:30] [PASSED] destination_pitch
[23:28:30] ======== [PASSED] drm_test_fb_xrgb8888_to_abgr8888 =========
[23:28:30] ================= drm_test_fb_clip_offset  =================
[23:28:30] [PASSED] pass through
[23:28:30] [PASSED] horizontal offset
[23:28:30] [PASSED] vertical offset
[23:28:30] [PASSED] horizontal and vertical offset
[23:28:30] [PASSED] horizontal offset (custom pitch)
[23:28:30] [PASSED] vertical offset (custom pitch)
[23:28:30] [PASSED] horizontal and vertical offset (custom pitch)
[23:28:30] ============= [PASSED] drm_test_fb_clip_offset =============
[23:28:30] ============== drm_test_fb_build_fourcc_list  ==============
[23:28:30] [PASSED] no native formats
[23:28:30] [PASSED] XRGB8888 as native format
[23:28:30] [PASSED] remove duplicates
[23:28:30] [PASSED] convert alpha formats
[23:28:30] [PASSED] random formats
[23:28:30] ========== [PASSED] drm_test_fb_build_fourcc_list ==========
[23:28:30] =================== drm_test_fb_memcpy  ====================
[23:28:30] [PASSED] single_pixel_source_buffer: XR24 little-endian (0x34325258)
[23:28:30] [PASSED] single_pixel_source_buffer: XRA8 little-endian (0x38415258)
[23:28:30] [PASSED] single_pixel_source_buffer: YU24 little-endian (0x34325559)
[23:28:30] [PASSED] single_pixel_clip_rectangle: XB24 little-endian (0x34324258)
[23:28:30] [PASSED] single_pixel_clip_rectangle: XRA8 little-endian (0x38415258)
[23:28:30] [PASSED] single_pixel_clip_rectangle: YU24 little-endian (0x34325559)
[23:28:30] [PASSED] well_known_colors: XB24 little-endian (0x34324258)
[23:28:30] [PASSED] well_known_colors: XRA8 little-endian (0x38415258)
[23:28:30] [PASSED] well_known_colors: YU24 little-endian (0x34325559)
[23:28:30] [PASSED] destination_pitch: XB24 little-endian (0x34324258)
[23:28:30] [PASSED] destination_pitch: XRA8 little-endian (0x38415258)
[23:28:30] [PASSED] destination_pitch: YU24 little-endian (0x34325559)
[23:28:30] =============== [PASSED] drm_test_fb_memcpy ================
[23:28:30] ============= [PASSED] drm_format_helper_test ==============
[23:28:30] ================= drm_format (18 subtests) =================
[23:28:30] [PASSED] drm_test_format_block_width_invalid
[23:28:30] [PASSED] drm_test_format_block_width_one_plane
[23:28:30] [PASSED] drm_test_format_block_width_two_plane
[23:28:30] [PASSED] drm_test_format_block_width_three_plane
[23:28:30] [PASSED] drm_test_format_block_width_tiled
[23:28:30] [PASSED] drm_test_format_block_height_invalid
[23:28:30] [PASSED] drm_test_format_block_height_one_plane
[23:28:30] [PASSED] drm_test_format_block_height_two_plane
[23:28:30] [PASSED] drm_test_format_block_height_three_plane
[23:28:30] [PASSED] drm_test_format_block_height_tiled
[23:28:30] [PASSED] drm_test_format_min_pitch_invalid
[23:28:30] [PASSED] drm_test_format_min_pitch_one_plane_8bpp
[23:28:30] [PASSED] drm_test_format_min_pitch_one_plane_16bpp
[23:28:30] [PASSED] drm_test_format_min_pitch_one_plane_24bpp
[23:28:30] [PASSED] drm_test_format_min_pitch_one_plane_32bpp
[23:28:30] [PASSED] drm_test_format_min_pitch_two_plane
[23:28:30] [PASSED] drm_test_format_min_pitch_three_plane_8bpp
[23:28:30] [PASSED] drm_test_format_min_pitch_tiled
[23:28:30] =================== [PASSED] drm_format ====================
[23:28:30] ============== drm_framebuffer (10 subtests) ===============
[23:28:30] ========== drm_test_framebuffer_check_src_coords  ==========
[23:28:30] [PASSED] Success: source fits into fb
[23:28:30] [PASSED] Fail: overflowing fb with x-axis coordinate
[23:28:30] [PASSED] Fail: overflowing fb with y-axis coordinate
[23:28:30] [PASSED] Fail: overflowing fb with source width
[23:28:30] [PASSED] Fail: overflowing fb with source height
[23:28:30] ====== [PASSED] drm_test_framebuffer_check_src_coords ======
[23:28:30] [PASSED] drm_test_framebuffer_cleanup
[23:28:30] =============== drm_test_framebuffer_create  ===============
[23:28:30] [PASSED] ABGR8888 normal sizes
[23:28:30] [PASSED] ABGR8888 max sizes
[23:28:30] [PASSED] ABGR8888 pitch greater than min required
[23:28:30] [PASSED] ABGR8888 pitch less than min required
[23:28:30] [PASSED] ABGR8888 Invalid width
[23:28:30] [PASSED] ABGR8888 Invalid buffer handle
[23:28:30] [PASSED] No pixel format
[23:28:30] [PASSED] ABGR8888 Width 0
[23:28:30] [PASSED] ABGR8888 Height 0
[23:28:30] [PASSED] ABGR8888 Out of bound height * pitch combination
[23:28:30] [PASSED] ABGR8888 Large buffer offset
[23:28:30] [PASSED] ABGR8888 Buffer offset for inexistent plane
[23:28:30] [PASSED] ABGR8888 Invalid flag
[23:28:30] [PASSED] ABGR8888 Set DRM_MODE_FB_MODIFIERS without modifiers
[23:28:30] [PASSED] ABGR8888 Valid buffer modifier
[23:28:30] [PASSED] ABGR8888 Invalid buffer modifier(DRM_FORMAT_MOD_SAMSUNG_64_32_TILE)
[23:28:30] [PASSED] ABGR8888 Extra pitches without DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] ABGR8888 Extra pitches with DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] NV12 Normal sizes
[23:28:30] [PASSED] NV12 Max sizes
[23:28:30] [PASSED] NV12 Invalid pitch
[23:28:30] [PASSED] NV12 Invalid modifier/missing DRM_MODE_FB_MODIFIERS flag
[23:28:30] [PASSED] NV12 different  modifier per-plane
[23:28:30] [PASSED] NV12 with DRM_FORMAT_MOD_SAMSUNG_64_32_TILE
[23:28:30] [PASSED] NV12 Valid modifiers without DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] NV12 Modifier for inexistent plane
[23:28:30] [PASSED] NV12 Handle for inexistent plane
[23:28:30] [PASSED] NV12 Handle for inexistent plane without DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] YVU420 DRM_MODE_FB_MODIFIERS set without modifier
[23:28:30] [PASSED] YVU420 Normal sizes
[23:28:30] [PASSED] YVU420 Max sizes
[23:28:30] [PASSED] YVU420 Invalid pitch
[23:28:30] [PASSED] YVU420 Different pitches
[23:28:30] [PASSED] YVU420 Different buffer offsets/pitches
[23:28:30] [PASSED] YVU420 Modifier set just for plane 0, without DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] YVU420 Modifier set just for planes 0, 1, without DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] YVU420 Modifier set just for plane 0, 1, with DRM_MODE_FB_MODIFIERS
[23:28:30] [PASSED] YVU420 Valid modifier
[23:28:30] [PASSED] YVU420 Different modifiers per plane
[23:28:30] [PASSED] YVU420 Modifier for inexistent plane
[23:28:30] [PASSED] YUV420_10BIT Invalid modifier(DRM_FORMAT_MOD_LINEAR)
[23:28:30] [PASSED] X0L2 Normal sizes
[23:28:30] [PASSED] X0L2 Max sizes
[23:28:30] [PASSED] X0L2 Invalid pitch
[23:28:30] [PASSED] X0L2 Pitch greater than minimum required
[23:28:30] [PASSED] X0L2 Handle for inexistent plane
[23:28:30] [PASSED] X0L2 Offset for inexistent plane, without DRM_MODE_FB_MODIFIERS set
[23:28:30] [PASSED] X0L2 Modifier without DRM_MODE_FB_MODIFIERS set
[23:28:30] [PASSED] X0L2 Valid modifier
[23:28:30] [PASSED] X0L2 Modifier for inexistent plane
[23:28:30] =========== [PASSED] drm_test_framebuffer_create ===========
[23:28:30] [PASSED] drm_test_framebuffer_free
[23:28:30] [PASSED] drm_test_framebuffer_init
[23:28:30] [PASSED] drm_test_framebuffer_init_bad_format
[23:28:30] [PASSED] drm_test_framebuffer_init_dev_mismatch
[23:28:30] [PASSED] drm_test_framebuffer_lookup
[23:28:30] [PASSED] drm_test_framebuffer_lookup_inexistent
[23:28:30] [PASSED] drm_test_framebuffer_modifiers_not_supported
[23:28:30] ================= [PASSED] drm_framebuffer =================
[23:28:30] ================ drm_gem_shmem (8 subtests) ================
[23:28:30] [PASSED] drm_gem_shmem_test_obj_create
[23:28:30] [PASSED] drm_gem_shmem_test_obj_create_private
[23:28:30] [PASSED] drm_gem_shmem_test_pin_pages
[23:28:30] [PASSED] drm_gem_shmem_test_vmap
[23:28:30] [PASSED] drm_gem_shmem_test_get_pages_sgt
[23:28:30] [PASSED] drm_gem_shmem_test_get_sg_table
[23:28:30] [PASSED] drm_gem_shmem_test_madvise
[23:28:30] [PASSED] drm_gem_shmem_test_purge
[23:28:30] ================== [PASSED] drm_gem_shmem ==================
[23:28:30] === drm_atomic_helper_connector_hdmi_check (23 subtests) ===
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode_vic_1
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode_vic_1
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode_vic_1
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_changed
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_not_changed
[23:28:30] [PASSED] drm_test_check_disable_connector
[23:28:30] [PASSED] drm_test_check_hdmi_funcs_reject_rate
[23:28:30] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback
[23:28:30] [PASSED] drm_test_check_max_tmds_rate_format_fallback
[23:28:30] [PASSED] drm_test_check_output_bpc_crtc_mode_changed
[23:28:30] [PASSED] drm_test_check_output_bpc_crtc_mode_not_changed
[23:28:30] [PASSED] drm_test_check_output_bpc_dvi
[23:28:30] [PASSED] drm_test_check_output_bpc_format_vic_1
[23:28:30] [PASSED] drm_test_check_output_bpc_format_display_8bpc_only
[23:28:30] [PASSED] drm_test_check_output_bpc_format_display_rgb_only
[23:28:30] [PASSED] drm_test_check_output_bpc_format_driver_8bpc_only
[23:28:30] [PASSED] drm_test_check_output_bpc_format_driver_rgb_only
[23:28:30] [PASSED] drm_test_check_tmds_char_rate_rgb_8bpc
[23:28:30] [PASSED] drm_test_check_tmds_char_rate_rgb_10bpc
[23:28:30] [PASSED] drm_test_check_tmds_char_rate_rgb_12bpc
[23:28:30] ===== [PASSED] drm_atomic_helper_connector_hdmi_check ======
[23:28:30] === drm_atomic_helper_connector_hdmi_reset (6 subtests) ====
[23:28:30] [PASSED] drm_test_check_broadcast_rgb_value
[23:28:30] [PASSED] drm_test_check_bpc_8_value
[23:28:30] [PASSED] drm_test_check_bpc_10_value
[23:28:30] [PASSED] drm_test_check_bpc_12_value
[23:28:30] [PASSED] drm_test_check_format_value
[23:28:30] [PASSED] drm_test_check_tmds_char_value
[23:28:30] ===== [PASSED] drm_atomic_helper_connector_hdmi_reset ======
[23:28:30] = drm_atomic_helper_connector_hdmi_mode_valid (4 subtests) =
[23:28:30] [PASSED] drm_test_check_mode_valid
[23:28:30] [PASSED] drm_test_check_mode_valid_reject
[23:28:30] [PASSED] drm_test_check_mode_valid_reject_rate
[23:28:30] [PASSED] drm_test_check_mode_valid_reject_max_clock
[23:28:30] === [PASSED] drm_atomic_helper_connector_hdmi_mode_valid ===
[23:28:30] ================= drm_managed (2 subtests) =================
[23:28:30] [PASSED] drm_test_managed_release_action
[23:28:30] [PASSED] drm_test_managed_run_action
[23:28:30] =================== [PASSED] drm_managed ===================
[23:28:30] =================== drm_mm (6 subtests) ====================
[23:28:30] [PASSED] drm_test_mm_init
[23:28:30] [PASSED] drm_test_mm_debug
[23:28:30] [PASSED] drm_test_mm_align32
[23:28:30] [PASSED] drm_test_mm_align64
[23:28:30] [PASSED] drm_test_mm_lowest
[23:28:30] [PASSED] drm_test_mm_highest
[23:28:30] ===================== [PASSED] drm_mm ======================
[23:28:30] ============= drm_modes_analog_tv (5 subtests) =============
[23:28:30] [PASSED] drm_test_modes_analog_tv_mono_576i
[23:28:30] [PASSED] drm_test_modes_analog_tv_ntsc_480i
[23:28:30] [PASSED] drm_test_modes_analog_tv_ntsc_480i_inlined
[23:28:30] [PASSED] drm_test_modes_analog_tv_pal_576i
[23:28:30] [PASSED] drm_test_modes_analog_tv_pal_576i_inlined
[23:28:30] =============== [PASSED] drm_modes_analog_tv ===============
[23:28:30] ============== drm_plane_helper (2 subtests) ===============
[23:28:30] =============== drm_test_check_plane_state  ================
[23:28:30] [PASSED] clipping_simple
[23:28:30] [PASSED] clipping_rotate_reflect
[23:28:30] [PASSED] positioning_simple
[23:28:30] [PASSED] upscaling
[23:28:30] [PASSED] downscaling
[23:28:30] [PASSED] rounding1
[23:28:30] [PASSED] rounding2
[23:28:30] [PASSED] rounding3
[23:28:30] [PASSED] rounding4
[23:28:30] =========== [PASSED] drm_test_check_plane_state ============
[23:28:30] =========== drm_test_check_invalid_plane_state  ============
[23:28:30] [PASSED] positioning_invalid
[23:28:30] [PASSED] upscaling_invalid
[23:28:30] [PASSED] downscaling_invalid
[23:28:30] ======= [PASSED] drm_test_check_invalid_plane_state ========
[23:28:30] ================ [PASSED] drm_plane_helper =================
[23:28:30] ====== drm_connector_helper_tv_get_modes (1 subtest) =======
[23:28:30] ====== drm_test_connector_helper_tv_get_modes_check  =======
[23:28:30] [PASSED] None
[23:28:30] [PASSED] PAL
[23:28:30] [PASSED] NTSC
[23:28:30] [PASSED] Both, NTSC Default
[23:28:30] [PASSED] Both, PAL Default
[23:28:30] [PASSED] Both, NTSC Default, with PAL on command-line
[23:28:30] [PASSED] Both, PAL Default, with NTSC on command-line
[23:28:30] == [PASSED] drm_test_connector_helper_tv_get_modes_check ===
[23:28:30] ======== [PASSED] drm_connector_helper_tv_get_modes ========
[23:28:30] ================== drm_rect (9 subtests) ===================
[23:28:30] [PASSED] drm_test_rect_clip_scaled_div_by_zero
[23:28:30] [PASSED] drm_test_rect_clip_scaled_not_clipped
[23:28:30] [PASSED] drm_test_rect_clip_scaled_clipped
[23:28:30] [PASSED] drm_test_rect_clip_scaled_signed_vs_unsigned
[23:28:30] ================= drm_test_rect_intersect  =================
[23:28:30] [PASSED] top-left x bottom-right: 2x2+1+1 x 2x2+0+0
[23:28:30] [PASSED] top-right x bottom-left: 2x2+0+0 x 2x2+1-1
[23:28:30] [PASSED] bottom-left x top-right: 2x2+1-1 x 2x2+0+0
[23:28:30] [PASSED] bottom-right x top-left: 2x2+0+0 x 2x2+1+1
[23:28:30] [PASSED] right x left: 2x1+0+0 x 3x1+1+0
[23:28:30] [PASSED] left x right: 3x1+1+0 x 2x1+0+0
[23:28:30] [PASSED] up x bottom: 1x2+0+0 x 1x3+0-1
[23:28:30] [PASSED] bottom x up: 1x3+0-1 x 1x2+0+0
[23:28:30] [PASSED] touching corner: 1x1+0+0 x 2x2+1+1
[23:28:30] [PASSED] touching side: 1x1+0+0 x 1x1+1+0
[23:28:30] [PASSED] equal rects: 2x2+0+0 x 2x2+0+0
[23:28:30] [PASSED] inside another: 2x2+0+0 x 1x1+1+1
[23:28:30] [PASSED] far away: 1x1+0+0 x 1x1+3+6
[23:28:30] [PASSED] points intersecting: 0x0+5+10 x 0x0+5+10
[23:28:30] [PASSED] points not intersecting: 0x0+0+0 x 0x0+5+10
[23:28:30] ============= [PASSED] drm_test_rect_intersect =============
[23:28:30] ================ drm_test_rect_calc_hscale  ================
[23:28:30] [PASSED] normal use
[23:28:30] [PASSED] out of max range
[23:28:30] [PASSED] out of min range
[23:28:30] [PASSED] zero dst
[23:28:30] [PASSED] negative src
[23:28:30] [PASSED] negative dst
[23:28:30] ============ [PASSED] drm_test_rect_calc_hscale ============
[23:28:30] ================ drm_test_rect_calc_vscale  ================
[23:28:30] [PASSED] normal use
[23:28:30] [PASSED] out of max range
[23:28:30] [PASSED] out of min range
[23:28:30] [PASSED] zero dst
[23:28:30] [PASSED] negative src
[23:28:30] [PASSED] negative dst
[23:28:30] ============ [PASSED] drm_test_rect_calc_vscale ============
[23:28:30] ================== drm_test_rect_rotate  ===================
[23:28:30] [PASSED] reflect-x
[23:28:30] [PASSED] reflect-y
[23:28:30] [PASSED] rotate-0
[23:28:30] [PASSED] rotate-90
[23:28:30] [PASSED] rotate-180
[23:28:30] [PASSED] rotate-270
stty: 'standard input': Inappropriate ioctl for device
[23:28:30] ============== [PASSED] drm_test_rect_rotate ===============
[23:28:30] ================ drm_test_rect_rotate_inv  =================
[23:28:30] [PASSED] reflect-x
[23:28:30] [PASSED] reflect-y
[23:28:30] [PASSED] rotate-0
[23:28:30] [PASSED] rotate-90
[23:28:30] [PASSED] rotate-180
[23:28:30] [PASSED] rotate-270
[23:28:30] ============ [PASSED] drm_test_rect_rotate_inv =============
[23:28:30] ==================== [PASSED] drm_rect =====================
[23:28:30] ============================================================
[23:28:30] Testing complete. Ran 598 tests: passed: 598
[23:28:30] Elapsed time: 22.878s total, 1.653s configuring, 21.004s building, 0.173s running

+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/ttm/tests/.kunitconfig
[23:28:30] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[23:28:31] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make all compile_commands.json ARCH=um O=.kunit --jobs=48
[23:28:39] Starting KUnit Kernel (1/1)...
[23:28:39] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[23:28:39] ================= ttm_device (5 subtests) ==================
[23:28:39] [PASSED] ttm_device_init_basic
[23:28:39] [PASSED] ttm_device_init_multiple
[23:28:39] [PASSED] ttm_device_fini_basic
[23:28:39] [PASSED] ttm_device_init_no_vma_man
[23:28:39] ================== ttm_device_init_pools  ==================
[23:28:39] [PASSED] No DMA allocations, no DMA32 required
[23:28:39] [PASSED] DMA allocations, DMA32 required
[23:28:39] [PASSED] No DMA allocations, DMA32 required
[23:28:39] [PASSED] DMA allocations, no DMA32 required
[23:28:39] ============== [PASSED] ttm_device_init_pools ==============
[23:28:39] =================== [PASSED] ttm_device ====================
[23:28:39] ================== ttm_pool (8 subtests) ===================
[23:28:39] ================== ttm_pool_alloc_basic  ===================
[23:28:39] [PASSED] One page
[23:28:39] [PASSED] More than one page
[23:28:39] [PASSED] Above the allocation limit
[23:28:39] [PASSED] One page, with coherent DMA mappings enabled
[23:28:39] [PASSED] Above the allocation limit, with coherent DMA mappings enabled
[23:28:39] ============== [PASSED] ttm_pool_alloc_basic ===============
[23:28:39] ============== ttm_pool_alloc_basic_dma_addr  ==============
[23:28:39] [PASSED] One page
[23:28:39] [PASSED] More than one page
[23:28:39] [PASSED] Above the allocation limit
[23:28:39] [PASSED] One page, with coherent DMA mappings enabled
[23:28:39] [PASSED] Above the allocation limit, with coherent DMA mappings enabled
[23:28:39] ========== [PASSED] ttm_pool_alloc_basic_dma_addr ==========
[23:28:39] [PASSED] ttm_pool_alloc_order_caching_match
[23:28:39] [PASSED] ttm_pool_alloc_caching_mismatch
[23:28:39] [PASSED] ttm_pool_alloc_order_mismatch
[23:28:39] [PASSED] ttm_pool_free_dma_alloc
[23:28:39] [PASSED] ttm_pool_free_no_dma_alloc
[23:28:39] [PASSED] ttm_pool_fini_basic
[23:28:39] ==================== [PASSED] ttm_pool =====================
[23:28:39] ================ ttm_resource (8 subtests) =================
[23:28:39] ================= ttm_resource_init_basic  =================
[23:28:39] [PASSED] Init resource in TTM_PL_SYSTEM
[23:28:39] [PASSED] Init resource in TTM_PL_VRAM
[23:28:39] [PASSED] Init resource in a private placement
[23:28:39] [PASSED] Init resource in TTM_PL_SYSTEM, set placement flags
[23:28:39] ============= [PASSED] ttm_resource_init_basic =============
[23:28:39] [PASSED] ttm_resource_init_pinned
[23:28:39] [PASSED] ttm_resource_fini_basic
[23:28:39] [PASSED] ttm_resource_manager_init_basic
[23:28:39] [PASSED] ttm_resource_manager_usage_basic
[23:28:39] [PASSED] ttm_resource_manager_set_used_basic
[23:28:39] [PASSED] ttm_sys_man_alloc_basic
[23:28:39] [PASSED] ttm_sys_man_free_basic
[23:28:39] ================== [PASSED] ttm_resource ===================
[23:28:39] =================== ttm_tt (15 subtests) ===================
[23:28:39] ==================== ttm_tt_init_basic  ====================
[23:28:39] [PASSED] Page-aligned size
[23:28:39] [PASSED] Extra pages requested
[23:28:39] ================ [PASSED] ttm_tt_init_basic ================
[23:28:39] [PASSED] ttm_tt_init_misaligned
[23:28:39] [PASSED] ttm_tt_fini_basic
[23:28:39] [PASSED] ttm_tt_fini_sg
[23:28:39] [PASSED] ttm_tt_fini_shmem
[23:28:39] [PASSED] ttm_tt_create_basic
[23:28:39] [PASSED] ttm_tt_create_invalid_bo_type
[23:28:39] [PASSED] ttm_tt_create_ttm_exists
[23:28:39] [PASSED] ttm_tt_create_failed
[23:28:39] [PASSED] ttm_tt_destroy_basic
[23:28:39] [PASSED] ttm_tt_populate_null_ttm
[23:28:39] [PASSED] ttm_tt_populate_populated_ttm
[23:28:39] [PASSED] ttm_tt_unpopulate_basic
[23:28:39] [PASSED] ttm_tt_unpopulate_empty_ttm
[23:28:39] [PASSED] ttm_tt_swapin_basic
[23:28:39] ===================== [PASSED] ttm_tt ======================
[23:28:39] =================== ttm_bo (14 subtests) ===================
[23:28:39] =========== ttm_bo_reserve_optimistic_no_ticket  ===========
[23:28:39] [PASSED] Cannot be interrupted and sleeps
[23:28:39] [PASSED] Cannot be interrupted, locks straight away
[23:28:39] [PASSED] Can be interrupted, sleeps
[23:28:39] ======= [PASSED] ttm_bo_reserve_optimistic_no_ticket =======
[23:28:39] [PASSED] ttm_bo_reserve_locked_no_sleep
[23:28:39] [PASSED] ttm_bo_reserve_no_wait_ticket
[23:28:39] [PASSED] ttm_bo_reserve_double_resv
[23:28:39] [PASSED] ttm_bo_reserve_interrupted
[23:28:39] [PASSED] ttm_bo_reserve_deadlock
[23:28:39] [PASSED] ttm_bo_unreserve_basic
[23:28:39] [PASSED] ttm_bo_unreserve_pinned
[23:28:39] [PASSED] ttm_bo_unreserve_bulk
[23:28:39] [PASSED] ttm_bo_put_basic
[23:28:39] [PASSED] ttm_bo_put_shared_resv
[23:28:39] [PASSED] ttm_bo_pin_basic
[23:28:39] [PASSED] ttm_bo_pin_unpin_resource
[23:28:39] [PASSED] ttm_bo_multiple_pin_one_unpin
[23:28:39] ===================== [PASSED] ttm_bo ======================
[23:28:39] ============== ttm_bo_validate (22 subtests) ===============
[23:28:39] ============== ttm_bo_init_reserved_sys_man  ===============
[23:28:39] [PASSED] Buffer object for userspace
[23:28:39] [PASSED] Kernel buffer object
[23:28:39] [PASSED] Shared buffer object
[23:28:39] ========== [PASSED] ttm_bo_init_reserved_sys_man ===========
[23:28:39] ============== ttm_bo_init_reserved_mock_man  ==============
[23:28:39] [PASSED] Buffer object for userspace
[23:28:39] [PASSED] Kernel buffer object
[23:28:39] [PASSED] Shared buffer object
[23:28:39] ========== [PASSED] ttm_bo_init_reserved_mock_man ==========
[23:28:39] [PASSED] ttm_bo_init_reserved_resv
[23:28:39] ================== ttm_bo_validate_basic  ==================
[23:28:39] [PASSED] Buffer object for userspace
[23:28:39] [PASSED] Kernel buffer object
[23:28:39] [PASSED] Shared buffer object
[23:28:39] ============== [PASSED] ttm_bo_validate_basic ==============
[23:28:39] [PASSED] ttm_bo_validate_invalid_placement
[23:28:39] ============= ttm_bo_validate_same_placement  ==============
[23:28:39] [PASSED] System manager
[23:28:39] [PASSED] VRAM manager
[23:28:39] ========= [PASSED] ttm_bo_validate_same_placement ==========
[23:28:39] [PASSED] ttm_bo_validate_failed_alloc
[23:28:39] [PASSED] ttm_bo_validate_pinned
[23:28:39] [PASSED] ttm_bo_validate_busy_placement
[23:28:39] ================ ttm_bo_validate_multihop  =================
[23:28:39] [PASSED] Buffer object for userspace
[23:28:39] [PASSED] Kernel buffer object
[23:28:39] [PASSED] Shared buffer object
[23:28:39] ============ [PASSED] ttm_bo_validate_multihop =============
[23:28:39] ========== ttm_bo_validate_no_placement_signaled  ==========
[23:28:39] [PASSED] Buffer object in system domain, no page vector
[23:28:39] [PASSED] Buffer object in system domain with an existing page vector
[23:28:39] ====== [PASSED] ttm_bo_validate_no_placement_signaled ======
[23:28:39] ======== ttm_bo_validate_no_placement_not_signaled  ========
[23:28:39] [PASSED] Buffer object for userspace
[23:28:39] [PASSED] Kernel buffer object
[23:28:39] [PASSED] Shared buffer object
[23:28:39] ==== [PASSED] ttm_bo_validate_no_placement_not_signaled ====
[23:28:39] [PASSED] ttm_bo_validate_move_fence_signaled
[23:28:39] ========= ttm_bo_validate_move_fence_not_signaled  =========
[23:28:39] [PASSED] Waits for GPU
[23:28:39] [PASSED] Tries to lock straight away
[23:28:39] ===== [PASSED] ttm_bo_validate_move_fence_not_signaled =====
[23:28:39] [PASSED] ttm_bo_validate_swapout
[23:28:39] [PASSED] ttm_bo_validate_happy_evict
[23:28:39] [PASSED] ttm_bo_validate_all_pinned_evict
[23:28:39] [PASSED] ttm_bo_validate_allowed_only_evict
[23:28:39] [PASSED] ttm_bo_validate_deleted_evict
[23:28:39] [PASSED] ttm_bo_validate_busy_domain_evict
[23:28:39] [PASSED] ttm_bo_validate_evict_gutting
[23:28:39] [PASSED] ttm_bo_validate_recrusive_evict
stty: 'standard input': Inappropriate ioctl for device
[23:28:39] ================= [PASSED] ttm_bo_validate =================
[23:28:39] ============================================================
[23:28:39] Testing complete. Ran 102 tests: passed: 102
[23:28:40] Elapsed time: 9.851s total, 1.632s configuring, 7.603s building, 0.536s running

+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✓ CI.Build: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (4 preceding siblings ...)
  2025-02-04 23:28 ` ✓ CI.KUnit: success " Patchwork
@ 2025-02-04 23:45 ` Patchwork
  2025-02-04 23:47 ` ✓ CI.Hooks: " Patchwork
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:45 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/events/amd/
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/events/amd/amd-uncore.ko
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/events/rapl.ko
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/kvm/
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/kvm/kvm.ko
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/kvm/kvm-intel.ko
lib/modules/6.14.0-rc1-xe+/kernel/arch/x86/kvm/kvm-amd.ko
lib/modules/6.14.0-rc1-xe+/kernel/kernel/
lib/modules/6.14.0-rc1-xe+/kernel/kernel/kheaders.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/
lib/modules/6.14.0-rc1-xe+/kernel/crypto/ecrdsa_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/xcbc.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/serpent_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/aria_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/crypto_simd.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/adiantum.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/tcrypt.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/crypto_engine.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/zstd.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/asymmetric_keys/
lib/modules/6.14.0-rc1-xe+/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/asymmetric_keys/pkcs8_key_parser.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/des_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/xctr.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/authenc.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/sm4_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/camellia_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/sm3.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/pcrypt.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/aegis128.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/af_alg.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/algif_aead.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/cmac.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/sm3_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/aes_ti.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/chacha_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/poly1305_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/nhpoly1305.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/crc32_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/essiv.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/ccm.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/wp512.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/streebog_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/authencesn.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/echainiv.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/lrw.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/cryptd.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/crypto_user.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/algif_hash.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/polyval-generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/hctr2.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/842.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/pcbc.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/ansi_cprng.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/cast6_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/twofish_common.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/twofish_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/lz4hc.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/blowfish_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/md4.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/chacha20poly1305.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/curve25519-generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/lz4.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/rmd160.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/algif_skcipher.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/cast5_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/fcrypt.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/ecdsa_generic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/sm4.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/cast_common.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/blowfish_common.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/michael_mic.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/async_xor.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/async_tx.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/async_memcpy.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/async_pq.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/async_tx/async_raid6_recov.ko
lib/modules/6.14.0-rc1-xe+/kernel/crypto/algif_rng.ko
lib/modules/6.14.0-rc1-xe+/kernel/block/
lib/modules/6.14.0-rc1-xe+/kernel/block/bfq.ko
lib/modules/6.14.0-rc1-xe+/kernel/block/kyber-iosched.ko
lib/modules/6.14.0-rc1-xe+/build
lib/modules/6.14.0-rc1-xe+/modules.alias.bin
lib/modules/6.14.0-rc1-xe+/modules.builtin
lib/modules/6.14.0-rc1-xe+/modules.softdep
lib/modules/6.14.0-rc1-xe+/modules.alias
lib/modules/6.14.0-rc1-xe+/modules.order
lib/modules/6.14.0-rc1-xe+/modules.symbols
lib/modules/6.14.0-rc1-xe+/modules.dep.bin
+ mv kernel-nodebug.tar.gz ..
+ cd ..
+ rm -rf archive
++ date +%s
+ echo -e '\e[0Ksection_end:1738712704:package_x86_64_nodebug\r\e[0K'
+ sync
^[[0Ksection_end:1738712704:package_x86_64_nodebug
^[[0K
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✓ CI.Hooks: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (5 preceding siblings ...)
  2025-02-04 23:45 ` ✓ CI.Build: " Patchwork
@ 2025-02-04 23:47 ` Patchwork
  2025-02-04 23:48 ` ✓ CI.checksparse: " Patchwork
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:47 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

run-parts: executing /workspace/ci/hooks/00-showenv
+ export
+ grep -Ei '(^|\W)CI_'
declare -x CI_KERNEL_BUILD_DIR="/workspace/kernel/build64-default"
declare -x CI_KERNEL_SRC_DIR="/workspace/kernel"
declare -x CI_TOOLS_SRC_DIR="/workspace/ci"
declare -x CI_WORKSPACE_DIR="/workspace"
run-parts: executing /workspace/ci/hooks/10-build-W1
+ SRC_DIR=/workspace/kernel
+ RESTORE_DISPLAY_CONFIG=0
+ '[' -n /workspace/kernel/build64-default ']'
+ BUILD_DIR=/workspace/kernel/build64-default
+ cd /workspace/kernel
++ nproc
+ make -j48 O=/workspace/kernel/build64-default modules_prepare
make[1]: Entering directory '/workspace/kernel/build64-default'
  GEN     Makefile
mkdir -p /workspace/kernel/build64-default/tools/objtool && make O=/workspace/kernel/build64-default subdir=tools/objtool --no-print-directory -C objtool 
  CALL    ../scripts/checksyscalls.sh
  INSTALL libsubcmd_headers
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/exec-cmd.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/help.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/pager.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/parse-options.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/run-command.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/sigchain.o
  CC      /workspace/kernel/build64-default/tools/objtool/libsubcmd/subcmd-config.o
  LD      /workspace/kernel/build64-default/tools/objtool/libsubcmd/libsubcmd-in.o
  AR      /workspace/kernel/build64-default/tools/objtool/libsubcmd/libsubcmd.a
  CC      /workspace/kernel/build64-default/tools/objtool/weak.o
  CC      /workspace/kernel/build64-default/tools/objtool/check.o
  CC      /workspace/kernel/build64-default/tools/objtool/special.o
  CC      /workspace/kernel/build64-default/tools/objtool/builtin-check.o
  CC      /workspace/kernel/build64-default/tools/objtool/elf.o
  CC      /workspace/kernel/build64-default/tools/objtool/objtool.o
  CC      /workspace/kernel/build64-default/tools/objtool/orc_gen.o
  CC      /workspace/kernel/build64-default/tools/objtool/orc_dump.o
  CC      /workspace/kernel/build64-default/tools/objtool/arch/x86/special.o
  CC      /workspace/kernel/build64-default/tools/objtool/libstring.o
  CC      /workspace/kernel/build64-default/tools/objtool/libctype.o
  CC      /workspace/kernel/build64-default/tools/objtool/arch/x86/decode.o
  CC      /workspace/kernel/build64-default/tools/objtool/arch/x86/orc.o
  CC      /workspace/kernel/build64-default/tools/objtool/str_error_r.o
  CC      /workspace/kernel/build64-default/tools/objtool/librbtree.o
  LD      /workspace/kernel/build64-default/tools/objtool/arch/x86/objtool-in.o
  LD      /workspace/kernel/build64-default/tools/objtool/objtool-in.o
  LINK    /workspace/kernel/build64-default/tools/objtool/objtool
make[1]: Leaving directory '/workspace/kernel/build64-default'
++ nproc
+ make -j48 O=/workspace/kernel/build64-default W=1 drivers/gpu/drm/xe
make[1]: Entering directory '/workspace/kernel/build64-default'
make[2]: Nothing to be done for 'drivers/gpu/drm/xe'.
make[1]: Leaving directory '/workspace/kernel/build64-default'
run-parts: executing /workspace/ci/hooks/11-build-32b
+++ realpath /workspace/ci/hooks/11-build-32b
++ dirname /workspace/ci/hooks/11-build-32b
+ THIS_SCRIPT_DIR=/workspace/ci/hooks
+ SRC_DIR=/workspace/kernel
+ TOOLS_SRC_DIR=/workspace/ci
+ '[' -n /workspace/kernel/build64-default ']'
+ BUILD_DIR=/workspace/kernel/build64-default
+ BUILD_DIR=/workspace/kernel/build64-default/build32
+ cd /workspace/kernel
+ mkdir -p /workspace/kernel/build64-default/build32
++ nproc
+ make -j48 ARCH=i386 O=/workspace/kernel/build64-default/build32 defconfig
make[1]: Entering directory '/workspace/kernel/build64-default/build32'
  GEN     Makefile
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  YACC    scripts/kconfig/parser.tab.[ch]
  LEX     scripts/kconfig/lexer.lex.c
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'i386_defconfig'
#
# configuration written to .config
#
make[1]: Leaving directory '/workspace/kernel/build64-default/build32'
+ cd /workspace/kernel/build64-default/build32
+ /workspace/kernel/scripts/kconfig/merge_config.sh .config /workspace/ci/kernel/fragments/10-xe.fragment
Using .config as base
Merging /workspace/ci/kernel/fragments/10-xe.fragment
Value of CONFIG_DRM_XE is redefined by fragment /workspace/ci/kernel/fragments/10-xe.fragment:
Previous value: # CONFIG_DRM_XE is not set
New value: CONFIG_DRM_XE=m

  GEN     Makefile

WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
  Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
  Selected by [m]:
  - DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m] && HAS_IOPORT [=y]
#
# configuration written to .config
#
Value requested for CONFIG_HAVE_UID16 not in final .config
Requested value:  CONFIG_HAVE_UID16=y
Actual value:     

Value requested for CONFIG_UID16 not in final .config
Requested value:  CONFIG_UID16=y
Actual value:     

Value requested for CONFIG_X86_32 not in final .config
Requested value:  CONFIG_X86_32=y
Actual value:     

Value requested for CONFIG_OUTPUT_FORMAT not in final .config
Requested value:  CONFIG_OUTPUT_FORMAT="elf32-i386"
Actual value:     CONFIG_OUTPUT_FORMAT="elf64-x86-64"

Value requested for CONFIG_ARCH_MMAP_RND_BITS_MIN not in final .config
Requested value:  CONFIG_ARCH_MMAP_RND_BITS_MIN=8
Actual value:     CONFIG_ARCH_MMAP_RND_BITS_MIN=28

Value requested for CONFIG_ARCH_MMAP_RND_BITS_MAX not in final .config
Requested value:  CONFIG_ARCH_MMAP_RND_BITS_MAX=16
Actual value:     CONFIG_ARCH_MMAP_RND_BITS_MAX=32

Value requested for CONFIG_PGTABLE_LEVELS not in final .config
Requested value:  CONFIG_PGTABLE_LEVELS=2
Actual value:     CONFIG_PGTABLE_LEVELS=5

Value requested for CONFIG_X86_BIGSMP not in final .config
Requested value:  # CONFIG_X86_BIGSMP is not set
Actual value:     

Value requested for CONFIG_X86_INTEL_QUARK not in final .config
Requested value:  # CONFIG_X86_INTEL_QUARK is not set
Actual value:     

Value requested for CONFIG_X86_RDC321X not in final .config
Requested value:  # CONFIG_X86_RDC321X is not set
Actual value:     

Value requested for CONFIG_X86_32_NON_STANDARD not in final .config
Requested value:  # CONFIG_X86_32_NON_STANDARD is not set
Actual value:     

Value requested for CONFIG_X86_32_IRIS not in final .config
Requested value:  # CONFIG_X86_32_IRIS is not set
Actual value:     

Value requested for CONFIG_M486SX not in final .config
Requested value:  # CONFIG_M486SX is not set
Actual value:     

Value requested for CONFIG_M486 not in final .config
Requested value:  # CONFIG_M486 is not set
Actual value:     

Value requested for CONFIG_M586 not in final .config
Requested value:  # CONFIG_M586 is not set
Actual value:     

Value requested for CONFIG_M586TSC not in final .config
Requested value:  # CONFIG_M586TSC is not set
Actual value:     

Value requested for CONFIG_M586MMX not in final .config
Requested value:  # CONFIG_M586MMX is not set
Actual value:     

Value requested for CONFIG_M686 not in final .config
Requested value:  CONFIG_M686=y
Actual value:     

Value requested for CONFIG_MPENTIUMII not in final .config
Requested value:  # CONFIG_MPENTIUMII is not set
Actual value:     

Value requested for CONFIG_MPENTIUMIII not in final .config
Requested value:  # CONFIG_MPENTIUMIII is not set
Actual value:     

Value requested for CONFIG_MPENTIUMM not in final .config
Requested value:  # CONFIG_MPENTIUMM is not set
Actual value:     

Value requested for CONFIG_MPENTIUM4 not in final .config
Requested value:  # CONFIG_MPENTIUM4 is not set
Actual value:     

Value requested for CONFIG_MK6 not in final .config
Requested value:  # CONFIG_MK6 is not set
Actual value:     

Value requested for CONFIG_MK7 not in final .config
Requested value:  # CONFIG_MK7 is not set
Actual value:     

Value requested for CONFIG_MCRUSOE not in final .config
Requested value:  # CONFIG_MCRUSOE is not set
Actual value:     

Value requested for CONFIG_MEFFICEON not in final .config
Requested value:  # CONFIG_MEFFICEON is not set
Actual value:     

Value requested for CONFIG_MWINCHIPC6 not in final .config
Requested value:  # CONFIG_MWINCHIPC6 is not set
Actual value:     

Value requested for CONFIG_MWINCHIP3D not in final .config
Requested value:  # CONFIG_MWINCHIP3D is not set
Actual value:     

Value requested for CONFIG_MELAN not in final .config
Requested value:  # CONFIG_MELAN is not set
Actual value:     

Value requested for CONFIG_MGEODEGX1 not in final .config
Requested value:  # CONFIG_MGEODEGX1 is not set
Actual value:     

Value requested for CONFIG_MGEODE_LX not in final .config
Requested value:  # CONFIG_MGEODE_LX is not set
Actual value:     

Value requested for CONFIG_MCYRIXIII not in final .config
Requested value:  # CONFIG_MCYRIXIII is not set
Actual value:     

Value requested for CONFIG_MVIAC3_2 not in final .config
Requested value:  # CONFIG_MVIAC3_2 is not set
Actual value:     

Value requested for CONFIG_MVIAC7 not in final .config
Requested value:  # CONFIG_MVIAC7 is not set
Actual value:     

Value requested for CONFIG_X86_GENERIC not in final .config
Requested value:  # CONFIG_X86_GENERIC is not set
Actual value:     

Value requested for CONFIG_X86_INTERNODE_CACHE_SHIFT not in final .config
Requested value:  CONFIG_X86_INTERNODE_CACHE_SHIFT=5
Actual value:     CONFIG_X86_INTERNODE_CACHE_SHIFT=6

Value requested for CONFIG_X86_L1_CACHE_SHIFT not in final .config
Requested value:  CONFIG_X86_L1_CACHE_SHIFT=5
Actual value:     CONFIG_X86_L1_CACHE_SHIFT=6

Value requested for CONFIG_X86_USE_PPRO_CHECKSUM not in final .config
Requested value:  CONFIG_X86_USE_PPRO_CHECKSUM=y
Actual value:     

Value requested for CONFIG_X86_MINIMUM_CPU_FAMILY not in final .config
Requested value:  CONFIG_X86_MINIMUM_CPU_FAMILY=6
Actual value:     CONFIG_X86_MINIMUM_CPU_FAMILY=64

Value requested for CONFIG_CPU_SUP_TRANSMETA_32 not in final .config
Requested value:  CONFIG_CPU_SUP_TRANSMETA_32=y
Actual value:     

Value requested for CONFIG_CPU_SUP_VORTEX_32 not in final .config
Requested value:  CONFIG_CPU_SUP_VORTEX_32=y
Actual value:     

Value requested for CONFIG_HPET_TIMER not in final .config
Requested value:  # CONFIG_HPET_TIMER is not set
Actual value:     CONFIG_HPET_TIMER=y

Value requested for CONFIG_NR_CPUS_RANGE_END not in final .config
Requested value:  CONFIG_NR_CPUS_RANGE_END=8
Actual value:     CONFIG_NR_CPUS_RANGE_END=512

Value requested for CONFIG_NR_CPUS_DEFAULT not in final .config
Requested value:  CONFIG_NR_CPUS_DEFAULT=8
Actual value:     CONFIG_NR_CPUS_DEFAULT=64

Value requested for CONFIG_X86_ANCIENT_MCE not in final .config
Requested value:  # CONFIG_X86_ANCIENT_MCE is not set
Actual value:     

Value requested for CONFIG_X86_LEGACY_VM86 not in final .config
Requested value:  # CONFIG_X86_LEGACY_VM86 is not set
Actual value:     

Value requested for CONFIG_X86_ESPFIX32 not in final .config
Requested value:  CONFIG_X86_ESPFIX32=y
Actual value:     

Value requested for CONFIG_TOSHIBA not in final .config
Requested value:  # CONFIG_TOSHIBA is not set
Actual value:     

Value requested for CONFIG_X86_REBOOTFIXUPS not in final .config
Requested value:  # CONFIG_X86_REBOOTFIXUPS is not set
Actual value:     

Value requested for CONFIG_MICROCODE_INITRD32 not in final .config
Requested value:  CONFIG_MICROCODE_INITRD32=y
Actual value:     

Value requested for CONFIG_NOHIGHMEM not in final .config
Requested value:  # CONFIG_NOHIGHMEM is not set
Actual value:     

Value requested for CONFIG_HIGHMEM4G not in final .config
Requested value:  CONFIG_HIGHMEM4G=y
Actual value:     

Value requested for CONFIG_HIGHMEM64G not in final .config
Requested value:  # CONFIG_HIGHMEM64G is not set
Actual value:     

Value requested for CONFIG_VMSPLIT_3G not in final .config
Requested value:  CONFIG_VMSPLIT_3G=y
Actual value:     

Value requested for CONFIG_VMSPLIT_3G_OPT not in final .config
Requested value:  # CONFIG_VMSPLIT_3G_OPT is not set
Actual value:     

Value requested for CONFIG_VMSPLIT_2G not in final .config
Requested value:  # CONFIG_VMSPLIT_2G is not set
Actual value:     

Value requested for CONFIG_VMSPLIT_2G_OPT not in final .config
Requested value:  # CONFIG_VMSPLIT_2G_OPT is not set
Actual value:     

Value requested for CONFIG_VMSPLIT_1G not in final .config
Requested value:  # CONFIG_VMSPLIT_1G is not set
Actual value:     

Value requested for CONFIG_PAGE_OFFSET not in final .config
Requested value:  CONFIG_PAGE_OFFSET=0xC0000000
Actual value:     

Value requested for CONFIG_HIGHMEM not in final .config
Requested value:  CONFIG_HIGHMEM=y
Actual value:     

Value requested for CONFIG_X86_PAE not in final .config
Requested value:  # CONFIG_X86_PAE is not set
Actual value:     

Value requested for CONFIG_ARCH_FLATMEM_ENABLE not in final .config
Requested value:  CONFIG_ARCH_FLATMEM_ENABLE=y
Actual value:     

Value requested for CONFIG_ARCH_SELECT_MEMORY_MODEL not in final .config
Requested value:  CONFIG_ARCH_SELECT_MEMORY_MODEL=y
Actual value:     

Value requested for CONFIG_ILLEGAL_POINTER_VALUE not in final .config
Requested value:  CONFIG_ILLEGAL_POINTER_VALUE=0
Actual value:     CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000

Value requested for CONFIG_HIGHPTE not in final .config
Requested value:  # CONFIG_HIGHPTE is not set
Actual value:     

Value requested for CONFIG_COMPAT_VDSO not in final .config
Requested value:  # CONFIG_COMPAT_VDSO is not set
Actual value:     

Value requested for CONFIG_FUNCTION_PADDING_CFI not in final .config
Requested value:  CONFIG_FUNCTION_PADDING_CFI=0
Actual value:     CONFIG_FUNCTION_PADDING_CFI=11

Value requested for CONFIG_FUNCTION_PADDING_BYTES not in final .config
Requested value:  CONFIG_FUNCTION_PADDING_BYTES=4
Actual value:     CONFIG_FUNCTION_PADDING_BYTES=16

Value requested for CONFIG_APM not in final .config
Requested value:  # CONFIG_APM is not set
Actual value:     

Value requested for CONFIG_X86_POWERNOW_K6 not in final .config
Requested value:  # CONFIG_X86_POWERNOW_K6 is not set
Actual value:     

Value requested for CONFIG_X86_POWERNOW_K7 not in final .config
Requested value:  # CONFIG_X86_POWERNOW_K7 is not set
Actual value:     

Value requested for CONFIG_X86_GX_SUSPMOD not in final .config
Requested value:  # CONFIG_X86_GX_SUSPMOD is not set
Actual value:     

Value requested for CONFIG_X86_SPEEDSTEP_ICH not in final .config
Requested value:  # CONFIG_X86_SPEEDSTEP_ICH is not set
Actual value:     

Value requested for CONFIG_X86_SPEEDSTEP_SMI not in final .config
Requested value:  # CONFIG_X86_SPEEDSTEP_SMI is not set
Actual value:     

Value requested for CONFIG_X86_CPUFREQ_NFORCE2 not in final .config
Requested value:  # CONFIG_X86_CPUFREQ_NFORCE2 is not set
Actual value:     

Value requested for CONFIG_X86_LONGRUN not in final .config
Requested value:  # CONFIG_X86_LONGRUN is not set
Actual value:     

Value requested for CONFIG_X86_LONGHAUL not in final .config
Requested value:  # CONFIG_X86_LONGHAUL is not set
Actual value:     

Value requested for CONFIG_X86_E_POWERSAVER not in final .config
Requested value:  # CONFIG_X86_E_POWERSAVER is not set
Actual value:     

Value requested for CONFIG_PCI_GOBIOS not in final .config
Requested value:  # CONFIG_PCI_GOBIOS is not set
Actual value:     

Value requested for CONFIG_PCI_GOMMCONFIG not in final .config
Requested value:  # CONFIG_PCI_GOMMCONFIG is not set
Actual value:     

Value requested for CONFIG_PCI_GODIRECT not in final .config
Requested value:  # CONFIG_PCI_GODIRECT is not set
Actual value:     

Value requested for CONFIG_PCI_GOANY not in final .config
Requested value:  CONFIG_PCI_GOANY=y
Actual value:     

Value requested for CONFIG_PCI_BIOS not in final .config
Requested value:  CONFIG_PCI_BIOS=y
Actual value:     

Value requested for CONFIG_ISA not in final .config
Requested value:  # CONFIG_ISA is not set
Actual value:     

Value requested for CONFIG_SCx200 not in final .config
Requested value:  # CONFIG_SCx200 is not set
Actual value:     

Value requested for CONFIG_OLPC not in final .config
Requested value:  # CONFIG_OLPC is not set
Actual value:     

Value requested for CONFIG_ALIX not in final .config
Requested value:  # CONFIG_ALIX is not set
Actual value:     

Value requested for CONFIG_NET5501 not in final .config
Requested value:  # CONFIG_NET5501 is not set
Actual value:     

Value requested for CONFIG_GEOS not in final .config
Requested value:  # CONFIG_GEOS is not set
Actual value:     

Value requested for CONFIG_COMPAT_32 not in final .config
Requested value:  CONFIG_COMPAT_32=y
Actual value:     

Value requested for CONFIG_HAVE_ATOMIC_IOMAP not in final .config
Requested value:  CONFIG_HAVE_ATOMIC_IOMAP=y
Actual value:     

Value requested for CONFIG_ARCH_32BIT_OFF_T not in final .config
Requested value:  CONFIG_ARCH_32BIT_OFF_T=y
Actual value:     

Value requested for CONFIG_ARCH_WANT_IPC_PARSE_VERSION not in final .config
Requested value:  CONFIG_ARCH_WANT_IPC_PARSE_VERSION=y
Actual value:     

Value requested for CONFIG_MODULES_USE_ELF_REL not in final .config
Requested value:  CONFIG_MODULES_USE_ELF_REL=y
Actual value:     

Value requested for CONFIG_ARCH_MMAP_RND_BITS not in final .config
Requested value:  CONFIG_ARCH_MMAP_RND_BITS=8
Actual value:     CONFIG_ARCH_MMAP_RND_BITS=28

Value requested for CONFIG_CLONE_BACKWARDS not in final .config
Requested value:  CONFIG_CLONE_BACKWARDS=y
Actual value:     

Value requested for CONFIG_OLD_SIGSUSPEND3 not in final .config
Requested value:  CONFIG_OLD_SIGSUSPEND3=y
Actual value:     

Value requested for CONFIG_OLD_SIGACTION not in final .config
Requested value:  CONFIG_OLD_SIGACTION=y
Actual value:     

Value requested for CONFIG_ARCH_SPLIT_ARG64 not in final .config
Requested value:  CONFIG_ARCH_SPLIT_ARG64=y
Actual value:     

Value requested for CONFIG_FUNCTION_ALIGNMENT not in final .config
Requested value:  CONFIG_FUNCTION_ALIGNMENT=4
Actual value:     CONFIG_FUNCTION_ALIGNMENT=16

Value requested for CONFIG_SELECT_MEMORY_MODEL not in final .config
Requested value:  CONFIG_SELECT_MEMORY_MODEL=y
Actual value:     

Value requested for CONFIG_FLATMEM_MANUAL not in final .config
Requested value:  CONFIG_FLATMEM_MANUAL=y
Actual value:     

Value requested for CONFIG_SPARSEMEM_MANUAL not in final .config
Requested value:  # CONFIG_SPARSEMEM_MANUAL is not set
Actual value:     

Value requested for CONFIG_FLATMEM not in final .config
Requested value:  CONFIG_FLATMEM=y
Actual value:     

Value requested for CONFIG_SPARSEMEM_STATIC not in final .config
Requested value:  CONFIG_SPARSEMEM_STATIC=y
Actual value:     

Value requested for CONFIG_BOUNCE not in final .config
Requested value:  CONFIG_BOUNCE=y
Actual value:     

Value requested for CONFIG_KMAP_LOCAL not in final .config
Requested value:  CONFIG_KMAP_LOCAL=y
Actual value:     

Value requested for CONFIG_HOTPLUG_PCI_COMPAQ not in final .config
Requested value:  # CONFIG_HOTPLUG_PCI_COMPAQ is not set
Actual value:     

Value requested for CONFIG_HOTPLUG_PCI_IBM not in final .config
Requested value:  # CONFIG_HOTPLUG_PCI_IBM is not set
Actual value:     

Value requested for CONFIG_EFI_CAPSULE_QUIRK_QUARK_CSH not in final .config
Requested value:  CONFIG_EFI_CAPSULE_QUIRK_QUARK_CSH=y
Actual value:     

Value requested for CONFIG_PCH_PHUB not in final .config
Requested value:  # CONFIG_PCH_PHUB is not set
Actual value:     

Value requested for CONFIG_SCSI_NSP32 not in final .config
Requested value:  # CONFIG_SCSI_NSP32 is not set
Actual value:     

Value requested for CONFIG_PATA_CS5520 not in final .config
Requested value:  # CONFIG_PATA_CS5520 is not set
Actual value:     

Value requested for CONFIG_PATA_CS5530 not in final .config
Requested value:  # CONFIG_PATA_CS5530 is not set
Actual value:     

Value requested for CONFIG_PATA_CS5535 not in final .config
Requested value:  # CONFIG_PATA_CS5535 is not set
Actual value:     

Value requested for CONFIG_PATA_CS5536 not in final .config
Requested value:  # CONFIG_PATA_CS5536 is not set
Actual value:     

Value requested for CONFIG_PATA_SC1200 not in final .config
Requested value:  # CONFIG_PATA_SC1200 is not set
Actual value:     

Value requested for CONFIG_PCH_GBE not in final .config
Requested value:  # CONFIG_PCH_GBE is not set
Actual value:     

Value requested for CONFIG_INPUT_WISTRON_BTNS not in final .config
Requested value:  # CONFIG_INPUT_WISTRON_BTNS is not set
Actual value:     

Value requested for CONFIG_SERIAL_TIMBERDALE not in final .config
Requested value:  # CONFIG_SERIAL_TIMBERDALE is not set
Actual value:     

Value requested for CONFIG_SERIAL_PCH_UART not in final .config
Requested value:  # CONFIG_SERIAL_PCH_UART is not set
Actual value:     

Value requested for CONFIG_HW_RANDOM_GEODE not in final .config
Requested value:  CONFIG_HW_RANDOM_GEODE=y
Actual value:     

Value requested for CONFIG_SONYPI not in final .config
Requested value:  # CONFIG_SONYPI is not set
Actual value:     

Value requested for CONFIG_PC8736x_GPIO not in final .config
Requested value:  # CONFIG_PC8736x_GPIO is not set
Actual value:     

Value requested for CONFIG_NSC_GPIO not in final .config
Requested value:  # CONFIG_NSC_GPIO is not set
Actual value:     

Value requested for CONFIG_I2C_EG20T not in final .config
Requested value:  # CONFIG_I2C_EG20T is not set
Actual value:     

Value requested for CONFIG_SCx200_ACB not in final .config
Requested value:  # CONFIG_SCx200_ACB is not set
Actual value:     

Value requested for CONFIG_PTP_1588_CLOCK_PCH not in final .config
Requested value:  # CONFIG_PTP_1588_CLOCK_PCH is not set
Actual value:     

Value requested for CONFIG_SBC8360_WDT not in final .config
Requested value:  # CONFIG_SBC8360_WDT is not set
Actual value:     

Value requested for CONFIG_SBC7240_WDT not in final .config
Requested value:  # CONFIG_SBC7240_WDT is not set
Actual value:     

Value requested for CONFIG_MFD_CS5535 not in final .config
Requested value:  # CONFIG_MFD_CS5535 is not set
Actual value:     

Value requested for CONFIG_AGP_ALI not in final .config
Requested value:  # CONFIG_AGP_ALI is not set
Actual value:     

Value requested for CONFIG_AGP_ATI not in final .config
Requested value:  # CONFIG_AGP_ATI is not set
Actual value:     

Value requested for CONFIG_AGP_AMD not in final .config
Requested value:  # CONFIG_AGP_AMD is not set
Actual value:     

Value requested for CONFIG_AGP_NVIDIA not in final .config
Requested value:  # CONFIG_AGP_NVIDIA is not set
Actual value:     

Value requested for CONFIG_AGP_SWORKS not in final .config
Requested value:  # CONFIG_AGP_SWORKS is not set
Actual value:     

Value requested for CONFIG_AGP_EFFICEON not in final .config
Requested value:  # CONFIG_AGP_EFFICEON is not set
Actual value:     

Value requested for CONFIG_SND_CS5530 not in final .config
Requested value:  # CONFIG_SND_CS5530 is not set
Actual value:     

Value requested for CONFIG_SND_CS5535AUDIO not in final .config
Requested value:  # CONFIG_SND_CS5535AUDIO is not set
Actual value:     

Value requested for CONFIG_SND_SIS7019 not in final .config
Requested value:  # CONFIG_SND_SIS7019 is not set
Actual value:     

Value requested for CONFIG_LEDS_OT200 not in final .config
Requested value:  # CONFIG_LEDS_OT200 is not set
Actual value:     

Value requested for CONFIG_PCH_DMA not in final .config
Requested value:  # CONFIG_PCH_DMA is not set
Actual value:     

Value requested for CONFIG_CLKSRC_I8253 not in final .config
Requested value:  CONFIG_CLKSRC_I8253=y
Actual value:     

Value requested for CONFIG_MAILBOX not in final .config
Requested value:  # CONFIG_MAILBOX is not set
Actual value:     CONFIG_MAILBOX=y

Value requested for CONFIG_CRYPTO_SERPENT_SSE2_586 not in final .config
Requested value:  # CONFIG_CRYPTO_SERPENT_SSE2_586 is not set
Actual value:     

Value requested for CONFIG_CRYPTO_TWOFISH_586 not in final .config
Requested value:  # CONFIG_CRYPTO_TWOFISH_586 is not set
Actual value:     

Value requested for CONFIG_CRYPTO_DEV_GEODE not in final .config
Requested value:  # CONFIG_CRYPTO_DEV_GEODE is not set
Actual value:     

Value requested for CONFIG_CRYPTO_DEV_HIFN_795X not in final .config
Requested value:  # CONFIG_CRYPTO_DEV_HIFN_795X is not set
Actual value:     

Value requested for CONFIG_CRYPTO_LIB_POLY1305_RSIZE not in final .config
Requested value:  CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
Actual value:     CONFIG_CRYPTO_LIB_POLY1305_RSIZE=11

Value requested for CONFIG_AUDIT_GENERIC not in final .config
Requested value:  CONFIG_AUDIT_GENERIC=y
Actual value:     

Value requested for CONFIG_GENERIC_VDSO_32 not in final .config
Requested value:  CONFIG_GENERIC_VDSO_32=y
Actual value:     

Value requested for CONFIG_DEBUG_KMAP_LOCAL not in final .config
Requested value:  # CONFIG_DEBUG_KMAP_LOCAL is not set
Actual value:     

Value requested for CONFIG_DEBUG_HIGHMEM not in final .config
Requested value:  # CONFIG_DEBUG_HIGHMEM is not set
Actual value:     

Value requested for CONFIG_HAVE_DEBUG_STACKOVERFLOW not in final .config
Requested value:  CONFIG_HAVE_DEBUG_STACKOVERFLOW=y
Actual value:     

Value requested for CONFIG_DEBUG_STACKOVERFLOW not in final .config
Requested value:  # CONFIG_DEBUG_STACKOVERFLOW is not set
Actual value:     

Value requested for CONFIG_HAVE_FUNCTION_GRAPH_TRACER not in final .config
Requested value:  CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
Actual value:     

Value requested for CONFIG_HAVE_FUNCTION_GRAPH_FREGS not in final .config
Requested value:  CONFIG_HAVE_FUNCTION_GRAPH_FREGS=y
Actual value:     

Value requested for CONFIG_HAVE_FTRACE_GRAPH_FUNC not in final .config
Requested value:  CONFIG_HAVE_FTRACE_GRAPH_FUNC=y
Actual value:     

Value requested for CONFIG_DRM_KUNIT_TEST not in final .config
Requested value:  CONFIG_DRM_KUNIT_TEST=m
Actual value:     

Value requested for CONFIG_DRM_XE_WERROR not in final .config
Requested value:  CONFIG_DRM_XE_WERROR=y
Actual value:     

Value requested for CONFIG_DRM_XE_DEBUG not in final .config
Requested value:  CONFIG_DRM_XE_DEBUG=y
Actual value:     

Value requested for CONFIG_DRM_XE_DEBUG_MEM not in final .config
Requested value:  CONFIG_DRM_XE_DEBUG_MEM=y
Actual value:     

Value requested for CONFIG_DRM_XE_KUNIT_TEST not in final .config
Requested value:  CONFIG_DRM_XE_KUNIT_TEST=m
Actual value:     

++ nproc
+ make -j48 ARCH=i386 olddefconfig
  GEN     Makefile

WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
  Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
  Selected by [m]:
  - DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m] && HAS_IOPORT [=y]
#
# configuration written to .config
#
++ nproc
+ make -j48 ARCH=i386
  SYNC    include/config/auto.conf.cmd
  GEN     Makefile

WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
  Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
  Selected by [m]:
  - DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m] && HAS_IOPORT [=y]

WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
  Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
  Selected by [m]:
  - DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m] && HAS_IOPORT [=y]

WARNING: unmet direct dependencies detected for FB_IOMEM_HELPERS
  Depends on [n]: HAS_IOMEM [=y] && FB_CORE [=n]
  Selected by [m]:
  - DRM_XE_DISPLAY [=y] && HAS_IOMEM [=y] && DRM [=y] && DRM_XE [=m] && DRM_XE [=m]=m [=m] && HAS_IOPORT [=y]
  GEN     Makefile
  WRAP    arch/x86/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/x86/include/generated/uapi/asm/errno.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctl.h
  WRAP    arch/x86/include/generated/uapi/asm/fcntl.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctls.h
  UPD     include/generated/uapi/linux/version.h
  WRAP    arch/x86/include/generated/uapi/asm/ipcbuf.h
  WRAP    arch/x86/include/generated/uapi/asm/param.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  WRAP    arch/x86/include/generated/uapi/asm/poll.h
  WRAP    arch/x86/include/generated/uapi/asm/resource.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  WRAP    arch/x86/include/generated/uapi/asm/socket.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  WRAP    arch/x86/include/generated/uapi/asm/sockios.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  WRAP    arch/x86/include/generated/uapi/asm/termbits.h
  WRAP    arch/x86/include/generated/uapi/asm/termios.h
  WRAP    arch/x86/include/generated/uapi/asm/types.h
  HOSTCC  arch/x86/tools/relocs_32.o
  HOSTCC  arch/x86/tools/relocs_64.o
  UPD     include/generated/compile.h
  HOSTCC  arch/x86/tools/relocs_common.o
  WRAP    arch/x86/include/generated/asm/early_ioremap.h
  WRAP    arch/x86/include/generated/asm/fprobe.h
  WRAP    arch/x86/include/generated/asm/mcs_spinlock.h
  WRAP    arch/x86/include/generated/asm/mmzone.h
  WRAP    arch/x86/include/generated/asm/irq_regs.h
  WRAP    arch/x86/include/generated/asm/kmap_size.h
  WRAP    arch/x86/include/generated/asm/local64.h
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/sorttable
  WRAP    arch/x86/include/generated/asm/mmiowb.h
  WRAP    arch/x86/include/generated/asm/module.lds.h
  WRAP    arch/x86/include/generated/asm/rwonce.h
  HOSTCC  scripts/asn1_compiler
  HOSTCC  scripts/selinux/mdp/mdp
  HOSTLD  arch/x86/tools/relocs
  UPD     include/config/kernel.release
  UPD     include/generated/utsrelease.h
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  CC      scripts/mod/devicetable-offsets.s
  UPD     scripts/mod/devicetable-offsets.h
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/mod/symsearch.o
  HOSTLD  scripts/mod/modpost
  CC      kernel/bounds.s
  CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-arch-fallback.h
  CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-instrumented.h
  CHKSHA1 /workspace/kernel/include/linux/atomic/atomic-long.h
  UPD     include/generated/timeconst.h
  UPD     include/generated/bounds.h
  CC      arch/x86/kernel/asm-offsets.s
  UPD     include/generated/asm-offsets.h
  CALL    /workspace/kernel/scripts/checksyscalls.sh
  LDS     scripts/module.lds
  HOSTCC  usr/gen_init_cpio
  CC      init/main.o
  CC      certs/system_keyring.o
  CC      init/do_mounts.o
  CC      init/do_mounts_initrd.o
  CC      ipc/util.o
  UPD     init/utsversion-tmp.h
  CC      init/initramfs.o
  CC      ipc/msgutil.o
  CC      init/calibrate.o
  CC      mm/filemap.o
  CC      ipc/msg.o
  CC      io_uring/io_uring.o
  CC      security/commoncap.o
  CC      init/init_task.o
  AS      arch/x86/entry/entry.o
  CC      mm/mempool.o
  CC      security/lsm_syscalls.o
  CC      ipc/sem.o
  CC      block/bdev.o
  CC      io_uring/opdef.o
  CC      arch/x86/power/cpu.o
  AS      arch/x86/lib/atomic64_cx8_32.o
  CC      arch/x86/realmode/init.o
  CC      arch/x86/pci/i386.o
  CC      security/min_addr.o
  AR      arch/x86/crypto/built-in.a
  AR      arch/x86/net/built-in.a
  CC      security/keys/gc.o
  CC      arch/x86/video/video-common.o
  AR      arch/x86/entry/vsyscall/built-in.a
  CC      security/integrity/iint.o
  HOSTCC  security/selinux/genheaders
  CC      block/partitions/core.o
  CC      fs/nfs_common/nfsacl.o
  CC      arch/x86/mm/pat/set_memory.o
  CC      arch/x86/events/amd/core.o
  AR      arch/x86/platform/atom/built-in.a
  AR      virt/lib/built-in.a
  CC      arch/x86/mm/pat/memtype.o
  CC      arch/x86/virt/svm/cmdline.o
  CC      lib/math/div64.o
  CC      net/core/sock.o
  CC      arch/x86/kernel/fpu/init.o
  AR      drivers/cache/built-in.a
  CC      fs/notify/dnotify/dnotify.o
  CC      fs/notify/inotify/inotify_fsnotify.o
  CC      sound/core/seq/seq.o
  AR      virt/built-in.a
  AR      arch/x86/virt/vmx/built-in.a
  AR      arch/x86/platform/ce4100/built-in.a
  CC      arch/x86/entry/vdso/vma.o
  AS      arch/x86/lib/checksum_32.o
  AR      drivers/irqchip/built-in.a
  CC      block/partitions/msdos.o
  CC      arch/x86/pci/init.o
  CC      lib/math/gcd.o
  CC      arch/x86/platform/efi/memmap.o
  CC      kernel/sched/core.o
  CC      arch/x86/lib/cmdline.o
  AR      drivers/bus/mhi/built-in.a
  AR      drivers/bus/built-in.a
  CC      crypto/asymmetric_keys/asymmetric_type.o
  AR      drivers/pwm/built-in.a
  CC      arch/x86/pci/pcbios.o
  AR      drivers/leds/trigger/built-in.a
  AR      arch/x86/virt/svm/built-in.a
  AR      arch/x86/virt/built-in.a
  AR      drivers/leds/blink/built-in.a
  CC      arch/x86/pci/mmconfig_32.o
  AR      drivers/leds/simple/built-in.a
  CC      drivers/leds/led-core.o
  AS      arch/x86/lib/cmpxchg8b_emu.o
  CC      lib/math/lcm.o
  GEN     security/selinux/flask.h security/selinux/av_permissions.h
  CC      security/selinux/avc.o
  CC      arch/x86/lib/cpu.o
  CC      lib/math/int_log.o
  CC      arch/x86/events/amd/lbr.o
  CC      lib/math/int_pow.o
  GEN     usr/initramfs_data.cpio
  COPY    usr/initramfs_inc_data
  AS      usr/initramfs_data.o
  CC      lib/math/int_sqrt.o
  AR      usr/built-in.a
  HOSTCC  certs/extract-cert
  CC      arch/x86/kernel/fpu/bugs.o
  CC      arch/x86/events/amd/ibs.o
  CC      lib/math/reciprocal_div.o
  CC      arch/x86/kernel/fpu/core.o
  CC      lib/math/rational.o
  CC      fs/notify/inotify/inotify_user.o
  CC      sound/core/seq/seq_lock.o
  CC      arch/x86/lib/delay.o
  AS      arch/x86/realmode/rm/header.o
  AS      arch/x86/realmode/rm/trampoline_32.o
  AR      arch/x86/video/built-in.a
  CC      sound/core/sound.o
  AS      arch/x86/realmode/rm/stack.o
  AS      arch/x86/realmode/rm/reboot.o
  CC      security/security.o
  CC      sound/core/init.o
  CERT    certs/x509_certificate_list
  CERT    certs/signing_key.x509
  AS      arch/x86/realmode/rm/wakeup_asm.o
  AS      certs/system_certificates.o
  CC      security/integrity/integrity_audit.o
  CC      net/ethernet/eth.o
  CC      fs/nfs_common/grace.o
  AR      certs/built-in.a
  CC      arch/x86/realmode/rm/wakemain.o
  AR      net/802/built-in.a
  CC      net/sched/sch_generic.o
  CC      sound/core/memory.o
  CC      drivers/leds/led-class.o
  AR      sound/i2c/other/built-in.a
  CC      arch/x86/kernel/cpu/mce/core.o
  CC      sound/core/control.o
  AR      sound/i2c/built-in.a
  AR      arch/x86/platform/geode/built-in.a
  CC      security/keys/key.o
  CC      arch/x86/kernel/cpu/mce/severity.o
  CC      arch/x86/kernel/cpu/mce/genpool.o
  CC      crypto/asymmetric_keys/restrict.o
  CC      arch/x86/realmode/rm/video-mode.o
  AS      arch/x86/entry/entry_32.o
  CC      sound/core/misc.o
  CC      arch/x86/entry/vdso/extable.o
  AR      fs/notify/dnotify/built-in.a
  CC      security/selinux/hooks.o
  CC      arch/x86/platform/efi/quirks.o
  CC      arch/x86/pci/direct.o
  CC      sound/core/seq/seq_clientmgr.o
  CC      arch/x86/power/hibernate_32.o
  CC      block/fops.o
  AS      arch/x86/lib/getuser.o
  GEN     arch/x86/lib/inat-tables.c
  CC      arch/x86/kernel/cpu/mce/intel.o
  AR      lib/math/built-in.a
  AS      arch/x86/realmode/rm/copy.o
  CC      block/partitions/efi.o
  CC      arch/x86/lib/insn-eval.o
  CC      lib/crypto/mpi/generic_mpih-lshift.o
  AS      arch/x86/realmode/rm/bioscall.o
  CC      arch/x86/realmode/rm/regs.o
  CC      lib/crypto/mpi/generic_mpih-mul1.o
  CC      arch/x86/realmode/rm/video-vga.o
  AR      arch/x86/platform/iris/built-in.a
  CC      block/bio.o
  CC      io_uring/kbuf.o
  CC      arch/x86/kernel/cpu/mce/amd.o
  CC      arch/x86/realmode/rm/video-vesa.o
  CC      arch/x86/entry/syscall_32.o
  CC      arch/x86/platform/intel/iosf_mbi.o
  CC      arch/x86/realmode/rm/video-bios.o
  CC      crypto/api.o
  AS      arch/x86/power/hibernate_asm_32.o
  CC      lib/crypto/memneq.o
  CC      crypto/asymmetric_keys/signature.o
  CC      io_uring/rsrc.o
  PASYMS  arch/x86/realmode/rm/pasyms.h
  CC      arch/x86/kernel/cpu/mtrr/mtrr.o
  CC      drivers/leds/led-triggers.o
  LDS     arch/x86/realmode/rm/realmode.lds
  LD      arch/x86/realmode/rm/realmode.elf
  RELOCS  arch/x86/realmode/rm/realmode.relocs
  OBJCOPY arch/x86/realmode/rm/realmode.bin
  AS      arch/x86/realmode/rmpiggy.o
  CC      arch/x86/lib/insn.o
  AR      arch/x86/realmode/built-in.a
  CC      arch/x86/mm/pat/memtype_interval.o
  CC      fs/iomap/trace.o
  CC      fs/quota/dquot.o
  AR      security/integrity/built-in.a
  CC      arch/x86/platform/efi/efi.o
  CC      fs/nfs_common/common.o
  CC      init/version.o
  AR      fs/notify/fanotify/built-in.a
  CC      ipc/shm.o
  CC      arch/x86/mm/init.o
  AR      fs/notify/inotify/built-in.a
  CC      fs/notify/fsnotify.o
  CC      fs/proc/task_mmu.o
  CC      arch/x86/pci/mmconfig-shared.o
  CC      lib/crypto/mpi/generic_mpih-mul2.o
  CC      arch/x86/power/hibernate.o
  LDS     arch/x86/entry/vdso/vdso32/vdso32.lds
  CC      arch/x86/events/amd/uncore.o
  AS      arch/x86/entry/vdso/vdso32/note.o
  CC      kernel/sched/fair.o
  CC      arch/x86/kernel/fpu/regset.o
  AS      arch/x86/entry/vdso/vdso32/system_call.o
  CC      security/lsm_audit.o
  AR      init/built-in.a
  AS      arch/x86/entry/vdso/vdso32/sigreturn.o
  CC      arch/x86/kernel/fpu/signal.o
  CC      arch/x86/entry/vdso/vdso32/vclock_gettime.o
  CC      net/netlink/af_netlink.o
  AR      net/bpf/built-in.a
  CC      crypto/asymmetric_keys/public_key.o
  CC      net/ethtool/ioctl.o
  CC      net/netfilter/core.o
  CC      arch/x86/lib/kaslr.o
  CC      security/keys/keyring.o
  AR      block/partitions/built-in.a
  CC      net/netfilter/nf_log.o
  CC      fs/proc/inode.o
  CC      net/core/request_sock.o
  CC      net/ethtool/common.o
  AR      arch/x86/platform/intel/built-in.a
  CC      kernel/locking/mutex.o
  CC      arch/x86/lib/memcpy_32.o
  AR      arch/x86/mm/pat/built-in.a
  CC      security/keys/keyctl.o
  AR      net/ethernet/built-in.a
  CC      lib/crypto/mpi/generic_mpih-mul3.o
  AS      arch/x86/lib/memmove_32.o
  AR      drivers/leds/built-in.a
  CC      arch/x86/kernel/cpu/mtrr/if.o
  CC      net/ipv4/netfilter/nf_defrag_ipv4.o
  CC      arch/x86/lib/misc.o
  CC      drivers/pci/msi/pcidev_msi.o
  CC      arch/x86/lib/pc-conf-reg.o
  CC      drivers/pci/pcie/portdrv.o
  AR      fs/nfs_common/built-in.a
  CC      sound/core/seq/seq_memory.o
  AR      sound/drivers/opl3/built-in.a
  AR      sound/drivers/opl4/built-in.a
  AR      sound/drivers/mpu401/built-in.a
  CC      arch/x86/mm/init_32.o
  AR      arch/x86/platform/intel-mid/built-in.a
  AR      sound/drivers/vx/built-in.a
  CC      arch/x86/entry/common.o
  AR      sound/drivers/pcsp/built-in.a
  CC      net/ipv4/route.o
  AR      sound/drivers/built-in.a
  AR      arch/x86/power/built-in.a
  AS      arch/x86/lib/putuser.o
  CC      net/ipv4/netfilter/nf_reject_ipv4.o
  CC      arch/x86/kernel/cpu/mtrr/generic.o
  AS      arch/x86/lib/retpoline.o
  CC      arch/x86/entry/vdso/vdso32/vgetcpu.o
  CC      arch/x86/lib/string_32.o
  ASN.1   crypto/asymmetric_keys/x509.asn1.[ch]
  ASN.1   crypto/asymmetric_keys/x509_akid.asn1.[ch]
  CC      crypto/asymmetric_keys/x509_loader.o
  HOSTCC  arch/x86/entry/vdso/vdso2c
  CC      arch/x86/entry/vdso/vdso32-setup.o
  CC      fs/notify/notification.o
  CC      arch/x86/lib/strstr_32.o
  CC      arch/x86/platform/efi/efi_32.o
  CC      block/elevator.o
  CC      arch/x86/lib/usercopy.o
  CC      arch/x86/pci/fixup.o
  CC      drivers/video/console/dummycon.o
  CC      arch/x86/kernel/fpu/xstate.o
  CC      fs/iomap/iter.o
  CC      lib/zlib_inflate/inffast.o
  CC      arch/x86/kernel/cpu/mce/threshold.o
  CC      crypto/asymmetric_keys/x509_public_key.o
  CC      net/core/skbuff.o
  CC      lib/crypto/mpi/generic_mpih-rshift.o
  CC      lib/zlib_deflate/deflate.o
  CC      lib/lzo/lzo1x_compress.o
  AR      arch/x86/events/amd/built-in.a
  CC      lib/lz4/lz4_decompress.o
  CC      arch/x86/events/intel/core.o
  CC      ipc/syscall.o
  CC      drivers/pci/msi/api.o
  CC      arch/x86/lib/usercopy_32.o
  CC      net/sched/sch_mq.o
  CC      lib/zlib_inflate/inflate.o
  VDSO    arch/x86/entry/vdso/vdso32.so.dbg
  CC      lib/zstd/zstd_decompress_module.o
  OBJCOPY arch/x86/entry/vdso/vdso32.so
  VDSO2C  arch/x86/entry/vdso/vdso-image-32.c
  CC      arch/x86/entry/vdso/vdso-image-32.o
  CC      lib/xz/xz_dec_syms.o
  CC      lib/dim/dim.o
  CC      drivers/pci/pcie/rcec.o
  CC      sound/core/seq/seq_queue.o
  CC      lib/zlib_deflate/deftree.o
  CC      arch/x86/lib/msr-smp.o
  AR      drivers/idle/built-in.a
  CC      net/ipv4/inetpeer.o
  CC      fs/notify/group.o
  CC      drivers/video/console/vgacon.o
  CC      kernel/sched/build_policy.o
  CC      mm/oom_kill.o
  AS      arch/x86/platform/efi/efi_stub_32.o
  CC      arch/x86/mm/fault.o
  CC      kernel/power/qos.o
  CC      net/netfilter/nf_queue.o
  CC      arch/x86/platform/efi/runtime-map.o
  ASN.1   crypto/asymmetric_keys/pkcs7.asn1.[ch]
  AR      arch/x86/entry/vdso/built-in.a
  CC      crypto/asymmetric_keys/pkcs7_trust.o
  AS      arch/x86/entry/thunk.o
  CC      lib/crypto/mpi/generic_mpih-sub1.o
  CC      lib/lzo/lzo1x_decompress_safe.o
  CC      security/keys/permission.o
  AR      arch/x86/entry/built-in.a
  CC      kernel/locking/semaphore.o
  AR      arch/x86/platform/intel-quark/built-in.a
  CC      lib/dim/net_dim.o
  CC      arch/x86/pci/acpi.o
  CC      lib/xz/xz_dec_stream.o
  CC      fs/proc/root.o
  CC      lib/zstd/decompress/huf_decompress.o
  CC      arch/x86/kernel/cpu/mtrr/cleanup.o
  CC      fs/iomap/buffered-io.o
  CC      drivers/pci/pcie/bwctrl.o
  CC      arch/x86/lib/cache-smp.o
  CC      fs/quota/quota_v2.o
  CC      lib/zlib_inflate/infutil.o
  CC      arch/x86/events/intel/bts.o
  CC      drivers/pci/msi/msi.o
  CC      lib/fonts/fonts.o
  CC      ipc/ipc_sysctl.o
  CC      arch/x86/lib/crc32-glue.o
  CC      ipc/mqueue.o
  CC      block/blk-core.o
  CC      crypto/asymmetric_keys/pkcs7_verify.o
  CC      net/ipv4/netfilter/ip_tables.o
  AR      arch/x86/kernel/cpu/mce/built-in.a
  CC      arch/x86/kernel/cpu/mtrr/amd.o
  CC      arch/x86/events/intel/ds.o
  CC      lib/zlib_deflate/deflate_syms.o
  AR      arch/x86/kernel/fpu/built-in.a
  AR      lib/lzo/built-in.a
  CC      io_uring/notif.o
  CC      arch/x86/pci/legacy.o
  CC      lib/zlib_inflate/inftrees.o
  CC      fs/notify/mark.o
  CC      lib/crypto/mpi/generic_mpih-add1.o
  CC      lib/xz/xz_dec_lzma2.o
  CC      sound/core/seq/seq_fifo.o
  CC      net/sched/sch_frag.o
  CC      lib/fonts/font_8x16.o
  AS      arch/x86/lib/crc32-pclmul.o
  AR      arch/x86/platform/efi/built-in.a
  CC      kernel/locking/rwsem.o
  CC      lib/zlib_inflate/inflate_syms.o
  CC      arch/x86/lib/msr.o
  AR      arch/x86/platform/olpc/built-in.a
  CC      security/keys/process_keys.o
  AR      arch/x86/platform/scx200/built-in.a
  AR      arch/x86/platform/ts5500/built-in.a
  AR      arch/x86/platform/uv/built-in.a
  AR      arch/x86/platform/built-in.a
  AR      lib/zlib_deflate/built-in.a
  CC      lib/xz/xz_dec_bcj.o
  CC      drivers/pci/msi/irqdomain.o
  CC      fs/iomap/direct-io.o
  CC      crypto/asymmetric_keys/x509.asn1.o
  AR      lib/lz4/built-in.a
  CC      ipc/namespace.o
  CC      crypto/asymmetric_keys/x509_akid.asn1.o
  CC      fs/proc/base.o
  CC      ipc/mq_sysctl.o
  CC      crypto/asymmetric_keys/x509_cert_parser.o
  CC      drivers/pci/pcie/aspm.o
  CC      fs/quota/quota_tree.o
  CC      fs/kernfs/mount.o
  AR      drivers/pci/pwrctrl/built-in.a
  CC      net/xfrm/xfrm_policy.o
  AR      drivers/video/console/built-in.a
  CC      net/netlink/genetlink.o
  CC      drivers/video/backlight/backlight.o
  AR      drivers/video/fbdev/core/built-in.a
  AR      drivers/video/fbdev/omap/built-in.a
  CC      kernel/power/main.o
  CC      arch/x86/kernel/cpu/mtrr/cyrix.o
  AR      drivers/video/fbdev/omap2/omapfb/dss/built-in.a
  CC      lib/dim/rdma_dim.o
  AR      drivers/video/fbdev/omap2/omapfb/displays/built-in.a
  AR      drivers/video/fbdev/omap2/omapfb/built-in.a
  AR      drivers/video/fbdev/omap2/built-in.a
  AR      lib/zlib_inflate/built-in.a
  AR      lib/fonts/built-in.a
  AR      drivers/video/fbdev/built-in.a
  CC      sound/core/seq/seq_prioq.o
  CC      fs/kernfs/inode.o
  CC      fs/iomap/fiemap.o
  CC      arch/x86/pci/irq.o
  CC      lib/crypto/mpi/mpicoder.o
  CC      net/ethtool/netlink.o
  CC      net/netfilter/nf_sockopt.o
  CC      security/selinux/selinuxfs.o
  CC      sound/core/seq/seq_timer.o
  CC      arch/x86/mm/ioremap.o
  CC      net/ipv4/protocol.o
  CC      net/ipv4/ip_input.o
  CC      drivers/pci/pcie/pme.o
  AR      lib/xz/built-in.a
  CC      fs/quota/quota.o
  CC      lib/zstd/decompress/zstd_ddict.o
  CC      net/ethtool/bitset.o
  CC      crypto/asymmetric_keys/pkcs7.asn1.o
  CC      crypto/asymmetric_keys/pkcs7_parser.o
  AR      lib/dim/built-in.a
  CC      arch/x86/kernel/cpu/microcode/core.o
  CC      fs/notify/fdinfo.o
  CC      lib/zstd/decompress/zstd_decompress.o
  CC      mm/fadvise.o
  CC      sound/core/device.o
  CC      kernel/locking/percpu-rwsem.o
  AS      arch/x86/lib/msr-reg.o
  AR      drivers/pci/msi/built-in.a
  CC      arch/x86/lib/msr-reg-export.o
  CC      io_uring/tctx.o
  CC      fs/sysfs/file.o
  CC      arch/x86/kernel/cpu/mtrr/centaur.o
  CC      security/keys/request_key.o
  CC      net/ethtool/strset.o
  CC      lib/zstd/decompress/zstd_decompress_block.o
  AR      drivers/video/backlight/built-in.a
  CC      drivers/video/aperture.o
  CC      arch/x86/events/intel/knc.o
  AS      arch/x86/lib/hweight.o
  CC      fs/proc/generic.o
  CC      arch/x86/lib/iomem.o
  CC      kernel/printk/printk.o
  CC      net/sched/sch_api.o
  CC      kernel/irq/irqdesc.o
  CC      lib/crypto/mpi/mpi-add.o
  CC      fs/kernfs/dir.o
  CC      net/sched/sch_blackhole.o
  AR      ipc/built-in.a
  AR      drivers/char/ipmi/built-in.a
  CC      lib/crypto/mpi/mpi-bit.o
  CC      net/ipv4/netfilter/iptable_filter.o
  CC      sound/core/seq/seq_system.o
  AR      crypto/asymmetric_keys/built-in.a
  CC      crypto/cipher.o
  CC      net/ipv4/netfilter/iptable_mangle.o
  CC      arch/x86/mm/extable.o
  CC      kernel/power/console.o
  CC      security/keys/request_key_auth.o
  CC      fs/iomap/seek.o
  AR      fs/notify/built-in.a
  AR      drivers/pci/pcie/built-in.a
  CC      fs/iomap/swapfile.o
  CC      drivers/pci/hotplug/pci_hotplug_core.o
  CC      net/netfilter/utils.o
  CC      arch/x86/kernel/cpu/microcode/intel.o
  CC      kernel/locking/spinlock.o
  CC      arch/x86/lib/atomic64_32.o
  CC      arch/x86/kernel/cpu/mtrr/legacy.o
  CC      net/ipv4/netfilter/ipt_REJECT.o
  CC      arch/x86/lib/inat.o
  CC [M]  net/ipv4/netfilter/iptable_nat.o
  CC      mm/maccess.o
  CC      block/blk-sysfs.o
  CC      arch/x86/pci/common.o
  CC      fs/sysfs/dir.o
  AR      arch/x86/lib/built-in.a
  CC      kernel/rcu/update.o
  AR      arch/x86/lib/lib.a
  CC      lib/crypto/utils.o
  CC      net/unix/af_unix.o
  CC      io_uring/filetable.o
  CC      fs/devpts/inode.o
  CC      sound/core/seq/seq_ports.o
  AR      arch/x86/kernel/cpu/mtrr/built-in.a
  CC      kernel/locking/osq_lock.o
  CC      kernel/rcu/sync.o
  CC      kernel/irq/handle.o
  CC      lib/crypto/mpi/mpi-cmp.o
  CC      net/netlink/policy.o
  CC      fs/quota/kqid.o
  CC      drivers/video/cmdline.o
  CC      lib/crypto/mpi/mpi-sub-ui.o
  CC      lib/crypto/mpi/mpi-div.o
  CC      crypto/compress.o
  CC      arch/x86/kernel/cpu/cacheinfo.o
  CC      kernel/irq/manage.o
  CC      fs/netfs/buffered_read.o
  CC      security/keys/user_defined.o
  CC      kernel/power/process.o
  CC      kernel/locking/qspinlock.o
  CC      fs/proc/array.o
  CC      net/ipv6/netfilter/ip6_tables.o
  CC      fs/proc/fd.o
  CC      arch/x86/events/intel/lbr.o
  CC      net/packet/af_packet.o
  CC      net/ethtool/linkinfo.o
  CC      fs/netfs/buffered_write.o
  AR      fs/iomap/built-in.a
  CC      fs/netfs/direct_read.o
  CC      arch/x86/mm/mmap.o
  CC      fs/sysfs/symlink.o
  CC      security/selinux/netlink.o
  CC      kernel/power/suspend.o
  CC      drivers/pci/hotplug/acpi_pcihp.o
  CC      arch/x86/kernel/cpu/microcode/amd.o
  CC      security/device_cgroup.o
  CC      kernel/sched/build_utility.o
  CC      mm/page-writeback.o
  CC      fs/quota/netlink.o
  CC      fs/kernfs/file.o
  AR      sound/isa/ad1816a/built-in.a
  AR      sound/isa/ad1848/built-in.a
  CC      crypto/algapi.o
  AR      sound/isa/cs423x/built-in.a
  AR      sound/isa/es1688/built-in.a
  CC      arch/x86/pci/early.o
  AR      sound/isa/galaxy/built-in.a
  AR      sound/isa/gus/built-in.a
  AR      sound/isa/msnd/built-in.a
  CC      arch/x86/kernel/acpi/boot.o
  CC      net/sched/cls_api.o
  AR      sound/isa/opti9xx/built-in.a
  AR      sound/isa/sb/built-in.a
  AR      sound/isa/wavefront/built-in.a
  AR      sound/isa/wss/built-in.a
  CC      kernel/locking/rtmutex_api.o
  AR      sound/isa/built-in.a
  CC      drivers/acpi/acpica/dsargs.o
  CC      fs/netfs/direct_write.o
  CC      lib/crypto/mpi/mpi-mod.o
  CC      security/keys/proc.o
  AR      fs/devpts/built-in.a
  CC      io_uring/rw.o
  CC      io_uring/net.o
  CC      block/blk-flush.o
  CC      drivers/video/nomodeset.o
  CC      sound/core/seq/seq_info.o
  CC      net/netfilter/nfnetlink.o
  AR      net/ipv4/netfilter/built-in.a
  CC      net/ipv4/ip_fragment.o
  AR      net/netlink/built-in.a
  CC      drivers/acpi/acpica/dscontrol.o
  CC      arch/x86/mm/pgtable.o
  CC      fs/sysfs/mount.o
  CC      net/unix/garbage.o
  CC      fs/proc/proc_tty.o
  CC      lib/zstd/zstd_common_module.o
  CC      drivers/video/hdmi.o
  AR      drivers/pci/hotplug/built-in.a
  AR      drivers/pci/controller/dwc/built-in.a
  AR      drivers/pci/controller/mobiveil/built-in.a
  CC      security/selinux/nlmsgtab.o
  CC      arch/x86/kernel/acpi/sleep.o
  AR      drivers/pci/controller/plda/built-in.a
  AR      drivers/acpi/pmic/built-in.a
  AR      drivers/pci/controller/built-in.a
  CC      kernel/power/hibernate.o
  AR      drivers/pci/switch/built-in.a
  CC      arch/x86/pci/bus_numa.o
  CC      drivers/pci/access.o
  CC      sound/core/info.o
  CC      net/ethtool/linkmodes.o
  CC      lib/crypto/mpi/mpi-mul.o
  CC      lib/crypto/mpi/mpih-cmp.o
  CC      mm/folio-compat.o
  CC      sound/core/seq/seq_dummy.o
  AR      fs/quota/built-in.a
  CC      drivers/acpi/acpica/dsdebug.o
  CC      net/ipv6/af_inet6.o
  AR      arch/x86/kernel/cpu/microcode/built-in.a
  CC      arch/x86/kernel/cpu/scattered.o
  CC      security/keys/sysctl.o
  AR      sound/pci/ac97/built-in.a
  AR      sound/pci/ali5451/built-in.a
  CC      arch/x86/kernel/apic/apic.o
  AR      sound/pci/asihpi/built-in.a
  AR      sound/pci/au88x0/built-in.a
  CC      kernel/irq/spurious.o
  CC      lib/zstd/common/debug.o
  AR      sound/pci/aw2/built-in.a
  AR      sound/pci/ctxfi/built-in.a
  CC      lib/zstd/common/entropy_common.o
  AR      sound/pci/ca0106/built-in.a
  CC      crypto/scatterwalk.o
  CC      kernel/printk/printk_safe.o
  AR      sound/pci/cs46xx/built-in.a
  CC      fs/kernfs/symlink.o
  AR      sound/pci/cs5535audio/built-in.a
  CC      arch/x86/events/intel/p4.o
  AR      sound/pci/lola/built-in.a
  AR      sound/pci/lx6464es/built-in.a
  AR      sound/pci/echoaudio/built-in.a
  AR      sound/pci/emu10k1/built-in.a
  CC      fs/netfs/iterator.o
  CC      sound/pci/hda/hda_bind.o
  CC      net/core/datagram.o
  CC      block/blk-settings.o
  CC      kernel/locking/qrwlock.o
  CC      net/xfrm/xfrm_state.o
  CC      drivers/acpi/acpica/dsfield.o
  CC      lib/zstd/common/error_private.o
  CC      arch/x86/mm/physaddr.o
  CC      kernel/printk/nbcon.o
  CC      lib/zstd/common/fse_decompress.o
  CC      fs/proc/cmdline.o
  CC      net/ethtool/rss.o
  CC      net/sched/act_api.o
  CC      arch/x86/kernel/cpu/topology_common.o
  CC      net/netfilter/nfnetlink_log.o
  CC      fs/sysfs/group.o
  CC      net/unix/sysctl_net_unix.o
  AS      arch/x86/kernel/acpi/wakeup_32.o
  CC      arch/x86/pci/amd_bus.o
  CC      lib/crypto/mpi/mpih-div.o
  CC      arch/x86/kernel/acpi/cstate.o
  AR      sound/core/seq/built-in.a
  CC      lib/crypto/chacha.o
  CC      net/ipv6/netfilter/ip6table_filter.o
  AR      drivers/video/built-in.a
  CC      fs/ext4/balloc.o
  CC      security/keys/keyctl_pkey.o
  CC      kernel/irq/resend.o
  CC      fs/jbd2/transaction.o
  CC      fs/ramfs/inode.o
  CC      fs/hugetlbfs/inode.o
  CC      net/ipv4/ip_forward.o
  AR      kernel/locking/built-in.a
  CC      fs/ext4/bitmap.o
  CC      sound/core/isadma.o
  CC      drivers/acpi/acpica/dsinit.o
  CC      fs/ramfs/file-mmu.o
  CC      security/selinux/netif.o
  CC      crypto/proc.o
  CC      drivers/pci/bus.o
  CC      drivers/acpi/dptf/int340x_thermal.o
  CC      lib/zstd/common/zstd_common.o
  CC      block/blk-ioc.o
  AR      lib/zstd/built-in.a
  CC      arch/x86/mm/tlb.o
  CC      arch/x86/kernel/cpu/topology_ext.o
  CC      kernel/power/snapshot.o
  CC      fs/proc/consoles.o
  CC      arch/x86/mm/cpu_entry_area.o
  AR      fs/kernfs/built-in.a
  CC      mm/readahead.o
  CC      mm/swap.o
  CC      sound/pci/hda/hda_codec.o
  CC      sound/pci/hda/hda_jack.o
  CC      kernel/rcu/srcutree.o
  AR      arch/x86/kernel/acpi/built-in.a
  CC      net/netfilter/nf_conntrack_core.o
  CC      kernel/irq/chip.o
  CC      mm/truncate.o
  AR      fs/sysfs/built-in.a
  CC      arch/x86/mm/maccess.o
  CC      drivers/acpi/acpica/dsmethod.o
  CC      fs/netfs/locking.o
  CC      arch/x86/mm/pgprot.o
  CC      kernel/printk/printk_ringbuffer.o
  CC      arch/x86/events/intel/p6.o
  CC      sound/core/vmaster.o
  AR      security/keys/built-in.a
  CC      lib/crypto/aes.o
  AR      drivers/acpi/dptf/built-in.a
  CC      io_uring/poll.o
  CC      net/core/stream.o
  AR      arch/x86/pci/built-in.a
  CC      arch/x86/kernel/cpu/topology_amd.o
  CC      mm/vmscan.o
  CC      crypto/aead.o
  AR      net/unix/built-in.a
  CC      lib/crypto/mpi/mpih-mul.o
  CC      net/ethtool/linkstate.o
  AR      sound/ppc/built-in.a
  CC      arch/x86/kernel/apic/apic_common.o
  CC      sound/core/ctljack.o
  CC      arch/x86/events/zhaoxin/core.o
  AR      fs/ramfs/built-in.a
  CC      drivers/pnp/pnpacpi/core.o
  CC      fs/proc/cpuinfo.o
  CC      drivers/pnp/core.o
  CC      drivers/pnp/card.o
  CC      drivers/acpi/acpica/dsmthdat.o
  CC      drivers/pci/probe.o
  CC      drivers/pnp/pnpacpi/rsparser.o
  CC      block/blk-map.o
  CC      mm/shrinker.o
  CC      net/sched/sch_fifo.o
  CC      net/ipv6/netfilter/ip6table_mangle.o
  CC      arch/x86/kernel/cpu/common.o
  CC      kernel/rcu/tree.o
  CC      drivers/pci/host-bridge.o
  CC      sound/core/jack.o
  CC      arch/x86/kernel/apic/apic_noop.o
  CC      fs/netfs/main.o
  CC      security/selinux/netnode.o
  CC      kernel/printk/sysctl.o
  CC      net/core/scm.o
  CC      fs/fat/cache.o
  CC      io_uring/eventfd.o
  CC      lib/crypto/arc4.o
  AR      sound/arm/built-in.a
  CC      arch/x86/events/intel/pt.o
  AR      net/packet/built-in.a
  CC      arch/x86/mm/pgtable_32.o
  CC      drivers/pnp/driver.o
  CC      kernel/rcu/rcu_segcblist.o
  CC      drivers/acpi/acpica/dsobject.o
  CC      kernel/irq/dummychip.o
  CC      net/ipv4/ip_options.o
  CC      fs/proc/devices.o
  CC      fs/netfs/misc.o
  CC      fs/jbd2/commit.o
  CC      block/blk-merge.o
  CC      net/sched/cls_cgroup.o
  CC      lib/crypto/gf128mul.o
  CC      lib/crypto/mpi/mpi-pow.o
  CC      arch/x86/kernel/apic/ipi.o
  AR      fs/hugetlbfs/built-in.a
  CC      arch/x86/events/core.o
  AR      kernel/printk/built-in.a
  CC      crypto/geniv.o
  CC      lib/argv_split.o
  CC      kernel/irq/devres.o
  CC      fs/ext4/block_validity.o
  CC      net/ipv6/netfilter/nf_defrag_ipv6_hooks.o
  CC      lib/bug.o
  AR      arch/x86/events/zhaoxin/built-in.a
  CC      net/ipv6/netfilter/nf_conntrack_reasm.o
  CC      net/ipv6/netfilter/nf_reject_ipv6.o
  CC      security/selinux/netport.o
  CC      net/ethtool/debug.o
  CC      drivers/acpi/acpica/dsopcode.o
  CC      block/blk-timeout.o
  CC      lib/crypto/mpi/mpiutil.o
  AR      kernel/sched/built-in.a
  CC      arch/x86/mm/iomap_32.o
  CC      fs/fat/dir.o
  CC      sound/core/hwdep.o
  CC      kernel/power/swap.o
  CC      io_uring/uring_cmd.o
  CC      arch/x86/events/probe.o
  CC      security/selinux/status.o
  AR      drivers/pnp/pnpacpi/built-in.a
  AR      drivers/amba/built-in.a
  CC      drivers/pnp/resource.o
  CC      arch/x86/kernel/apic/vector.o
  CC      arch/x86/events/intel/uncore.o
  CC      crypto/lskcipher.o
  CC      fs/proc/interrupts.o
  CC      kernel/irq/kexec.o
  AR      kernel/livepatch/built-in.a
  CC      drivers/pci/remove.o
  CC      net/core/gen_stats.o
  CC      drivers/pci/pci.o
  CC      kernel/power/user.o
  CC      net/netfilter/nf_conntrack_standalone.o
  CC      drivers/acpi/acpica/dspkginit.o
  CC      arch/x86/kernel/kprobes/core.o
  CC      arch/x86/events/utils.o
  CC      block/blk-lib.o
  CC      security/selinux/ss/ebitmap.o
  CC      io_uring/openclose.o
  CC      arch/x86/mm/hugetlbpage.o
  CC      net/ethtool/wol.o
  AR      lib/crypto/mpi/built-in.a
  CC      lib/crypto/blake2s.o
  CC      crypto/skcipher.o
  CC      fs/ext4/dir.o
  CC      sound/pci/hda/hda_auto_parser.o
  AR      net/dsa/built-in.a
  AR      sound/pci/ice1712/built-in.a
  CC      net/ipv6/netfilter/ip6t_ipv6header.o
  CC      kernel/irq/autoprobe.o
  CC      security/selinux/ss/hashtab.o
  CC      fs/proc/loadavg.o
  CC      net/sched/ematch.o
  CC      arch/x86/events/rapl.o
  AR      drivers/clk/actions/built-in.a
  AR      drivers/clk/analogbits/built-in.a
  CC      arch/x86/kernel/cpu/rdrand.o
  AR      drivers/clk/bcm/built-in.a
  CC      io_uring/sqpoll.o
  CC      drivers/acpi/acpica/dsutils.o
  CC      sound/core/timer.o
  CC      fs/jbd2/recovery.o
  AR      drivers/clk/imgtec/built-in.a
  AR      drivers/clk/imx/built-in.a
  AR      drivers/clk/ingenic/built-in.a
  AR      drivers/clk/mediatek/built-in.a
  CC      net/ipv4/ip_output.o
  CC      net/ipv6/netfilter/ip6t_REJECT.o
  AR      drivers/clk/microchip/built-in.a
  AR      drivers/clk/mstar/built-in.a
  AR      drivers/clk/mvebu/built-in.a
  CC      arch/x86/kernel/cpu/match.o
  AR      drivers/clk/ralink/built-in.a
  CC      arch/x86/kernel/apic/init.o
  AR      drivers/clk/renesas/built-in.a
  CC      fs/netfs/objects.o
  AR      drivers/clk/socfpga/built-in.a
  CC      net/xfrm/xfrm_hash.o
  AR      drivers/clk/sophgo/built-in.a
  AR      drivers/clk/sprd/built-in.a
  CC      lib/crypto/blake2s-generic.o
  CC      mm/shmem.o
  AR      drivers/clk/starfive/built-in.a
  CC      fs/netfs/read_collect.o
  AR      drivers/clk/sunxi-ng/built-in.a
  CC      mm/util.o
  AR      drivers/clk/ti/built-in.a
  AR      drivers/clk/versatile/built-in.a
  CC      fs/netfs/read_pgpriv2.o
  AR      drivers/clk/xilinx/built-in.a
  AR      drivers/clk/built-in.a
  CC      kernel/irq/irqdomain.o
  CC      fs/jbd2/checkpoint.o
  LDS     arch/x86/kernel/vmlinux.lds
  CC      drivers/acpi/acpica/dswexec.o
  CC      drivers/pnp/manager.o
  AR      sound/sh/built-in.a
  CC      drivers/acpi/acpica/dswload.o
  AR      sound/synth/emux/built-in.a
  AR      sound/synth/built-in.a
  CC      arch/x86/kernel/kprobes/opt.o
  CC      arch/x86/mm/dump_pagetables.o
  AS      arch/x86/kernel/head_32.o
  CC      arch/x86/mm/highmem_32.o
  CC      kernel/dma/mapping.o
  CC      block/blk-mq.o
  CC      block/blk-mq-tag.o
  CC      crypto/seqiv.o
  CC      fs/proc/meminfo.o
  CC      kernel/power/poweroff.o
  CC      net/core/gen_estimator.o
  CC      arch/x86/kernel/cpu/bugs.o
  CC      lib/crypto/sha1.o
  CC      arch/x86/kernel/apic/hw_nmi.o
  CC      fs/ext4/ext4_jbd2.o
  CC      lib/crypto/sha256.o
  CC      net/netfilter/nf_conntrack_expect.o
  CC      net/netfilter/nf_conntrack_helper.o
  CC      fs/fat/fatent.o
  AR      kernel/power/built-in.a
  CC      net/ethtool/features.o
  CC      fs/fat/file.o
  CC      drivers/acpi/x86/apple.o
  CC      drivers/acpi/acpica/dswload2.o
  CC      net/xfrm/xfrm_input.o
  CC      drivers/pci/pci-driver.o
  CC      security/selinux/ss/symtab.o
  CC      kernel/dma/direct.o
  CC      drivers/acpi/tables.o
  CC      fs/fat/inode.o
  CC      sound/pci/hda/hda_sysfs.o
  CC      arch/x86/events/intel/uncore_nhmex.o
  CC      net/sunrpc/clnt.o
  CC      kernel/irq/proc.o
  CC      net/sunrpc/auth_gss/auth_gss.o
  CC      drivers/pnp/support.o
  AR      net/sched/built-in.a
  CC      net/sunrpc/auth_gss/gss_mech_switch.o
  CC      net/sunrpc/xprt.o
  CC      net/sunrpc/auth_gss/svcauth_gss.o
  CC      security/selinux/ss/sidtab.o
  CC      net/xfrm/xfrm_output.o
  CC      net/sunrpc/auth_gss/gss_rpc_upcall.o
  CC      crypto/echainiv.o
  CC      fs/fat/misc.o
  AR      arch/x86/mm/built-in.a
  CC      fs/proc/stat.o
  CC      fs/proc/uptime.o
  CC      arch/x86/kernel/apic/io_apic.o
  CC      drivers/acpi/acpica/dswscope.o
  AR      lib/crypto/built-in.a
  CC      lib/buildid.o
  AR      net/ipv6/netfilter/built-in.a
  CC      net/ipv6/anycast.o
  AR      arch/x86/kernel/kprobes/built-in.a
  CC      drivers/pci/search.o
  CC      net/ethtool/privflags.o
  CC      fs/jbd2/revoke.o
  CC      drivers/acpi/x86/cmos_rtc.o
  CC      net/sunrpc/socklib.o
  CC      fs/netfs/read_retry.o
  CC      arch/x86/events/msr.o
  CC      net/core/net_namespace.o
  CC      sound/core/hrtimer.o
  CC      drivers/pnp/interface.o
  CC      io_uring/xattr.o
  CC      drivers/acpi/acpica/dswstate.o
  CC      sound/pci/hda/hda_controller.o
  CC      kernel/irq/migration.o
  CC      net/core/secure_seq.o
  CC      net/core/flow_dissector.o
  CC      crypto/ahash.o
  CC      net/core/sysctl_net_core.o
  CC      block/blk-stat.o
  CC      fs/proc/util.o
  CC      net/netfilter/nf_conntrack_proto.o
  CC      kernel/entry/common.o
  CC      lib/clz_tab.o
  CC      sound/core/pcm.o
  CC      drivers/acpi/x86/lpss.o
  CC      net/netfilter/nf_conntrack_proto_generic.o
  CC      lib/cmdline.o
  CC      net/ethtool/rings.o
  CC      fs/ext4/extents.o
  CC      drivers/acpi/acpica/evevent.o
  CC      fs/netfs/read_single.o
  CC      kernel/entry/syscall_user_dispatch.o
  CC      lib/cpumask.o
  CC      lib/ctype.o
  CC      arch/x86/events/intel/uncore_snb.o
  CC      kernel/irq/cpuhotplug.o
  CC      fs/ext4/extents_status.o
  CC      fs/jbd2/journal.o
  CC      drivers/pnp/quirks.o
  CC      kernel/irq/pm.o
  CC      arch/x86/kernel/cpu/aperfmperf.o
  CC      drivers/pci/rom.o
  CC      net/ipv4/ip_sockglue.o
  CC      fs/netfs/rolling_buffer.o
  CC      io_uring/nop.o
  CC      fs/fat/nfs.o
  AR      kernel/rcu/built-in.a
  CC      security/selinux/ss/avtab.o
  CC      net/xfrm/xfrm_sysctl.o
  CC      fs/proc/version.o
  CC      io_uring/fs.o
  CC      drivers/acpi/osi.o
  CC      drivers/acpi/acpica/evgpe.o
  CC      kernel/module/main.o
  CC      net/sunrpc/auth_gss/gss_rpc_xdr.o
  CC      net/xfrm/xfrm_replay.o
  CC      net/ipv6/ip6_output.o
  CC      net/core/dev.o
  CC      kernel/dma/ops_helpers.o
  CC      security/selinux/ss/policydb.o
  CC      drivers/acpi/x86/s2idle.o
  CC      lib/dec_and_lock.o
  CC      security/selinux/ss/services.o
  CC      mm/mmzone.o
  CC      crypto/shash.o
  CC      arch/x86/kernel/cpu/cpuid-deps.o
  CC      net/netfilter/nf_conntrack_proto_tcp.o
  CC      io_uring/splice.o
  CC      mm/vmstat.o
  CC      drivers/acpi/acpica/evgpeblk.o
  CC      fs/proc/softirqs.o
  CC      arch/x86/kernel/apic/msi.o
  CC      sound/core/pcm_native.o
  CC      mm/backing-dev.o
  CC      drivers/pci/setup-res.o
  CC      lib/decompress.o
  CC      kernel/irq/msi.o
  CC      kernel/irq/affinity.o
  CC      drivers/pnp/system.o
  CC      lib/decompress_bunzip2.o
  AR      kernel/entry/built-in.a
  CC      mm/mm_init.o
  CC      kernel/irq/matrix.o
  CC      drivers/acpi/x86/utils.o
  CC      net/ethtool/channels.o
  CC      net/xfrm/xfrm_device.o
  CC      fs/isofs/namei.o
  CC      sound/pci/hda/hda_proc.o
  CC      kernel/module/strict_rwx.o
  CC      fs/fat/namei_vfat.o
  CC      arch/x86/kernel/cpu/umwait.o
  CC      kernel/dma/remap.o
  CC      arch/x86/events/intel/uncore_snbep.o
  CC      fs/netfs/write_collect.o
  CC      net/ipv4/inet_hashtables.o
  CC      drivers/acpi/osl.o
  CC      drivers/acpi/acpica/evgpeinit.o
  CC      kernel/time/time.o
  CC      sound/core/pcm_lib.o
  CC      net/ipv6/ip6_input.o
  CC      kernel/module/kmod.o
  AR      drivers/pnp/built-in.a
  CC      fs/fat/namei_msdos.o
  CC      fs/proc/namespaces.o
  AR      net/wireless/tests/built-in.a
  CC      net/wireless/core.o
  CC      net/sunrpc/auth_gss/trace.o
  CC      drivers/pci/irq.o
  CC      arch/x86/kernel/apic/probe_32.o
  CC      lib/decompress_inflate.o
  CC      net/ipv4/inet_timewait_sock.o
  CC      io_uring/sync.o
  CC      net/netfilter/nf_conntrack_proto_udp.o
  CC      drivers/acpi/acpica/evgpeutil.o
  CC      drivers/acpi/x86/blacklist.o
  CC      net/ethtool/coalesce.o
  CC      crypto/akcipher.o
  CC      fs/isofs/inode.o
  CC      block/blk-mq-sysfs.o
  AR      kernel/dma/built-in.a
  CC      net/netfilter/nf_conntrack_proto_icmp.o
  CC      net/ethtool/pause.o
  CC      sound/pci/hda/hda_hwdep.o
  MKCAP   arch/x86/kernel/cpu/capflags.c
  CC      net/ethtool/eee.o
  AR      sound/pci/korg1212/built-in.a
  CC      drivers/acpi/acpica/evglock.o
  CC      net/sunrpc/auth_gss/gss_krb5_mech.o
  CC      net/xfrm/xfrm_nat_keepalive.o
  AR      arch/x86/kernel/apic/built-in.a
  CC      net/ipv6/addrconf.o
  CC      sound/core/pcm_misc.o
  CC      net/ethtool/tsinfo.o
  CC      block/blk-mq-cpumap.o
  AR      drivers/acpi/x86/built-in.a
  CC      block/blk-mq-sched.o
  CC      fs/proc/self.o
  CC      lib/decompress_unlz4.o
  CC      io_uring/msg_ring.o
  AR      kernel/irq/built-in.a
  CC      crypto/sig.o
  CC      drivers/pci/vpd.o
  CC      kernel/futex/core.o
  CC      kernel/time/timer.o
  CC      mm/percpu.o
  CC      fs/netfs/write_issue.o
  CC      arch/x86/events/intel/uncore_discovery.o
  CC      sound/pci/hda/hda_intel.o
  AR      fs/fat/built-in.a
  CC      drivers/acpi/acpica/evhandler.o
  CC      kernel/futex/syscalls.o
  AR      sound/pci/mixart/built-in.a
  AR      sound/usb/misc/built-in.a
  AR      sound/firewire/built-in.a
  AR      sound/usb/usx2y/built-in.a
  CC      net/ipv6/addrlabel.o
  CC      kernel/time/hrtimer.o
  AR      sound/usb/caiaq/built-in.a
  CC      kernel/module/tree_lookup.o
  CC      sound/core/pcm_memory.o
  AR      sound/usb/6fire/built-in.a
  AR      sound/usb/hiface/built-in.a
  CC      crypto/kpp.o
  AR      sound/usb/bcd2000/built-in.a
  AR      sound/usb/built-in.a
  AR      fs/jbd2/built-in.a
  CC      net/sunrpc/xprtsock.o
  CC      arch/x86/kernel/head32.o
  CC      net/core/dev_addr_lists.o
  CC      lib/decompress_unlzma.o
  CC      fs/proc/thread_self.o
  CC      net/netfilter/nf_conntrack_extend.o
  CC      net/core/dst.o
  CC      drivers/acpi/acpica/evmisc.o
  CC      net/core/netevent.o
  CC      kernel/futex/pi.o
  CC      fs/nfs/client.o
  CC      fs/isofs/dir.o
  CC      fs/isofs/util.o
  CC      kernel/cgroup/cgroup.o
  CC      net/netfilter/nf_conntrack_acct.o
  CC      fs/ext4/file.o
  CC      io_uring/advise.o
  CC      net/xfrm/xfrm_algo.o
  CC      lib/decompress_unlzo.o
  CC      drivers/acpi/acpica/evregion.o
  CC      kernel/module/kallsyms.o
  CC      drivers/pci/setup-bus.o
  CC      net/ipv4/inet_connection_sock.o
  CC      block/ioctl.o
  CC      drivers/pci/vc.o
  CC      fs/nfs/dir.o
  CC      drivers/dma/dw/core.o
  CC      fs/proc/proc_sysctl.o
  CC      net/wireless/sysfs.o
  CC      mm/slab_common.o
  CC      fs/proc/proc_net.o
  CC      security/selinux/ss/conditional.o
  CC      sound/core/memalloc.o
  CC      net/ethtool/cabletest.o
  CC      arch/x86/kernel/ebda.o
  CC      fs/exportfs/expfs.o
  AR      sound/sparc/built-in.a
  ASN.1   crypto/rsapubkey.asn1.[ch]
  ASN.1   crypto/rsaprivkey.asn1.[ch]
  CC      drivers/dma/dw/dw.o
  CC      crypto/rsa.o
  CC      arch/x86/events/intel/cstate.o
  CC      drivers/acpi/utils.o
  CC      fs/netfs/write_retry.o
  CC      drivers/acpi/acpica/evrgnini.o
  AR      drivers/soc/apple/built-in.a
  AR      drivers/soc/aspeed/built-in.a
  AR      drivers/soc/bcm/built-in.a
  AR      drivers/soc/fsl/built-in.a
  AR      drivers/soc/fujitsu/built-in.a
  AR      drivers/soc/hisilicon/built-in.a
  CC      kernel/futex/requeue.o
  CC      lib/decompress_unxz.o
  CC      drivers/virtio/virtio.o
  AR      drivers/soc/imx/built-in.a
  CC      net/sunrpc/auth_gss/gss_krb5_seal.o
  AR      drivers/soc/ixp4xx/built-in.a
  CC      fs/isofs/rock.o
  AR      drivers/soc/loongson/built-in.a
  CC      crypto/rsa_helper.o
  AR      drivers/soc/mediatek/built-in.a
  AR      drivers/soc/microchip/built-in.a
  CC      drivers/acpi/acpica/evsci.o
  AR      drivers/soc/nuvoton/built-in.a
  AR      drivers/soc/pxa/built-in.a
  AR      drivers/soc/amlogic/built-in.a
  AR      drivers/soc/qcom/built-in.a
  CC      net/netfilter/nf_conntrack_seqadj.o
  AR      drivers/soc/renesas/built-in.a
  CC      io_uring/epoll.o
  AR      drivers/soc/rockchip/built-in.a
  AR      drivers/soc/sunxi/built-in.a
  AR      drivers/soc/ti/built-in.a
  AR      drivers/soc/versatile/built-in.a
  AR      sound/pci/hda/built-in.a
  CC      net/sunrpc/sched.o
  AR      drivers/soc/xilinx/built-in.a
  AR      sound/pci/nm256/built-in.a
  CC      io_uring/statx.o
  AR      drivers/soc/built-in.a
  AR      sound/pci/oxygen/built-in.a
  CC      kernel/module/procfs.o
  AR      sound/pci/pcxhr/built-in.a
  AR      sound/pci/riptide/built-in.a
  CC      kernel/futex/waitwake.o
  CC      net/sunrpc/auth_gss/gss_krb5_unseal.o
  AR      sound/pci/rme9652/built-in.a
  AR      sound/pci/trident/built-in.a
  CC      kernel/module/sysfs.o
  AR      sound/pci/ymfpci/built-in.a
  AR      sound/pci/vx222/built-in.a
  AR      sound/pci/built-in.a
  CC      fs/isofs/export.o
  AR      fs/exportfs/built-in.a
  CC      net/netlabel/netlabel_user.o
  AR      net/mac80211/tests/built-in.a
  CC      net/rfkill/core.o
  CC      net/mac80211/main.o
  CC      net/mac80211/status.o
  CC      fs/ext4/fsmap.o
  CC      net/9p/mod.o
  CC      net/dns_resolver/dns_key.o
  CC      net/handshake/alert.o
  CC      kernel/time/sleep_timeout.o
  CC      crypto/rsa-pkcs1pad.o
  CC      drivers/acpi/acpica/evxface.o
  CC      block/genhd.o
  CC      sound/core/pcm_timer.o
  CC      lib/decompress_unzstd.o
  CC      net/xfrm/xfrm_user.o
  CC      block/ioprio.o
  CC      net/9p/client.o
  AR      arch/x86/events/intel/built-in.a
  AR      arch/x86/events/built-in.a
  CC      drivers/dma/dw/idma32.o
  CC      drivers/dma/dw/acpi.o
  CC      net/ethtool/tunnels.o
  CC      net/dns_resolver/dns_query.o
  CC      drivers/virtio/virtio_ring.o
  AR      fs/netfs/built-in.a
  CC      fs/isofs/joliet.o
  CC      net/core/neighbour.o
  CC      security/selinux/ss/mls.o
  CC      security/selinux/ss/context.o
  CC      kernel/cgroup/rstat.o
  CC      fs/nfs/file.o
  CC      kernel/time/timekeeping.o
  CC      lib/dump_stack.o
  CC      fs/isofs/compress.o
  CC      drivers/acpi/acpica/evxfevnt.o
  CC      io_uring/timeout.o
  CC      kernel/trace/trace_clock.o
  AR      kernel/futex/built-in.a
  AR      kernel/module/built-in.a
  CC      fs/lockd/clntlock.o
  CC      kernel/time/ntp.o
  CC      fs/lockd/clntproc.o
  CC      arch/x86/kernel/cpu/powerflags.o
  CC      fs/proc/kcore.o
  CC      kernel/trace/ring_buffer.o
  CC      crypto/rsassa-pkcs1.o
  CC      drivers/pci/mmap.o
  CC      net/sunrpc/auth.o
  CC      sound/core/seq_device.o
  CC      net/sunrpc/auth_gss/gss_krb5_wrap.o
  CC      net/rfkill/input.o
  CC      block/badblocks.o
  CC      net/netlabel/netlabel_kapi.o
  AR      drivers/dma/dw/built-in.a
  CC      drivers/dma/hsu/hsu.o
  CC      drivers/acpi/acpica/evxfgpe.o
  CC      kernel/cgroup/namespace.o
  CC      net/mac80211/driver-ops.o
  CC      net/netfilter/nf_conntrack_proto_icmpv6.o
  CC      net/ipv4/tcp.o
  AR      net/dns_resolver/built-in.a
  CC      fs/lockd/clntxdr.o
  CC      arch/x86/kernel/platform-quirks.o
  CC      lib/earlycpio.o
  CC      mm/compaction.o
  CC      mm/show_mem.o
  CC      net/wireless/radiotap.o
  CC      net/handshake/genl.o
  CC      lib/extable.o
  AR      sound/core/built-in.a
  AR      sound/spi/built-in.a
  AR      fs/isofs/built-in.a
  AR      sound/parisc/built-in.a
  CC      fs/lockd/host.o
  CC      drivers/pci/devres.o
  CC      drivers/virtio/virtio_anchor.o
  AR      sound/pcmcia/vx/built-in.a
  AR      sound/pcmcia/pdaudiocf/built-in.a
  AR      sound/pcmcia/built-in.a
  CC      crypto/acompress.o
  CC      net/netlabel/netlabel_domainhash.o
  AR      sound/mips/built-in.a
  AR      sound/soc/built-in.a
  AR      sound/atmel/built-in.a
  CC      drivers/acpi/acpica/evxfregn.o
  CC      net/core/rtnetlink.o
  CC      sound/hda/hda_bus_type.o
  AR      net/rfkill/built-in.a
  CC      net/ethtool/fec.o
  CC      arch/x86/kernel/cpu/topology.o
  CC      fs/ext4/fsync.o
  CC      net/sunrpc/auth_gss/gss_krb5_crypto.o
  CC      io_uring/fdinfo.o
  CC      net/sunrpc/auth_gss/gss_krb5_keys.o
  CC      security/selinux/netlabel.o
  CC      fs/proc/vmcore.o
  CC      lib/flex_proportions.o
  CC      fs/nls/nls_base.o
  CC      fs/nls/nls_cp437.o
  CC      block/blk-rq-qos.o
  AR      drivers/dma/hsu/built-in.a
  CC      drivers/acpi/acpica/exconcat.o
  AR      drivers/dma/idxd/built-in.a
  AR      drivers/dma/amd/built-in.a
  AR      drivers/dma/mediatek/built-in.a
  AR      drivers/dma/qcom/built-in.a
  AR      drivers/dma/stm32/built-in.a
  AR      drivers/dma/ti/built-in.a
  AR      drivers/dma/xilinx/built-in.a
  CC      drivers/dma/dmaengine.o
  CC      net/9p/error.o
  CC      fs/nls/nls_ascii.o
  CC      net/ipv6/route.o
  CC      kernel/cgroup/cgroup-v1.o
  CC      kernel/cgroup/freezer.o
  CC      kernel/cgroup/legacy_freezer.o
  CC      net/handshake/netlink.o
  CC      kernel/time/clocksource.o
  CC      lib/idr.o
  CC      drivers/virtio/virtio_pci_modern_dev.o
  CC      sound/hda/hdac_bus.o
  CC      kernel/cgroup/pids.o
  CC      net/mac80211/sta_info.o
  CC      block/disk-events.o
  CC      net/ipv6/ip6_fib.o
  CC      drivers/pci/proc.o
  CC      crypto/scompress.o
  CC      block/blk-ia-ranges.o
  CC      net/wireless/util.o
  CC      drivers/acpi/acpica/exconfig.o
  CC      net/mac80211/wep.o
  CC      net/ethtool/eeprom.o
  CC      fs/nfs/getroot.o
  CC      net/netfilter/nf_conntrack_netlink.o
  CC      fs/nls/nls_iso8859-1.o
  CC      net/9p/protocol.o
  CC      io_uring/cancel.o
  CC      kernel/trace/trace.o
  CC      fs/ext4/hash.o
  CC      drivers/virtio/virtio_pci_legacy_dev.o
  CC      fs/lockd/svc.o
  CC      arch/x86/kernel/cpu/proc.o
  CC      fs/nls/nls_utf8.o
  CC      lib/iomem_copy.o
  CC      fs/proc/kmsg.o
  CC      drivers/acpi/acpica/exconvrt.o
  CC      lib/irq_regs.o
  AR      net/xfrm/built-in.a
  CC      fs/proc/page.o
  CC      net/netlabel/netlabel_addrlist.o
  CC      io_uring/waitid.o
  CC      net/mac80211/aead_api.o
  CC      net/wireless/reg.o
  CC      net/netfilter/nf_conntrack_ftp.o
  CC      sound/hda/hdac_device.o
  AR      net/sunrpc/auth_gss/built-in.a
  CC      kernel/trace/trace_output.o
  AR      sound/x86/built-in.a
  CC      drivers/dma/virt-dma.o
  CC      io_uring/register.o
  CC      lib/is_single_threaded.o
  CC      fs/nfs/inode.o
  AR      sound/xen/built-in.a
  CC      arch/x86/kernel/cpu/feat_ctl.o
  CC      kernel/time/jiffies.o
  CC      block/early-lookup.o
  AR      security/selinux/built-in.a
  CC      drivers/pci/pci-sysfs.o
  AR      fs/nls/built-in.a
  AR      security/built-in.a
  CC      net/wireless/scan.o
  CC      net/handshake/request.o
  CC      crypto/algboss.o
  CC      net/devres.o
  CC      net/ipv4/tcp_input.o
  CC      sound/hda/hdac_sysfs.o
  CC      net/9p/trans_common.o
  CC      drivers/acpi/acpica/excreate.o
  CC      drivers/acpi/reboot.o
  CC      drivers/virtio/virtio_pci_modern.o
  CC      fs/ext4/ialloc.o
  CC      fs/ext4/indirect.o
  CC      kernel/cgroup/rdma.o
  CC      net/ethtool/stats.o
  CC      lib/klist.o
  AR      sound/virtio/built-in.a
  CC      arch/x86/kernel/cpu/intel.o
  CC      net/ipv4/tcp_output.o
  CC      net/socket.o
  CC      kernel/time/timer_list.o
  CC      kernel/time/timeconv.o
  CC      kernel/cgroup/cpuset.o
  CC      net/core/utils.o
  CC      drivers/acpi/nvs.o
  CC      arch/x86/kernel/process_32.o
  AR      fs/proc/built-in.a
  CC      net/ipv6/ipv6_sockglue.o
  CC      drivers/dma/acpi-dma.o
  CC      drivers/acpi/acpica/exdebug.o
  CC      lib/kobject.o
  AR      fs/unicode/built-in.a
  CC      net/ethtool/phc_vclocks.o
  CC      block/bounce.o
  CC      net/sunrpc/auth_null.o
  CC      mm/interval_tree.o
  CC      net/9p/trans_fd.o
  CC      net/ipv6/ndisc.o
  CC      fs/lockd/svclock.o
  CC      net/sysctl_net.o
  CC      net/netlabel/netlabel_mgmt.o
  CC      sound/sound_core.o
  CC      drivers/acpi/acpica/exdump.o
  CC      sound/hda/hdac_regmap.o
  CC      fs/ext4/inline.o
  CC      arch/x86/kernel/cpu/tsx.o
  CC      drivers/pci/slot.o
  CC      crypto/testmgr.o
  CC      kernel/time/timecounter.o
  CC      drivers/virtio/virtio_pci_common.o
  CC      drivers/virtio/virtio_pci_legacy.o
  CC      kernel/time/alarmtimer.o
  CC      crypto/cmac.o
  CC      fs/lockd/svcshare.o
  CC      lib/kobject_uevent.o
  CC      kernel/cgroup/misc.o
  CC      net/netlabel/netlabel_unlabeled.o
  CC      drivers/acpi/acpica/exfield.o
  AR      drivers/dma/built-in.a
  CC      drivers/acpi/acpica/exfldio.o
  CC      net/netlabel/netlabel_cipso_v4.o
  CC      net/9p/trans_virtio.o
  CC      net/handshake/tlshd.o
  CC      arch/x86/kernel/cpu/intel_epb.o
  CC      io_uring/truncate.o
  CC      net/wireless/nl80211.o
  CC      net/netfilter/nf_conntrack_irc.o
  CC      net/ethtool/mm.o
  CC      net/sunrpc/auth_tls.o
  CC      mm/list_lru.o
  CC      kernel/cgroup/debug.o
  CC      crypto/hmac.o
  CC      net/core/link_watch.o
  CC      block/bsg.o
  CC      crypto/crypto_null.o
  CC      drivers/pci/pci-acpi.o
  CC      drivers/tty/vt/vt_ioctl.o
  CC      sound/hda/hdac_controller.o
  CC      drivers/acpi/acpica/exmisc.o
  CC      net/core/filter.o
  CC      net/netlabel/netlabel_calipso.o
  CC      arch/x86/kernel/cpu/amd.o
  CC      drivers/tty/vt/vc_screen.o
  CC      drivers/virtio/virtio_pci_admin_legacy_io.o
  CC      drivers/tty/vt/selection.o
  CC      kernel/bpf/core.o
  CC      drivers/pci/iomap.o
  CC      net/sunrpc/auth_unix.o
  CC      drivers/acpi/acpica/exmutex.o
  CC      fs/lockd/svcproc.o
  CC      fs/lockd/svcsubs.o
  CC      io_uring/memmap.o
  CC      net/ipv6/udp.o
  CC      kernel/time/posix-timers.o
  CC      mm/workingset.o
  CC      block/blk-cgroup.o
  CC      crypto/md5.o
  CC      lib/logic_pio.o
  CC      arch/x86/kernel/cpu/hygon.o
  CC      kernel/trace/trace_seq.o
  CC      drivers/tty/hvc/hvc_console.o
  AR      kernel/cgroup/built-in.a
  CC      net/ipv4/tcp_timer.o
  CC      net/ethtool/module.o
  CC      fs/nfs/super.o
  CC      net/handshake/trace.o
  CC      kernel/time/posix-cpu-timers.o
  CC      fs/ext4/inode.o
  CC      lib/maple_tree.o
  CC      drivers/acpi/acpica/exnames.o
  AR      net/9p/built-in.a
  CC      lib/memcat_p.o
  CC      arch/x86/kernel/signal.o
  CC      sound/hda/hdac_stream.o
  CC      net/netfilter/nf_conntrack_sip.o
  CC      net/netfilter/nf_nat_core.o
  CC      drivers/virtio/virtio_input.o
  CC      net/netfilter/nf_nat_proto.o
  CC      net/mac80211/wpa.o
  CC      drivers/char/hw_random/core.o
  AR      drivers/iommu/amd/built-in.a
  AR      drivers/iommu/intel/built-in.a
  CC      drivers/tty/vt/keyboard.o
  AR      drivers/iommu/arm/arm-smmu/built-in.a
  AR      drivers/gpu/host1x/built-in.a
  CC      drivers/char/agp/backend.o
  CC      arch/x86/kernel/cpu/centaur.o
  AR      drivers/iommu/arm/arm-smmu-v3/built-in.a
  AR      drivers/gpu/vga/built-in.a
  AR      drivers/iommu/arm/built-in.a
  CC      drivers/char/mem.o
  AR      drivers/iommu/iommufd/built-in.a
  CC      kernel/time/posix-clock.o
  AR      drivers/iommu/riscv/built-in.a
  CC      drivers/pci/quirks.o
  CC      drivers/iommu/iommu.o
  CC      crypto/sha256_generic.o
  AR      net/netlabel/built-in.a
  CC      drivers/connector/cn_queue.o
  AR      drivers/gpu/drm/tests/built-in.a
  CC      drivers/base/power/sysfs.o
  CC      net/netfilter/nf_nat_helper.o
  AR      drivers/gpu/drm/arm/built-in.a
  CC      sound/last.o
  AR      drivers/gpu/drm/clients/built-in.a
  CC      net/ethtool/cmis_fw_update.o
  CC      drivers/gpu/drm/display/drm_display_helper_mod.o
  CC      drivers/acpi/acpica/exoparg1.o
  CC      io_uring/alloc_cache.o
  CC      drivers/block/loop.o
  CC      mm/debug.o
  CC      drivers/base/firmware_loader/builtin/main.o
  CC      net/sunrpc/svc.o
  CC      arch/x86/kernel/cpu/transmeta.o
  AR      drivers/tty/hvc/built-in.a
  CC      drivers/char/agp/generic.o
  CC      drivers/gpu/drm/display/drm_dp_dual_mode_helper.o
  CC      drivers/virtio/virtio_dma_buf.o
  CC      kernel/trace/trace_stat.o
  CC      drivers/connector/connector.o
  CC      fs/lockd/mon.o
  CC      crypto/sha512_generic.o
  CC      drivers/char/random.o
  CC      drivers/connector/cn_proc.o
  CC      drivers/acpi/acpica/exoparg2.o
  CC      mm/gup.o
  CC      drivers/base/power/generic_ops.o
  AR      drivers/base/firmware_loader/builtin/built-in.a
  CC      net/netfilter/nf_nat_masquerade.o
  CC      drivers/char/hw_random/intel-rng.o
  CC      drivers/base/firmware_loader/main.o
  CC      net/ethtool/cmis_cdb.o
  CC      fs/lockd/trace.o
  CC      net/ipv6/udplite.o
  CC      sound/hda/array.o
  CC      io_uring/io-wq.o
  CC      kernel/time/itimer.o
  CC      drivers/pci/pci-label.o
  CC      fs/autofs/init.o
  CC      block/blk-ioprio.o
  CC      arch/x86/kernel/cpu/zhaoxin.o
  CC      drivers/iommu/iommu-traces.o
  CC      drivers/acpi/acpica/exoparg3.o
  CC      drivers/tty/vt/vt.o
  AR      net/handshake/built-in.a
  CC      drivers/char/misc.o
  CC      fs/nfs/io.o
  AR      drivers/virtio/built-in.a
  CC      drivers/char/hw_random/amd-rng.o
  CC      drivers/misc/eeprom/eeprom_93cx6.o
  CC      drivers/base/power/common.o
  CC      kernel/time/clockevents.o
  CC      drivers/block/virtio_blk.o
  CC      drivers/gpu/drm/display/drm_dp_helper.o
  CC      kernel/trace/trace_printk.o
  CC      net/mac80211/scan.o
  CC      drivers/acpi/wakeup.o
  CC      crypto/sha3_generic.o
  CC      fs/lockd/xdr.o
  CC      drivers/base/regmap/regmap.o
  CC      arch/x86/kernel/cpu/vortex.o
  CC      fs/autofs/inode.o
  CC      sound/hda/hdmi_chmap.o
  CC      drivers/char/agp/isoch.o
  CC      drivers/acpi/acpica/exoparg6.o
  AR      drivers/misc/eeprom/built-in.a
  AR      drivers/misc/cb710/built-in.a
  AR      drivers/misc/lis3lv02d/built-in.a
  CC      drivers/acpi/sleep.o
  AR      drivers/misc/cardreader/built-in.a
  AR      drivers/misc/keba/built-in.a
  COPY    drivers/tty/vt/defkeymap.c
  CC      drivers/acpi/device_sysfs.o
  CC      drivers/char/hw_random/geode-rng.o
  AR      drivers/misc/built-in.a
  CC      fs/9p/vfs_super.o
  CC      drivers/base/power/qos.o
  AR      drivers/base/firmware_loader/built-in.a
  AR      fs/hostfs/built-in.a
  CC      block/blk-iolatency.o
  CC      net/sunrpc/svcsock.o
  CC      drivers/char/virtio_console.o
  AR      drivers/connector/built-in.a
  CC      fs/autofs/root.o
  CC      fs/debugfs/inode.o
  CC      drivers/gpu/drm/ttm/ttm_tt.o
  CC      net/ipv4/tcp_ipv4.o
  CC      kernel/time/tick-common.o
  CC      drivers/gpu/drm/ttm/ttm_bo.o
  CC      net/netfilter/nf_nat_ftp.o
  CC      arch/x86/kernel/cpu/perfctr-watchdog.o
  CC      net/ethtool/pse-pd.o
  CC      net/ipv6/raw.o
  CC      drivers/tty/vt/consolemap.o
  CC      block/blk-iocost.o
  CC      net/ipv6/icmp.o
  CC      drivers/acpi/acpica/exprep.o
  CC      crypto/ecb.o
  CC      net/mac80211/offchannel.o
  CC      drivers/gpu/drm/ttm/ttm_bo_util.o
  CC      crypto/cbc.o
  AR      kernel/bpf/built-in.a
  CC      fs/lockd/clnt4xdr.o
  CC      drivers/gpu/drm/ttm/ttm_bo_vm.o
  CC      drivers/iommu/iommu-sysfs.o
  CC      drivers/pci/vgaarb.o
  CC      fs/nfs/direct.o
  CC      io_uring/futex.o
  CC      kernel/trace/pid_list.o
  CC      drivers/char/hw_random/via-rng.o
  CC      drivers/acpi/acpica/exregion.o
  CC      drivers/char/agp/amd64-agp.o
  AR      drivers/gpu/drm/renesas/rcar-du/built-in.a
  AR      drivers/gpu/drm/renesas/rz-du/built-in.a
  AR      drivers/gpu/drm/renesas/built-in.a
  CC      drivers/gpu/drm/i915/i915_config.o
  CC      lib/nmi_backtrace.o
  AR      drivers/mfd/built-in.a
  CC      fs/9p/vfs_inode.o
  CC      drivers/base/power/runtime.o
  CC      crypto/ctr.o
  CC      sound/hda/trace.o
  CC      arch/x86/kernel/cpu/vmware.o
  AR      drivers/block/built-in.a
  CC      kernel/events/core.o
  CC      drivers/gpu/drm/i915/i915_driver.o
  CC      fs/lockd/xdr4.o
  CC      fs/debugfs/file.o
  AR      drivers/char/hw_random/built-in.a
  AR      drivers/base/test/built-in.a
  CC      drivers/acpi/acpica/exresnte.o
  CC      net/ethtool/plca.o
  CC      fs/tracefs/inode.o
  CC      mm/mmap_lock.o
  CC      drivers/iommu/dma-iommu.o
  CC      fs/autofs/symlink.o
  CC [M]  fs/efivarfs/inode.o
  CC [M]  fs/efivarfs/file.o
  CC      drivers/iommu/iova.o
  CC      fs/tracefs/event_inode.o
  HOSTCC  drivers/tty/vt/conmakehash
  CC      mm/highmem.o
  CC      drivers/gpu/drm/ttm/ttm_module.o
  CC      kernel/time/tick-broadcast.o
  CC      crypto/gcm.o
  CC      net/netfilter/nf_nat_irc.o
  CC      mm/memory.o
  CC      net/ipv4/tcp_minisocks.o
  CC      arch/x86/kernel/signal_32.o
  CC      drivers/gpu/drm/display/drm_dp_mst_topology.o
  CC      drivers/gpu/drm/ttm/ttm_execbuf_util.o
  CC      drivers/acpi/acpica/exresolv.o
  CC      drivers/char/agp/intel-agp.o
  CC      net/core/sock_diag.o
  CC      kernel/trace/trace_sched_switch.o
  CC      arch/x86/kernel/cpu/hypervisor.o
  CC      io_uring/napi.o
  AR      drivers/pci/built-in.a
  CC      drivers/tty/serial/8250/8250_core.o
  CC      kernel/events/ring_buffer.o
  CC      mm/mincore.o
  CC      drivers/char/agp/intel-gtt.o
  CC [M]  fs/efivarfs/super.o
  CC      fs/autofs/waitq.o
  CC      sound/hda/hdac_component.o
  CC      drivers/base/power/wakeirq.o
  CC      arch/x86/kernel/cpu/mshyperv.o
  CC      drivers/acpi/acpica/exresop.o
  CC      kernel/time/tick-broadcast-hrtimer.o
  CC      fs/9p/vfs_inode_dotl.o
  CC      drivers/tty/vt/defkeymap.o
  CC      net/core/dev_ioctl.o
  AR      drivers/tty/ipwireless/built-in.a
  CC      net/ipv6/mcast.o
  CC      drivers/char/hpet.o
  CC      fs/ext4/ioctl.o
  CONMK   drivers/tty/vt/consolemap_deftbl.c
  CC      drivers/tty/vt/consolemap_deftbl.o
  CC      net/ethtool/phy.o
  CC      drivers/gpu/drm/ttm/ttm_range_manager.o
  AR      drivers/tty/vt/built-in.a
  AR      fs/tracefs/built-in.a
  CC      drivers/base/power/main.o
  CC      drivers/base/power/wakeup.o
  CC      drivers/base/power/wakeup_stats.o
  AR      fs/debugfs/built-in.a
  CC      drivers/base/power/trace.o
  CC      drivers/acpi/acpica/exserial.o
  CC      fs/lockd/svc4proc.o
  CC      drivers/base/component.o
  CC      drivers/gpu/drm/i915/i915_drm_client.o
  CC      drivers/gpu/drm/i915/i915_getparam.o
  CC      crypto/ccm.o
  CC [M]  fs/efivarfs/vars.o
  CC      lib/objpool.o
  CC      kernel/time/tick-oneshot.o
  CC      fs/lockd/procfs.o
  CC      drivers/base/regmap/regcache.o
  CC      net/netfilter/nf_nat_sip.o
  CC      net/mac80211/ht.o
  CC      kernel/trace/trace_nop.o
  CC      sound/hda/hdac_i915.o
  CC      drivers/acpi/acpica/exstore.o
  CC      net/sunrpc/svcauth.o
  CC      fs/nfs/pagelist.o
  CC      fs/autofs/expire.o
  CC      sound/hda/intel-dsp-config.o
  CC      net/mac80211/agg-tx.o
  AR      drivers/iommu/built-in.a
  CC      net/netfilter/x_tables.o
  CC      drivers/tty/serial/8250/8250_platform.o
  AR      drivers/nfc/built-in.a
  CC      net/netfilter/xt_tcpudp.o
  CC      drivers/base/regmap/regcache-rbtree.o
  CC      fs/autofs/dev-ioctl.o
  AR      drivers/char/agp/built-in.a
  CC      arch/x86/kernel/cpu/debugfs.o
  CC      lib/plist.o
  CC      kernel/time/tick-sched.o
  CC      kernel/time/timer_migration.o
  AR      drivers/gpu/drm/omapdrm/built-in.a
  CC      sound/hda/intel-nhlt.o
  CC      drivers/gpu/drm/ttm/ttm_resource.o
  CC      block/mq-deadline.o
  CC      lib/radix-tree.o
  CC      fs/9p/vfs_addr.o
  CC      net/ipv4/tcp_cong.o
  CC      drivers/char/nvram.o
  CC      net/sunrpc/svcauth_unix.o
  CC      fs/9p/vfs_file.o
  CC      drivers/acpi/acpica/exstoren.o
  AR      io_uring/built-in.a
  CC      fs/9p/vfs_dir.o
  CC      drivers/gpu/drm/ttm/ttm_pool.o
  LD [M]  fs/efivarfs/efivarfs.o
  CC      arch/x86/kernel/traps.o
  CC      fs/ext4/mballoc.o
  CC      net/ethtool/tsconfig.o
  CC      drivers/tty/serial/serial_core.o
  CC      crypto/aes_generic.o
  CC      block/kyber-iosched.o
  CC      drivers/acpi/device_pm.o
  CC      drivers/gpu/drm/display/drm_dsc_helper.o
  CC      kernel/trace/blktrace.o
  CC      drivers/tty/serial/8250/8250_pnp.o
  CC      drivers/acpi/acpica/exstorob.o
  CC      arch/x86/kernel/cpu/bus_lock.o
  AR      fs/lockd/built-in.a
  CC      sound/hda/intel-sdw-acpi.o
  AR      drivers/dax/hmem/built-in.a
  CC      net/mac80211/agg-rx.o
  AR      drivers/dax/built-in.a
  CC      drivers/tty/tty_io.o
  CC      drivers/gpu/drm/i915/i915_ioctl.o
  CC      mm/mlock.o
  CC      drivers/gpu/drm/ttm/ttm_device.o
  CC      drivers/base/regmap/regcache-flat.o
  AR      fs/autofs/built-in.a
  CC      net/ipv4/tcp_metrics.o
  CC      drivers/acpi/acpica/exsystem.o
  AR      drivers/base/power/built-in.a
  CC      fs/ext4/migrate.o
  AR      drivers/gpu/drm/tilcdc/built-in.a
  CC      arch/x86/kernel/idt.o
  CC      crypto/authenc.o
  CC      drivers/acpi/proc.o
  CC      fs/9p/vfs_dentry.o
  CC      net/core/tso.o
  CC      kernel/fork.o
  AR      drivers/char/built-in.a
  CC      kernel/exec_domain.o
  CC      drivers/dma-buf/dma-buf.o
  CC      lib/ratelimit.o
  CC      drivers/gpu/drm/virtio/virtgpu_drv.o
  AR      drivers/gpu/drm/imx/built-in.a
  CC      drivers/acpi/acpica/extrace.o
  CC      drivers/gpu/drm/display/drm_hdcp_helper.o
  AR      sound/hda/built-in.a
  CC      net/mac80211/vht.o
  AR      sound/built-in.a
  CC      drivers/gpu/drm/virtio/virtgpu_kms.o
  CC      kernel/trace/trace_events.o
  CC      lib/rbtree.o
  CC      drivers/tty/serial/8250/8250_rsa.o
  CC      drivers/base/regmap/regcache-maple.o
  CC      drivers/gpu/drm/ttm/ttm_sys_manager.o
  CC      kernel/trace/trace_export.o
  CC      net/ipv6/reassembly.o
  CC      net/ipv4/tcp_fastopen.o
  AR      net/ethtool/built-in.a
  CC      arch/x86/kernel/cpu/capflags.o
  CC      net/wireless/mlme.o
  AR      arch/x86/kernel/cpu/built-in.a
  CC      fs/9p/v9fs.o
  CC      drivers/base/core.o
  CC      fs/ext4/mmp.o
  CC      drivers/acpi/acpica/exutils.o
  CC      drivers/gpu/drm/i915/i915_irq.o
  CC      fs/open.o
  CC      kernel/time/vsyscall.o
  CC      drivers/tty/n_tty.o
  CC      lib/seq_buf.o
  CC      drivers/base/regmap/regmap-debugfs.o
  AR      drivers/gpu/drm/i2c/built-in.a
  CC      net/sunrpc/addr.o
  CC      drivers/tty/serial/serial_base_bus.o
  CC      arch/x86/kernel/irq.o
  CC      net/netfilter/xt_CONNSECMARK.o
  CC      drivers/gpu/drm/display/drm_hdmi_helper.o
  CC      drivers/gpu/drm/ttm/ttm_agp_backend.o
  CC      drivers/tty/tty_ioctl.o
  CC      drivers/dma-buf/dma-fence.o
  CC      fs/nfs/read.o
  CC      fs/read_write.o
  CC      crypto/authencesn.o
  CC      kernel/events/callchain.o
  CC      fs/nfs/symlink.o
  CC      drivers/tty/serial/8250/8250_port.o
  CC      block/blk-mq-debugfs.o
  CC      drivers/acpi/acpica/hwacpi.o
  CC      fs/ext4/move_extent.o
  CC      kernel/trace/trace_event_perf.o
  CC      kernel/time/timekeeping_debug.o
  CC      net/wireless/ibss.o
  CC      net/core/sock_reuseport.o
  CC      drivers/gpu/drm/virtio/virtgpu_gem.o
  CC      drivers/gpu/drm/display/drm_scdc_helper.o
  CC      mm/mmap.o
  CC      lib/siphash.o
  CC      kernel/panic.o
  CC      net/netfilter/xt_NFLOG.o
  CC      drivers/base/bus.o
  CC      net/wireless/sme.o
  CC      fs/9p/fid.o
  CC      drivers/acpi/acpica/hwesleep.o
  CC      drivers/gpu/drm/i915/i915_mitigations.o
  AR      drivers/gpu/drm/panel/built-in.a
  CC      fs/file_table.o
  CC      drivers/gpu/drm/virtio/virtgpu_vram.o
  CC      drivers/base/dd.o
  AR      drivers/gpu/drm/ttm/built-in.a
  AR      drivers/base/regmap/built-in.a
  AR      drivers/gpu/drm/hisilicon/built-in.a
  AR      drivers/gpu/drm/bridge/analogix/built-in.a
  CC      drivers/tty/serial/serial_ctrl.o
  AR      drivers/gpu/drm/bridge/cadence/built-in.a
  AR      drivers/gpu/drm/bridge/imx/built-in.a
  AR      drivers/gpu/drm/bridge/synopsys/built-in.a
  AR      drivers/gpu/drm/bridge/built-in.a
  CC      lib/string.o
  CC      net/ipv4/tcp_rate.o
  CC      fs/ext4/namei.o
  CC      drivers/acpi/acpica/hwgpe.o
  CC      fs/9p/xattr.o
  CC      drivers/gpu/drm/i915/i915_module.o
  CC      net/sunrpc/rpcb_clnt.o
  CC      kernel/time/namespace.o
  CC      net/ipv6/tcp_ipv6.o
  CC      lib/timerqueue.o
  CC      drivers/tty/serial/8250/8250_dma.o
  CC      drivers/dma-buf/dma-fence-array.o
  CC      kernel/cpu.o
  AR      drivers/gpu/drm/display/built-in.a
  CC      crypto/lzo.o
  CC      drivers/gpu/drm/i915/i915_params.o
  CC      drivers/acpi/bus.o
  CC      block/blk-pm.o
  CC      drivers/gpu/drm/virtio/virtgpu_display.o
  CC      lib/union_find.o
  CC      kernel/exit.o
  CC      fs/super.o
  CC      net/sunrpc/timer.o
  CC      kernel/softirq.o
  CC      lib/vsprintf.o
  CC      kernel/events/hw_breakpoint.o
  CC      drivers/dma-buf/dma-fence-chain.o
  CC      kernel/events/uprobes.o
  CC      kernel/resource.o
  CC      kernel/trace/trace_events_filter.o
  AR      drivers/cxl/core/built-in.a
  CC      net/netfilter/xt_SECMARK.o
  CC      kernel/trace/trace_events_trigger.o
  AR      drivers/cxl/built-in.a
  CC      kernel/sysctl.o
  CC      drivers/acpi/acpica/hwregs.o
  CC      net/mac80211/he.o
  CC      kernel/trace/trace_eprobe.o
  CC      kernel/trace/trace_kprobe.o
  CC      kernel/trace/error_report-traces.o
  CC      drivers/macintosh/mac_hid.o
  AR      fs/9p/built-in.a
  CC      net/core/fib_notifier.o
  CC      arch/x86/kernel/irq_32.o
  CC      kernel/trace/power-traces.o
  CC      kernel/trace/rpm-traces.o
  CC      crypto/lzo-rle.o
  CC      block/holder.o
  CC      drivers/tty/tty_ldisc.o
  AR      kernel/time/built-in.a
  CC      arch/x86/kernel/dumpstack_32.o
  CC      drivers/acpi/acpica/hwsleep.o
  CC      fs/nfs/unlink.o
  CC      kernel/trace/trace_dynevent.o
  CC      drivers/gpu/drm/virtio/virtgpu_vq.o
  CC      mm/mmu_gather.o
  CC      net/netfilter/xt_TCPMSS.o
  CC      drivers/dma-buf/dma-fence-unwrap.o
  CC      net/ipv4/tcp_recovery.o
  CC      kernel/trace/trace_probe.o
  CC      drivers/gpu/drm/i915/i915_pci.o
  CC      drivers/tty/serial/8250/8250_dwlib.o
  AR      drivers/macintosh/built-in.a
  CC      net/mac80211/s1g.o
  CC      drivers/tty/serial/8250/8250_pcilib.o
  CC      drivers/acpi/acpica/hwvalid.o
  CC      drivers/gpu/drm/i915/i915_scatterlist.o
  CC      kernel/capability.o
  CC      drivers/tty/tty_buffer.o
  CC      crypto/rng.o
  CC      drivers/base/syscore.o
  CC      drivers/dma-buf/dma-resv.o
  AR      block/built-in.a
  CC      fs/nfs/write.o
  CC      net/ipv4/tcp_ulp.o
  CC      lib/win_minmax.o
  CC      arch/x86/kernel/time.o
  CC      arch/x86/kernel/ioport.o
  CC      lib/xarray.o
  CC      drivers/tty/tty_port.o
  CC      drivers/acpi/glue.o
  CC      net/core/xdp.o
  CC      fs/char_dev.o
  CC      net/mac80211/ibss.o
  CC      net/mac80211/iface.o
  CC      drivers/acpi/acpica/hwxface.o
  AR      drivers/scsi/pcmcia/built-in.a
  CC      drivers/scsi/scsi.o
  CC      mm/mprotect.o
  CC      drivers/acpi/scan.o
  CC      drivers/gpu/drm/i915/i915_switcheroo.o
  CC      drivers/tty/serial/serial_port.o
  CC      kernel/ptrace.o
  CC      drivers/gpu/drm/virtio/virtgpu_fence.o
  CC      crypto/drbg.o
  CC      net/sunrpc/xdr.o
  CC      net/wireless/chan.o
  CC      drivers/tty/serial/8250/8250_early.o
  CC      drivers/scsi/hosts.o
  CC      drivers/tty/serial/earlycon.o
  CC      drivers/base/driver.o
  CC      drivers/dma-buf/sync_file.o
  CC      lib/lockref.o
  CC      net/netfilter/xt_conntrack.o
  CC      drivers/acpi/acpica/hwxfsleep.o
  AR      kernel/events/built-in.a
  CC      net/netfilter/xt_policy.o
  CC      crypto/jitterentropy.o
  CC      net/wireless/ethtool.o
  CC      net/ipv6/ping.o
  CC      drivers/scsi/scsi_ioctl.o
  CC      net/sunrpc/sunrpc_syms.o
  CC      net/core/flow_offload.o
  CC      lib/bcd.o
  CC      drivers/tty/tty_mutex.o
  CC      drivers/tty/serial/8250/8250_exar.o
  CC      drivers/tty/serial/8250/8250_lpss.o
  CC      drivers/tty/tty_ldsem.o
  CC      lib/sort.o
  CC      arch/x86/kernel/dumpstack.o
  CC      net/netfilter/xt_state.o
  CC      drivers/acpi/mipi-disco-img.o
  CC      drivers/tty/serial/8250/8250_mid.o
  CC      fs/nfs/namespace.o
  CC      drivers/gpu/drm/i915/i915_sysfs.o
  CC      drivers/base/class.o
  CC      drivers/acpi/acpica/hwpci.o
  CC      drivers/scsi/scsicam.o
  CC      kernel/user.o
  CC      net/mac80211/link.o
  CC      drivers/tty/serial/8250/8250_pci.o
  CC      lib/parser.o
  CC [M]  net/netfilter/nf_log_syslog.o
  CC      kernel/trace/trace_uprobe.o
  AR      drivers/dma-buf/built-in.a
  CC      net/ipv4/tcp_offload.o
  CC      kernel/trace/rethook.o
  CC [M]  net/netfilter/xt_mark.o
  CC      drivers/gpu/drm/virtio/virtgpu_object.o
  CC      crypto/jitterentropy-kcapi.o
  CC      drivers/acpi/resource.o
  CC      net/wireless/mesh.o
  CC      mm/mremap.o
  CC      net/ipv6/exthdrs.o
  CC      drivers/acpi/acpica/nsaccess.o
  AR      drivers/gpu/drm/mxsfb/built-in.a
  CC      net/mac80211/rate.o
  CC      fs/nfs/mount_clnt.o
  CC      fs/ext4/page-io.o
  CC      drivers/tty/serial/8250/8250_pericom.o
  CC      net/wireless/ap.o
  AR      drivers/nvme/common/built-in.a
  AR      drivers/nvme/host/built-in.a
  CC      net/wireless/trace.o
  CC      arch/x86/kernel/nmi.o
  AR      drivers/nvme/target/built-in.a
  CC      arch/x86/kernel/ldt.o
  AR      drivers/nvme/built-in.a
  CC      lib/debug_locks.o
  CC      net/mac80211/michael.o
  CC [M]  net/netfilter/xt_nat.o
  CC      fs/stat.o
  CC      net/mac80211/tkip.o
  CC      fs/nfs/nfstrace.o
  CC      fs/nfs/export.o
  CC      net/wireless/ocb.o
  CC      drivers/scsi/scsi_error.o
  CC      lib/random32.o
  CC      net/ipv4/tcp_plb.o
  CC      net/core/gro.o
  CC      net/wireless/pmsr.o
  AR      drivers/gpu/drm/tiny/built-in.a
  CC      drivers/scsi/scsi_lib.o
  CC      crypto/ghash-generic.o
  CC      drivers/acpi/acpica/nsalloc.o
  CC      drivers/base/platform.o
  CC      drivers/base/cpu.o
  CC      drivers/gpu/drm/i915/i915_utils.o
  CC      drivers/acpi/acpi_processor.o
  CC      arch/x86/kernel/setup.o
  AR      drivers/gpu/drm/xlnx/built-in.a
  CC      net/mac80211/aes_cmac.o
  CC      drivers/gpu/drm/virtio/virtgpu_debugfs.o
  CC      lib/bust_spinlocks.o
  CC      net/ipv4/datagram.o
  CC      net/sunrpc/cache.o
  GEN     net/wireless/shipped-certs.c
  CC      drivers/acpi/acpica/nsarguments.o
  CC      fs/nfs/sysfs.o
  CC      crypto/hash_info.o
  CC      drivers/scsi/constants.o
  CC      crypto/rsapubkey.asn1.o
  CC      crypto/rsaprivkey.asn1.o
  AR      crypto/built-in.a
  CC      net/ipv6/datagram.o
  CC      fs/ext4/readpage.o
  CC      mm/msync.o
  CC      fs/ext4/resize.o
  AR      drivers/gpu/drm/gud/built-in.a
  AR      drivers/tty/serial/8250/built-in.a
  CC      drivers/gpu/drm/i915/intel_clock_gating.o
  AR      drivers/tty/serial/built-in.a
  CC      drivers/gpu/drm/virtio/virtgpu_plane.o
  CC      drivers/tty/tty_baudrate.o
  CC      drivers/acpi/acpica/nsconvert.o
  CC      net/ipv6/ip6_flowlabel.o
  CC      fs/nfs/fs_context.o
  CC [M]  net/netfilter/xt_LOG.o
  CC      lib/kasprintf.o
  CC      fs/exec.o
  CC      arch/x86/kernel/x86_init.o
  CC      drivers/ata/libata-core.o
  CC      drivers/gpu/drm/i915/intel_cpu_info.o
  CC      arch/x86/kernel/i8259.o
  CC      fs/ext4/super.o
  CC      drivers/acpi/processor_core.o
  CC      drivers/acpi/acpica/nsdump.o
  CC      net/ipv6/inet6_connection_sock.o
  CC      kernel/signal.o
  CC      net/core/netdev-genl.o
  CC      net/mac80211/aes_gmac.o
  CC      lib/bitmap.o
  CC      drivers/base/firmware.o
  CC      net/mac80211/fils_aead.o
  CC      drivers/acpi/acpica/nseval.o
  AR      kernel/trace/built-in.a
  CC      drivers/gpu/drm/i915/intel_device_info.o
  CC      drivers/acpi/processor_pdc.o
  CC      drivers/acpi/ec.o
  CC      mm/page_vma_mapped.o
  CC      drivers/gpu/drm/virtio/virtgpu_ioctl.o
  CC      drivers/tty/tty_jobctrl.o
  CC      drivers/gpu/drm/i915/intel_memory_region.o
  CC [M]  net/netfilter/xt_MASQUERADE.o
  CC      net/ipv6/udp_offload.o
  CC      net/ipv4/raw.o
  CC      kernel/sys.o
  CC      drivers/gpu/drm/i915/intel_pcode.o
  CC      lib/scatterlist.o
  CC      arch/x86/kernel/irqinit.o
  CC [M]  net/netfilter/xt_addrtype.o
  AR      drivers/net/pse-pd/built-in.a
  AR      drivers/net/phy/mediatek/built-in.a
  CC      drivers/tty/n_null.o
  CC      drivers/gpu/drm/virtio/virtgpu_prime.o
  CC      drivers/base/init.o
  AR      drivers/net/phy/qcom/built-in.a
  CC      drivers/acpi/acpica/nsinit.o
  CC      fs/ext4/symlink.o
  CC      drivers/net/phy/realtek/realtek_main.o
  CC      net/ipv6/seg6.o
  CC      drivers/net/mdio/acpi_mdio.o
  AR      drivers/net/pcs/built-in.a
  CC      mm/pagewalk.o
  CC      net/mac80211/cfg.o
  CC      drivers/ata/libata-scsi.o
  CC      drivers/scsi/scsi_lib_dma.o
  CC      net/wireless/shipped-certs.o
  CC      drivers/gpu/drm/virtio/virtgpu_trace_points.o
  CC      net/sunrpc/rpc_pipe.o
  CC      drivers/acpi/dock.o
  AR      drivers/gpu/drm/solomon/built-in.a
  CC      net/sunrpc/sysfs.o
  CC      drivers/acpi/acpica/nsload.o
  CC      drivers/net/phy/realtek/realtek_hwmon.o
  CC      drivers/scsi/scsi_scan.o
  CC      drivers/tty/pty.o
  CC      drivers/gpu/drm/i915/intel_region_ttm.o
  CC      fs/pipe.o
  CC      fs/namei.o
  CC      drivers/base/map.o
  CC      drivers/net/phy/mdio-boardinfo.o
  AR      drivers/net/ethernet/3com/built-in.a
  AR      drivers/net/wireless/admtek/built-in.a
  CC      drivers/net/ethernet/8390/ne2k-pci.o
  AR      drivers/net/wireless/ath/built-in.a
  AR      drivers/net/wireless/atmel/built-in.a
  AR      drivers/net/wireless/broadcom/built-in.a
  AR      drivers/net/wireless/intel/built-in.a
  CC      arch/x86/kernel/jump_label.o
  AR      drivers/net/wireless/intersil/built-in.a
  GEN     drivers/scsi/scsi_devinfo_tbl.c
  AR      drivers/net/wireless/marvell/built-in.a
  CC      drivers/net/mdio/fwnode_mdio.o
  CC      mm/pgtable-generic.o
  AR      drivers/net/wireless/mediatek/built-in.a
  CC      lib/list_sort.o
  AR      drivers/net/wireless/microchip/built-in.a
  CC      drivers/base/devres.o
  CC      drivers/acpi/acpica/nsnames.o
  AR      drivers/net/wireless/purelifi/built-in.a
  AR      drivers/net/wireless/quantenna/built-in.a
  CC      drivers/base/attribute_container.o
  CC      net/ipv4/udp.o
  AR      drivers/net/wireless/ralink/built-in.a
  AR      drivers/net/wireless/realtek/built-in.a
  CC      net/mac80211/ethtool.o
  AR      drivers/net/wireless/rsi/built-in.a
  CC      drivers/net/phy/stubs.o
  AR      drivers/net/wireless/silabs/built-in.a
  AR      drivers/net/wireless/st/built-in.a
  CC      drivers/acpi/pci_root.o
  AR      drivers/net/wireless/ti/built-in.a
  AR      drivers/net/wireless/zydas/built-in.a
  AR      drivers/net/wireless/virtual/built-in.a
  CC      mm/rmap.o
  AR      drivers/net/wireless/built-in.a
  CC      fs/nfs/nfsroot.o
  CC      net/mac80211/rx.o
  CC      drivers/firewire/init_ohci1394_dma.o
  CC      arch/x86/kernel/irq_work.o
  CC      arch/x86/kernel/probe_roms.o
  CC      net/core/netdev-genl-gen.o
  CC      net/core/gso.o
  CC      net/mac80211/spectmgmt.o
  CC      drivers/gpu/drm/virtio/virtgpu_submit.o
  CC      lib/uuid.o
  CC      net/ipv6/fib6_notifier.o
  CC      net/ipv6/rpl.o
  CC      net/mac80211/tx.o
  CC      lib/iov_iter.o
  CC      drivers/tty/tty_audit.o
  CC      drivers/net/ethernet/8390/8390.o
  CC      drivers/acpi/acpica/nsobject.o
  AR      drivers/net/ethernet/adaptec/built-in.a
  AR      drivers/net/ethernet/agere/built-in.a
  CC      fs/nfs/sysctl.o
  CC      drivers/scsi/scsi_devinfo.o
  AR      net/netfilter/built-in.a
  CC [M]  drivers/gpu/drm/scheduler/sched_main.o
  AR      drivers/net/phy/realtek/built-in.a
  CC      arch/x86/kernel/sys_ia32.o
  CC      fs/fcntl.o
  CC      kernel/umh.o
  AR      drivers/net/usb/built-in.a
  CC      mm/vmalloc.o
  CC      drivers/scsi/scsi_sysctl.o
  CC      drivers/acpi/pci_link.o
  CC      fs/ioctl.o
  CC      drivers/gpu/drm/i915/intel_runtime_pm.o
  CC      drivers/base/transport_class.o
  CC      drivers/acpi/acpica/nsparse.o
  AR      drivers/firewire/built-in.a
  CC      net/core/net-sysfs.o
  CC      lib/clz_ctz.o
  CC      drivers/net/phy/mdio_devres.o
  CC      drivers/net/mii.o
  AR      drivers/net/mdio/built-in.a
  CC      fs/ext4/sysfs.o
  CC [M]  drivers/gpu/drm/scheduler/sched_fence.o
  CC      net/ipv4/udplite.o
  CC      net/mac80211/key.o
  CC      net/ipv6/ioam6.o
  CC      net/ipv6/sysctl_net_ipv6.o
  CC      drivers/cdrom/cdrom.o
  CC      net/sunrpc/svc_xprt.o
  CC      drivers/tty/sysrq.o
  AR      drivers/auxdisplay/built-in.a
  AR      drivers/gpu/drm/virtio/built-in.a
  CC      net/core/hotdata.o
  CC      net/core/netdev_rx_queue.o
  CC      net/sunrpc/xprtmultipath.o
  CC      drivers/acpi/acpica/nspredef.o
  CC      drivers/base/topology.o
  CC      drivers/ata/libata-eh.o
  CC      fs/ext4/xattr.o
  CC      arch/x86/kernel/ksysfs.o
  CC      net/mac80211/util.o
  CC      drivers/scsi/scsi_proc.o
  CC      fs/nfs/nfs3super.o
  CC      drivers/acpi/acpica/nsprepkg.o
  CC      kernel/workqueue.o
  CC      lib/bsearch.o
  CC      net/ipv4/udp_offload.o
  AR      drivers/net/ethernet/alacritech/built-in.a
  CC      fs/ext4/xattr_hurd.o
  CC      net/sunrpc/stats.o
  CC      arch/x86/kernel/bootflag.o
  HOSTCC  drivers/gpu/drm/xe/xe_gen_wa_oob
  AR      drivers/net/ethernet/8390/built-in.a
  AR      drivers/net/ethernet/alteon/built-in.a
  CC      net/ipv6/xfrm6_policy.o
  AR      drivers/net/ethernet/amazon/built-in.a
  CC      mm/vma.o
  AR      drivers/net/ethernet/amd/built-in.a
  CC      drivers/scsi/scsi_debugfs.o
  AR      drivers/net/ethernet/aquantia/built-in.a
  AR      drivers/net/ethernet/arc/built-in.a
  AR      drivers/net/ethernet/asix/built-in.a
  AR      drivers/net/ethernet/atheros/built-in.a
  GEN     xe_wa_oob.c xe_wa_oob.h
  CC [M]  drivers/gpu/drm/xe/xe_bb.o
  AR      drivers/net/ethernet/cadence/built-in.a
  CC      drivers/net/ethernet/broadcom/bnx2.o
  CC      drivers/net/phy/phy.o
  CC      net/sunrpc/sysctl.o
  AR      drivers/net/ethernet/brocade/built-in.a
  CC      net/mac80211/parse.o
  CC      drivers/acpi/acpica/nsrepair.o
  CC      drivers/gpu/drm/i915/intel_sbi.o
  CC [M]  drivers/gpu/drm/scheduler/sched_entity.o
  CC      drivers/base/container.o
  CC      drivers/ata/libata-transport.o
  CC      drivers/pcmcia/cs.o
  CC      drivers/net/loopback.o
  CC      kernel/pid.o
  CC      net/mac80211/wme.o
  CC      fs/ext4/xattr_trusted.o
  CC      arch/x86/kernel/e820.o
  AR      drivers/tty/built-in.a
  CC      drivers/scsi/scsi_trace.o
  CC      drivers/acpi/acpica/nsrepair2.o
  CC      net/ipv6/xfrm6_state.o
  CC [M]  drivers/gpu/drm/xe/xe_bo.o
  CC      drivers/net/netconsole.o
  CC      drivers/pcmcia/socket_sysfs.o
  CC      mm/process_vm_access.o
  CC      drivers/base/property.o
  CC      fs/readdir.o
  CC      fs/nfs/nfs3client.o
  CC      drivers/net/ethernet/broadcom/tg3.o
  CC      drivers/ata/libata-trace.o
  LD [M]  drivers/gpu/drm/scheduler/gpu-sched.o
  CC      drivers/acpi/acpica/nssearch.o
  CC      lib/find_bit.o
  CC      drivers/net/virtio_net.o
  AR      drivers/net/ethernet/cavium/common/built-in.a
  AR      drivers/net/ethernet/cavium/thunder/built-in.a
  CC      fs/select.o
  AR      drivers/net/ethernet/cavium/liquidio/built-in.a
  CC      drivers/acpi/pci_irq.o
  AR      drivers/net/ethernet/cavium/octeon/built-in.a
  AR      drivers/net/ethernet/cavium/built-in.a
  CC      net/ipv4/arp.o
  CC      drivers/usb/common/common.o
  CC      net/core/net-procfs.o
  CC      drivers/gpu/drm/i915/intel_step.o
  CC      fs/nfs/nfs3proc.o
  AR      drivers/cdrom/built-in.a
  CC      drivers/scsi/scsi_logging.o
  CC      drivers/usb/core/usb.o
  AR      drivers/usb/phy/built-in.a
  CC      net/ipv4/icmp.o
  CC      lib/llist.o
  CC      drivers/pcmcia/cardbus.o
  CC      drivers/base/cacheinfo.o
  CC      fs/dcache.o
  CC      drivers/acpi/acpica/nsutils.o
  CC      drivers/net/phy/phy-c45.o
  CC      lib/lwq.o
  CC      drivers/gpu/drm/drm_atomic.o
  CC [M]  drivers/gpu/drm/xe/xe_bo_evict.o
  CC      drivers/pcmcia/ds.o
  CC      net/ipv6/xfrm6_input.o
  CC      lib/memweight.o
  CC      fs/inode.o
  CC      drivers/usb/common/debug.o
  CC      mm/page_alloc.o
  CC      arch/x86/kernel/pci-dma.o
  CC      net/mac80211/chan.o
  CC      lib/kfifo.o
  CC      arch/x86/kernel/quirks.o
  CC      net/mac80211/trace.o
  CC      net/mac80211/mlme.o
  CC      drivers/input/serio/serio.o
  AR      net/sunrpc/built-in.a
  CC      drivers/input/keyboard/atkbd.o
  CC      drivers/input/mouse/psmouse-base.o
  AR      drivers/input/joystick/built-in.a
  CC      drivers/ata/libata-sata.o
  CC      drivers/acpi/acpica/nswalk.o
  CC      lib/percpu-refcount.o
  CC      drivers/pcmcia/pcmcia_resource.o
  AR      drivers/usb/common/built-in.a
  CC      drivers/rtc/lib.o
  CC      mm/page_frag_cache.o
  CC      drivers/net/phy/phy-core.o
  CC      drivers/gpu/drm/i915/intel_uncore.o
  CC      net/mac80211/tdls.o
  CC      drivers/scsi/scsi_pm.o
  CC      drivers/usb/core/hub.o
  CC      drivers/base/swnode.o
  CC      drivers/net/net_failover.o
  CC      drivers/acpi/acpica/nsxfeval.o
  CC      net/mac80211/ocb.o
  CC      net/core/netpoll.o
  CC      drivers/usb/mon/mon_main.o
  CC      net/mac80211/airtime.o
  CC      drivers/i2c/algos/i2c-algo-bit.o
  CC      arch/x86/kernel/kdebugfs.o
  CC      drivers/input/serio/i8042.o
  AR      drivers/i3c/built-in.a
  CC      drivers/rtc/class.o
  CC      drivers/scsi/scsi_bsg.o
  CC      lib/rhashtable.o
  CC      fs/ext4/xattr_user.o
  CC      fs/nfs/nfs3xdr.o
  CC      kernel/task_work.o
  CC [M]  drivers/gpu/drm/xe/xe_devcoredump.o
  CC      drivers/gpu/drm/i915/intel_uncore_trace.o
  CC      drivers/usb/mon/mon_stat.o
  CC      drivers/acpi/acpica/nsxfname.o
  CC      net/core/fib_rules.o
  CC      net/ipv4/devinet.o
  CC      drivers/usb/core/hcd.o
  AR      drivers/input/keyboard/built-in.a
  CC      drivers/usb/mon/mon_text.o
  CC      drivers/input/mouse/synaptics.o
  CC      drivers/ata/libata-sff.o
  CC      net/ipv6/xfrm6_output.o
  CC      drivers/pcmcia/cistpl.o
  CC      drivers/base/auxiliary.o
  CC      lib/base64.o
  CC      drivers/acpi/acpi_apd.o
  CC      drivers/net/phy/phy_device.o
  AR      drivers/media/i2c/built-in.a
  AR      drivers/media/tuners/built-in.a
  CC      arch/x86/kernel/alternative.o
  AR      drivers/media/rc/keymaps/built-in.a
  AR      drivers/media/rc/built-in.a
  CC      lib/once.o
  AR      drivers/media/common/b2c2/built-in.a
  CC      net/core/net-traces.o
  AR      drivers/media/common/saa7146/built-in.a
  AR      drivers/media/common/siano/built-in.a
  AR      drivers/media/common/v4l2-tpg/built-in.a
  CC      drivers/acpi/acpica/nsxfobj.o
  CC      drivers/gpu/drm/i915/intel_wakeref.o
  CC      drivers/rtc/interface.o
  AR      drivers/media/common/videobuf2/built-in.a
  AR      drivers/media/common/built-in.a
  AR      drivers/media/platform/allegro-dvt/built-in.a
  CC      drivers/scsi/scsi_common.o
  AR      drivers/media/platform/amlogic/meson-ge2d/built-in.a
  AR      drivers/media/pci/ttpci/built-in.a
  CC      fs/ext4/fast_commit.o
  AR      drivers/media/platform/amlogic/built-in.a
  AR      drivers/media/pci/b2c2/built-in.a
  AR      drivers/media/platform/amphion/built-in.a
  AR      drivers/media/pci/pluto2/built-in.a
  CC      kernel/extable.o
  AR      drivers/media/platform/aspeed/built-in.a
  AR      drivers/media/pci/dm1105/built-in.a
  CC      arch/x86/kernel/i8253.o
  AR      drivers/media/platform/atmel/built-in.a
  AR      drivers/media/pci/pt1/built-in.a
  AR      drivers/media/platform/broadcom/built-in.a
  AR      drivers/media/pci/pt3/built-in.a
  AR      drivers/media/platform/cadence/built-in.a
  AR      drivers/media/pci/mantis/built-in.a
  AR      drivers/media/pci/ngene/built-in.a
  CC      drivers/base/devtmpfs.o
  AR      drivers/media/platform/chips-media/coda/built-in.a
  AR      drivers/media/pci/ddbridge/built-in.a
  AR      drivers/media/platform/chips-media/wave5/built-in.a
  AR      drivers/media/platform/chips-media/built-in.a
  AR      drivers/media/pci/saa7146/built-in.a
  AR      drivers/i2c/algos/built-in.a
  AR      drivers/media/pci/smipcie/built-in.a
  AR      drivers/media/platform/imagination/built-in.a
  CC      drivers/i2c/busses/i2c-i801.o
  AR      drivers/media/platform/intel/built-in.a
  AR      drivers/media/pci/netup_unidvb/built-in.a
  CC      drivers/base/module.o
  AR      drivers/media/platform/marvell/built-in.a
  AR      drivers/media/pci/intel/ipu3/built-in.a
  AR      drivers/media/platform/mediatek/jpeg/built-in.a
  AR      drivers/media/pci/intel/ivsc/built-in.a
  AR      drivers/media/pci/intel/built-in.a
  AR      drivers/media/platform/mediatek/mdp/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_device.o
  AR      drivers/media/pci/built-in.a
  AR      drivers/media/platform/mediatek/vcodec/common/built-in.a
  CC      drivers/usb/mon/mon_bin.o
  AR      drivers/media/platform/mediatek/vcodec/encoder/built-in.a
  AR      drivers/media/platform/mediatek/vcodec/decoder/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_device_sysfs.o
  AR      drivers/media/platform/mediatek/vcodec/built-in.a
  CC      drivers/ata/libata-pmp.o
  AR      drivers/media/platform/mediatek/vpu/built-in.a
  CC      drivers/acpi/acpica/psargs.o
  AR      drivers/media/platform/mediatek/mdp3/built-in.a
  AR      drivers/media/platform/mediatek/built-in.a
  AR      drivers/media/platform/microchip/built-in.a
  AR      drivers/media/platform/nuvoton/built-in.a
  CC      drivers/net/phy/linkmode.o
  AR      drivers/media/platform/nvidia/tegra-vde/built-in.a
  AR      drivers/media/platform/nvidia/built-in.a
  CC      lib/refcount.o
  AR      drivers/media/platform/nxp/dw100/built-in.a
  AR      drivers/media/platform/nxp/imx-jpeg/built-in.a
  CC      drivers/gpu/drm/drm_atomic_uapi.o
  CC      drivers/scsi/scsi_transport_spi.o
  AR      drivers/net/ethernet/chelsio/built-in.a
  AR      drivers/media/platform/nxp/imx8-isi/built-in.a
  AR      drivers/media/platform/nxp/built-in.a
  CC      drivers/scsi/virtio_scsi.o
  CC      net/ipv4/af_inet.o
  AR      drivers/media/platform/qcom/camss/built-in.a
  AR      drivers/input/tablet/built-in.a
  AR      drivers/media/platform/qcom/venus/built-in.a
  AR      drivers/media/platform/qcom/built-in.a
  AR      drivers/media/platform/raspberrypi/pisp_be/built-in.a
  CC      drivers/base/auxiliary_sysfs.o
  AR      drivers/media/platform/raspberrypi/rp1-cfe/built-in.a
  AR      drivers/media/platform/raspberrypi/built-in.a
  CC      drivers/input/serio/serport.o
  CC      drivers/pcmcia/pcmcia_cis.o
  AR      drivers/media/platform/renesas/rcar-vin/built-in.a
  AR      drivers/media/platform/renesas/rzg2l-cru/built-in.a
  AR      drivers/media/platform/renesas/vsp1/built-in.a
  AR      drivers/media/platform/renesas/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_dma_buf.o
  CC      lib/rcuref.o
  AR      drivers/media/platform/rockchip/rga/built-in.a
  AR      drivers/media/platform/samsung/exynos-gsc/built-in.a
  AR      drivers/media/platform/rockchip/rkisp1/built-in.a
  AR      drivers/media/platform/samsung/exynos4-is/built-in.a
  AR      drivers/media/platform/rockchip/built-in.a
  AR      drivers/media/platform/samsung/s3c-camif/built-in.a
  CC      drivers/input/serio/libps2.o
  AR      drivers/media/platform/samsung/s5p-g2d/built-in.a
  AR      drivers/media/platform/samsung/s5p-jpeg/built-in.a
  AR      drivers/media/platform/samsung/s5p-mfc/built-in.a
  AR      drivers/media/platform/samsung/built-in.a
  AR      drivers/media/platform/st/sti/bdisp/built-in.a
  AR      drivers/media/platform/st/sti/c8sectpfe/built-in.a
  AR      drivers/media/platform/st/sti/delta/built-in.a
  AR      drivers/media/platform/st/sti/hva/built-in.a
  CC      drivers/acpi/acpica/psloop.o
  AR      drivers/media/platform/st/stm32/built-in.a
  AR      drivers/media/platform/st/built-in.a
  CC      lib/usercopy.o
  AR      drivers/media/platform/sunxi/sun4i-csi/built-in.a
  AR      drivers/media/platform/sunxi/sun6i-csi/built-in.a
  AR      drivers/media/platform/sunxi/sun6i-mipi-csi2/built-in.a
  AR      drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/built-in.a
  AR      drivers/media/platform/sunxi/sun8i-di/built-in.a
  AR      drivers/i2c/muxes/built-in.a
  CC      drivers/pcmcia/rsrc_mgr.o
  AR      drivers/media/platform/sunxi/sun8i-rotate/built-in.a
  AR      drivers/media/platform/sunxi/built-in.a
  CC      drivers/input/mouse/focaltech.o
  AR      drivers/media/platform/ti/am437x/built-in.a
  AR      drivers/media/platform/ti/cal/built-in.a
  AR      drivers/media/platform/ti/vpe/built-in.a
  CC      net/ipv6/xfrm6_protocol.o
  CC      mm/init-mm.o
  AR      drivers/media/platform/ti/davinci/built-in.a
  CC      net/ipv4/igmp.o
  CC      drivers/base/devcoredump.o
  AR      drivers/media/platform/ti/j721e-csi2rx/built-in.a
  AR      drivers/media/platform/ti/omap/built-in.a
  AR      drivers/media/platform/ti/omap3isp/built-in.a
  AR      drivers/media/platform/ti/built-in.a
  CC      net/mac80211/eht.o
  AR      drivers/media/platform/verisilicon/built-in.a
  CC      drivers/gpu/drm/i915/vlv_sideband.o
  CC      kernel/params.o
  AR      drivers/media/platform/via/built-in.a
  AR      drivers/media/platform/xilinx/built-in.a
  CC      drivers/usb/host/pci-quirks.o
  AR      drivers/media/platform/built-in.a
  CC      arch/x86/kernel/hw_breakpoint.o
  CC      fs/attr.o
  CC      fs/nfs/nfs3acl.o
  CC      lib/errseq.o
  CC      net/ipv4/fib_frontend.o
  CC      net/mac80211/led.o
  AR      drivers/media/usb/b2c2/built-in.a
  AR      drivers/media/usb/dvb-usb/built-in.a
  AR      drivers/media/usb/dvb-usb-v2/built-in.a
  AR      drivers/media/usb/s2255/built-in.a
  CC      drivers/acpi/acpi_platform.o
  AR      drivers/media/usb/siano/built-in.a
  AR      drivers/media/usb/ttusb-budget/built-in.a
  CC      lib/bucket_locks.o
  AR      drivers/media/usb/ttusb-dec/built-in.a
  AR      drivers/media/usb/built-in.a
  CC      drivers/acpi/acpica/psobject.o
  AR      drivers/net/ethernet/cisco/built-in.a
  AR      drivers/media/mmc/siano/built-in.a
  CC      drivers/rtc/nvmem.o
  AR      drivers/media/mmc/built-in.a
  AR      drivers/usb/mon/built-in.a
  AR      drivers/media/firewire/built-in.a
  AR      drivers/media/spi/built-in.a
  CC      fs/ext4/orphan.o
  CC      drivers/input/mouse/alps.o
  AR      drivers/media/test-drivers/built-in.a
  AR      drivers/media/built-in.a
  CC      drivers/net/phy/phy_link_topology.o
  AR      drivers/i2c/busses/built-in.a
  CC      drivers/gpu/drm/drm_auth.o
  CC      drivers/i2c/i2c-boardinfo.o
  CC      lib/generic-radix-tree.o
  CC      net/core/selftests.o
  AR      drivers/input/serio/built-in.a
  CC      drivers/ata/libata-acpi.o
  CC      drivers/acpi/acpi_pnp.o
  CC      drivers/usb/host/ehci-hcd.o
  CC      arch/x86/kernel/tsc.o
  CC      drivers/usb/core/urb.o
  AR      drivers/net/ethernet/cortina/built-in.a
  CC      drivers/rtc/dev.o
  CC      drivers/base/platform-msi.o
  CC [M]  drivers/gpu/drm/xe/xe_drm_client.o
  CC      drivers/pcmcia/rsrc_nonstatic.o
  AR      drivers/pps/clients/built-in.a
  CC      drivers/pps/pps.o
  CC      net/ipv4/fib_semantics.o
  CC      drivers/acpi/acpica/psopcode.o
  CC      drivers/pcmcia/yenta_socket.o
  CC      drivers/scsi/sd.o
  CC      mm/memblock.o
  CC      drivers/usb/host/ehci-pci.o
  CC      drivers/usb/class/usblp.o
  CC      drivers/scsi/sr.o
  CC      lib/bitmap-str.o
  CC      lib/string_helpers.o
  AR      drivers/input/touchscreen/built-in.a
  CC      drivers/usb/core/message.o
  CC      drivers/net/phy/mdio_bus.o
  CC      net/mac80211/pm.o
  CC      kernel/kthread.o
  CC      drivers/usb/storage/scsiglue.o
  CC      drivers/base/physical_location.o
  CC      drivers/usb/host/ohci-hcd.o
  CC      drivers/usb/core/driver.o
  CC      drivers/acpi/acpica/psopinfo.o
  CC      drivers/gpu/drm/i915/vlv_suspend.o
  CC      drivers/i2c/i2c-core-base.o
  CC      net/core/ptp_classifier.o
  CC      mm/slub.o
  CC      net/ipv4/fib_trie.o
  CC      net/ipv6/netfilter.o
  CC      net/mac80211/rc80211_minstrel_ht.o
  AR      drivers/net/ethernet/dec/tulip/built-in.a
  AR      drivers/net/ethernet/dec/built-in.a
  CC      fs/nfs/nfs4proc.o
  CC      arch/x86/kernel/tsc_msr.o
  CC      drivers/pps/kapi.o
  CC      drivers/rtc/proc.o
  CC      fs/ext4/acl.o
  CC      arch/x86/kernel/io_delay.o
  CC      drivers/acpi/power.o
  CC      drivers/acpi/acpica/psparse.o
  CC      lib/hexdump.o
  CC      net/core/netprio_cgroup.o
  CC      drivers/base/trace.o
  CC      drivers/ata/libata-pata-timings.o
  CC      drivers/i2c/i2c-core-smbus.o
  CC [M]  drivers/gpu/drm/xe/xe_exec.o
  CC      drivers/input/mouse/byd.o
  CC      net/ipv6/proc.o
  AR      drivers/net/ethernet/dlink/built-in.a
  CC      drivers/net/phy/mdio_device.o
  CC      fs/bad_inode.o
  CC      arch/x86/kernel/rtc.o
  CC      arch/x86/kernel/resource.o
  CC      drivers/gpu/drm/i915/soc/intel_dram.o
  CC      lib/kstrtox.o
  AR      drivers/usb/class/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_exec_queue.o
  CC      drivers/usb/storage/protocol.o
  CC      drivers/acpi/event.o
  CC      drivers/acpi/acpica/psscope.o
  CC      drivers/pps/sysfs.o
  CC      drivers/rtc/sysfs.o
  CC      drivers/rtc/rtc-mc146818-lib.o
  CC      mm/madvise.o
  AR      drivers/pcmcia/built-in.a
  CC      drivers/gpu/drm/drm_blend.o
  CC      drivers/net/phy/swphy.o
  AR      drivers/net/ethernet/emulex/built-in.a
  CC      fs/ext4/xattr_security.o
  CC      net/mac80211/wbrf.o
  CC      drivers/i2c/i2c-core-acpi.o
  CC      drivers/i2c/i2c-smbus.o
  CC      drivers/input/mouse/logips2pp.o
  CC      fs/file.o
  CC      drivers/acpi/acpica/pstree.o
  CC      lib/iomap.o
  AR      drivers/pps/built-in.a
  CC      lib/iomap_copy.o
  CC      drivers/rtc/rtc-cmos.o
  AR      drivers/input/misc/built-in.a
  CC      drivers/ptp/ptp_clock.o
  CC      drivers/ata/ahci.o
  AR      drivers/usb/misc/built-in.a
  CC      kernel/sys_ni.o
  CC      drivers/acpi/evged.o
  AR      net/wireless/built-in.a
  AR      drivers/base/built-in.a
  AR      drivers/net/ethernet/engleder/built-in.a
  AR      drivers/net/ethernet/ezchip/built-in.a
  CC      drivers/ptp/ptp_chardev.o
  CC      net/ipv4/fib_notifier.o
  AS      arch/x86/kernel/irqflags.o
  CC      drivers/usb/core/config.o
  CC      arch/x86/kernel/static_call.o
  CC      net/core/netclassid_cgroup.o
  CC      drivers/usb/storage/transport.o
  CC      drivers/usb/storage/usb.o
  CC      drivers/gpu/drm/drm_bridge.o
  CC      net/ipv6/syncookies.o
  CC      fs/nfs/nfs4xdr.o
  CC      drivers/usb/core/file.o
  CC      drivers/usb/storage/initializers.o
  CC      drivers/acpi/acpica/psutils.o
  CC      drivers/input/mouse/lifebook.o
  CC      kernel/nsproxy.o
  CC      drivers/scsi/sr_ioctl.o
  CC      mm/page_io.o
  CC      arch/x86/kernel/process.o
  CC      drivers/power/supply/power_supply_core.o
  CC      drivers/net/phy/fixed_phy.o
  CC      drivers/hwmon/hwmon.o
  CC      drivers/ptp/ptp_sysfs.o
  CC      drivers/scsi/sr_vendor.o
  AR      fs/ext4/built-in.a
  CC      drivers/gpu/drm/i915/soc/intel_gmch.o
  CC      drivers/scsi/sg.o
  CC [M]  drivers/gpu/drm/xe/xe_execlist.o
  CC      drivers/power/supply/power_supply_sysfs.o
  CC      drivers/acpi/sysfs.o
  CC      drivers/acpi/acpica/pswalk.o
  CC      arch/x86/kernel/ptrace.o
  CC      lib/devres.o
  CC      drivers/usb/early/ehci-dbgp.o
  AR      drivers/i2c/built-in.a
  CC      net/ipv6/calipso.o
  CC      fs/filesystems.o
  CC      drivers/usb/storage/sierra_ms.o
  CC      drivers/ptp/ptp_vclock.o
  CC      drivers/input/mouse/trackpoint.o
  CC      drivers/input/input.o
  AR      drivers/net/ethernet/fujitsu/built-in.a
  CC      drivers/ata/libahci.o
  CC      drivers/input/input-compat.o
  CC      lib/check_signature.o
  CC      drivers/ata/ata_piix.o
  AR      drivers/rtc/built-in.a
  CC      drivers/usb/host/ohci-pci.o
  CC      fs/nfs/nfs4state.o
  CC      drivers/acpi/acpica/psxface.o
  CC      net/core/dst_cache.o
  CC      drivers/power/supply/power_supply_leds.o
  CC      drivers/acpi/property.o
  CC      fs/namespace.o
  CC      net/ipv6/ah6.o
  CC      lib/interval_tree.o
  CC      drivers/usb/host/uhci-hcd.o
  CC      net/ipv4/inet_fragment.o
  CC      arch/x86/kernel/tls.o
  CC      fs/seq_file.o
  CC      drivers/usb/core/buffer.o
  CC      drivers/ptp/ptp_kvm_x86.o
  CC      net/core/gro_cells.o
  CC      arch/x86/kernel/step.o
  CC      drivers/ata/pata_amd.o
  CC      kernel/notifier.o
  CC [M]  drivers/gpu/drm/xe/xe_force_wake.o
  CC      drivers/ata/pata_oldpiix.o
  CC      lib/assoc_array.o
  CC      net/core/failover.o
  CC      drivers/usb/core/sysfs.o
  CC      drivers/acpi/acpica/rsaddr.o
  CC      kernel/ksysfs.o
  AR      drivers/net/ethernet/fungible/built-in.a
  CC      drivers/power/supply/power_supply_hwmon.o
  CC      drivers/gpu/drm/i915/soc/intel_pch.o
  CC      drivers/usb/storage/option_ms.o
  AR      drivers/net/phy/built-in.a
  CC      arch/x86/kernel/i8237.o
  CC      kernel/cred.o
  CC      drivers/input/input-mt.o
  CC      drivers/input/mouse/cypress_ps2.o
  AR      drivers/usb/early/built-in.a
  CC      mm/swap_state.o
  CC      drivers/ptp/ptp_kvm_common.o
  CC      lib/bitrev.o
  CC      drivers/acpi/acpica/rscalc.o
  CC      fs/nfs/nfs4renewd.o
  CC      drivers/ata/pata_sch.o
  CC      kernel/reboot.o
  CC      net/ipv6/esp6.o
  AR      drivers/hwmon/built-in.a
  CC      drivers/gpu/drm/drm_cache.o
  CC      drivers/usb/core/endpoint.o
  CC      arch/x86/kernel/stacktrace.o
  CC      arch/x86/kernel/reboot.o
  CC [M]  drivers/gpu/drm/xe/xe_ggtt.o
  CC [M]  drivers/gpu/drm/xe/xe_gpu_scheduler.o
  AR      drivers/power/supply/built-in.a
  CC      drivers/input/input-poller.o
  AR      drivers/power/built-in.a
  CC      fs/xattr.o
  CC      net/ipv6/sit.o
  CC      drivers/acpi/acpica/rscreate.o
  AR      drivers/net/ethernet/google/built-in.a
  CC      drivers/usb/host/xhci.o
  CC      fs/nfs/nfs4super.o
  CC      drivers/usb/core/devio.o
  CC      drivers/scsi/scsi_sysfs.o
  CC      net/ipv6/addrconf_core.o
  AR      drivers/net/ethernet/hisilicon/built-in.a
  CC      drivers/gpu/drm/i915/soc/intel_rom.o
  AR      drivers/thermal/broadcom/built-in.a
  CC      drivers/usb/storage/usual-tables.o
  CC      kernel/async.o
  AR      drivers/thermal/renesas/built-in.a
  AR      drivers/thermal/samsung/built-in.a
  CC      drivers/thermal/intel/intel_tcc.o
  CC      lib/crc-ccitt.o
  CC      drivers/acpi/acpica/rsdumpinfo.o
  CC      drivers/acpi/acpica/rsinfo.o
  AR      drivers/net/ethernet/huawei/built-in.a
  CC      net/ipv4/ping.o
  CC      drivers/input/mouse/psmouse-smbus.o
  CC      lib/crc16.o
  AR      drivers/ptp/built-in.a
  CC      net/ipv6/exthdrs_core.o
  CC      fs/nfs/nfs4file.o
  CC      drivers/acpi/acpica/rsio.o
  AR      net/core/built-in.a
  CC      mm/swapfile.o
  CC      kernel/range.o
  CC      drivers/gpu/drm/drm_color_mgmt.o
  CC      drivers/gpu/drm/drm_connector.o
  CC      net/ipv4/ip_tunnel_core.o
  CC      drivers/input/ff-core.o
  HOSTCC  lib/gen_crc32table
  CC      drivers/input/touchscreen.o
  CC      drivers/thermal/intel/therm_throt.o
  CC      drivers/input/ff-memless.o
  CC      drivers/input/sparse-keymap.o
  CC      net/ipv6/ip6_checksum.o
  CC      drivers/acpi/debugfs.o
  AR      drivers/watchdog/built-in.a
  CC      lib/xxhash.o
  CC      drivers/usb/host/xhci-mem.o
  CC      drivers/gpu/drm/i915/i915_memcpy.o
  CC      drivers/usb/core/notify.o
  CC      net/ipv4/gre_offload.o
  CC      drivers/gpu/drm/drm_crtc.o
  CC      drivers/ata/pata_mpiix.o
  CC      drivers/acpi/acpi_lpat.o
  CC      drivers/input/vivaldi-fmap.o
  CC      drivers/acpi/acpica/rsirq.o
  CC      mm/swap_slots.o
  AR      drivers/usb/storage/built-in.a
  CC      drivers/gpu/drm/i915/i915_mm.o
  CC      net/ipv4/metrics.o
  CC      drivers/ata/ata_generic.o
  CC      arch/x86/kernel/msr.o
  CC [M]  drivers/gpu/drm/xe/xe_gsc.o
  CC      kernel/smpboot.o
  CC      drivers/gpu/drm/drm_displayid.o
  CC      mm/dmapool.o
  CC      drivers/acpi/acpica/rslist.o
  CC      drivers/md/md.o
  AR      net/mac80211/built-in.a
  CC      drivers/acpi/acpi_pcc.o
  CC      fs/nfs/delegation.o
  AR      drivers/thermal/st/built-in.a
  CC      lib/genalloc.o
  AR      drivers/input/mouse/built-in.a
  CC      lib/percpu_counter.o
  CC      mm/hugetlb.o
  CC      net/ipv4/netlink.o
  CC [M]  drivers/thermal/intel/x86_pkg_temp_thermal.o
  CC      drivers/gpu/drm/drm_drv.o
  AR      drivers/net/ethernet/broadcom/built-in.a
  CC      drivers/cpufreq/cpufreq.o
  CC      drivers/net/ethernet/intel/e1000/e1000_main.o
  CC      drivers/net/ethernet/intel/e1000e/82571.o
  CC      drivers/net/ethernet/intel/e100.o
  CC      drivers/net/ethernet/intel/e1000/e1000_hw.o
  CC      drivers/cpufreq/freq_table.o
  CC      lib/audit.o
  CC      drivers/net/ethernet/intel/e1000e/ich8lan.o
  CC      drivers/input/input-leds.o
  CC      drivers/md/md-bitmap.o
  CC      kernel/ucount.o
  AR      drivers/scsi/built-in.a
  AR      drivers/net/ethernet/microsoft/built-in.a
  AR      drivers/net/ethernet/i825xx/built-in.a
  CC      drivers/usb/core/generic.o
  CC      drivers/input/evdev.o
  CC      drivers/gpu/drm/drm_dumb_buffers.o
  CC      drivers/cpufreq/cpufreq_performance.o
  CC      drivers/acpi/acpica/rsmemory.o
  CC      drivers/acpi/acpica/rsmisc.o
  AR      drivers/thermal/qcom/built-in.a
  CC      drivers/md/md-autodetect.o
  AR      drivers/ata/built-in.a
  CC      drivers/net/ethernet/intel/e1000e/80003es2lan.o
  CC      arch/x86/kernel/cpuid.o
  CC      drivers/usb/host/xhci-ext-caps.o
  CC      drivers/usb/core/quirks.o
  CC      fs/nfs/nfs4idmap.o
  CC      drivers/cpufreq/cpufreq_userspace.o
  CC      fs/libfs.o
  CC      drivers/gpu/drm/i915/i915_sw_fence.o
  CC      net/ipv4/nexthop.o
  CC      drivers/net/ethernet/intel/e1000/e1000_ethtool.o
  AR      drivers/thermal/tegra/built-in.a
  CC      drivers/gpu/drm/drm_edid.o
  CC      drivers/md/dm.o
  CC      lib/syscall.o
  CC      drivers/usb/host/xhci-ring.o
  CC      net/ipv4/udp_tunnel_stub.o
  CC      drivers/gpu/drm/i915/i915_sw_fence_work.o
  CC      drivers/acpi/acpica/rsserial.o
  CC      kernel/regset.o
  CC      drivers/usb/core/devices.o
  CC      net/ipv6/ip6_icmp.o
  CC      drivers/usb/host/xhci-hub.o
  CC [M]  drivers/gpu/drm/xe/xe_gsc_debugfs.o
  AR      drivers/thermal/intel/built-in.a
  CC      drivers/net/ethernet/intel/e1000e/mac.o
  AR      drivers/thermal/mediatek/built-in.a
  CC      drivers/thermal/thermal_core.o
  CC      drivers/usb/core/phy.o
  CC      drivers/usb/host/xhci-dbg.o
  CC      lib/errname.o
  CC      drivers/cpufreq/cpufreq_ondemand.o
  CC      drivers/md/dm-table.o
  CC      arch/x86/kernel/early-quirks.o
  CC      mm/mmu_notifier.o
  CC      fs/fs-writeback.o
  AR      drivers/net/ethernet/litex/built-in.a
  CC      drivers/net/ethernet/intel/e1000e/manage.o
  CC      drivers/thermal/thermal_sysfs.o
  CC      drivers/usb/host/xhci-trace.o
  CC      drivers/acpi/acpica/rsutils.o
  CC [M]  drivers/gpu/drm/xe/xe_gsc_proxy.o
  CC      kernel/ksyms_common.o
  CC      kernel/groups.o
  CC      drivers/net/ethernet/intel/e1000/e1000_param.o
  CC      drivers/gpu/drm/i915/i915_syncmap.o
  AR      drivers/net/ethernet/marvell/octeon_ep/built-in.a
  AR      drivers/net/ethernet/marvell/octeon_ep_vf/built-in.a
  AR      drivers/net/ethernet/marvell/octeontx2/built-in.a
  CC      lib/nlattr.o
  CC      mm/migrate.o
  AR      drivers/net/ethernet/marvell/prestera/built-in.a
  CC      drivers/net/ethernet/marvell/sky2.o
  AR      drivers/input/built-in.a
  CC      drivers/acpi/ac.o
  CC      fs/nfs/callback.o
  CC      drivers/gpu/drm/drm_eld.o
  CC      drivers/acpi/acpica/rsxface.o
  CC      drivers/gpu/drm/drm_encoder.o
  CC      fs/pnode.o
  CC      drivers/usb/core/port.o
  CC [M]  drivers/gpu/drm/xe/xe_gsc_submit.o
  CC      drivers/md/dm-target.o
  AR      drivers/net/ethernet/mellanox/built-in.a
  AR      drivers/net/ethernet/meta/built-in.a
  CC      lib/cpu_rmap.o
  CC      drivers/gpu/drm/i915/i915_user_extensions.o
  CC      drivers/acpi/acpica/tbdata.o
  CC      fs/nfs/callback_xdr.o
  CC      drivers/cpuidle/governors/menu.o
  AR      drivers/mmc/built-in.a
  CC      lib/dynamic_queue_limits.o
  CC      drivers/acpi/button.o
  CC      drivers/gpu/drm/drm_file.o
  CC      net/ipv6/output_core.o
  CC      arch/x86/kernel/smp.o
  CC      drivers/acpi/acpica/tbfadt.o
  CC      drivers/net/ethernet/intel/e1000e/nvm.o
  CC      drivers/cpufreq/cpufreq_governor.o
  CC      kernel/kcmp.o
  CC      drivers/acpi/fan_core.o
  CC      lib/glob.o
  AR      drivers/ufs/built-in.a
  CC      drivers/cpuidle/governors/haltpoll.o
  CC      drivers/usb/core/hcd-pci.o
  CC      drivers/gpu/drm/i915/i915_debugfs.o
  CC      drivers/gpu/drm/i915/i915_debugfs_params.o
  CC      drivers/usb/host/xhci-debugfs.o
  CC      drivers/gpu/drm/i915/i915_pmu.o
  CC      drivers/md/dm-linear.o
  CC      drivers/acpi/fan_attr.o
  CC      net/ipv6/protocol.o
  CC      lib/strncpy_from_user.o
  CC      arch/x86/kernel/smpboot.o
  CC      drivers/gpu/drm/i915/gt/gen2_engine_cs.o
  AR      drivers/net/ethernet/micrel/built-in.a
  CC      drivers/cpuidle/cpuidle.o
  CC      drivers/thermal/thermal_trip.o
  CC      drivers/usb/core/usb-acpi.o
  CC      mm/page_counter.o
  CC      drivers/acpi/acpica/tbfind.o
  CC      lib/strnlen_user.o
  CC      net/ipv4/ip_tunnel.o
  CC      drivers/gpu/drm/drm_fourcc.o
  CC      drivers/md/dm-stripe.o
  CC      drivers/thermal/thermal_helpers.o
  CC [M]  drivers/gpu/drm/xe/xe_gt.o
  CC      kernel/freezer.o
  CC      fs/nfs/callback_proc.o
  CC      fs/splice.o
  CC      drivers/usb/host/xhci-pci.o
  CC      lib/net_utils.o
  AR      drivers/firmware/arm_ffa/built-in.a
  AR      drivers/crypto/stm32/built-in.a
  AR      drivers/firmware/arm_scmi/built-in.a
  AR      drivers/crypto/xilinx/built-in.a
  CC      drivers/clocksource/acpi_pm.o
  AR      drivers/firmware/broadcom/built-in.a
  AR      drivers/crypto/hisilicon/built-in.a
  AR      drivers/firmware/cirrus/test/built-in.a
  AR      drivers/crypto/intel/keembay/built-in.a
  AR      drivers/firmware/cirrus/built-in.a
  CC      drivers/cpufreq/cpufreq_governor_attr_set.o
  AR      drivers/crypto/intel/ixp4xx/built-in.a
  AR      drivers/firmware/meson/built-in.a
  AR      drivers/firmware/microchip/built-in.a
  AR      drivers/crypto/intel/built-in.a
  AR      drivers/crypto/starfive/built-in.a
  CC      drivers/hid/usbhid/hid-core.o
  AR      drivers/crypto/built-in.a
  CC      fs/nfs/nfs4namespace.o
  CC      drivers/md/dm-ioctl.o
  CC      drivers/acpi/acpica/tbinstal.o
  AR      drivers/firmware/imx/built-in.a
  AR      drivers/firmware/psci/built-in.a
  CC      drivers/cpufreq/acpi-cpufreq.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_ccs_mode.o
  CC      drivers/clocksource/i8253.o
  CC      drivers/firmware/efi/libstub/efi-stub-helper.o
  CC      net/ipv6/ip6_offload.o
  CC      drivers/net/ethernet/intel/e1000e/phy.o
  CC      fs/nfs/nfs4getroot.o
  CC      drivers/hid/usbhid/hiddev.o
  AR      drivers/cpuidle/governors/built-in.a
  CC      drivers/acpi/acpica/tbprint.o
  CC      net/ipv6/tcpv6_offload.o
  CC      drivers/acpi/fan_hwmon.o
  CC      net/ipv4/sysctl_net_ipv4.o
  CC      drivers/thermal/thermal_thresholds.o
  AR      drivers/platform/x86/amd/built-in.a
  CC      drivers/mailbox/mailbox.o
  AR      drivers/platform/x86/intel/built-in.a
  CC      drivers/platform/x86/wmi.o
  AR      drivers/usb/core/built-in.a
  AR      drivers/perf/built-in.a
  CC      drivers/firmware/efi/efi-bgrt.o
  AR      drivers/platform/surface/built-in.a
  AR      drivers/net/ethernet/intel/e1000/built-in.a
  AR      drivers/net/ethernet/microchip/built-in.a
  AR      drivers/net/ethernet/mscc/built-in.a
  CC      drivers/acpi/acpi_video.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_clock.o
  CC      drivers/mailbox/pcc.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_freq.o
  CC      drivers/firmware/efi/efi.o
  CC      drivers/firmware/efi/libstub/gop.o
  CC      mm/hugetlb_cgroup.o
  CC      drivers/cpuidle/driver.o
  AR      drivers/firmware/qcom/built-in.a
  CC      drivers/firmware/efi/vars.o
  CC      lib/sg_pool.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_idle.o
  CC      lib/stackdepot.o
  AR      drivers/hwtracing/intel_th/built-in.a
  CC      kernel/profile.o
  CC      drivers/md/dm-io.o
  CC      lib/asn1_decoder.o
  CC      drivers/net/ethernet/intel/e1000e/param.o
  CC      drivers/acpi/acpica/tbutils.o
  AR      drivers/clocksource/built-in.a
  CC      drivers/acpi/video_detect.o
  AR      drivers/firmware/smccc/built-in.a
  CC      drivers/firmware/efi/reboot.o
  CC      drivers/platform/x86/wmi-bmof.o
  CC      arch/x86/kernel/tsc_sync.o
  AR      drivers/net/ethernet/myricom/built-in.a
  CC      net/ipv4/proc.o
  CC      drivers/thermal/thermal_netlink.o
  CC      drivers/acpi/acpica/tbxface.o
  CC      drivers/gpu/drm/i915/gt/gen6_engine_cs.o
  CC      drivers/gpu/drm/drm_framebuffer.o
  CC      drivers/cpufreq/amd-pstate.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_mcr.o
  CC      drivers/cpuidle/governor.o
  GEN     lib/oid_registry_data.c
  CC      fs/nfs/nfs4client.o
  CC      drivers/md/dm-kcopyd.o
  CC      net/ipv6/exthdrs_offload.o
  CC      kernel/stacktrace.o
  CC      drivers/cpufreq/amd-pstate-trace.o
  CC      drivers/firmware/efi/libstub/secureboot.o
  CC      drivers/acpi/processor_driver.o
  AR      drivers/mailbox/built-in.a
  AR      drivers/firmware/tegra/built-in.a
  CC      drivers/md/dm-sysfs.o
  AR      drivers/firmware/xilinx/built-in.a
  CC      net/ipv6/inet6_hashtables.o
  CC      drivers/thermal/thermal_hwmon.o
  CC      drivers/acpi/acpica/tbxfload.o
  CC      drivers/hid/hid-core.o
  AR      drivers/usb/host/built-in.a
  AR      drivers/usb/built-in.a
  CC      drivers/md/dm-stats.o
  CC      drivers/hid/usbhid/hid-pidff.o
  CC      drivers/gpu/drm/drm_gem.o
  CC      arch/x86/kernel/setup_percpu.o
  CC      arch/x86/kernel/mpparse.o
  CC      drivers/cpufreq/intel_pstate.o
  AR      drivers/net/ethernet/natsemi/built-in.a
  CC      lib/ucs2_string.o
  CC      drivers/firmware/efi/libstub/tpm.o
  CC      drivers/firmware/efi/libstub/file.o
  CC      drivers/net/ethernet/intel/e1000e/ethtool.o
  CC      drivers/platform/x86/eeepc-laptop.o
  CC      mm/early_ioremap.o
  CC      drivers/firmware/efi/memattr.o
  CC      net/ipv4/fib_rules.o
  CC      drivers/cpuidle/sysfs.o
  CC      fs/nfs/nfs4session.o
  CC      drivers/hid/hid-input.o
  CC      drivers/gpu/drm/i915/gt/gen6_ppgtt.o
  AR      drivers/net/ethernet/marvell/built-in.a
  AR      drivers/android/built-in.a
  CC      drivers/platform/x86/p2sb.o
  CC      kernel/dma.o
  CC      drivers/md/dm-rq.o
  CC      drivers/firmware/dmi_scan.o
  CC      drivers/acpi/processor_thermal.o
  CC      drivers/acpi/acpica/tbxfroot.o
  AR      drivers/nvmem/layouts/built-in.a
  CC      net/ipv6/mcast_snoop.o
  CC      drivers/nvmem/core.o
  CC      fs/nfs/dns_resolve.o
  CC      drivers/firmware/efi/libstub/mem.o
  CC      drivers/firmware/dmi-id.o
  CC      drivers/net/ethernet/intel/e1000e/netdev.o
  CC      lib/sbitmap.o
  CC      drivers/md/dm-io-rewind.o
  CC      fs/nfs/nfs4trace.o
  CC      fs/sync.o
  CC      drivers/gpu/drm/i915/gt/gen7_renderclear.o
  CC      drivers/firmware/memmap.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_pagefault.o
  CC      drivers/gpu/drm/drm_ioctl.o
  CC      net/ipv4/ipmr.o
  CC      drivers/firmware/efi/libstub/random.o
  CC      fs/utimes.o
  CC      drivers/cpuidle/poll_state.o
  CC      fs/d_path.o
  CC      drivers/acpi/acpica/utaddress.o
  CC      mm/secretmem.o
  CC      drivers/hid/hid-quirks.o
  CC      kernel/smp.o
  CC      drivers/net/ethernet/intel/e1000e/ptp.o
  CC      drivers/thermal/gov_step_wise.o
  CC      lib/group_cpus.o
  CC      drivers/gpu/drm/i915/gt/gen8_engine_cs.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_sysfs.o
  CC      arch/x86/kernel/trace_clock.o
  CC      kernel/uid16.o
  CC      drivers/firmware/efi/tpm.o
  CC      drivers/acpi/acpica/utalloc.o
  CC      drivers/cpuidle/cpuidle-haltpoll.o
  CC      lib/fw_table.o
  CC      drivers/gpu/drm/drm_lease.o
  CC      arch/x86/kernel/trace.o
  CC      drivers/hid/hid-debug.o
  CC      drivers/md/dm-builtin.o
  AR      drivers/platform/x86/built-in.a
  CC      mm/hmm.o
  AR      drivers/platform/built-in.a
  CC      drivers/gpu/drm/i915/gt/gen8_ppgtt.o
  CC      fs/stack.o
  AR      drivers/hid/usbhid/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_gt_throttle.o
  CC      drivers/acpi/processor_idle.o
  CC      drivers/firmware/efi/memmap.o
  AR      drivers/thermal/built-in.a
  CC      drivers/acpi/acpica/utascii.o
  AR      drivers/net/ethernet/neterion/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_gt_tlb_invalidation.o
  CC      drivers/acpi/processor_throttling.o
  CC      drivers/firmware/efi/libstub/randomalloc.o
  AR      lib/lib.a
  CC      drivers/md/dm-raid1.o
  CC      arch/x86/kernel/rethook.o
  CC      net/ipv4/ipmr_base.o
  CC      drivers/firmware/efi/capsule.o
  CC      fs/fs_struct.o
  CC      drivers/gpu/drm/drm_managed.o
  CC      drivers/acpi/acpica/utbuffer.o
  AR      drivers/cpuidle/built-in.a
  CC      arch/x86/kernel/vmcore_info_32.o
  CC      drivers/gpu/drm/i915/gt/intel_breadcrumbs.o
  GEN     lib/crc32table.h
  CC      mm/memfd.o
  CC      lib/oid_registry.o
  AR      net/ipv6/built-in.a
  CC      drivers/firmware/efi/esrt.o
  AR      drivers/nvmem/built-in.a
  CC      drivers/gpu/drm/drm_mm.o
  CC      drivers/gpu/drm/i915/gt/intel_context.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_topology.o
  CC      drivers/acpi/acpica/utcksum.o
  CC      net/ipv4/syncookies.o
  CC      drivers/firmware/efi/libstub/pci.o
  CC      drivers/acpi/processor_perflib.o
  CC      drivers/gpu/drm/drm_mode_config.o
  CC      fs/nfs/nfs4sysctl.o
  CC      arch/x86/kernel/machine_kexec_32.o
  CC      drivers/gpu/drm/i915/gt/intel_context_sseu.o
  CC      net/ipv4/tunnel4.o
  CC      drivers/firmware/efi/libstub/skip_spaces.o
  CC      drivers/acpi/container.o
  CC      mm/ptdump.o
  CC      fs/statfs.o
  AR      drivers/net/ethernet/netronome/built-in.a
  CC      drivers/md/dm-log.o
  CC      lib/crc32.o
  CC      drivers/md/dm-region-hash.o
  CC      drivers/firmware/efi/libstub/lib-cmdline.o
  CC      kernel/kallsyms.o
  CC      drivers/acpi/acpica/utcopy.o
  CC      drivers/hid/hidraw.o
  CC [M]  drivers/gpu/drm/xe/xe_guc.o
  CC      drivers/hid/hid-generic.o
  CC      drivers/acpi/thermal_lib.o
  AR      drivers/net/ethernet/ni/built-in.a
  CC      drivers/firmware/efi/runtime-wrappers.o
  CC      kernel/acct.o
  CC      drivers/gpu/drm/i915/gt/intel_engine_cs.o
  CC      drivers/hid/hid-a4tech.o
  CC      fs/fs_pin.o
  CC      drivers/gpu/drm/drm_mode_object.o
  AR      drivers/cpufreq/built-in.a
  CC      drivers/gpu/drm/i915/gt/intel_engine_heartbeat.o
  CC      net/ipv4/ipconfig.o
  CC      drivers/md/dm-zero.o
  CC      net/ipv4/netfilter.o
  CC      drivers/acpi/acpica/utexcep.o
  CC      net/ipv4/tcp_cubic.o
  CC      drivers/firmware/efi/libstub/lib-ctype.o
  CC      kernel/vmcore_info.o
  CC      drivers/net/ethernet/nvidia/forcedeth.o
  CC      drivers/firmware/efi/libstub/alignedmem.o
  CC      drivers/hid/hid-apple.o
  AR      lib/built-in.a
  CC      drivers/acpi/acpica/utdebug.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_ads.o
  CC      mm/execmem.o
  CC      fs/nsfs.o
  AS      arch/x86/kernel/relocate_kernel_32.o
  CC      net/ipv4/tcp_sigpool.o
  CC      arch/x86/kernel/crash_dump_32.o
  CC      drivers/gpu/drm/i915/gt/intel_engine_pm.o
  CC      kernel/elfcorehdr.o
  CC      drivers/firmware/efi/libstub/relocate.o
  AR      drivers/net/ethernet/oki-semi/built-in.a
  CC      drivers/gpu/drm/i915/gt/intel_engine_user.o
  CC      drivers/acpi/thermal.o
  CC      drivers/hid/hid-belkin.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_buf.o
  CC      arch/x86/kernel/crash.o
  CC      net/ipv4/cipso_ipv4.o
  CC      drivers/firmware/efi/libstub/printk.o
  CC      kernel/crash_reserve.o
  CC      drivers/firmware/efi/capsule-loader.o
  CC      drivers/gpu/drm/i915/gt/intel_execlists_submission.o
  CC      arch/x86/kernel/module.o
  CC      drivers/acpi/acpica/utdecode.o
  CC      drivers/gpu/drm/i915/gt/intel_ggtt.o
  CC      drivers/gpu/drm/drm_modes.o
  CC      fs/fs_types.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_capture.o
  CC      drivers/firmware/efi/libstub/vsprintf.o
  CC      drivers/firmware/efi/earlycon.o
  CC      drivers/acpi/acpica/utdelete.o
  CC      net/ipv4/xfrm4_policy.o
  AR      drivers/md/built-in.a
  CC      drivers/acpi/nhlt.o
  CC      drivers/acpi/acpi_memhotplug.o
  AR      drivers/net/ethernet/packetengines/built-in.a
  CC      net/ipv4/xfrm4_state.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_ct.o
  CC      drivers/acpi/acpica/uterror.o
  CC      fs/fs_context.o
  CC      drivers/acpi/ioapic.o
  CC      drivers/firmware/efi/libstub/x86-stub.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_db_mgr.o
  AR      drivers/net/ethernet/qlogic/built-in.a
  CC      drivers/gpu/drm/drm_modeset_lock.o
  CC      drivers/gpu/drm/i915/gt/intel_ggtt_fencing.o
  CC      kernel/kexec_core.o
  CC      drivers/firmware/efi/libstub/smbios.o
  AR      mm/built-in.a
  CC      drivers/gpu/drm/drm_plane.o
  CC      drivers/hid/hid-cherry.o
  CC      drivers/acpi/acpica/uteval.o
  CC      net/ipv4/xfrm4_input.o
  CC      drivers/acpi/battery.o
  CC      drivers/gpu/drm/i915/gt/intel_gt.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_hwconfig.o
  CC      kernel/crash_core.o
  CC      arch/x86/kernel/doublefault_32.o
  CC      drivers/acpi/bgrt.o
  CC      drivers/acpi/acpica/utglobal.o
  CC      net/ipv4/xfrm4_output.o
  CC      kernel/kexec.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.o
  CC      drivers/acpi/spcr.o
  CC      drivers/hid/hid-chicony.o
  CC      drivers/acpi/acpica/uthex.o
  AR      drivers/net/ethernet/qualcomm/emac/built-in.a
  CC      drivers/net/ethernet/realtek/8139too.o
  AR      drivers/net/ethernet/qualcomm/built-in.a
  CC      drivers/gpu/drm/drm_prime.o
  CC      drivers/net/ethernet/realtek/r8169_main.o
  AR      drivers/net/ethernet/renesas/built-in.a
  CC      net/ipv4/xfrm4_protocol.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.o
  STUBCPY drivers/firmware/efi/libstub/alignedmem.stub.o
  CC      drivers/gpu/drm/drm_print.o
  CC      fs/fs_parser.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_id_mgr.o
  CC      arch/x86/kernel/early_printk.o
  CC      drivers/net/ethernet/realtek/r8169_firmware.o
  AR      fs/nfs/built-in.a
  AR      drivers/net/ethernet/rdc/built-in.a
  CC      kernel/utsname.o
  CC      drivers/acpi/acpica/utids.o
  STUBCPY drivers/firmware/efi/libstub/efi-stub-helper.stub.o
  CC      drivers/hid/hid-cypress.o
  AR      drivers/firmware/efi/built-in.a
  CC      drivers/gpu/drm/drm_property.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_clock_utils.o
  CC      arch/x86/kernel/hpet.o
  CC      drivers/net/ethernet/realtek/r8169_phy_config.o
  AR      drivers/net/ethernet/rocker/built-in.a
  AR      drivers/net/ethernet/samsung/built-in.a
  CC      kernel/pid_namespace.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_klv_helpers.o
  STUBCPY drivers/firmware/efi/libstub/file.stub.o
  STUBCPY drivers/firmware/efi/libstub/gop.stub.o
  STUBCPY drivers/firmware/efi/libstub/lib-cmdline.stub.o
  STUBCPY drivers/firmware/efi/libstub/lib-ctype.stub.o
  CC      drivers/gpu/drm/drm_rect.o
  STUBCPY drivers/firmware/efi/libstub/mem.stub.o
  STUBCPY drivers/firmware/efi/libstub/pci.stub.o
  CC      arch/x86/kernel/amd_nb.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_debugfs.o
  STUBCPY drivers/firmware/efi/libstub/printk.stub.o
  STUBCPY drivers/firmware/efi/libstub/random.stub.o
  STUBCPY drivers/firmware/efi/libstub/randomalloc.stub.o
  AR      drivers/net/ethernet/seeq/built-in.a
  STUBCPY drivers/firmware/efi/libstub/relocate.stub.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.o
  AR      drivers/net/ethernet/silan/built-in.a
  STUBCPY drivers/firmware/efi/libstub/secureboot.stub.o
  CC      drivers/acpi/acpica/utinit.o
  STUBCPY drivers/firmware/efi/libstub/skip_spaces.stub.o
  STUBCPY drivers/firmware/efi/libstub/smbios.stub.o
  CC      kernel/stop_machine.o
  STUBCPY drivers/firmware/efi/libstub/tpm.stub.o
  STUBCPY drivers/firmware/efi/libstub/vsprintf.stub.o
  STUBCPY drivers/firmware/efi/libstub/x86-stub.stub.o
  AR      drivers/firmware/efi/libstub/lib.a
  CC      drivers/hid/hid-ezkey.o
  AR      drivers/firmware/built-in.a
  CC      arch/x86/kernel/amd_node.o
  CC      arch/x86/kernel/kvm.o
  CC      drivers/gpu/drm/drm_syncobj.o
  CC      kernel/audit.o
  CC      drivers/acpi/acpica/utlock.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_log.o
  CC      arch/x86/kernel/kvmclock.o
  CC      drivers/acpi/acpica/utmath.o
  AR      drivers/net/ethernet/sis/built-in.a
  CC      arch/x86/kernel/paravirt.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_pc.o
  CC      drivers/hid/hid-gyration.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_irq.o
  CC      fs/fsopen.o
  CC      drivers/gpu/drm/drm_sysfs.o
  CC      drivers/acpi/acpica/utmisc.o
  CC      drivers/gpu/drm/drm_trace_points.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_mcr.o
  CC      drivers/gpu/drm/drm_vblank.o
  CC      kernel/auditfilter.o
  CC      drivers/hid/hid-ite.o
  CC      kernel/auditsc.o
  CC      drivers/acpi/acpica/utmutex.o
  AR      drivers/net/ethernet/sfc/built-in.a
  CC      fs/init.o
  CC      drivers/gpu/drm/drm_vblank_work.o
  CC      kernel/audit_watch.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_pm.o
  AR      drivers/net/ethernet/smsc/built-in.a
  CC      drivers/hid/hid-kensington.o
  CC      drivers/gpu/drm/drm_vma_manager.o
  CC      fs/kernel_read_file.o
  CC      drivers/acpi/acpica/utnonansi.o
  CC      kernel/audit_fsnotify.o
  AR      drivers/net/ethernet/socionext/built-in.a
  CC      arch/x86/kernel/pvclock.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_submit.o
  AR      net/ipv4/built-in.a
  CC      kernel/audit_tree.o
  AR      net/built-in.a
  AR      drivers/net/ethernet/stmicro/built-in.a
  CC      drivers/acpi/acpica/utobject.o
  CC      fs/mnt_idmapping.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_pm_irq.o
  CC      drivers/acpi/acpica/utosi.o
  CC      drivers/gpu/drm/drm_writeback.o
  CC      drivers/hid/hid-lg.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_requests.o
  CC      fs/remap_range.o
  AR      drivers/net/ethernet/sun/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_heci_gsc.o
  CC      drivers/gpu/drm/drm_panel.o
  CC      drivers/hid/hid-lgff.o
  CC      arch/x86/kernel/pcspeaker.o
  CC      fs/pidfs.o
  AR      drivers/net/ethernet/tehuti/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_huc.o
  CC      drivers/hid/hid-lg4ff.o
  CC      kernel/kprobes.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_sysfs.o
  CC      drivers/acpi/acpica/utownerid.o
  AR      drivers/net/ethernet/intel/e1000e/built-in.a
  AR      drivers/net/ethernet/intel/built-in.a
  CC      arch/x86/kernel/check.o
  CC      drivers/gpu/drm/drm_pci.o
  AR      drivers/net/ethernet/ti/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_hw_engine.o
  CC      drivers/hid/hid-lg-g15.o
  CC      drivers/hid/hid-microsoft.o
  CC      drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.o
  CC      fs/buffer.o
  CC      drivers/gpu/drm/drm_debugfs.o
  CC      arch/x86/kernel/uprobes.o
  CC      fs/mpage.o
  CC      drivers/gpu/drm/i915/gt/intel_gtt.o
  CC      drivers/acpi/acpica/utpredef.o
  CC      drivers/gpu/drm/drm_debugfs_crc.o
  CC      fs/proc_namespace.o
  CC      fs/direct-io.o
  CC      fs/eventpoll.o
  CC      kernel/seccomp.o
  CC [M]  drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.o
  AR      drivers/net/ethernet/nvidia/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_hw_engine_group.o
  CC      drivers/hid/hid-monterey.o
  CC      drivers/acpi/acpica/utresdecode.o
  CC      drivers/gpu/drm/drm_panel_orientation_quirks.o
  CC      arch/x86/kernel/perf_regs.o
  CC      drivers/acpi/acpica/utresrc.o
  CC      fs/anon_inodes.o
  AR      drivers/net/ethernet/vertexcom/built-in.a
  AR      drivers/net/ethernet/via/built-in.a
  CC      arch/x86/kernel/tracepoint.o
  CC      drivers/gpu/drm/drm_buddy.o
  CC      drivers/hid/hid-ntrig.o
  CC      kernel/relay.o
  CC      arch/x86/kernel/itmt.o
  CC      fs/signalfd.o
  AR      drivers/net/ethernet/wangxun/built-in.a
  CC      drivers/hid/hid-pl.o
  CC      arch/x86/kernel/umip.o
  CC      kernel/utsname_sysctl.o
  CC      drivers/acpi/acpica/utstate.o
  CC [M]  drivers/gpu/drm/xe/xe_hw_fence.o
  CC      arch/x86/kernel/unwind_frame.o
  CC      fs/timerfd.o
  CC      drivers/acpi/acpica/utstring.o
  CC      drivers/gpu/drm/i915/gt/intel_llc.o
  AR      drivers/net/ethernet/wiznet/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_irq.o
  CC      kernel/delayacct.o
  AR      drivers/net/ethernet/xilinx/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_lrc.o
  CC      drivers/gpu/drm/drm_gem_shmem_helper.o
  CC      drivers/gpu/drm/i915/gt/intel_lrc.o
  CC      drivers/acpi/acpica/utstrsuppt.o
  CC      drivers/hid/hid-petalynx.o
  CC      drivers/gpu/drm/i915/gt/intel_migrate.o
  CC      drivers/acpi/acpica/utstrtoul64.o
  CC      drivers/gpu/drm/drm_atomic_helper.o
  AR      drivers/net/ethernet/xircom/built-in.a
  CC      kernel/taskstats.o
  CC [M]  drivers/gpu/drm/xe/xe_migrate.o
  CC      fs/eventfd.o
  CC      kernel/tsacct.o
  AR      drivers/net/ethernet/synopsys/built-in.a
  CC      drivers/acpi/acpica/utxface.o
  CC      drivers/gpu/drm/i915/gt/intel_mocs.o
  CC      drivers/hid/hid-redragon.o
  CC      drivers/gpu/drm/drm_atomic_state_helper.o
  CC      drivers/acpi/acpica/utxfinit.o
  CC [M]  drivers/gpu/drm/xe/xe_mmio.o
  CC      fs/aio.o
  CC      fs/locks.o
  CC [M]  drivers/gpu/drm/xe/xe_mocs.o
  AR      drivers/net/ethernet/pensando/built-in.a
  CC      drivers/gpu/drm/i915/gt/intel_ppgtt.o
  CC      fs/binfmt_misc.o
  CC      drivers/hid/hid-samsung.o
  CC      drivers/gpu/drm/drm_crtc_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_module.o
  CC      drivers/acpi/acpica/utxferror.o
  CC      drivers/gpu/drm/i915/gt/intel_rc6.o
  CC      fs/binfmt_script.o
  CC      kernel/tracepoint.o
  CC      drivers/hid/hid-sony.o
  CC      drivers/gpu/drm/drm_damage_helper.o
  CC      drivers/gpu/drm/i915/gt/intel_region_lmem.o
  CC [M]  drivers/gpu/drm/xe/xe_oa.o
  CC [M]  drivers/gpu/drm/xe/xe_observation.o
  CC      drivers/acpi/acpica/utxfmutex.o
  CC      drivers/gpu/drm/drm_flip_work.o
  CC      kernel/irq_work.o
  CC [M]  drivers/gpu/drm/xe/xe_pat.o
  AR      drivers/net/ethernet/realtek/built-in.a
  AR      arch/x86/kernel/built-in.a
  CC      drivers/gpu/drm/drm_format_helper.o
  AR      drivers/net/ethernet/built-in.a
  CC      kernel/static_call.o
  AR      arch/x86/built-in.a
  CC      fs/binfmt_elf.o
  CC [M]  drivers/gpu/drm/xe/xe_pci.o
  CC      drivers/gpu/drm/i915/gt/intel_renderstate.o
  AR      drivers/net/built-in.a
  CC      fs/mbcache.o
  CC      drivers/gpu/drm/i915/gt/intel_reset.o
  CC      drivers/gpu/drm/drm_gem_atomic_helper.o
  CC      kernel/padata.o
  CC [M]  drivers/gpu/drm/xe/xe_pcode.o
  CC      drivers/hid/hid-sunplus.o
  CC      fs/posix_acl.o
  AR      drivers/acpi/acpica/built-in.a
  CC      fs/coredump.o
  CC      kernel/jump_label.o
  AR      drivers/acpi/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_pm.o
  CC      fs/drop_caches.o
  CC      drivers/hid/hid-topseed.o
  CC      drivers/gpu/drm/drm_gem_framebuffer_helper.o
  CC      fs/sysctls.o
  CC      drivers/gpu/drm/i915/gt/intel_ring.o
  CC      drivers/gpu/drm/drm_kms_helper_common.o
  CC      fs/fhandle.o
  CC      drivers/gpu/drm/drm_modeset_helper.o
  CC      drivers/gpu/drm/i915/gt/intel_ring_submission.o
  CC [M]  drivers/gpu/drm/xe/xe_preempt_fence.o
  CC      kernel/context_tracking.o
  CC      drivers/gpu/drm/i915/gt/intel_rps.o
  CC      drivers/gpu/drm/drm_plane_helper.o
  CC      drivers/gpu/drm/i915/gt/intel_sa_media.o
  CC      drivers/gpu/drm/drm_probe_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_pt.o
  CC      kernel/iomem.o
  CC      drivers/gpu/drm/drm_self_refresh_helper.o
  CC      drivers/gpu/drm/drm_simple_kms_helper.o
  CC      kernel/rseq.o
  CC      drivers/gpu/drm/i915/gt/intel_sseu.o
  CC      drivers/gpu/drm/i915/gt/intel_sseu_debugfs.o
  CC [M]  drivers/gpu/drm/xe/xe_pt_walk.o
  CC [M]  drivers/gpu/drm/xe/xe_pxp.o
  CC      drivers/gpu/drm/bridge/panel.o
  CC      drivers/gpu/drm/i915/gt/intel_timeline.o
  CC      drivers/gpu/drm/i915/gt/intel_tlb.o
  CC      drivers/gpu/drm/i915/gt/intel_wopcm.o
  CC [M]  drivers/gpu/drm/xe/xe_pxp_debugfs.o
  CC      drivers/gpu/drm/i915/gt/intel_workarounds.o
  CC      drivers/gpu/drm/drm_mipi_dsi.o
  CC [M]  drivers/gpu/drm/xe/xe_pxp_submit.o
  CC [M]  drivers/gpu/drm/drm_exec.o
  CC [M]  drivers/gpu/drm/xe/xe_query.o
  CC      drivers/gpu/drm/i915/gt/shmem_utils.o
  CC [M]  drivers/gpu/drm/xe/xe_range_fence.o
  AR      drivers/hid/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_reg_sr.o
  CC      drivers/gpu/drm/i915/gt/sysfs_engines.o
  CC      drivers/gpu/drm/i915/gt/intel_ggtt_gmch.o
  CC [M]  drivers/gpu/drm/xe/xe_reg_whitelist.o
  CC [M]  drivers/gpu/drm/drm_gpuvm.o
  CC      drivers/gpu/drm/i915/gt/gen6_renderstate.o
  CC [M]  drivers/gpu/drm/xe/xe_ring_ops.o
  CC      drivers/gpu/drm/i915/gt/gen7_renderstate.o
  CC [M]  drivers/gpu/drm/xe/xe_rtp.o
  CC [M]  drivers/gpu/drm/drm_suballoc.o
  CC      drivers/gpu/drm/i915/gt/gen8_renderstate.o
  CC      drivers/gpu/drm/i915/gt/gen9_renderstate.o
  CC [M]  drivers/gpu/drm/xe/xe_sa.o
  CC [M]  drivers/gpu/drm/xe/xe_sched_job.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_busy.o
  CC [M]  drivers/gpu/drm/drm_gem_ttm_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_step.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_clflush.o
  CC [M]  drivers/gpu/drm/xe/xe_survivability_mode.o
  CC [M]  drivers/gpu/drm/xe/xe_sync.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_context.o
  CC [M]  drivers/gpu/drm/xe/xe_tile.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_create.o
  CC [M]  drivers/gpu/drm/xe/xe_tile_sysfs.o
  CC [M]  drivers/gpu/drm/xe/xe_trace.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_dmabuf.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_domain.o
  AR      kernel/built-in.a
  CC      drivers/gpu/drm/i915/gem/i915_gem_execbuffer.o
  CC [M]  drivers/gpu/drm/xe/xe_trace_bo.o
  CC [M]  drivers/gpu/drm/xe/xe_trace_guc.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_internal.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_lmem.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_mman.o
  CC [M]  drivers/gpu/drm/xe/xe_trace_lrc.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_stolen_mgr.o
  AR      fs/built-in.a
  CC      drivers/gpu/drm/i915/gem/i915_gem_object.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_pages.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_sys_mgr.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_phys.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_vram_mgr.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_pm.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_region.o
  LD [M]  drivers/gpu/drm/drm_suballoc_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_tuning.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_shmem.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_shrinker.o
  LD [M]  drivers/gpu/drm/drm_ttm_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_uc.o
  CC [M]  drivers/gpu/drm/xe/xe_uc_fw.o
  CC [M]  drivers/gpu/drm/xe/xe_vm.o
  CC [M]  drivers/gpu/drm/xe/xe_vram.o
  CC [M]  drivers/gpu/drm/xe/xe_vram_freq.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_stolen.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_throttle.o
  CC [M]  drivers/gpu/drm/xe/xe_vsec.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_tiling.o
  CC [M]  drivers/gpu/drm/xe/xe_wa.o
  CC [M]  drivers/gpu/drm/xe/xe_wait_user_fence.o
  CC [M]  drivers/gpu/drm/xe/xe_wopcm.o
  CC [M]  drivers/gpu/drm/xe/xe_hmm.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_ttm.o
  CC [M]  drivers/gpu/drm/xe/xe_hwmon.o
  CC [M]  drivers/gpu/drm/xe/xe_pmu.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_sriov_vf.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_ttm_move.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_relay.o
  CC [M]  drivers/gpu/drm/xe/xe_memirq.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_userptr.o
  CC      drivers/gpu/drm/i915/gem/i915_gem_wait.o
  CC [M]  drivers/gpu/drm/xe/xe_sriov.o
  CC      drivers/gpu/drm/i915/gem/i915_gemfs.o
  CC [M]  drivers/gpu/drm/xe/xe_sriov_vf.o
  CC      drivers/gpu/drm/i915/i915_active.o
  CC [M]  drivers/gpu/drm/xe/display/ext/i915_irq.o
  CC [M]  drivers/gpu/drm/xe/display/ext/i915_utils.o
  CC      drivers/gpu/drm/i915/i915_cmd_parser.o
  CC      drivers/gpu/drm/i915/i915_deps.o
  CC [M]  drivers/gpu/drm/xe/display/intel_bo.o
  CC [M]  drivers/gpu/drm/xe/display/intel_fb_bo.o
  CC      drivers/gpu/drm/i915/i915_gem.o
  CC [M]  drivers/gpu/drm/xe/display/intel_fbdev_fb.o
  CC      drivers/gpu/drm/i915/i915_gem_evict.o
  CC [M]  drivers/gpu/drm/xe/display/xe_display.o
  CC [M]  drivers/gpu/drm/xe/display/xe_display_misc.o
  CC      drivers/gpu/drm/i915/i915_gem_gtt.o
  CC [M]  drivers/gpu/drm/xe/display/xe_display_rps.o
  CC      drivers/gpu/drm/i915/i915_gem_ww.o
  CC [M]  drivers/gpu/drm/xe/display/xe_display_wa.o
  CC      drivers/gpu/drm/i915/i915_query.o
  CC [M]  drivers/gpu/drm/xe/display/xe_dsb_buffer.o
  CC [M]  drivers/gpu/drm/xe/display/xe_fb_pin.o
  CC      drivers/gpu/drm/i915/i915_request.o
  CC      drivers/gpu/drm/i915/i915_scheduler.o
  CC [M]  drivers/gpu/drm/xe/display/xe_hdcp_gsc.o
  CC [M]  drivers/gpu/drm/xe/display/xe_plane_initial.o
  CC [M]  drivers/gpu/drm/xe/display/xe_tdf.o
  CC      drivers/gpu/drm/i915/i915_trace_points.o
  CC [M]  drivers/gpu/drm/xe/i915-soc/intel_dram.o
  CC [M]  drivers/gpu/drm/xe/i915-soc/intel_pch.o
  CC      drivers/gpu/drm/i915/i915_ttm_buddy_manager.o
  CC [M]  drivers/gpu/drm/xe/i915-soc/intel_rom.o
  CC      drivers/gpu/drm/i915/i915_vma.o
  CC      drivers/gpu/drm/i915/i915_vma_resource.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_gsc_proxy.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/icl_dsi.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_debugfs.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_alpm.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_atomic.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_atomic_plane.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_audio.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_backlight.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_ads.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_capture.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_ct.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_bios.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_bw.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_fw.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cdclk.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cmtg.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_color.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_combo_phy.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_connector.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_hwconfig.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_log.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_crtc.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_rc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_crtc_state_dump.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cursor.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_guc_submission.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cx0_phy.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_huc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_ddi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_ddi_buf_trans.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_huc_debugfs.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_huc_fw.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_conversion.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_uc.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_uc_debugfs.o
  CC      drivers/gpu/drm/i915/gt/uc/intel_uc_fw.o
  CC      drivers/gpu/drm/i915/gt/intel_gsc.o
  CC      drivers/gpu/drm/i915/i915_hwmon.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_device.o
  CC      drivers/gpu/drm/i915/display/hsw_ips.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_driver.o
  CC      drivers/gpu/drm/i915/display/i9xx_plane.o
  CC      drivers/gpu/drm/i915/display/i9xx_display_sr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_irq.o
  CC      drivers/gpu/drm/i915/display/i9xx_wm.o
  CC      drivers/gpu/drm/i915/display/intel_alpm.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_params.o
  CC      drivers/gpu/drm/i915/display/intel_atomic.o
  CC      drivers/gpu/drm/i915/display/intel_atomic_plane.o
  CC      drivers/gpu/drm/i915/display/intel_audio.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power.o
  CC      drivers/gpu/drm/i915/display/intel_bios.o
  CC      drivers/gpu/drm/i915/display/intel_bo.o
  CC      drivers/gpu/drm/i915/display/intel_bw.o
  CC      drivers/gpu/drm/i915/display/intel_cdclk.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power_map.o
  CC      drivers/gpu/drm/i915/display/intel_cmtg.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power_well.o
  CC      drivers/gpu/drm/i915/display/intel_color.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_trace.o
  CC      drivers/gpu/drm/i915/display/intel_combo_phy.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_wa.o
  CC      drivers/gpu/drm/i915/display/intel_connector.o
  CC      drivers/gpu/drm/i915/display/intel_crtc.o
  CC      drivers/gpu/drm/i915/display/intel_crtc_state_dump.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dkl_phy.o
  CC      drivers/gpu/drm/i915/display/intel_cursor.o
  CC      drivers/gpu/drm/i915/display/intel_display.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dmc.o
  CC      drivers/gpu/drm/i915/display/intel_display_conversion.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dmc_wl.o
  CC      drivers/gpu/drm/i915/display/intel_display_driver.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_aux.o
  CC      drivers/gpu/drm/i915/display/intel_display_irq.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_aux_backlight.o
  CC      drivers/gpu/drm/i915/display/intel_display_params.o
  CC      drivers/gpu/drm/i915/display/intel_display_power.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_hdcp.o
  CC      drivers/gpu/drm/i915/display/intel_display_power_map.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_link_training.o
  CC      drivers/gpu/drm/i915/display/intel_display_power_well.o
  CC      drivers/gpu/drm/i915/display/intel_display_reset.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_mst.o
  CC      drivers/gpu/drm/i915/display/intel_display_rps.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_test.o
  CC      drivers/gpu/drm/i915/display/intel_display_snapshot.o
  CC      drivers/gpu/drm/i915/display/intel_display_wa.o
  CC      drivers/gpu/drm/i915/display/intel_dmc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpll.o
  CC      drivers/gpu/drm/i915/display/intel_dmc_wl.o
  CC      drivers/gpu/drm/i915/display/intel_dpio_phy.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpll_mgr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpt_common.o
  CC      drivers/gpu/drm/i915/display/intel_dpll.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_drrs.o
  CC      drivers/gpu/drm/i915/display/intel_dpll_mgr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsb.o
  CC      drivers/gpu/drm/i915/display/intel_dpt.o
  CC      drivers/gpu/drm/i915/display/intel_dpt_common.o
  CC      drivers/gpu/drm/i915/display/intel_drrs.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi.o
  CC      drivers/gpu/drm/i915/display/intel_dsb.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi_dcs_backlight.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi_vbt.o
  CC      drivers/gpu/drm/i915/display/intel_dsb_buffer.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_encoder.o
  CC      drivers/gpu/drm/i915/display/intel_fb.o
  CC      drivers/gpu/drm/i915/display/intel_fb_bo.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fb.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fbc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fdi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fifo_underrun.o
  CC      drivers/gpu/drm/i915/display/intel_fb_pin.o
  CC      drivers/gpu/drm/i915/display/intel_fbc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_frontbuffer.o
  CC      drivers/gpu/drm/i915/display/intel_fdi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_global_state.o
  CC      drivers/gpu/drm/i915/display/intel_fifo_underrun.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_gmbus.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hdcp.o
  CC      drivers/gpu/drm/i915/display/intel_frontbuffer.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hdcp_gsc_message.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hdmi.o
  CC      drivers/gpu/drm/i915/display/intel_global_state.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hotplug.o
  CC      drivers/gpu/drm/i915/display/intel_hdcp.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hotplug_irq.o
  CC      drivers/gpu/drm/i915/display/intel_hdcp_gsc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hti.o
  CC      drivers/gpu/drm/i915/display/intel_hdcp_gsc_message.o
  CC      drivers/gpu/drm/i915/display/intel_hotplug.o
  CC      drivers/gpu/drm/i915/display/intel_hotplug_irq.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_link_bw.o
  CC      drivers/gpu/drm/i915/display/intel_hti.o
  CC      drivers/gpu/drm/i915/display/intel_link_bw.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_lspcon.o
  CC      drivers/gpu/drm/i915/display/intel_load_detect.o
  CC      drivers/gpu/drm/i915/display/intel_lpe_audio.o
  CC      drivers/gpu/drm/i915/display/intel_modeset_lock.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_modeset_lock.o
  CC      drivers/gpu/drm/i915/display/intel_modeset_setup.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_modeset_setup.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_modeset_verify.o
  CC      drivers/gpu/drm/i915/display/intel_modeset_verify.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_panel.o
  CC      drivers/gpu/drm/i915/display/intel_overlay.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pfit.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pmdemand.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pps.o
  CC      drivers/gpu/drm/i915/display/intel_pch_display.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_psr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_qp_tables.o
  CC      drivers/gpu/drm/i915/display/intel_pch_refclk.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_quirks.o
  CC      drivers/gpu/drm/i915/display/intel_plane_initial.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_snps_hdmi_pll.o
  CC      drivers/gpu/drm/i915/display/intel_pmdemand.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_snps_phy.o
  CC      drivers/gpu/drm/i915/display/intel_psr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_tc.o
  CC      drivers/gpu/drm/i915/display/intel_quirks.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vblank.o
  CC      drivers/gpu/drm/i915/display/intel_sprite.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vdsc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vga.o
  CC      drivers/gpu/drm/i915/display/intel_sprite_uapi.o
  CC      drivers/gpu/drm/i915/display/intel_tc.o
  CC      drivers/gpu/drm/i915/display/intel_vblank.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vrr.o
  CC      drivers/gpu/drm/i915/display/intel_vga.o
  CC      drivers/gpu/drm/i915/display/intel_wm.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_wm.o
  CC      drivers/gpu/drm/i915/display/skl_scaler.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_scaler.o
  CC      drivers/gpu/drm/i915/display/skl_universal_plane.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_universal_plane.o
  CC      drivers/gpu/drm/i915/display/skl_watermark.o
  CC      drivers/gpu/drm/i915/display/intel_acpi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_watermark.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_acpi.o
  CC      drivers/gpu/drm/i915/display/intel_opregion.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_opregion.o
  CC [M]  drivers/gpu/drm/xe/xe_debugfs.o
  CC      drivers/gpu/drm/i915/display/intel_display_debugfs.o
  CC      drivers/gpu/drm/i915/display/intel_display_debugfs_params.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_debugfs.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.o
  CC      drivers/gpu/drm/i915/display/intel_pipe_crc.o
  CC      drivers/gpu/drm/i915/display/dvo_ch7017.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_stats.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_debugfs.o
  CC [M]  drivers/gpu/drm/xe/xe_huc_debugfs.o
  CC [M]  drivers/gpu/drm/xe/xe_uc_debugfs.o
  CC      drivers/gpu/drm/i915/display/dvo_ch7xxx.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_debugfs.o
  CC      drivers/gpu/drm/i915/display/dvo_ivch.o
  CC      drivers/gpu/drm/i915/display/dvo_ns2501.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_debugfs_params.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pipe_crc.o
  CC      drivers/gpu/drm/i915/display/dvo_sil164.o
  CC      drivers/gpu/drm/i915/display/dvo_tfp410.o
  CC      drivers/gpu/drm/i915/display/g4x_dp.o
  CC      drivers/gpu/drm/i915/display/g4x_hdmi.o
  CC      drivers/gpu/drm/i915/display/icl_dsi.o
  CC      drivers/gpu/drm/i915/display/intel_backlight.o
  CC      drivers/gpu/drm/i915/display/intel_crt.o
  CC      drivers/gpu/drm/i915/display/intel_cx0_phy.o
  CC      drivers/gpu/drm/i915/display/intel_ddi.o
  CC      drivers/gpu/drm/i915/display/intel_ddi_buf_trans.o
  CC      drivers/gpu/drm/i915/display/intel_display_device.o
  CC      drivers/gpu/drm/i915/display/intel_display_trace.o
  CC      drivers/gpu/drm/i915/display/intel_dkl_phy.o
  CC      drivers/gpu/drm/i915/display/intel_dp.o
  CC      drivers/gpu/drm/i915/display/intel_dp_aux.o
  CC      drivers/gpu/drm/i915/display/intel_dp_aux_backlight.o
  CC      drivers/gpu/drm/i915/display/intel_dp_hdcp.o
  CC      drivers/gpu/drm/i915/display/intel_dp_link_training.o
  CC      drivers/gpu/drm/i915/display/intel_dp_mst.o
  CC      drivers/gpu/drm/i915/display/intel_dp_test.o
  CC      drivers/gpu/drm/i915/display/intel_dsi.o
  CC      drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.o
  CC      drivers/gpu/drm/i915/display/intel_dsi_vbt.o
  CC      drivers/gpu/drm/i915/display/intel_dvo.o
  CC      drivers/gpu/drm/i915/display/intel_encoder.o
  CC      drivers/gpu/drm/i915/display/intel_gmbus.o
  CC      drivers/gpu/drm/i915/display/intel_hdmi.o
  CC      drivers/gpu/drm/i915/display/intel_lspcon.o
  CC      drivers/gpu/drm/i915/display/intel_lvds.o
  CC      drivers/gpu/drm/i915/display/intel_panel.o
  CC      drivers/gpu/drm/i915/display/intel_pfit.o
  CC      drivers/gpu/drm/i915/display/intel_pps.o
  CC      drivers/gpu/drm/i915/display/intel_qp_tables.o
  CC      drivers/gpu/drm/i915/display/intel_sdvo.o
  CC      drivers/gpu/drm/i915/display/intel_snps_hdmi_pll.o
  CC      drivers/gpu/drm/i915/display/intel_snps_phy.o
  CC      drivers/gpu/drm/i915/display/intel_tv.o
  CC      drivers/gpu/drm/i915/display/intel_vdsc.o
  CC      drivers/gpu/drm/i915/display/intel_vrr.o
  CC      drivers/gpu/drm/i915/display/vlv_dsi.o
  CC      drivers/gpu/drm/i915/display/vlv_dsi_pll.o
  CC      drivers/gpu/drm/i915/i915_perf.o
  CC      drivers/gpu/drm/i915/pxp/intel_pxp.o
  CC      drivers/gpu/drm/i915/pxp/intel_pxp_huc.o
  CC      drivers/gpu/drm/i915/pxp/intel_pxp_tee.o
  CC      drivers/gpu/drm/i915/i915_gpu_error.o
  CC      drivers/gpu/drm/i915/i915_vgpu.o
  LD [M]  drivers/gpu/drm/xe/xe.o
  AR      drivers/gpu/drm/i915/built-in.a
  AR      drivers/gpu/drm/built-in.a
  AR      drivers/gpu/built-in.a
  AR      drivers/built-in.a
  AR      built-in.a
  AR      vmlinux.a
  LD      vmlinux.o
  OBJCOPY modules.builtin.modinfo
  GEN     modules.builtin
  MODPOST Module.symvers
  CC      .vmlinux.export.o
  CC [M]  fs/efivarfs/efivarfs.mod.o
  CC [M]  .module-common.o
  CC [M]  drivers/gpu/drm/drm_exec.mod.o
  CC [M]  drivers/gpu/drm/drm_gpuvm.mod.o
  CC [M]  drivers/gpu/drm/drm_suballoc_helper.mod.o
  CC [M]  drivers/gpu/drm/drm_ttm_helper.mod.o
  CC [M]  drivers/gpu/drm/scheduler/gpu-sched.mod.o
  CC [M]  drivers/gpu/drm/xe/xe.mod.o
  CC [M]  drivers/thermal/intel/x86_pkg_temp_thermal.mod.o
  CC [M]  net/netfilter/nf_log_syslog.mod.o
  CC [M]  net/netfilter/xt_mark.mod.o
  CC [M]  net/netfilter/xt_nat.mod.o
  CC [M]  net/netfilter/xt_LOG.mod.o
  CC [M]  net/netfilter/xt_MASQUERADE.mod.o
  CC [M]  net/netfilter/xt_addrtype.mod.o
  CC [M]  net/ipv4/netfilter/iptable_nat.mod.o
  LD [M]  fs/efivarfs/efivarfs.ko
  LD [M]  drivers/gpu/drm/drm_exec.ko
  LD [M]  net/netfilter/nf_log_syslog.ko
  LD [M]  drivers/gpu/drm/drm_suballoc_helper.ko
  LD [M]  drivers/gpu/drm/scheduler/gpu-sched.ko
  LD [M]  net/netfilter/xt_mark.ko
  LD [M]  drivers/gpu/drm/drm_gpuvm.ko
  LD [M]  drivers/gpu/drm/drm_ttm_helper.ko
  LD [M]  net/netfilter/xt_addrtype.ko
  LD [M]  net/netfilter/xt_nat.ko
  LD [M]  drivers/thermal/intel/x86_pkg_temp_thermal.ko
  LD [M]  net/ipv4/netfilter/iptable_nat.ko
  LD [M]  drivers/gpu/drm/xe/xe.ko
  LD [M]  net/netfilter/xt_LOG.ko
  LD [M]  net/netfilter/xt_MASQUERADE.ko
  UPD     include/generated/utsversion.h
  CC      init/version-timestamp.o
  KSYMS   .tmp_vmlinux0.kallsyms.S
  AS      .tmp_vmlinux0.kallsyms.o
  LD      .tmp_vmlinux1
  NM      .tmp_vmlinux1.syms
  KSYMS   .tmp_vmlinux1.kallsyms.S
  AS      .tmp_vmlinux1.kallsyms.o
  LD      .tmp_vmlinux2
  NM      .tmp_vmlinux2.syms
  KSYMS   .tmp_vmlinux2.kallsyms.S
  AS      .tmp_vmlinux2.kallsyms.o
  LD      vmlinux
  NM      System.map
  SORTTAB vmlinux
  RELOCS  arch/x86/boot/compressed/vmlinux.relocs
  RSTRIP  vmlinux
  CC      arch/x86/boot/a20.o
  AS      arch/x86/boot/bioscall.o
  CC      arch/x86/boot/cmdline.o
  AS      arch/x86/boot/copy.o
  HOSTCC  arch/x86/boot/mkcpustr
  CC      arch/x86/boot/cpuflags.o
  CC      arch/x86/boot/cpucheck.o
  CC      arch/x86/boot/early_serial_console.o
  CC      arch/x86/boot/edd.o
  CC      arch/x86/boot/main.o
  CC      arch/x86/boot/memory.o
  CC      arch/x86/boot/pm.o
  AS      arch/x86/boot/pmjump.o
  CC      arch/x86/boot/printf.o
  CC      arch/x86/boot/regs.o
  CC      arch/x86/boot/string.o
  CC      arch/x86/boot/tty.o
  CC      arch/x86/boot/video.o
  CC      arch/x86/boot/video-mode.o
  CC      arch/x86/boot/version.o
  CC      arch/x86/boot/video-vga.o
  CC      arch/x86/boot/video-vesa.o
  CC      arch/x86/boot/video-bios.o
  HOSTCC  arch/x86/boot/tools/build
  CPUSTR  arch/x86/boot/cpustr.h
  CC      arch/x86/boot/cpu.o
  LDS     arch/x86/boot/compressed/vmlinux.lds
  AS      arch/x86/boot/compressed/kernel_info.o
  AS      arch/x86/boot/compressed/head_32.o
  VOFFSET arch/x86/boot/compressed/../voffset.h
  CC      arch/x86/boot/compressed/string.o
  CC      arch/x86/boot/compressed/cmdline.o
  CC      arch/x86/boot/compressed/error.o
  OBJCOPY arch/x86/boot/compressed/vmlinux.bin
  HOSTCC  arch/x86/boot/compressed/mkpiggy
  CC      arch/x86/boot/compressed/cpuflags.o
  CC      arch/x86/boot/compressed/early_serial_console.o
  CC      arch/x86/boot/compressed/kaslr.o
  CC      arch/x86/boot/compressed/acpi.o
  CC      arch/x86/boot/compressed/efi.o
  GZIP    arch/x86/boot/compressed/vmlinux.bin.gz
  CC      arch/x86/boot/compressed/misc.o
  MKPIGGY arch/x86/boot/compressed/piggy.S
  AS      arch/x86/boot/compressed/piggy.o
  LD      arch/x86/boot/compressed/vmlinux
  ZOFFSET arch/x86/boot/zoffset.h
  OBJCOPY arch/x86/boot/vmlinux.bin
  AS      arch/x86/boot/header.o
  LD      arch/x86/boot/setup.elf
  OBJCOPY arch/x86/boot/setup.bin
  BUILD   arch/x86/boot/bzImage
Kernel: arch/x86/boot/bzImage is ready  (#1)
run-parts: executing /workspace/ci/hooks/20-kernel-doc
+ SRC_DIR=/workspace/kernel
+ cd /workspace/kernel
+ xargs ./scripts/kernel-doc -Werror -none include/uapi/drm/xe_drm.h
+ find drivers/gpu/drm/xe/ -name '*.[ch]' -not -path 'drivers/gpu/drm/xe/display/*'
All hooks done



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✓ CI.checksparse: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (6 preceding siblings ...)
  2025-02-04 23:47 ` ✓ CI.Hooks: " Patchwork
@ 2025-02-04 23:48 ` Patchwork
  2025-02-05  6:05 ` ✓ Xe.CI.BAT: " Patchwork
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-04 23:48 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

+ trap cleanup EXIT
+ KERNEL=/kernel
+ MT=/root/linux/maintainer-tools
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools /root/linux/maintainer-tools
Cloning into '/root/linux/maintainer-tools'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ make -C /root/linux/maintainer-tools
make: Entering directory '/root/linux/maintainer-tools'
cc -O2 -g -Wextra -o remap-log remap-log.c
make: Leaving directory '/root/linux/maintainer-tools'
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ /root/linux/maintainer-tools/dim sparse --fast a19d8731db07e41101ed00b9d86ac8868df2a763
Sparse version: 0.6.4 (Ubuntu: 0.6.4-4ubuntu3)
Fast mode used, each commit won't be checked separately.
Okay!

+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel



^ permalink raw reply	[flat|nested] 38+ messages in thread

* ✓ Xe.CI.BAT: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (7 preceding siblings ...)
  2025-02-04 23:48 ` ✓ CI.checksparse: " Patchwork
@ 2025-02-05  6:05 ` Patchwork
  2025-02-05 13:16 ` [PATCH 1/3] " Thomas Hellström
  2025-02-25 21:37 ` Matthew Brost
  10 siblings, 0 replies; 38+ messages in thread
From: Patchwork @ 2025-02-05  6:05 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

== Series Details ==

Series: series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor
URL   : https://patchwork.freedesktop.org/series/144338/
State : success

== Summary ==

CI Bug Log - changes from xe-2596-61ea57f296816a0c8d068c32043f18dbff95e056_BAT -> xe-pw-144338v1_BAT
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  

Participating hosts (9 -> 8)
------------------------------

  Missing    (1): bat-adlp-vm 


Changes
-------

  No changes found


Build changes
-------------

  * Linux: xe-2596-61ea57f296816a0c8d068c32043f18dbff95e056 -> xe-pw-144338v1

  IGT_8223: ccfe042787b082c06402ff9af257f8338b8edd5e @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  xe-2596-61ea57f296816a0c8d068c32043f18dbff95e056: 61ea57f296816a0c8d068c32043f18dbff95e056
  xe-pw-144338v1: 144338v1

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-144338v1/index.html

[-- Attachment #2: Type: text/html, Size: 1539 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-04 18:45 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
@ 2025-02-05 13:14   ` Thomas Hellström
  2025-02-06  1:54     ` Zeng, Oak
  2025-02-25 22:05   ` Matthew Brost
  1 sibling, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-05 13:14 UTC (permalink / raw)
  To: Oak Zeng, intel-xe; +Cc: matthew.brost, jonathan.cavitt

On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> Normally scratch page is not allowed when a vm is operate under page
> fault mode, i.e., in the existing codes,
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> is fault mode relies on recoverable page to work, while scratch page
> can mute recoverable page fault.
> 
> On xe2 and xe3, out of bound prefetch can cause page fault and
> further
> system hang because xekmd can't resolve such page fault. SYCL and OCL
> language runtime requires out of bound prefetch to be silently
> dropped
> without causing any functional problem, thus the existing behavior
> doesn't meet language runtime requirement.
> 
> At the same time, HW prefetching can cause page fault interrupt. Due
> to
> page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> the page fault), HW prefetching can be slowed by many orders of
> magnitude.
> 
> Fix those problems by allowing scratch page under fault mode for xe2
> and
> xe3. With scratch page in place, HW prefetching could always hit
> scratch
> page instead of causing interrupt.
> 
> A side effect is, scratch page could hide application program error.
> Application out of bound accesses are hided
s/hided/hidden/

>  by scratch page mapping,
> instead of get reported to user.
> 
> igt test: https://patchwork.freedesktop.org/series/144334/. Test
> result on
> BMG:
> 
> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests#
> ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738684805 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> Subtest scratch-fault: SUCCESS (0.080s)
> 
> Without this series, the test result is:
> 
> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests#
> ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738686046 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function
> test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> exec_queues[i % n_exec_queues], &timeout) == 0
> (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> Stack trace:
>   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
>   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
>   #2 ../tests/intel/xe_exec_fault_mode.c:537
> __igt_unique____real_main407()
>   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
>   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> __libc_start_call_main()
>   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
>   #6 [_start+0x2e]
> Subtest scratch-fault failed.
> 
> v2: Refine commit message (Thomas)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 813d893d9b63..c0372f083d42 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> void *data,
>  	if (XE_IOCTL_DBG(xe, args->extensions))
>  		return -EINVAL;
>  
> +	if (XE_IOCTL_DBG(xe, args->flags &
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> +			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> +			 !(NEEDS_SCRATCH(xe))))
> +		return -EINVAL;
> +

We should probably move this test to where the old test were below,
since the WA below enables scratch pages.

/Thomas


>  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
>  		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
>  
> @@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct drm_device *dev,
> void *data,
>  	if (XE_IOCTL_DBG(xe, args->flags &
> ~ALL_DRM_XE_VM_CREATE_FLAGS))
>  		return -EINVAL;
>  
> -	if (XE_IOCTL_DBG(xe, args->flags &
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> -			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> -		return -EINVAL;
> -
>  	if (XE_IOCTL_DBG(xe, !(args->flags &
> DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
>  			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>  		return -EINVAL;


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (8 preceding siblings ...)
  2025-02-05  6:05 ` ✓ Xe.CI.BAT: " Patchwork
@ 2025-02-05 13:16 ` Thomas Hellström
  2025-02-25 21:37 ` Matthew Brost
  10 siblings, 0 replies; 38+ messages in thread
From: Thomas Hellström @ 2025-02-05 13:16 UTC (permalink / raw)
  To: Oak Zeng, intel-xe; +Cc: matthew.brost, jonathan.cavitt

On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> On some platform, scratch page is needed for out of bound prefetch
> to work. Introduce a bit in device descriptor to specify whether
> this device needs scratch page to work.
> 
> v2: introduce a needs_scratch bit in device info (Thomas, Jonathan)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
LGTM.
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> ---
>  drivers/gpu/drm/xe/xe_device_types.h | 3 +++
>  drivers/gpu/drm/xe/xe_pci.c          | 5 +++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h
> b/drivers/gpu/drm/xe/xe_device_types.h
> index c0e886bac183..1197ab31e528 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -45,6 +45,7 @@ struct xe_pxp;
>  #define GRAPHICS_VERx100(xe) ((xe)->info.graphics_verx100)
>  #define MEDIA_VERx100(xe) ((xe)->info.media_verx100)
>  #define IS_DGFX(xe) ((xe)->info.is_dgfx)
> +#define NEEDS_SCRATCH(xe) ((xe)->info.needs_scratch)
>  
>  #define XE_VRAM_FLAGS_NEED64K		BIT(0)
>  
> @@ -318,6 +319,8 @@ struct xe_device {
>  		u8 has_usm:1;
>  		/** @info.is_dgfx: is discrete device */
>  		u8 is_dgfx:1;
> +		/** @info.needs_scratch: needs scratch page for oob
> prefetch to work */
> +		u8 needs_scratch:1;
>  		/**
>  		 * @info.probe_display: Probe display hardware.  If
> set to
>  		 * false, the driver will behave as if there is no
> display
> diff --git a/drivers/gpu/drm/xe/xe_pci.c
> b/drivers/gpu/drm/xe/xe_pci.c
> index e8ef7d6b4db8..09736674a77e 100644
> --- a/drivers/gpu/drm/xe/xe_pci.c
> +++ b/drivers/gpu/drm/xe/xe_pci.c
> @@ -67,6 +67,7 @@ struct xe_device_desc {
>  	u8 has_llc:1;
>  	u8 has_pxp:1;
>  	u8 has_sriov:1;
> +	u8 needs_scratch:1;
>  	u8 skip_guc_pc:1;
>  	u8 skip_mtcfg:1;
>  	u8 skip_pcode:1;
> @@ -353,6 +354,7 @@ static const struct xe_device_desc lnl_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.has_pxp = true,
> +	.needs_scratch = true,
>  };
>  
>  static const struct xe_device_desc bmg_desc = {
> @@ -361,6 +363,7 @@ static const struct xe_device_desc bmg_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.has_heci_cscfi = 1,
> +	.needs_scratch = true,
>  };
>  
>  static const struct xe_device_desc ptl_desc = {
> @@ -368,6 +371,7 @@ static const struct xe_device_desc ptl_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.require_force_probe = true,
> +	.needs_scratch = true,
>  };
>  
>  #undef PLATFORM
> @@ -643,6 +647,7 @@ static int xe_info_init_early(struct xe_device
> *xe,
>  	xe->info.skip_guc_pc = desc->skip_guc_pc;
>  	xe->info.skip_mtcfg = desc->skip_mtcfg;
>  	xe->info.skip_pcode = desc->skip_pcode;
> +	xe->info.needs_scratch = desc->needs_scratch;
>  
>  	xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
> &&
>  				 xe_modparam.probe_display &&


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
@ 2025-02-05 14:35   ` Thomas Hellström
  2025-02-06  1:52     ` Zeng, Oak
  2025-02-06 10:34   ` Matthew Brost
  1 sibling, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-05 14:35 UTC (permalink / raw)
  To: Oak Zeng, intel-xe; +Cc: matthew.brost, jonathan.cavitt

On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> When a vm runs under fault mode, if scratch page is enabled, we need
> to clear the scratch page mapping before vm_bind for the vm_bind
> address range. Under fault mode, we depend on recoverable page fault
> to establish mapping in page table. If scratch page is not cleared,
> GPU access of address won't cause page fault because it always hits
> the existing scratch page mapping.

I think we need to ephasize that we clear AT vm_bind time, not before,
both in the commit description and the patch title.

> 
> When vm_bind with IMMEDIATE flag, there is no need of clearing as
> immediate bind can overwrite the scratch page mapping.
> 
> So far only is xe2 and xe3 products are allowed to enable scratch
> page
> under fault mode. On other platform we don't allow scratch page under
> fault mode, so no need of such clearing.
> 
> v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> similar
> to a map operation, with the exception that PTEs are cleared instead
> of
> pointing to valid physical pages. (Matt, Thomas)
> 
> TLB invalidation is needed after clear scratch page mapping as larger
> scratch page mapping could be backed by physical page and cached in
> TLB. (Matt, Thomas)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++--------
> --
>  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
>  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
>  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
>  4 files changed, 75 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
> index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> --- a/drivers/gpu/drm/xe/xe_pt.c
> +++ b/drivers/gpu/drm/xe/xe_pt.c
> @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
>  	 * granularity.
>  	 */
>  	bool needs_64K;
> +	/* @clear_pt: clear page table entries during the bind walk
> */
> +	bool clear_pt;
>  	/**
>  	 * @vma: VMA being mapped
>  	 */
> @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> pgoff_t offset,
>  
>  		XE_WARN_ON(xe_walk->va_curs_start != addr);
>  
> -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> -						 xe_res_dma(curs) +
> xe_walk->dma_offset,
> -						 xe_walk->vma,
> pat_index, level);
> -		pte |= xe_walk->default_pte;
> +		if (xe_walk->clear_pt) {
> +			pte = 0;
> +		} else {
> +			pte = vm->pt_ops->pte_encode_vma(is_null ? 0
> :
> +					xe_res_dma(curs) + xe_walk-
> >dma_offset,
> +					xe_walk->vma, pat_index,
> level);
> +			pte |= xe_walk->default_pte;
>  
> -		/*
> -		 * Set the XE_PTE_PS64 hint if possible, otherwise
> if
> -		 * this device *requires* 64K PTE size for VRAM,
> fail.
> -		 */
> -		if (level == 0 && !xe_parent->is_compact) {
> -			if (xe_pt_is_pte_ps64K(addr, next, xe_walk))
> {
> -				xe_walk->vma->gpuva.flags |=
> XE_VMA_PTE_64K;
> -				pte |= XE_PTE_PS64;
> -			} else if (XE_WARN_ON(xe_walk->needs_64K)) {
> -				return -EINVAL;
> +			/*
> +			 * Set the XE_PTE_PS64 hint if possible,
> otherwise if
> +			 * this device *requires* 64K PTE size for
> VRAM, fail.
> +			 */
> +			if (level == 0 && !xe_parent->is_compact) {
> +				if (xe_pt_is_pte_ps64K(addr, next,
> xe_walk)) {
> +					xe_walk->vma->gpuva.flags |=
> XE_VMA_PTE_64K;
> +					pte |= XE_PTE_PS64;
> +				} else if (XE_WARN_ON(xe_walk-
> >needs_64K)) {
> +					return -EINVAL;
> +				}
>  			}
>  		}
>  

I think there might be some things missing here.
The bind should look exactly as a is_null bind, with the exception that
the pte should be 0, so might be able to benefit from is_null, for
example xe_pg_hugepte_possible() short-circuit the dma-address check.
With clear_pt we don't want to use the XE_PTE_PS64K, though, like you
identified above.

Will get back with a look at the invalidation.


> @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> pgoff_t offset,
>  		if (unlikely(ret))
>  			return ret;
>  
> -		if (!is_null)
> +		if (!is_null && !xe_walk->clear_pt)
>  			xe_res_next(curs, next - addr);
>  		xe_walk->va_curs_start = next;
>  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> level);
> @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> xe_pt_stage_bind_ops = {
>   * @vma: The vma indicating the address range.
>   * @entries: Storage for the update entries used for connecting the
> tree to
>   * the main tree at commit time.
> + * @clear_pt: Clear the page table entries.
>   * @num_entries: On output contains the number of @entries used.
>   *
>   * This function builds a disconnected page-table tree for a given
> address
> @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> xe_pt_stage_bind_ops = {
>   */
>  static int
>  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> -		 struct xe_vm_pgtable_update *entries, u32
> *num_entries)
> +		 struct xe_vm_pgtable_update *entries,
> +		 bool clear_pt, u32 *num_entries)
>  {
>  	struct xe_device *xe = tile_to_xe(tile);
>  	struct xe_bo *bo = xe_vma_bo(vma);
> @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile, struct
> xe_vma *vma,
>  		.vma = vma,
>  		.wupd.entries = entries,
>  		.needs_64K = (xe_vma_vm(vma)->flags &
> XE_VM_FLAG_64K) && is_devmem,
> +		.clear_pt = clear_pt,
>  	};
>  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
>  	int ret;
>  
> +	if (clear_pt) {
> +		ret = xe_pt_walk_range(&pt->base, pt->level,
> xe_vma_start(vma),
> +				       xe_vma_end(vma),
> &xe_walk.base);
> +
> +		*num_entries = xe_walk.wupd.num_used_entries;
> +		return ret;
> +	}
> +
>  	/**
>  	 * Default atomic expectations for different allocation
> scenarios are as follows:
>  	 *
> @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> xe_vm_pgtable_update *entries,
>  
>  static int
>  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> -		   struct xe_vm_pgtable_update *entries, u32
> *num_entries)
> +		   struct xe_vm_pgtable_update *entries,
> +		   bool invalidate_on_bind, u32 *num_entries)
>  {
>  	int err;
>  
>  	*num_entries = 0;
> -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> +	err = xe_pt_stage_bind(tile, vma, entries,
> invalidate_on_bind,
> +			       num_entries);
>  	if (!err)
>  		xe_tile_assert(tile, *num_entries);
>  
> @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> struct xe_tile *tile,
>  		return err;
>  
>  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> +				 pt_update_ops->invalidate_on_bind,
>  				 &pt_op->num_entries);
>  	if (!err) {
>  		xe_tile_assert(tile, pt_op->num_entries <=
> @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> struct xe_tile *tile,
>  		 * it needs to be done here.
>  		 */
>  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> -		     xe_vm_in_preempt_fence_mode(vm)))
> +		     xe_vm_in_preempt_fence_mode(vm)) ||
> pt_update_ops->invalidate_on_bind)
>  			pt_update_ops->needs_invalidation = true;
>  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
>  			/* We bump also if batch_invalidate_tlb is
> true */
> @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
>  
>  	switch (op->base.op) {
>  	case DRM_GPUVA_OP_MAP:
> -		if (!op->map.immediate && xe_vm_in_fault_mode(vm))
> +		if (!op->map.immediate && xe_vm_in_fault_mode(vm) &&
> +		    !op->map.invalidate_on_bind)
>  			break;
>  
> +		if (op->map.invalidate_on_bind)
> +			pt_update_ops->invalidate_on_bind = true;
> +
>  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> >map.vma);
>  		pt_update_ops->wait_vm_kernel = true;
>  		break;
> @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm *vm,
> struct xe_tile *tile,
>  	}
>  	vma->tile_present |= BIT(tile->id);
>  	vma->tile_staged &= ~BIT(tile->id);
> +	if (pt_update_ops->invalidate_on_bind)
> +		vma->tile_invalidated |= BIT(tile->id);
>  	if (xe_vma_is_userptr(vma)) {
>  		lockdep_assert_held_read(&vm-
> >userptr.notifier_lock);
>  		to_userptr_vma(vma)->userptr.initial_bind = true;
> diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> b/drivers/gpu/drm/xe/xe_pt_types.h
> index 384cc04de719..3d0aa2a5102e 100644
> --- a/drivers/gpu/drm/xe/xe_pt_types.h
> +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
>  	bool needs_userptr_lock;
>  	/** @needs_invalidation: Needs invalidation */
>  	bool needs_invalidation;
> +	/** @invalidate_on_bind: Invalidate the range before bind */
> +	bool invalidate_on_bind;
>  	/**
>  	 * @wait_vm_bookkeep: PT operations need to wait until VM is
> idle
>  	 * (bookkeep dma-resv slots are idle) and stage all future
> VM activity
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index d664f2e418b2..813d893d9b63 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> struct drm_gpuva_op *op)
>  }
>  #endif
>  
> +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm, u32
> bind_flags)
> +{
> +	if (!xe_vm_in_fault_mode(vm))
> +		return false;
> +
> +	if (!NEEDS_SCRATCH(vm->xe))
> +		return false;
> +
> +	if (!xe_vm_has_scratch(vm))
> +		return false;
> +
> +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> +		return false;
> +
> +	return true;
> +}
> +
>  /*
>   * Create operations list from IOCTL arguments, setup operations
> fields so parse
>   * and commit steps are decoupled from IOCTL arguments. This step
> can fail.
> @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm,
> struct xe_bo *bo,
>  			op->map.is_null = flags &
> DRM_XE_VM_BIND_FLAG_NULL;
>  			op->map.dumpable = flags &
> DRM_XE_VM_BIND_FLAG_DUMPABLE;
>  			op->map.pat_index = pat_index;
> +			op->map.invalidate_on_bind =
> +				__xe_vm_needs_clear_scratch_pages(vm
> , flags);
>  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
>  			op->prefetch.region = prefetch_region;
>  		}
> @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm
> *vm, struct drm_gpuva_ops *ops,
>  				return PTR_ERR(vma);
>  
>  			op->map.vma = vma;
> -			if (op->map.immediate ||
> !xe_vm_in_fault_mode(vm))
> +			if (op->map.immediate ||
> !xe_vm_in_fault_mode(vm) ||
> +			    op->map.invalidate_on_bind)
>  				xe_vma_ops_incr_pt_update_ops(vops,
>  							      op-
> >tile_mask);
>  			break;
> @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct drm_exec
> *exec, struct xe_vm *vm,
>  
>  	switch (op->base.op) {
>  	case DRM_GPUVA_OP_MAP:
> -		err = vma_lock_and_validate(exec, op->map.vma,
> -					    !xe_vm_in_fault_mode(vm)
> ||
> -					    op->map.immediate);
> +		if (!op->map.invalidate_on_bind)
> +			err = vma_lock_and_validate(exec, op-
> >map.vma,
> +						   
> !xe_vm_in_fault_mode(vm) ||
> +						    op-
> >map.immediate);
>  		break;
>  	case DRM_GPUVA_OP_REMAP:
>  		err = check_ufence(gpuva_to_vma(op-
> >base.remap.unmap->va));
> diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> b/drivers/gpu/drm/xe/xe_vm_types.h
> index 52467b9b5348..dace04f4ea5e 100644
> --- a/drivers/gpu/drm/xe/xe_vm_types.h
> +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> @@ -297,6 +297,8 @@ struct xe_vma_op_map {
>  	bool is_null;
>  	/** @dumpable: whether BO is dumped on GPU hang */
>  	bool dumpable;
> +	/** @invalidate: invalidate the VMA before bind */
> +	bool invalidate_on_bind;
>  	/** @pat_index: The pat index to use for this operation. */
>  	u16 pat_index;
>  };


^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-05 14:35   ` Thomas Hellström
@ 2025-02-06  1:52     ` Zeng, Oak
  2025-02-06 12:51       ` Thomas Hellström
  0 siblings, 1 reply; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06  1:52 UTC (permalink / raw)
  To: Thomas Hellström, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan



> -----Original Message-----
> From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Sent: February 5, 2025 9:36 AM
> To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> 
> On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > When a vm runs under fault mode, if scratch page is enabled, we
> need
> > to clear the scratch page mapping before vm_bind for the vm_bind
> > address range. Under fault mode, we depend on recoverable page
> fault
> > to establish mapping in page table. If scratch page is not cleared,
> > GPU access of address won't cause page fault because it always hits
> > the existing scratch page mapping.
> 
> I think we need to ephasize that we clear AT vm_bind time, not
> before,
> both in the commit description and the patch title.

Will fix.

> 
> >
> > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > immediate bind can overwrite the scratch page mapping.
> >
> > So far only is xe2 and xe3 products are allowed to enable scratch
> > page
> > under fault mode. On other platform we don't allow scratch page
> under
> > fault mode, so no need of such clearing.
> >
> > v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> > similar
> > to a map operation, with the exception that PTEs are cleared
> instead
> > of
> > pointing to valid physical pages. (Matt, Thomas)
> >
> > TLB invalidation is needed after clear scratch page mapping as larger
> > scratch page mapping could be backed by physical page and cached
> in
> > TLB. (Matt, Thomas)
> >
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++--
> ------
> > --
> >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> >  4 files changed, 75 insertions(+), 24 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> b/drivers/gpu/drm/xe/xe_pt.c
> > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > --- a/drivers/gpu/drm/xe/xe_pt.c
> > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> >  	 * granularity.
> >  	 */
> >  	bool needs_64K;
> > +	/* @clear_pt: clear page table entries during the bind walk
> > */
> > +	bool clear_pt;
> >  	/**
> >  	 * @vma: VMA being mapped
> >  	 */
> > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw
> *parent,
> > pgoff_t offset,
> >
> >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> >
> > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > -						 xe_res_dma(curs) +
> > xe_walk->dma_offset,
> > -						 xe_walk->vma,
> > pat_index, level);
> > -		pte |= xe_walk->default_pte;
> > +		if (xe_walk->clear_pt) {
> > +			pte = 0;
> > +		} else {
> > +			pte = vm->pt_ops->pte_encode_vma(is_null ?
> 0
> > :
> > +					xe_res_dma(curs) + xe_walk-
> > >dma_offset,
> > +					xe_walk->vma, pat_index,
> > level);
> > +			pte |= xe_walk->default_pte;
> >
> > -		/*
> > -		 * Set the XE_PTE_PS64 hint if possible, otherwise
> > if
> > -		 * this device *requires* 64K PTE size for VRAM,
> > fail.
> > -		 */
> > -		if (level == 0 && !xe_parent->is_compact) {
> > -			if (xe_pt_is_pte_ps64K(addr, next, xe_walk))
> > {
> > -				xe_walk->vma->gpuva.flags |=
> > XE_VMA_PTE_64K;
> > -				pte |= XE_PTE_PS64;
> > -			} else if (XE_WARN_ON(xe_walk-
> >needs_64K)) {
> > -				return -EINVAL;
> > +			/*
> > +			 * Set the XE_PTE_PS64 hint if possible,
> > otherwise if
> > +			 * this device *requires* 64K PTE size for
> > VRAM, fail.
> > +			 */
> > +			if (level == 0 && !xe_parent->is_compact) {
> > +				if (xe_pt_is_pte_ps64K(addr, next,
> > xe_walk)) {
> > +					xe_walk->vma->gpuva.flags
> |=
> > XE_VMA_PTE_64K;
> > +					pte |= XE_PTE_PS64;
> > +				} else if (XE_WARN_ON(xe_walk-
> > >needs_64K)) {
> > +					return -EINVAL;
> > +				}
> >  			}
> >  		}
> >
> 
> I think there might be some things missing here.
> The bind should look exactly as a is_null bind, with the exception that
> the pte should be 0, so might be able to benefit from is_null, for
> example xe_pg_hugepte_possible() short-circuit the dma-address
> check.

You are right. Above code has a problem for huge pte cases: it wouldn't detect
A huge pte for clear_pt case. I will fix it.

I will not make is_null be true for clear_pt case, but will make similar logic to
Let clear_pt benefit from is_null logic.

> With clear_pt we don't want to use the XE_PTE_PS64K, though, like
> you
> identified above.
> 
> Will get back with a look at the invalidation.

I don't quite get here. Please do let me know if you find anything.

Oak
> 
> 
> > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> *parent,
> > pgoff_t offset,
> >  		if (unlikely(ret))
> >  			return ret;
> >
> > -		if (!is_null)
> > +		if (!is_null && !xe_walk->clear_pt)
> >  			xe_res_next(curs, next - addr);
> >  		xe_walk->va_curs_start = next;
> >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > level);
> > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> >   * @vma: The vma indicating the address range.
> >   * @entries: Storage for the update entries used for connecting the
> > tree to
> >   * the main tree at commit time.
> > + * @clear_pt: Clear the page table entries.
> >   * @num_entries: On output contains the number of @entries
> used.
> >   *
> >   * This function builds a disconnected page-table tree for a given
> > address
> > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> >   */
> >  static int
> >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		 struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > +		 struct xe_vm_pgtable_update *entries,
> > +		 bool clear_pt, u32 *num_entries)
> >  {
> >  	struct xe_device *xe = tile_to_xe(tile);
> >  	struct xe_bo *bo = xe_vma_bo(vma);
> > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> struct
> > xe_vma *vma,
> >  		.vma = vma,
> >  		.wupd.entries = entries,
> >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > XE_VM_FLAG_64K) && is_devmem,
> > +		.clear_pt = clear_pt,
> >  	};
> >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> >  	int ret;
> >
> > +	if (clear_pt) {
> > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > xe_vma_start(vma),
> > +				       xe_vma_end(vma),
> > &xe_walk.base);
> > +
> > +		*num_entries = xe_walk.wupd.num_used_entries;
> > +		return ret;
> > +	}
> > +
> >  	/**
> >  	 * Default atomic expectations for different allocation
> > scenarios are as follows:
> >  	 *
> > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > xe_vm_pgtable_update *entries,
> >
> >  static int
> >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		   struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > +		   struct xe_vm_pgtable_update *entries,
> > +		   bool invalidate_on_bind, u32 *num_entries)
> >  {
> >  	int err;
> >
> >  	*num_entries = 0;
> > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > +	err = xe_pt_stage_bind(tile, vma, entries,
> > invalidate_on_bind,
> > +			       num_entries);
> >  	if (!err)
> >  		xe_tile_assert(tile, *num_entries);
> >
> > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm
> *vm,
> > struct xe_tile *tile,
> >  		return err;
> >
> >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > +				 pt_update_ops->invalidate_on_bind,
> >  				 &pt_op->num_entries);
> >  	if (!err) {
> >  		xe_tile_assert(tile, pt_op->num_entries <=
> > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm
> *vm,
> > struct xe_tile *tile,
> >  		 * it needs to be done here.
> >  		 */
> >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > -		     xe_vm_in_preempt_fence_mode(vm)))
> > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > pt_update_ops->invalidate_on_bind)
> >  			pt_update_ops->needs_invalidation = true;
> >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> >  			/* We bump also if batch_invalidate_tlb is
> > true */
> > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> >
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		if (!op->map.immediate &&
> xe_vm_in_fault_mode(vm))
> > +		if (!op->map.immediate &&
> xe_vm_in_fault_mode(vm) &&
> > +		    !op->map.invalidate_on_bind)
> >  			break;
> >
> > +		if (op->map.invalidate_on_bind)
> > +			pt_update_ops->invalidate_on_bind = true;
> > +
> >  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> > >map.vma);
> >  		pt_update_ops->wait_vm_kernel = true;
> >  		break;
> > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm
> *vm,
> > struct xe_tile *tile,
> >  	}
> >  	vma->tile_present |= BIT(tile->id);
> >  	vma->tile_staged &= ~BIT(tile->id);
> > +	if (pt_update_ops->invalidate_on_bind)
> > +		vma->tile_invalidated |= BIT(tile->id);
> >  	if (xe_vma_is_userptr(vma)) {
> >  		lockdep_assert_held_read(&vm-
> > >userptr.notifier_lock);
> >  		to_userptr_vma(vma)->userptr.initial_bind = true;
> > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > b/drivers/gpu/drm/xe/xe_pt_types.h
> > index 384cc04de719..3d0aa2a5102e 100644
> > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> >  	bool needs_userptr_lock;
> >  	/** @needs_invalidation: Needs invalidation */
> >  	bool needs_invalidation;
> > +	/** @invalidate_on_bind: Invalidate the range before bind */
> > +	bool invalidate_on_bind;
> >  	/**
> >  	 * @wait_vm_bookkeep: PT operations need to wait until VM
> is
> > idle
> >  	 * (bookkeep dma-resv slots are idle) and stage all future
> > VM activity
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> b/drivers/gpu/drm/xe/xe_vm.c
> > index d664f2e418b2..813d893d9b63 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> > struct drm_gpuva_op *op)
> >  }
> >  #endif
> >
> > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm
> *vm, u32
> > bind_flags)
> > +{
> > +	if (!xe_vm_in_fault_mode(vm))
> > +		return false;
> > +
> > +	if (!NEEDS_SCRATCH(vm->xe))
> > +		return false;
> > +
> > +	if (!xe_vm_has_scratch(vm))
> > +		return false;
> > +
> > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> >  /*
> >   * Create operations list from IOCTL arguments, setup operations
> > fields so parse
> >   * and commit steps are decoupled from IOCTL arguments. This
> step
> > can fail.
> > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm
> *vm,
> > struct xe_bo *bo,
> >  			op->map.is_null = flags &
> > DRM_XE_VM_BIND_FLAG_NULL;
> >  			op->map.dumpable = flags &
> > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> >  			op->map.pat_index = pat_index;
> > +			op->map.invalidate_on_bind =
> > +
> 	__xe_vm_needs_clear_scratch_pages(vm
> > , flags);
> >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> >  			op->prefetch.region = prefetch_region;
> >  		}
> > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> xe_vm
> > *vm, struct drm_gpuva_ops *ops,
> >  				return PTR_ERR(vma);
> >
> >  			op->map.vma = vma;
> > -			if (op->map.immediate ||
> > !xe_vm_in_fault_mode(vm))
> > +			if (op->map.immediate ||
> > !xe_vm_in_fault_mode(vm) ||
> > +			    op->map.invalidate_on_bind)
> >
> 	xe_vma_ops_incr_pt_update_ops(vops,
> >  							      op-
> > >tile_mask);
> >  			break;
> > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> drm_exec
> > *exec, struct xe_vm *vm,
> >
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		err = vma_lock_and_validate(exec, op->map.vma,
> > -					    !xe_vm_in_fault_mode(vm)
> > ||
> > -					    op->map.immediate);
> > +		if (!op->map.invalidate_on_bind)
> > +			err = vma_lock_and_validate(exec, op-
> > >map.vma,
> > +
> > !xe_vm_in_fault_mode(vm) ||
> > +						    op-
> > >map.immediate);
> >  		break;
> >  	case DRM_GPUVA_OP_REMAP:
> >  		err = check_ufence(gpuva_to_vma(op-
> > >base.remap.unmap->va));
> > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > b/drivers/gpu/drm/xe/xe_vm_types.h
> > index 52467b9b5348..dace04f4ea5e 100644
> > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> >  	bool is_null;
> >  	/** @dumpable: whether BO is dumped on GPU hang */
> >  	bool dumpable;
> > +	/** @invalidate: invalidate the VMA before bind */
> > +	bool invalidate_on_bind;
> >  	/** @pat_index: The pat index to use for this operation. */
> >  	u16 pat_index;
> >  };


^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-05 13:14   ` Thomas Hellström
@ 2025-02-06  1:54     ` Zeng, Oak
  2025-02-06  9:29       ` Thomas Hellström
  0 siblings, 1 reply; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06  1:54 UTC (permalink / raw)
  To: Thomas Hellström, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan



> -----Original Message-----
> From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Sent: February 5, 2025 8:14 AM
> To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> mode for certain platform
> 
> On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > Normally scratch page is not allowed when a vm is operate under
> page
> > fault mode, i.e., in the existing codes,
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual
> exclusive. The reason
> > is fault mode relies on recoverable page to work, while scratch page
> > can mute recoverable page fault.
> >
> > On xe2 and xe3, out of bound prefetch can cause page fault and
> > further
> > system hang because xekmd can't resolve such page fault. SYCL and
> OCL
> > language runtime requires out of bound prefetch to be silently
> > dropped
> > without causing any functional problem, thus the existing behavior
> > doesn't meet language runtime requirement.
> >
> > At the same time, HW prefetching can cause page fault interrupt.
> Due
> > to
> > page fault interrupt overhead (i.e., need Guc and KMD involved to
> fix
> > the page fault), HW prefetching can be slowed by many orders of
> > magnitude.
> >
> > Fix those problems by allowing scratch page under fault mode for
> xe2
> > and
> > xe3. With scratch page in place, HW prefetching could always hit
> > scratch
> > page instead of causing interrupt.
> >
> > A side effect is, scratch page could hide application program error.
> > Application out of bound accesses are hided
> s/hided/hidden/
> 
> >  by scratch page mapping,
> > instead of get reported to user.
> >
> > igt test: https://patchwork.freedesktop.org/series/144334/. Test
> > result on
> > BMG:
> >
> > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> public/build/tests#
> > ./xe_exec_fault_mode --run-subtest scratch-fault
> > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > Using IGT_SRANDOM=1738684805 for randomisation
> > Opened device: /dev/dri/card0
> > Starting subtest: scratch-fault
> > Subtest scratch-fault: SUCCESS (0.080s)
> >
> > Without this series, the test result is:
> >
> > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> public/build/tests#
> > ./xe_exec_fault_mode --run-subtest scratch-fault
> > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > Using IGT_SRANDOM=1738686046 for randomisation
> > Opened device: /dev/dri/card0
> > Starting subtest: scratch-fault
> > (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure
> function
> > test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> > (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> > __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> > exec_queues[i % n_exec_queues], &timeout) == 0
> > (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> > (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> > Stack trace:
> >   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
> >   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
> >   #2 ../tests/intel/xe_exec_fault_mode.c:537
> > __igt_unique____real_main407()
> >   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
> >   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> > __libc_start_call_main()
> >   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
> >   #6 [_start+0x2e]
> > Subtest scratch-fault failed.
> >
> > v2: Refine commit message (Thomas)
> >
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> b/drivers/gpu/drm/xe/xe_vm.c
> > index 813d893d9b63..c0372f083d42 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct
> drm_device *dev,
> > void *data,
> >  	if (XE_IOCTL_DBG(xe, args->extensions))
> >  		return -EINVAL;
> >
> > +	if (XE_IOCTL_DBG(xe, args->flags &
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > +			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > +			 !(NEEDS_SCRATCH(xe))))
> > +		return -EINVAL;
> > +
> 
> We should probably move this test to where the old test were below,
> since the WA below enables scratch pages.


Below wa code is for dg2.
If we move this test below the wa code, then on dg2, if people create vm
With FAULT_MODE | LR_MODE, above check would fail user.

So I do think we should keep this check above wa codes.

Oak


> 
> /Thomas
> 
> 
> >  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
> >  		args->flags |=
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
> >
> > @@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct
> drm_device *dev,
> > void *data,
> >  	if (XE_IOCTL_DBG(xe, args->flags &
> > ~ALL_DRM_XE_VM_CREATE_FLAGS))
> >  		return -EINVAL;
> >
> > -	if (XE_IOCTL_DBG(xe, args->flags &
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > -			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > -		return -EINVAL;
> > -
> >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> > DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> >  			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> >  		return -EINVAL;


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-06  2:11 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
@ 2025-02-06  2:11 ` Oak Zeng
  0 siblings, 0 replies; 38+ messages in thread
From: Oak Zeng @ 2025-02-06  2:11 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

Normally scratch page is not allowed when a vm is operate under page
fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
is fault mode relies on recoverable page to work, while scratch page
can mute recoverable page fault.

On xe2 and xe3, out of bound prefetch can cause page fault and further
system hang because xekmd can't resolve such page fault. SYCL and OCL
language runtime requires out of bound prefetch to be silently dropped
without causing any functional problem, thus the existing behavior
doesn't meet language runtime requirement.

At the same time, HW prefetching can cause page fault interrupt. Due to
page fault interrupt overhead (i.e., need Guc and KMD involved to fix
the page fault), HW prefetching can be slowed by many orders of magnitude.

Fix those problems by allowing scratch page under fault mode for xe2 and
xe3. With scratch page in place, HW prefetching could always hit scratch
page instead of causing interrupt.

A side effect is, scratch page could hide application program error.
Application out of bound accesses are hided by scratch page mapping,
instead of get reported to user.

igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
BMG:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738684805 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
Subtest scratch-fault: SUCCESS (0.080s)

Without this series, the test result is:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738686046 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
(xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
(xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
(xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
(xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
Stack trace:
  #0 ../lib/igt_core.c:2266 __igt_fail_assert()
  #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
  #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
  #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
  #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
  #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
  #6 [_start+0x2e]
Subtest scratch-fault failed.

v2: Refine commit message (Thomas)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 813d893d9b63..c0372f083d42 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->extensions))
 		return -EINVAL;
 
+	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
+			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
+			 !(NEEDS_SCRATCH(xe))))
+		return -EINVAL;
+
 	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
 		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
 
@@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	if (XE_IOCTL_DBG(xe, args->flags & ~ALL_DRM_XE_VM_CREATE_FLAGS))
 		return -EINVAL;
 
-	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
-			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
-		return -EINVAL;
-
 	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
 			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
 		return -EINVAL;
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-06  1:54     ` Zeng, Oak
@ 2025-02-06  9:29       ` Thomas Hellström
  2025-02-06 15:14         ` Zeng, Oak
  0 siblings, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-06  9:29 UTC (permalink / raw)
  To: Zeng, Oak, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan

On Thu, 2025-02-06 at 01:54 +0000, Zeng, Oak wrote:
> 
> 
> > -----Original Message-----
> > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > Sent: February 5, 2025 8:14 AM
> > To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > <jonathan.cavitt@intel.com>
> > Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> > mode for certain platform
> > 
> > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > Normally scratch page is not allowed when a vm is operate under
> > page
> > > fault mode, i.e., in the existing codes,
> > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual
> > exclusive. The reason
> > > is fault mode relies on recoverable page to work, while scratch
> > > page
> > > can mute recoverable page fault.
> > > 
> > > On xe2 and xe3, out of bound prefetch can cause page fault and
> > > further
> > > system hang because xekmd can't resolve such page fault. SYCL and
> > OCL
> > > language runtime requires out of bound prefetch to be silently
> > > dropped
> > > without causing any functional problem, thus the existing
> > > behavior
> > > doesn't meet language runtime requirement.
> > > 
> > > At the same time, HW prefetching can cause page fault interrupt.
> > Due
> > > to
> > > page fault interrupt overhead (i.e., need Guc and KMD involved to
> > fix
> > > the page fault), HW prefetching can be slowed by many orders of
> > > magnitude.
> > > 
> > > Fix those problems by allowing scratch page under fault mode for
> > xe2
> > > and
> > > xe3. With scratch page in place, HW prefetching could always hit
> > > scratch
> > > page instead of causing interrupt.
> > > 
> > > A side effect is, scratch page could hide application program
> > > error.
> > > Application out of bound accesses are hided
> > s/hided/hidden/
> > 
> > >  by scratch page mapping,
> > > instead of get reported to user.
> > > 
> > > igt test: https://patchwork.freedesktop.org/series/144334/. Test
> > > result on
> > > BMG:
> > > 
> > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > public/build/tests#
> > > ./xe_exec_fault_mode --run-subtest scratch-fault
> > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > Using IGT_SRANDOM=1738684805 for randomisation
> > > Opened device: /dev/dri/card0
> > > Starting subtest: scratch-fault
> > > Subtest scratch-fault: SUCCESS (0.080s)
> > > 
> > > Without this series, the test result is:
> > > 
> > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > public/build/tests#
> > > ./xe_exec_fault_mode --run-subtest scratch-fault
> > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > Using IGT_SRANDOM=1738686046 for randomisation
> > > Opened device: /dev/dri/card0
> > > Starting subtest: scratch-fault
> > > (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure
> > function
> > > test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> > > (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> > > __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> > > exec_queues[i % n_exec_queues], &timeout) == 0
> > > (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> > > (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> > > Stack trace:
> > >   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
> > >   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
> > >   #2 ../tests/intel/xe_exec_fault_mode.c:537
> > > __igt_unique____real_main407()
> > >   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
> > >   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> > > __libc_start_call_main()
> > >   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
> > >   #6 [_start+0x2e]
> > > Subtest scratch-fault failed.
> > > 
> > > v2: Refine commit message (Thomas)
> > > 
> > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > ---
> > >  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
> > >  1 file changed, 5 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > > index 813d893d9b63..c0372f083d42 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct
> > drm_device *dev,
> > > void *data,
> > >  	if (XE_IOCTL_DBG(xe, args->extensions))
> > >  		return -EINVAL;
> > > 
> > > +	if (XE_IOCTL_DBG(xe, args->flags &
> > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > > +			 args->flags &
> > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > > +			 !(NEEDS_SCRATCH(xe))))
> > > +		return -EINVAL;
> > > +
> > 
> > We should probably move this test to where the old test were below,
> > since the WA below enables scratch pages.
> 
> 
> Below wa code is for dg2.
> If we move this test below the wa code, then on dg2, if people create
> vm
> With FAULT_MODE | LR_MODE, above check would fail user.

Though on DG2 we don't support fault mode.

> 
> So I do think we should keep this check above wa codes.

I think what matters here is readability of the code. A naive reader
(like me in this case) would start to wonder why enabling scratch pages
*after* the sanity check for them is done, and whether that is a bug or
an oversight, and that would require looking up what the WA actually
means.

So IMO in general sanity checks should be done after all implicit
enabling of flags if at all possible. And if not, they should be
accompanied by a comment that explains why the enabling of the flag is
not included in the sanity check.

/Thomas


> 
> Oak
> 
> 
> > 
> > /Thomas
> > 
> > 
> > >  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
> > >  		args->flags |=
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
> > > 
> > > @@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct
> > drm_device *dev,
> > > void *data,
> > >  	if (XE_IOCTL_DBG(xe, args->flags &
> > > ~ALL_DRM_XE_VM_CREATE_FLAGS))
> > >  		return -EINVAL;
> > > 
> > > -	if (XE_IOCTL_DBG(xe, args->flags &
> > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > > -			 args->flags &
> > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > > -		return -EINVAL;
> > > -
> > >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> > > DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> > >  			 args->flags &
> > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > >  		return -EINVAL;
> 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
  2025-02-05 14:35   ` Thomas Hellström
@ 2025-02-06 10:34   ` Matthew Brost
  2025-02-06 10:43     ` Thomas Hellström
  2025-02-06 15:16     ` Zeng, Oak
  1 sibling, 2 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-06 10:34 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe, Thomas.Hellstrom, jonathan.cavitt

On Tue, Feb 04, 2025 at 01:45:57PM -0500, Oak Zeng wrote:
> When a vm runs under fault mode, if scratch page is enabled, we need
> to clear the scratch page mapping before vm_bind for the vm_bind
> address range. Under fault mode, we depend on recoverable page fault
> to establish mapping in page table. If scratch page is not cleared,
> GPU access of address won't cause page fault because it always hits
> the existing scratch page mapping.
> 
> When vm_bind with IMMEDIATE flag, there is no need of clearing as
> immediate bind can overwrite the scratch page mapping.
> 
> So far only is xe2 and xe3 products are allowed to enable scratch page
> under fault mode. On other platform we don't allow scratch page under
> fault mode, so no need of such clearing.
> 
> v2: Rework vm_bind pipeline to clear scratch page mapping. This is similar
> to a map operation, with the exception that PTEs are cleared instead of
> pointing to valid physical pages. (Matt, Thomas)
> 
> TLB invalidation is needed after clear scratch page mapping as larger
> scratch page mapping could be backed by physical page and cached in
> TLB. (Matt, Thomas)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>

Given the complexity of the VM bind path, I think we need an IGT posted
with this series before merging as I suggested in v1. Without it, it
will be fairly difficult to ensure correctness by reviews only.

Matt

> ---
>  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++----------
>  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
>  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
>  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
>  4 files changed, 75 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
> index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> --- a/drivers/gpu/drm/xe/xe_pt.c
> +++ b/drivers/gpu/drm/xe/xe_pt.c
> @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
>  	 * granularity.
>  	 */
>  	bool needs_64K;
> +	/* @clear_pt: clear page table entries during the bind walk */
> +	bool clear_pt;
>  	/**
>  	 * @vma: VMA being mapped
>  	 */
> @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,
>  
>  		XE_WARN_ON(xe_walk->va_curs_start != addr);
>  
> -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> -						 xe_res_dma(curs) + xe_walk->dma_offset,
> -						 xe_walk->vma, pat_index, level);
> -		pte |= xe_walk->default_pte;
> +		if (xe_walk->clear_pt) {
> +			pte = 0;
> +		} else {
> +			pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> +					xe_res_dma(curs) + xe_walk->dma_offset,
> +					xe_walk->vma, pat_index, level);
> +			pte |= xe_walk->default_pte;
>  
> -		/*
> -		 * Set the XE_PTE_PS64 hint if possible, otherwise if
> -		 * this device *requires* 64K PTE size for VRAM, fail.
> -		 */
> -		if (level == 0 && !xe_parent->is_compact) {
> -			if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
> -				xe_walk->vma->gpuva.flags |= XE_VMA_PTE_64K;
> -				pte |= XE_PTE_PS64;
> -			} else if (XE_WARN_ON(xe_walk->needs_64K)) {
> -				return -EINVAL;
> +			/*
> +			 * Set the XE_PTE_PS64 hint if possible, otherwise if
> +			 * this device *requires* 64K PTE size for VRAM, fail.
> +			 */
> +			if (level == 0 && !xe_parent->is_compact) {
> +				if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
> +					xe_walk->vma->gpuva.flags |= XE_VMA_PTE_64K;
> +					pte |= XE_PTE_PS64;
> +				} else if (XE_WARN_ON(xe_walk->needs_64K)) {
> +					return -EINVAL;
> +				}
>  			}
>  		}
>  
> @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,
>  		if (unlikely(ret))
>  			return ret;
>  
> -		if (!is_null)
> +		if (!is_null && !xe_walk->clear_pt)
>  			xe_res_next(curs, next - addr);
>  		xe_walk->va_curs_start = next;
>  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);
> @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = {
>   * @vma: The vma indicating the address range.
>   * @entries: Storage for the update entries used for connecting the tree to
>   * the main tree at commit time.
> + * @clear_pt: Clear the page table entries.
>   * @num_entries: On output contains the number of @entries used.
>   *
>   * This function builds a disconnected page-table tree for a given address
> @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = {
>   */
>  static int
>  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> -		 struct xe_vm_pgtable_update *entries, u32 *num_entries)
> +		 struct xe_vm_pgtable_update *entries,
> +		 bool clear_pt, u32 *num_entries)
>  {
>  	struct xe_device *xe = tile_to_xe(tile);
>  	struct xe_bo *bo = xe_vma_bo(vma);
> @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
>  		.vma = vma,
>  		.wupd.entries = entries,
>  		.needs_64K = (xe_vma_vm(vma)->flags & XE_VM_FLAG_64K) && is_devmem,
> +		.clear_pt = clear_pt,
>  	};
>  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
>  	int ret;
>  
> +	if (clear_pt) {
> +		ret = xe_pt_walk_range(&pt->base, pt->level, xe_vma_start(vma),
> +				       xe_vma_end(vma), &xe_walk.base);
> +
> +		*num_entries = xe_walk.wupd.num_used_entries;
> +		return ret;
> +	}
> +
>  	/**
>  	 * Default atomic expectations for different allocation scenarios are as follows:
>  	 *
> @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct xe_vm_pgtable_update *entries,
>  
>  static int
>  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> -		   struct xe_vm_pgtable_update *entries, u32 *num_entries)
> +		   struct xe_vm_pgtable_update *entries,
> +		   bool invalidate_on_bind, u32 *num_entries)
>  {
>  	int err;
>  
>  	*num_entries = 0;
> -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> +	err = xe_pt_stage_bind(tile, vma, entries, invalidate_on_bind,
> +			       num_entries);
>  	if (!err)
>  		xe_tile_assert(tile, *num_entries);
>  
> @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
>  		return err;
>  
>  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> +				 pt_update_ops->invalidate_on_bind,
>  				 &pt_op->num_entries);
>  	if (!err) {
>  		xe_tile_assert(tile, pt_op->num_entries <=
> @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
>  		 * it needs to be done here.
>  		 */
>  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> -		     xe_vm_in_preempt_fence_mode(vm)))
> +		     xe_vm_in_preempt_fence_mode(vm)) || pt_update_ops->invalidate_on_bind)
>  			pt_update_ops->needs_invalidation = true;
>  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
>  			/* We bump also if batch_invalidate_tlb is true */
> @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
>  
>  	switch (op->base.op) {
>  	case DRM_GPUVA_OP_MAP:
> -		if (!op->map.immediate && xe_vm_in_fault_mode(vm))
> +		if (!op->map.immediate && xe_vm_in_fault_mode(vm) &&
> +		    !op->map.invalidate_on_bind)
>  			break;
>  
> +		if (op->map.invalidate_on_bind)
> +			pt_update_ops->invalidate_on_bind = true;
> +
>  		err = bind_op_prepare(vm, tile, pt_update_ops, op->map.vma);
>  		pt_update_ops->wait_vm_kernel = true;
>  		break;
> @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm *vm, struct xe_tile *tile,
>  	}
>  	vma->tile_present |= BIT(tile->id);
>  	vma->tile_staged &= ~BIT(tile->id);
> +	if (pt_update_ops->invalidate_on_bind)
> +		vma->tile_invalidated |= BIT(tile->id);
>  	if (xe_vma_is_userptr(vma)) {
>  		lockdep_assert_held_read(&vm->userptr.notifier_lock);
>  		to_userptr_vma(vma)->userptr.initial_bind = true;
> diff --git a/drivers/gpu/drm/xe/xe_pt_types.h b/drivers/gpu/drm/xe/xe_pt_types.h
> index 384cc04de719..3d0aa2a5102e 100644
> --- a/drivers/gpu/drm/xe/xe_pt_types.h
> +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
>  	bool needs_userptr_lock;
>  	/** @needs_invalidation: Needs invalidation */
>  	bool needs_invalidation;
> +	/** @invalidate_on_bind: Invalidate the range before bind */
> +	bool invalidate_on_bind;
>  	/**
>  	 * @wait_vm_bookkeep: PT operations need to wait until VM is idle
>  	 * (bookkeep dma-resv slots are idle) and stage all future VM activity
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index d664f2e418b2..813d893d9b63 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op)
>  }
>  #endif
>  
> +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm, u32 bind_flags)
> +{
> +	if (!xe_vm_in_fault_mode(vm))
> +		return false;
> +
> +	if (!NEEDS_SCRATCH(vm->xe))
> +		return false;
> +
> +	if (!xe_vm_has_scratch(vm))
> +		return false;
> +
> +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> +		return false;
> +
> +	return true;
> +}
> +
>  /*
>   * Create operations list from IOCTL arguments, setup operations fields so parse
>   * and commit steps are decoupled from IOCTL arguments. This step can fail.
> @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo,
>  			op->map.is_null = flags & DRM_XE_VM_BIND_FLAG_NULL;
>  			op->map.dumpable = flags & DRM_XE_VM_BIND_FLAG_DUMPABLE;
>  			op->map.pat_index = pat_index;
> +			op->map.invalidate_on_bind =
> +				__xe_vm_needs_clear_scratch_pages(vm, flags);
>  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
>  			op->prefetch.region = prefetch_region;
>  		}
> @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct drm_gpuva_ops *ops,
>  				return PTR_ERR(vma);
>  
>  			op->map.vma = vma;
> -			if (op->map.immediate || !xe_vm_in_fault_mode(vm))
> +			if (op->map.immediate || !xe_vm_in_fault_mode(vm) ||
> +			    op->map.invalidate_on_bind)
>  				xe_vma_ops_incr_pt_update_ops(vops,
>  							      op->tile_mask);
>  			break;
> @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
>  
>  	switch (op->base.op) {
>  	case DRM_GPUVA_OP_MAP:
> -		err = vma_lock_and_validate(exec, op->map.vma,
> -					    !xe_vm_in_fault_mode(vm) ||
> -					    op->map.immediate);
> +		if (!op->map.invalidate_on_bind)
> +			err = vma_lock_and_validate(exec, op->map.vma,
> +						    !xe_vm_in_fault_mode(vm) ||
> +						    op->map.immediate);
>  		break;
>  	case DRM_GPUVA_OP_REMAP:
>  		err = check_ufence(gpuva_to_vma(op->base.remap.unmap->va));
> diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
> index 52467b9b5348..dace04f4ea5e 100644
> --- a/drivers/gpu/drm/xe/xe_vm_types.h
> +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> @@ -297,6 +297,8 @@ struct xe_vma_op_map {
>  	bool is_null;
>  	/** @dumpable: whether BO is dumped on GPU hang */
>  	bool dumpable;
> +	/** @invalidate: invalidate the VMA before bind */
> +	bool invalidate_on_bind;
>  	/** @pat_index: The pat index to use for this operation. */
>  	u16 pat_index;
>  };
> -- 
> 2.26.3
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 10:34   ` Matthew Brost
@ 2025-02-06 10:43     ` Thomas Hellström
  2025-02-10 17:08       ` Matthew Brost
  2025-02-06 15:16     ` Zeng, Oak
  1 sibling, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-06 10:43 UTC (permalink / raw)
  To: Matthew Brost, Oak Zeng; +Cc: intel-xe, jonathan.cavitt

Hi, Matt,

On Thu, 2025-02-06 at 02:34 -0800, Matthew Brost wrote:
> On Tue, Feb 04, 2025 at 01:45:57PM -0500, Oak Zeng wrote:
> > When a vm runs under fault mode, if scratch page is enabled, we
> > need
> > to clear the scratch page mapping before vm_bind for the vm_bind
> > address range. Under fault mode, we depend on recoverable page
> > fault
> > to establish mapping in page table. If scratch page is not cleared,
> > GPU access of address won't cause page fault because it always hits
> > the existing scratch page mapping.
> > 
> > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > immediate bind can overwrite the scratch page mapping.
> > 
> > So far only is xe2 and xe3 products are allowed to enable scratch
> > page
> > under fault mode. On other platform we don't allow scratch page
> > under
> > fault mode, so no need of such clearing.
> > 
> > v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> > similar
> > to a map operation, with the exception that PTEs are cleared
> > instead of
> > pointing to valid physical pages. (Matt, Thomas)
> > 
> > TLB invalidation is needed after clear scratch page mapping as
> > larger
> > scratch page mapping could be backed by physical page and cached in
> > TLB. (Matt, Thomas)
> > 
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> 
> Given the complexity of the VM bind path, I think we need an IGT
> posted
> with this series before merging as I suggested in v1. Without it, it
> will be fairly difficult to ensure correctness by reviews only.

There is an igt posted that exercises xe_exec_fault_mode + scratch
pages. Although you might have had something more elaborate in mind?

/Thomas


> 
> Matt
> 
> > ---
> >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++------
> > ----
> >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> >  4 files changed, 75 insertions(+), 24 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > b/drivers/gpu/drm/xe/xe_pt.c
> > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > --- a/drivers/gpu/drm/xe/xe_pt.c
> > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> >  	 * granularity.
> >  	 */
> >  	bool needs_64K;
> > +	/* @clear_pt: clear page table entries during the bind
> > walk */
> > +	bool clear_pt;
> >  	/**
> >  	 * @vma: VMA being mapped
> >  	 */
> > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > pgoff_t offset,
> >  
> >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> >  
> > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > -						 xe_res_dma(curs)
> > + xe_walk->dma_offset,
> > -						 xe_walk->vma,
> > pat_index, level);
> > -		pte |= xe_walk->default_pte;
> > +		if (xe_walk->clear_pt) {
> > +			pte = 0;
> > +		} else {
> > +			pte = vm->pt_ops->pte_encode_vma(is_null ?
> > 0 :
> > +					xe_res_dma(curs) +
> > xe_walk->dma_offset,
> > +					xe_walk->vma, pat_index,
> > level);
> > +			pte |= xe_walk->default_pte;
> >  
> > -		/*
> > -		 * Set the XE_PTE_PS64 hint if possible, otherwise
> > if
> > -		 * this device *requires* 64K PTE size for VRAM,
> > fail.
> > -		 */
> > -		if (level == 0 && !xe_parent->is_compact) {
> > -			if (xe_pt_is_pte_ps64K(addr, next,
> > xe_walk)) {
> > -				xe_walk->vma->gpuva.flags |=
> > XE_VMA_PTE_64K;
> > -				pte |= XE_PTE_PS64;
> > -			} else if (XE_WARN_ON(xe_walk->needs_64K))
> > {
> > -				return -EINVAL;
> > +			/*
> > +			 * Set the XE_PTE_PS64 hint if possible,
> > otherwise if
> > +			 * this device *requires* 64K PTE size for
> > VRAM, fail.
> > +			 */
> > +			if (level == 0 && !xe_parent->is_compact)
> > {
> > +				if (xe_pt_is_pte_ps64K(addr, next,
> > xe_walk)) {
> > +					xe_walk->vma->gpuva.flags
> > |= XE_VMA_PTE_64K;
> > +					pte |= XE_PTE_PS64;
> > +				} else if (XE_WARN_ON(xe_walk-
> > >needs_64K)) {
> > +					return -EINVAL;
> > +				}
> >  			}
> >  		}
> >  
> > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > pgoff_t offset,
> >  		if (unlikely(ret))
> >  			return ret;
> >  
> > -		if (!is_null)
> > +		if (!is_null && !xe_walk->clear_pt)
> >  			xe_res_next(curs, next - addr);
> >  		xe_walk->va_curs_start = next;
> >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > level);
> > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> >   * @vma: The vma indicating the address range.
> >   * @entries: Storage for the update entries used for connecting
> > the tree to
> >   * the main tree at commit time.
> > + * @clear_pt: Clear the page table entries.
> >   * @num_entries: On output contains the number of @entries used.
> >   *
> >   * This function builds a disconnected page-table tree for a given
> > address
> > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> >   */
> >  static int
> >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		 struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > +		 struct xe_vm_pgtable_update *entries,
> > +		 bool clear_pt, u32 *num_entries)
> >  {
> >  	struct xe_device *xe = tile_to_xe(tile);
> >  	struct xe_bo *bo = xe_vma_bo(vma);
> > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile, struct
> > xe_vma *vma,
> >  		.vma = vma,
> >  		.wupd.entries = entries,
> >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > XE_VM_FLAG_64K) && is_devmem,
> > +		.clear_pt = clear_pt,
> >  	};
> >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> >  	int ret;
> >  
> > +	if (clear_pt) {
> > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > xe_vma_start(vma),
> > +				       xe_vma_end(vma),
> > &xe_walk.base);
> > +
> > +		*num_entries = xe_walk.wupd.num_used_entries;
> > +		return ret;
> > +	}
> > +
> >  	/**
> >  	 * Default atomic expectations for different allocation
> > scenarios are as follows:
> >  	 *
> > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > xe_vm_pgtable_update *entries,
> >  
> >  static int
> >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		   struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > +		   struct xe_vm_pgtable_update *entries,
> > +		   bool invalidate_on_bind, u32 *num_entries)
> >  {
> >  	int err;
> >  
> >  	*num_entries = 0;
> > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > +	err = xe_pt_stage_bind(tile, vma, entries,
> > invalidate_on_bind,
> > +			       num_entries);
> >  	if (!err)
> >  		xe_tile_assert(tile, *num_entries);
> >  
> > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> > struct xe_tile *tile,
> >  		return err;
> >  
> >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > +				 pt_update_ops-
> > >invalidate_on_bind,
> >  				 &pt_op->num_entries);
> >  	if (!err) {
> >  		xe_tile_assert(tile, pt_op->num_entries <=
> > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> > struct xe_tile *tile,
> >  		 * it needs to be done here.
> >  		 */
> >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > -		     xe_vm_in_preempt_fence_mode(vm)))
> > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > pt_update_ops->invalidate_on_bind)
> >  			pt_update_ops->needs_invalidation = true;
> >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> >  			/* We bump also if batch_invalidate_tlb is
> > true */
> > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> >  
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		if (!op->map.immediate && xe_vm_in_fault_mode(vm))
> > +		if (!op->map.immediate && xe_vm_in_fault_mode(vm)
> > &&
> > +		    !op->map.invalidate_on_bind)
> >  			break;
> >  
> > +		if (op->map.invalidate_on_bind)
> > +			pt_update_ops->invalidate_on_bind = true;
> > +
> >  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> > >map.vma);
> >  		pt_update_ops->wait_vm_kernel = true;
> >  		break;
> > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm *vm,
> > struct xe_tile *tile,
> >  	}
> >  	vma->tile_present |= BIT(tile->id);
> >  	vma->tile_staged &= ~BIT(tile->id);
> > +	if (pt_update_ops->invalidate_on_bind)
> > +		vma->tile_invalidated |= BIT(tile->id);
> >  	if (xe_vma_is_userptr(vma)) {
> >  		lockdep_assert_held_read(&vm-
> > >userptr.notifier_lock);
> >  		to_userptr_vma(vma)->userptr.initial_bind = true;
> > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > b/drivers/gpu/drm/xe/xe_pt_types.h
> > index 384cc04de719..3d0aa2a5102e 100644
> > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> >  	bool needs_userptr_lock;
> >  	/** @needs_invalidation: Needs invalidation */
> >  	bool needs_invalidation;
> > +	/** @invalidate_on_bind: Invalidate the range before bind
> > */
> > +	bool invalidate_on_bind;
> >  	/**
> >  	 * @wait_vm_bookkeep: PT operations need to wait until VM
> > is idle
> >  	 * (bookkeep dma-resv slots are idle) and stage all future
> > VM activity
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > index d664f2e418b2..813d893d9b63 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> > struct drm_gpuva_op *op)
> >  }
> >  #endif
> >  
> > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm,
> > u32 bind_flags)
> > +{
> > +	if (!xe_vm_in_fault_mode(vm))
> > +		return false;
> > +
> > +	if (!NEEDS_SCRATCH(vm->xe))
> > +		return false;
> > +
> > +	if (!xe_vm_has_scratch(vm))
> > +		return false;
> > +
> > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> >  /*
> >   * Create operations list from IOCTL arguments, setup operations
> > fields so parse
> >   * and commit steps are decoupled from IOCTL arguments. This step
> > can fail.
> > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm,
> > struct xe_bo *bo,
> >  			op->map.is_null = flags &
> > DRM_XE_VM_BIND_FLAG_NULL;
> >  			op->map.dumpable = flags &
> > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> >  			op->map.pat_index = pat_index;
> > +			op->map.invalidate_on_bind =
> > +				__xe_vm_needs_clear_scratch_pages(
> > vm, flags);
> >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> >  			op->prefetch.region = prefetch_region;
> >  		}
> > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> > xe_vm *vm, struct drm_gpuva_ops *ops,
> >  				return PTR_ERR(vma);
> >  
> >  			op->map.vma = vma;
> > -			if (op->map.immediate ||
> > !xe_vm_in_fault_mode(vm))
> > +			if (op->map.immediate ||
> > !xe_vm_in_fault_mode(vm) ||
> > +			    op->map.invalidate_on_bind)
> >  				xe_vma_ops_incr_pt_update_ops(vops
> > ,
> >  							      op-
> > >tile_mask);
> >  			break;
> > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct drm_exec
> > *exec, struct xe_vm *vm,
> >  
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		err = vma_lock_and_validate(exec, op->map.vma,
> > -					   
> > !xe_vm_in_fault_mode(vm) ||
> > -					    op->map.immediate);
> > +		if (!op->map.invalidate_on_bind)
> > +			err = vma_lock_and_validate(exec, op-
> > >map.vma,
> > +						   
> > !xe_vm_in_fault_mode(vm) ||
> > +						    op-
> > >map.immediate);
> >  		break;
> >  	case DRM_GPUVA_OP_REMAP:
> >  		err = check_ufence(gpuva_to_vma(op-
> > >base.remap.unmap->va));
> > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > b/drivers/gpu/drm/xe/xe_vm_types.h
> > index 52467b9b5348..dace04f4ea5e 100644
> > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> >  	bool is_null;
> >  	/** @dumpable: whether BO is dumped on GPU hang */
> >  	bool dumpable;
> > +	/** @invalidate: invalidate the VMA before bind */
> > +	bool invalidate_on_bind;
> >  	/** @pat_index: The pat index to use for this operation.
> > */
> >  	u16 pat_index;
> >  };
> > -- 
> > 2.26.3
> > 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06  1:52     ` Zeng, Oak
@ 2025-02-06 12:51       ` Thomas Hellström
  2025-02-06 18:56         ` Zeng, Oak
  0 siblings, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-06 12:51 UTC (permalink / raw)
  To: Zeng, Oak, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan

On Thu, 2025-02-06 at 01:52 +0000, Zeng, Oak wrote:
> 
> 
> > -----Original Message-----
> > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > Sent: February 5, 2025 9:36 AM
> > To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > <jonathan.cavitt@intel.com>
> > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> > 
> > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > When a vm runs under fault mode, if scratch page is enabled, we
> > need
> > > to clear the scratch page mapping before vm_bind for the vm_bind
> > > address range. Under fault mode, we depend on recoverable page
> > fault
> > > to establish mapping in page table. If scratch page is not
> > > cleared,
> > > GPU access of address won't cause page fault because it always
> > > hits
> > > the existing scratch page mapping.
> > 
> > I think we need to ephasize that we clear AT vm_bind time, not
> > before,
> > both in the commit description and the patch title.
> 
> Will fix.
> 
> > 
> > > 
> > > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > > immediate bind can overwrite the scratch page mapping.
> > > 
> > > So far only is xe2 and xe3 products are allowed to enable scratch
> > > page
> > > under fault mode. On other platform we don't allow scratch page
> > under
> > > fault mode, so no need of such clearing.
> > > 
> > > v2: Rework vm_bind pipeline to clear scratch page mapping. This
> > > is
> > > similar
> > > to a map operation, with the exception that PTEs are cleared
> > instead
> > > of
> > > pointing to valid physical pages. (Matt, Thomas)
> > > 
> > > TLB invalidation is needed after clear scratch page mapping as
> > > larger
> > > scratch page mapping could be backed by physical page and cached
> > in
> > > TLB. (Matt, Thomas)
> > > 
> > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > ---
> > >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++--
> > ------
> > > --
> > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > b/drivers/gpu/drm/xe/xe_pt.c
> > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > >  	 * granularity.
> > >  	 */
> > >  	bool needs_64K;
> > > +	/* @clear_pt: clear page table entries during the bind
> > > walk
> > > */
> > > +	bool clear_pt;
> > >  	/**
> > >  	 * @vma: VMA being mapped
> > >  	 */
> > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > *parent,
> > > pgoff_t offset,
> > > 
> > >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> > > 
> > > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > > -						
> > > xe_res_dma(curs) +
> > > xe_walk->dma_offset,
> > > -						 xe_walk->vma,
> > > pat_index, level);
> > > -		pte |= xe_walk->default_pte;
> > > +		if (xe_walk->clear_pt) {
> > > +			pte = 0;
> > > +		} else {
> > > +			pte = vm->pt_ops->pte_encode_vma(is_null
> > > ?
> > 0
> > > :
> > > +					xe_res_dma(curs) +
> > > xe_walk-
> > > > dma_offset,
> > > +					xe_walk->vma, pat_index,
> > > level);
> > > +			pte |= xe_walk->default_pte;
> > > 
> > > -		/*
> > > -		 * Set the XE_PTE_PS64 hint if possible,
> > > otherwise
> > > if
> > > -		 * this device *requires* 64K PTE size for VRAM,
> > > fail.
> > > -		 */
> > > -		if (level == 0 && !xe_parent->is_compact) {
> > > -			if (xe_pt_is_pte_ps64K(addr, next,
> > > xe_walk))
> > > {
> > > -				xe_walk->vma->gpuva.flags |=
> > > XE_VMA_PTE_64K;
> > > -				pte |= XE_PTE_PS64;
> > > -			} else if (XE_WARN_ON(xe_walk-
> > > needs_64K)) {
> > > -				return -EINVAL;
> > > +			/*
> > > +			 * Set the XE_PTE_PS64 hint if possible,
> > > otherwise if
> > > +			 * this device *requires* 64K PTE size
> > > for
> > > VRAM, fail.
> > > +			 */
> > > +			if (level == 0 && !xe_parent-
> > > >is_compact) {
> > > +				if (xe_pt_is_pte_ps64K(addr,
> > > next,
> > > xe_walk)) {
> > > +					xe_walk->vma-
> > > >gpuva.flags
> > > =
> > > XE_VMA_PTE_64K;
> > > +					pte |= XE_PTE_PS64;
> > > +				} else if (XE_WARN_ON(xe_walk-
> > > > needs_64K)) {
> > > +					return -EINVAL;
> > > +				}
> > >  			}
> > >  		}
> > > 
> > 
> > I think there might be some things missing here.
> > The bind should look exactly as a is_null bind, with the exception
> > that
> > the pte should be 0, so might be able to benefit from is_null, for
> > example xe_pg_hugepte_possible() short-circuit the dma-address
> > check.
> 
> You are right. Above code has a problem for huge pte cases: it
> wouldn't detect
> A huge pte for clear_pt case. I will fix it.
> 
> I will not make is_null be true for clear_pt case, but will make
> similar logic to
> Let clear_pt benefit from is_null logic.
> 
> > With clear_pt we don't want to use the XE_PTE_PS64K, though, like
> > you
> > identified above.
> > 
> > Will get back with a look at the invalidation.
> 
> I don't quite get here. Please do let me know if you find anything.

Yes, I meant I hadn't had time to look at this part yet.

For binding with scratch, it looks like *all* binds need TLB
invalidation, not just clearing binds, So in the below code-path, can't
we just replace the xe_vm_in_preempt_fence_mode() with
xe_vm_in_lr_mode()?


		/*
		 * If rebind, we have to invalidate TLB on !LR vms to
invalidate
		 * cached PTEs point to freed memory. On LR vms this
is done
		 * automatically when the context is re-enabled by the
rebind worker,
		 * or in fault mode it was invalidated on PTE zapping.
		 *
		 * If !rebind, and scratch enabled VMs, there is a
chance the scratch
		 * PTE is already cached in the TLB so it needs to be
invalidated.
		 * On !LR VMs this is done in the ring ops preceding a
batch, but on
		 * non-faulting LR, in particular on user-space batch
buffer chaining,
		 * it needs to be done here.
		 */
		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
		     xe_vm_in_preempt_fence_mode(vm)))

/Thomas




















> 
> Oak
> > 
> > 
> > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > *parent,
> > > pgoff_t offset,
> > >  		if (unlikely(ret))
> > >  			return ret;
> > > 
> > > -		if (!is_null)
> > > +		if (!is_null && !xe_walk->clear_pt)
> > >  			xe_res_next(curs, next - addr);
> > >  		xe_walk->va_curs_start = next;
> > >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > > level);
> > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > > xe_pt_stage_bind_ops = {
> > >   * @vma: The vma indicating the address range.
> > >   * @entries: Storage for the update entries used for connecting
> > > the
> > > tree to
> > >   * the main tree at commit time.
> > > + * @clear_pt: Clear the page table entries.
> > >   * @num_entries: On output contains the number of @entries
> > used.
> > >   *
> > >   * This function builds a disconnected page-table tree for a
> > > given
> > > address
> > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > > xe_pt_stage_bind_ops = {
> > >   */
> > >  static int
> > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		 struct xe_vm_pgtable_update *entries, u32
> > > *num_entries)
> > > +		 struct xe_vm_pgtable_update *entries,
> > > +		 bool clear_pt, u32 *num_entries)
> > >  {
> > >  	struct xe_device *xe = tile_to_xe(tile);
> > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> > struct
> > > xe_vma *vma,
> > >  		.vma = vma,
> > >  		.wupd.entries = entries,
> > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > > XE_VM_FLAG_64K) && is_devmem,
> > > +		.clear_pt = clear_pt,
> > >  	};
> > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> > >  	int ret;
> > > 
> > > +	if (clear_pt) {
> > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > > xe_vma_start(vma),
> > > +				       xe_vma_end(vma),
> > > &xe_walk.base);
> > > +
> > > +		*num_entries = xe_walk.wupd.num_used_entries;
> > > +		return ret;
> > > +	}
> > > +
> > >  	/**
> > >  	 * Default atomic expectations for different allocation
> > > scenarios are as follows:
> > >  	 *
> > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > > xe_vm_pgtable_update *entries,
> > > 
> > >  static int
> > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		   struct xe_vm_pgtable_update *entries, u32
> > > *num_entries)
> > > +		   struct xe_vm_pgtable_update *entries,
> > > +		   bool invalidate_on_bind, u32 *num_entries)
> > >  {
> > >  	int err;
> > > 
> > >  	*num_entries = 0;
> > > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > > +	err = xe_pt_stage_bind(tile, vma, entries,
> > > invalidate_on_bind,
> > > +			       num_entries);
> > >  	if (!err)
> > >  		xe_tile_assert(tile, *num_entries);
> > > 
> > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm
> > *vm,
> > > struct xe_tile *tile,
> > >  		return err;
> > > 
> > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > +				 pt_update_ops-
> > > >invalidate_on_bind,
> > >  				 &pt_op->num_entries);
> > >  	if (!err) {
> > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm
> > *vm,
> > > struct xe_tile *tile,
> > >  		 * it needs to be done here.
> > >  		 */
> > >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > > pt_update_ops->invalidate_on_bind)
> > >  			pt_update_ops->needs_invalidation =
> > > true;
> > >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> > >  			/* We bump also if batch_invalidate_tlb
> > > is
> > > true */
> > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> > > 
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		if (!op->map.immediate &&
> > xe_vm_in_fault_mode(vm))
> > > +		if (!op->map.immediate &&
> > xe_vm_in_fault_mode(vm) &&
> > > +		    !op->map.invalidate_on_bind)
> > >  			break;
> > > 
> > > +		if (op->map.invalidate_on_bind)
> > > +			pt_update_ops->invalidate_on_bind =
> > > true;
> > > +
> > >  		err = bind_op_prepare(vm, tile, pt_update_ops,
> > > op-
> > > > map.vma);
> > >  		pt_update_ops->wait_vm_kernel = true;
> > >  		break;
> > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm
> > *vm,
> > > struct xe_tile *tile,
> > >  	}
> > >  	vma->tile_present |= BIT(tile->id);
> > >  	vma->tile_staged &= ~BIT(tile->id);
> > > +	if (pt_update_ops->invalidate_on_bind)
> > > +		vma->tile_invalidated |= BIT(tile->id);
> > >  	if (xe_vma_is_userptr(vma)) {
> > >  		lockdep_assert_held_read(&vm-
> > > > userptr.notifier_lock);
> > >  		to_userptr_vma(vma)->userptr.initial_bind =
> > > true;
> > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > index 384cc04de719..3d0aa2a5102e 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > >  	bool needs_userptr_lock;
> > >  	/** @needs_invalidation: Needs invalidation */
> > >  	bool needs_invalidation;
> > > +	/** @invalidate_on_bind: Invalidate the range before
> > > bind */
> > > +	bool invalidate_on_bind;
> > >  	/**
> > >  	 * @wait_vm_bookkeep: PT operations need to wait until
> > > VM
> > is
> > > idle
> > >  	 * (bookkeep dma-resv slots are idle) and stage all
> > > future
> > > VM activity
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > > index d664f2e418b2..813d893d9b63 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> > > struct drm_gpuva_op *op)
> > >  }
> > >  #endif
> > > 
> > > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm
> > *vm, u32
> > > bind_flags)
> > > +{
> > > +	if (!xe_vm_in_fault_mode(vm))
> > > +		return false;
> > > +
> > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > +		return false;
> > > +
> > > +	if (!xe_vm_has_scratch(vm))
> > > +		return false;
> > > +
> > > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > +		return false;
> > > +
> > > +	return true;
> > > +}
> > > +
> > >  /*
> > >   * Create operations list from IOCTL arguments, setup operations
> > > fields so parse
> > >   * and commit steps are decoupled from IOCTL arguments. This
> > step
> > > can fail.
> > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm
> > *vm,
> > > struct xe_bo *bo,
> > >  			op->map.is_null = flags &
> > > DRM_XE_VM_BIND_FLAG_NULL;
> > >  			op->map.dumpable = flags &
> > > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > >  			op->map.pat_index = pat_index;
> > > +			op->map.invalidate_on_bind =
> > > +
> > 	__xe_vm_needs_clear_scratch_pages(vm
> > > , flags);
> > >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> > >  			op->prefetch.region = prefetch_region;
> > >  		}
> > > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> > xe_vm
> > > *vm, struct drm_gpuva_ops *ops,
> > >  				return PTR_ERR(vma);
> > > 
> > >  			op->map.vma = vma;
> > > -			if (op->map.immediate ||
> > > !xe_vm_in_fault_mode(vm))
> > > +			if (op->map.immediate ||
> > > !xe_vm_in_fault_mode(vm) ||
> > > +			    op->map.invalidate_on_bind)
> > > 
> > 	xe_vma_ops_incr_pt_update_ops(vops,
> > >  							     
> > > op-
> > > > tile_mask);
> > >  			break;
> > > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> > drm_exec
> > > *exec, struct xe_vm *vm,
> > > 
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		err = vma_lock_and_validate(exec, op->map.vma,
> > > -					   
> > > !xe_vm_in_fault_mode(vm)
> > > > > 
> > > -					    op->map.immediate);
> > > +		if (!op->map.invalidate_on_bind)
> > > +			err = vma_lock_and_validate(exec, op-
> > > > map.vma,
> > > +
> > > !xe_vm_in_fault_mode(vm) ||
> > > +						    op-
> > > > map.immediate);
> > >  		break;
> > >  	case DRM_GPUVA_OP_REMAP:
> > >  		err = check_ufence(gpuva_to_vma(op-
> > > > base.remap.unmap->va));
> > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > index 52467b9b5348..dace04f4ea5e 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > >  	bool is_null;
> > >  	/** @dumpable: whether BO is dumped on GPU hang */
> > >  	bool dumpable;
> > > +	/** @invalidate: invalidate the VMA before bind */
> > > +	bool invalidate_on_bind;
> > >  	/** @pat_index: The pat index to use for this operation.
> > > */
> > >  	u16 pat_index;
> > >  };
> 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-06  9:29       ` Thomas Hellström
@ 2025-02-06 15:14         ` Zeng, Oak
  0 siblings, 0 replies; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06 15:14 UTC (permalink / raw)
  To: Thomas Hellström, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan



> -----Original Message-----
> From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Sent: February 6, 2025 4:30 AM
> To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> mode for certain platform
> 
> On Thu, 2025-02-06 at 01:54 +0000, Zeng, Oak wrote:
> >
> >
> > > -----Original Message-----
> > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > Sent: February 5, 2025 8:14 AM
> > > To: Zeng, Oak <oak.zeng@intel.com>; intel-
> xe@lists.freedesktop.org
> > > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > > <jonathan.cavitt@intel.com>
> > > Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> > > mode for certain platform
> > >
> > > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > > Normally scratch page is not allowed when a vm is operate
> under
> > > page
> > > > fault mode, i.e., in the existing codes,
> > > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > > > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual
> > > exclusive. The reason
> > > > is fault mode relies on recoverable page to work, while scratch
> > > > page
> > > > can mute recoverable page fault.
> > > >
> > > > On xe2 and xe3, out of bound prefetch can cause page fault and
> > > > further
> > > > system hang because xekmd can't resolve such page fault. SYCL
> and
> > > OCL
> > > > language runtime requires out of bound prefetch to be silently
> > > > dropped
> > > > without causing any functional problem, thus the existing
> > > > behavior
> > > > doesn't meet language runtime requirement.
> > > >
> > > > At the same time, HW prefetching can cause page fault interrupt.
> > > Due
> > > > to
> > > > page fault interrupt overhead (i.e., need Guc and KMD involved
> to
> > > fix
> > > > the page fault), HW prefetching can be slowed by many orders
> of
> > > > magnitude.
> > > >
> > > > Fix those problems by allowing scratch page under fault mode
> for
> > > xe2
> > > > and
> > > > xe3. With scratch page in place, HW prefetching could always hit
> > > > scratch
> > > > page instead of causing interrupt.
> > > >
> > > > A side effect is, scratch page could hide application program
> > > > error.
> > > > Application out of bound accesses are hided
> > > s/hided/hidden/
> > >
> > > >  by scratch page mapping,
> > > > instead of get reported to user.
> > > >
> > > > igt test: https://patchwork.freedesktop.org/series/144334/.
> Test
> > > > result on
> > > > BMG:
> > > >
> > > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > > public/build/tests#
> > > > ./xe_exec_fault_mode --run-subtest scratch-fault
> > > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > > Using IGT_SRANDOM=1738684805 for randomisation
> > > > Opened device: /dev/dri/card0
> > > > Starting subtest: scratch-fault
> > > > Subtest scratch-fault: SUCCESS (0.080s)
> > > >
> > > > Without this series, the test result is:
> > > >
> > > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > > public/build/tests#
> > > > ./xe_exec_fault_mode --run-subtest scratch-fault
> > > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > > Using IGT_SRANDOM=1738686046 for randomisation
> > > > Opened device: /dev/dri/card0
> > > > Starting subtest: scratch-fault
> > > > (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure
> > > function
> > > > test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> > > > (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> > > > __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> > > > exec_queues[i % n_exec_queues], &timeout) == 0
> > > > (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer
> expired
> > > > (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> > > > Stack trace:
> > > >   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
> > > >   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
> > > >   #2 ../tests/intel/xe_exec_fault_mode.c:537
> > > > __igt_unique____real_main407()
> > > >   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
> > > >   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> > > > __libc_start_call_main()
> > > >   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
> > > >   #6 [_start+0x2e]
> > > > Subtest scratch-fault failed.
> > > >
> > > > v2: Refine commit message (Thomas)
> > > >
> > > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > > ---
> > > >  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
> > > >  1 file changed, 5 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > b/drivers/gpu/drm/xe/xe_vm.c
> > > > index 813d893d9b63..c0372f083d42 100644
> > > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > > @@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct
> > > drm_device *dev,
> > > > void *data,
> > > >  	if (XE_IOCTL_DBG(xe, args->extensions))
> > > >  		return -EINVAL;
> > > >
> > > > +	if (XE_IOCTL_DBG(xe, args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > > > +			 args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > > > +			 !(NEEDS_SCRATCH(xe))))
> > > > +		return -EINVAL;
> > > > +
> > >
> > > We should probably move this test to where the old test were
> below,
> > > since the WA below enables scratch pages.
> >
> >
> > Below wa code is for dg2.
> > If we move this test below the wa code, then on dg2, if people
> create
> > vm
> > With FAULT_MODE | LR_MODE, above check would fail user.
> 
> Though on DG2 we don't support fault mode.
> 
> >
> > So I do think we should keep this check above wa codes.
> 
> I think what matters here is readability of the code. A naive reader
> (like me in this case) would start to wonder why enabling scratch
> pages
> *after* the sanity check for them is done, and whether that is a bug
> or
> an oversight, and that would require looking up what the WA actually
> means.
> 
> So IMO in general sanity checks should be done after all implicit
> enabling of flags if at all possible. And if not, they should be
> accompanied by a comment that explains why the enabling of the flag
> is
> not included in the sanity check.

Ok, make sense to me. Will move the check below the wa then.

Oak

> 
> /Thomas
> 
> 
> >
> > Oak
> >
> >
> > >
> > > /Thomas
> > >
> > >
> > > >  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
> > > >  		args->flags |=
> > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
> > > >
> > > > @@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct
> > > drm_device *dev,
> > > > void *data,
> > > >  	if (XE_IOCTL_DBG(xe, args->flags &
> > > > ~ALL_DRM_XE_VM_CREATE_FLAGS))
> > > >  		return -EINVAL;
> > > >
> > > > -	if (XE_IOCTL_DBG(xe, args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > > > -			 args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > > > -		return -EINVAL;
> > > > -
> > > >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> > > >  			 args->flags &
> > > > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > > >  		return -EINVAL;
> >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 10:34   ` Matthew Brost
  2025-02-06 10:43     ` Thomas Hellström
@ 2025-02-06 15:16     ` Zeng, Oak
  2025-02-10 17:09       ` Matthew Brost
  1 sibling, 1 reply; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06 15:16 UTC (permalink / raw)
  To: Brost, Matthew
  Cc: intel-xe@lists.freedesktop.org, Thomas.Hellstrom@linux.intel.com,
	Cavitt, Jonathan



> -----Original Message-----
> From: Brost, Matthew <matthew.brost@intel.com>
> Sent: February 6, 2025 5:35 AM
> To: Zeng, Oak <oak.zeng@intel.com>
> Cc: intel-xe@lists.freedesktop.org;
> Thomas.Hellstrom@linux.intel.com; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> 
> On Tue, Feb 04, 2025 at 01:45:57PM -0500, Oak Zeng wrote:
> > When a vm runs under fault mode, if scratch page is enabled, we
> need
> > to clear the scratch page mapping before vm_bind for the vm_bind
> > address range. Under fault mode, we depend on recoverable page
> fault
> > to establish mapping in page table. If scratch page is not cleared,
> > GPU access of address won't cause page fault because it always hits
> > the existing scratch page mapping.
> >
> > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > immediate bind can overwrite the scratch page mapping.
> >
> > So far only is xe2 and xe3 products are allowed to enable scratch
> page
> > under fault mode. On other platform we don't allow scratch page
> under
> > fault mode, so no need of such clearing.
> >
> > v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> similar
> > to a map operation, with the exception that PTEs are cleared
> instead of
> > pointing to valid physical pages. (Matt, Thomas)
> >
> > TLB invalidation is needed after clear scratch page mapping as larger
> > scratch page mapping could be backed by physical page and cached
> in
> > TLB. (Matt, Thomas)
> >
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> 
> Given the complexity of the VM bind path, I think we need an IGT
> posted
> with this series before merging as I suggested in v1. Without it, it
> will be fairly difficult to ensure correctness by reviews only.

I did have an igt posted in patch 3 of this series.

The reason I posted on patch 3 is, without patch 3 this "feature"
Is not enabled. 

Oak

> 
> Matt
> 
> > ---
> >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++--
> --------
> >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> >  4 files changed, 75 insertions(+), 24 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> b/drivers/gpu/drm/xe/xe_pt.c
> > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > --- a/drivers/gpu/drm/xe/xe_pt.c
> > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> >  	 * granularity.
> >  	 */
> >  	bool needs_64K;
> > +	/* @clear_pt: clear page table entries during the bind walk */
> > +	bool clear_pt;
> >  	/**
> >  	 * @vma: VMA being mapped
> >  	 */
> > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw
> *parent, pgoff_t offset,
> >
> >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> >
> > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > -						 xe_res_dma(curs) +
> xe_walk->dma_offset,
> > -						 xe_walk->vma,
> pat_index, level);
> > -		pte |= xe_walk->default_pte;
> > +		if (xe_walk->clear_pt) {
> > +			pte = 0;
> > +		} else {
> > +			pte = vm->pt_ops->pte_encode_vma(is_null ?
> 0 :
> > +					xe_res_dma(curs) + xe_walk-
> >dma_offset,
> > +					xe_walk->vma, pat_index,
> level);
> > +			pte |= xe_walk->default_pte;
> >
> > -		/*
> > -		 * Set the XE_PTE_PS64 hint if possible, otherwise if
> > -		 * this device *requires* 64K PTE size for VRAM, fail.
> > -		 */
> > -		if (level == 0 && !xe_parent->is_compact) {
> > -			if (xe_pt_is_pte_ps64K(addr, next, xe_walk))
> {
> > -				xe_walk->vma->gpuva.flags |=
> XE_VMA_PTE_64K;
> > -				pte |= XE_PTE_PS64;
> > -			} else if (XE_WARN_ON(xe_walk-
> >needs_64K)) {
> > -				return -EINVAL;
> > +			/*
> > +			 * Set the XE_PTE_PS64 hint if possible,
> otherwise if
> > +			 * this device *requires* 64K PTE size for
> VRAM, fail.
> > +			 */
> > +			if (level == 0 && !xe_parent->is_compact) {
> > +				if (xe_pt_is_pte_ps64K(addr, next,
> xe_walk)) {
> > +					xe_walk->vma->gpuva.flags
> |= XE_VMA_PTE_64K;
> > +					pte |= XE_PTE_PS64;
> > +				} else if (XE_WARN_ON(xe_walk-
> >needs_64K)) {
> > +					return -EINVAL;
> > +				}
> >  			}
> >  		}
> >
> > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> *parent, pgoff_t offset,
> >  		if (unlikely(ret))
> >  			return ret;
> >
> > -		if (!is_null)
> > +		if (!is_null && !xe_walk->clear_pt)
> >  			xe_res_next(curs, next - addr);
> >  		xe_walk->va_curs_start = next;
> >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> level);
> > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> xe_pt_stage_bind_ops = {
> >   * @vma: The vma indicating the address range.
> >   * @entries: Storage for the update entries used for connecting the
> tree to
> >   * the main tree at commit time.
> > + * @clear_pt: Clear the page table entries.
> >   * @num_entries: On output contains the number of @entries
> used.
> >   *
> >   * This function builds a disconnected page-table tree for a given
> address
> > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> xe_pt_stage_bind_ops = {
> >   */
> >  static int
> >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		 struct xe_vm_pgtable_update *entries, u32
> *num_entries)
> > +		 struct xe_vm_pgtable_update *entries,
> > +		 bool clear_pt, u32 *num_entries)
> >  {
> >  	struct xe_device *xe = tile_to_xe(tile);
> >  	struct xe_bo *bo = xe_vma_bo(vma);
> > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> struct xe_vma *vma,
> >  		.vma = vma,
> >  		.wupd.entries = entries,
> >  		.needs_64K = (xe_vma_vm(vma)->flags &
> XE_VM_FLAG_64K) && is_devmem,
> > +		.clear_pt = clear_pt,
> >  	};
> >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> >  	int ret;
> >
> > +	if (clear_pt) {
> > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> xe_vma_start(vma),
> > +				       xe_vma_end(vma),
> &xe_walk.base);
> > +
> > +		*num_entries = xe_walk.wupd.num_used_entries;
> > +		return ret;
> > +	}
> > +
> >  	/**
> >  	 * Default atomic expectations for different allocation
> scenarios are as follows:
> >  	 *
> > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> xe_vm_pgtable_update *entries,
> >
> >  static int
> >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > -		   struct xe_vm_pgtable_update *entries, u32
> *num_entries)
> > +		   struct xe_vm_pgtable_update *entries,
> > +		   bool invalidate_on_bind, u32 *num_entries)
> >  {
> >  	int err;
> >
> >  	*num_entries = 0;
> > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > +	err = xe_pt_stage_bind(tile, vma, entries, invalidate_on_bind,
> > +			       num_entries);
> >  	if (!err)
> >  		xe_tile_assert(tile, *num_entries);
> >
> > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm
> *vm, struct xe_tile *tile,
> >  		return err;
> >
> >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > +				 pt_update_ops->invalidate_on_bind,
> >  				 &pt_op->num_entries);
> >  	if (!err) {
> >  		xe_tile_assert(tile, pt_op->num_entries <=
> > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm
> *vm, struct xe_tile *tile,
> >  		 * it needs to be done here.
> >  		 */
> >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > -		     xe_vm_in_preempt_fence_mode(vm)))
> > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> pt_update_ops->invalidate_on_bind)
> >  			pt_update_ops->needs_invalidation = true;
> >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> >  			/* We bump also if batch_invalidate_tlb is
> true */
> > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> >
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		if (!op->map.immediate &&
> xe_vm_in_fault_mode(vm))
> > +		if (!op->map.immediate &&
> xe_vm_in_fault_mode(vm) &&
> > +		    !op->map.invalidate_on_bind)
> >  			break;
> >
> > +		if (op->map.invalidate_on_bind)
> > +			pt_update_ops->invalidate_on_bind = true;
> > +
> >  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> >map.vma);
> >  		pt_update_ops->wait_vm_kernel = true;
> >  		break;
> > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm
> *vm, struct xe_tile *tile,
> >  	}
> >  	vma->tile_present |= BIT(tile->id);
> >  	vma->tile_staged &= ~BIT(tile->id);
> > +	if (pt_update_ops->invalidate_on_bind)
> > +		vma->tile_invalidated |= BIT(tile->id);
> >  	if (xe_vma_is_userptr(vma)) {
> >  		lockdep_assert_held_read(&vm-
> >userptr.notifier_lock);
> >  		to_userptr_vma(vma)->userptr.initial_bind = true;
> > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> b/drivers/gpu/drm/xe/xe_pt_types.h
> > index 384cc04de719..3d0aa2a5102e 100644
> > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> >  	bool needs_userptr_lock;
> >  	/** @needs_invalidation: Needs invalidation */
> >  	bool needs_invalidation;
> > +	/** @invalidate_on_bind: Invalidate the range before bind */
> > +	bool invalidate_on_bind;
> >  	/**
> >  	 * @wait_vm_bookkeep: PT operations need to wait until VM
> is idle
> >  	 * (bookkeep dma-resv slots are idle) and stage all future VM
> activity
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> b/drivers/gpu/drm/xe/xe_vm.c
> > index d664f2e418b2..813d893d9b63 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> struct drm_gpuva_op *op)
> >  }
> >  #endif
> >
> > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm
> *vm, u32 bind_flags)
> > +{
> > +	if (!xe_vm_in_fault_mode(vm))
> > +		return false;
> > +
> > +	if (!NEEDS_SCRATCH(vm->xe))
> > +		return false;
> > +
> > +	if (!xe_vm_has_scratch(vm))
> > +		return false;
> > +
> > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> >  /*
> >   * Create operations list from IOCTL arguments, setup operations
> fields so parse
> >   * and commit steps are decoupled from IOCTL arguments. This
> step can fail.
> > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm
> *vm, struct xe_bo *bo,
> >  			op->map.is_null = flags &
> DRM_XE_VM_BIND_FLAG_NULL;
> >  			op->map.dumpable = flags &
> DRM_XE_VM_BIND_FLAG_DUMPABLE;
> >  			op->map.pat_index = pat_index;
> > +			op->map.invalidate_on_bind =
> > +
> 	__xe_vm_needs_clear_scratch_pages(vm, flags);
> >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> >  			op->prefetch.region = prefetch_region;
> >  		}
> > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> xe_vm *vm, struct drm_gpuva_ops *ops,
> >  				return PTR_ERR(vma);
> >
> >  			op->map.vma = vma;
> > -			if (op->map.immediate
> || !xe_vm_in_fault_mode(vm))
> > +			if (op->map.immediate
> || !xe_vm_in_fault_mode(vm) ||
> > +			    op->map.invalidate_on_bind)
> >
> 	xe_vma_ops_incr_pt_update_ops(vops,
> >  							      op-
> >tile_mask);
> >  			break;
> > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> drm_exec *exec, struct xe_vm *vm,
> >
> >  	switch (op->base.op) {
> >  	case DRM_GPUVA_OP_MAP:
> > -		err = vma_lock_and_validate(exec, op->map.vma,
> > -					    !xe_vm_in_fault_mode(vm)
> ||
> > -					    op->map.immediate);
> > +		if (!op->map.invalidate_on_bind)
> > +			err = vma_lock_and_validate(exec, op-
> >map.vma,
> > +
> 	    !xe_vm_in_fault_mode(vm) ||
> > +						    op-
> >map.immediate);
> >  		break;
> >  	case DRM_GPUVA_OP_REMAP:
> >  		err = check_ufence(gpuva_to_vma(op-
> >base.remap.unmap->va));
> > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> b/drivers/gpu/drm/xe/xe_vm_types.h
> > index 52467b9b5348..dace04f4ea5e 100644
> > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> >  	bool is_null;
> >  	/** @dumpable: whether BO is dumped on GPU hang */
> >  	bool dumpable;
> > +	/** @invalidate: invalidate the VMA before bind */
> > +	bool invalidate_on_bind;
> >  	/** @pat_index: The pat index to use for this operation. */
> >  	u16 pat_index;
> >  };
> > --
> > 2.26.3
> >

^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 12:51       ` Thomas Hellström
@ 2025-02-06 18:56         ` Zeng, Oak
  2025-02-06 20:11           ` Thomas Hellström
  0 siblings, 1 reply; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06 18:56 UTC (permalink / raw)
  To: Thomas Hellström, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan

Hi Thomas!

> -----Original Message-----
> From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Sent: February 6, 2025 7:52 AM
> To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> 
> On Thu, 2025-02-06 at 01:52 +0000, Zeng, Oak wrote:
> >
> >
> > > -----Original Message-----
> > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > Sent: February 5, 2025 9:36 AM
> > > To: Zeng, Oak <oak.zeng@intel.com>; intel-
> xe@lists.freedesktop.org
> > > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > > <jonathan.cavitt@intel.com>
> > > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before
> vm_bind
> > >
> > > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > > When a vm runs under fault mode, if scratch page is enabled,
> we
> > > need
> > > > to clear the scratch page mapping before vm_bind for the
> vm_bind
> > > > address range. Under fault mode, we depend on recoverable
> page
> > > fault
> > > > to establish mapping in page table. If scratch page is not
> > > > cleared,
> > > > GPU access of address won't cause page fault because it always
> > > > hits
> > > > the existing scratch page mapping.
> > >
> > > I think we need to ephasize that we clear AT vm_bind time, not
> > > before,
> > > both in the commit description and the patch title.
> >
> > Will fix.
> >
> > >
> > > >
> > > > When vm_bind with IMMEDIATE flag, there is no need of
> clearing as
> > > > immediate bind can overwrite the scratch page mapping.
> > > >
> > > > So far only is xe2 and xe3 products are allowed to enable scratch
> > > > page
> > > > under fault mode. On other platform we don't allow scratch
> page
> > > under
> > > > fault mode, so no need of such clearing.
> > > >
> > > > v2: Rework vm_bind pipeline to clear scratch page mapping. This
> > > > is
> > > > similar
> > > > to a map operation, with the exception that PTEs are cleared
> > > instead
> > > > of
> > > > pointing to valid physical pages. (Matt, Thomas)
> > > >
> > > > TLB invalidation is needed after clear scratch page mapping as
> > > > larger
> > > > scratch page mapping could be backed by physical page and
> cached
> > > in
> > > > TLB. (Matt, Thomas)
> > > >
> > > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > > ---
> > > >  drivers/gpu/drm/xe/xe_pt.c       | 66
> ++++++++++++++++++++++--
> > > ------
> > > > --
> > > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > > b/drivers/gpu/drm/xe/xe_pt.c
> > > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > > >  	 * granularity.
> > > >  	 */
> > > >  	bool needs_64K;
> > > > +	/* @clear_pt: clear page table entries during the bind
> > > > walk
> > > > */
> > > > +	bool clear_pt;
> > > >  	/**
> > > >  	 * @vma: VMA being mapped
> > > >  	 */
> > > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct
> xe_ptw
> > > *parent,
> > > > pgoff_t offset,
> > > >
> > > >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> > > >
> > > > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > > > -
> > > > xe_res_dma(curs) +
> > > > xe_walk->dma_offset,
> > > > -						 xe_walk->vma,
> > > > pat_index, level);
> > > > -		pte |= xe_walk->default_pte;
> > > > +		if (xe_walk->clear_pt) {
> > > > +			pte = 0;
> > > > +		} else {
> > > > +			pte = vm->pt_ops->pte_encode_vma(is_null
> > > > ?
> > > 0
> > > > :
> > > > +					xe_res_dma(curs) +
> > > > xe_walk-
> > > > > dma_offset,
> > > > +					xe_walk->vma, pat_index,
> > > > level);
> > > > +			pte |= xe_walk->default_pte;
> > > >
> > > > -		/*
> > > > -		 * Set the XE_PTE_PS64 hint if possible,
> > > > otherwise
> > > > if
> > > > -		 * this device *requires* 64K PTE size for VRAM,
> > > > fail.
> > > > -		 */
> > > > -		if (level == 0 && !xe_parent->is_compact) {
> > > > -			if (xe_pt_is_pte_ps64K(addr, next,
> > > > xe_walk))
> > > > {
> > > > -				xe_walk->vma->gpuva.flags |=
> > > > XE_VMA_PTE_64K;
> > > > -				pte |= XE_PTE_PS64;
> > > > -			} else if (XE_WARN_ON(xe_walk-
> > > > needs_64K)) {
> > > > -				return -EINVAL;
> > > > +			/*
> > > > +			 * Set the XE_PTE_PS64 hint if possible,
> > > > otherwise if
> > > > +			 * this device *requires* 64K PTE size
> > > > for
> > > > VRAM, fail.
> > > > +			 */
> > > > +			if (level == 0 && !xe_parent-
> > > > >is_compact) {
> > > > +				if (xe_pt_is_pte_ps64K(addr,
> > > > next,
> > > > xe_walk)) {
> > > > +					xe_walk->vma-
> > > > >gpuva.flags
> > > > =
> > > > XE_VMA_PTE_64K;
> > > > +					pte |= XE_PTE_PS64;
> > > > +				} else if (XE_WARN_ON(xe_walk-
> > > > > needs_64K)) {
> > > > +					return -EINVAL;
> > > > +				}
> > > >  			}
> > > >  		}
> > > >
> > >
> > > I think there might be some things missing here.
> > > The bind should look exactly as a is_null bind, with the exception
> > > that
> > > the pte should be 0, so might be able to benefit from is_null, for
> > > example xe_pg_hugepte_possible() short-circuit the dma-
> address
> > > check.
> >
> > You are right. Above code has a problem for huge pte cases: it
> > wouldn't detect
> > A huge pte for clear_pt case. I will fix it.
> >
> > I will not make is_null be true for clear_pt case, but will make
> > similar logic to
> > Let clear_pt benefit from is_null logic.
> >
> > > With clear_pt we don't want to use the XE_PTE_PS64K, though,
> like
> > > you
> > > identified above.
> > >
> > > Will get back with a look at the invalidation.
> >
> > I don't quite get here. Please do let me know if you find anything.
> 
> Yes, I meant I hadn't had time to look at this part yet.
> 
> For binding with scratch, it looks like *all* binds need TLB
> invalidation, not just clearing binds, So in the below code-path, can't
> we just replace the xe_vm_in_preempt_fence_mode() with
> xe_vm_in_lr_mode()?

I think you are correct. 

For long run vm, regardless it is in fault mode or not, we need to invalidate
TLB scratch page caching during bind. Original codes only did TLB inv on non-
Fault mode. On fault mode + LR,  this runs into two sub-cases:
1.  immediate bind
2. bind triggered by page fault: note this can actually runs into the !rebind case
When the fault address is first time accessed by gpu. But the TLB cache for
Scratch page mapping should have been invalidated by the "invalidate_on_bind"
Condition.

So this leaves case #1 problematic.

It would be better for others to comment also, as in the comment below, it explicitly
Called out "non-faulting LR". My understanding is, people didn't think of case #1 above.

Since it is an existing issue, I will make it a separate patch for better tracking purpose.

Oak 

> 
> 
> 		/*
> 		 * If rebind, we have to invalidate TLB on !LR vms to
> invalidate
> 		 * cached PTEs point to freed memory. On LR vms this
> is done
> 		 * automatically when the context is re-enabled by
> the
> rebind worker,
> 		 * or in fault mode it was invalidated on PTE zapping.
> 		 *
> 		 * If !rebind, and scratch enabled VMs, there is a
> chance the scratch
> 		 * PTE is already cached in the TLB so it needs to be
> invalidated.
> 		 * On !LR VMs this is done in the ring ops preceding a
> batch, but on
> 		 * non-faulting LR, in particular on user-space batch
> buffer chaining,
> 		 * it needs to be done here.
> 		 */
> 		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> 		     xe_vm_in_preempt_fence_mode(vm)))
> 
> /Thomas
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> >
> > Oak
> > >
> > >
> > > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > > *parent,
> > > > pgoff_t offset,
> > > >  		if (unlikely(ret))
> > > >  			return ret;
> > > >
> > > > -		if (!is_null)
> > > > +		if (!is_null && !xe_walk->clear_pt)
> > > >  			xe_res_next(curs, next - addr);
> > > >  		xe_walk->va_curs_start = next;
> > > >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > > > level);
> > > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > > > xe_pt_stage_bind_ops = {
> > > >   * @vma: The vma indicating the address range.
> > > >   * @entries: Storage for the update entries used for connecting
> > > > the
> > > > tree to
> > > >   * the main tree at commit time.
> > > > + * @clear_pt: Clear the page table entries.
> > > >   * @num_entries: On output contains the number of @entries
> > > used.
> > > >   *
> > > >   * This function builds a disconnected page-table tree for a
> > > > given
> > > > address
> > > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > > > xe_pt_stage_bind_ops = {
> > > >   */
> > > >  static int
> > > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > > -		 struct xe_vm_pgtable_update *entries, u32
> > > > *num_entries)
> > > > +		 struct xe_vm_pgtable_update *entries,
> > > > +		 bool clear_pt, u32 *num_entries)
> > > >  {
> > > >  	struct xe_device *xe = tile_to_xe(tile);
> > > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> > > struct
> > > > xe_vma *vma,
> > > >  		.vma = vma,
> > > >  		.wupd.entries = entries,
> > > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > > > XE_VM_FLAG_64K) && is_devmem,
> > > > +		.clear_pt = clear_pt,
> > > >  	};
> > > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> > > >  	int ret;
> > > >
> > > > +	if (clear_pt) {
> > > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > > > xe_vma_start(vma),
> > > > +				       xe_vma_end(vma),
> > > > &xe_walk.base);
> > > > +
> > > > +		*num_entries = xe_walk.wupd.num_used_entries;
> > > > +		return ret;
> > > > +	}
> > > > +
> > > >  	/**
> > > >  	 * Default atomic expectations for different allocation
> > > > scenarios are as follows:
> > > >  	 *
> > > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > > > xe_vm_pgtable_update *entries,
> > > >
> > > >  static int
> > > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > > -		   struct xe_vm_pgtable_update *entries, u32
> > > > *num_entries)
> > > > +		   struct xe_vm_pgtable_update *entries,
> > > > +		   bool invalidate_on_bind, u32 *num_entries)
> > > >  {
> > > >  	int err;
> > > >
> > > >  	*num_entries = 0;
> > > > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > > > +	err = xe_pt_stage_bind(tile, vma, entries,
> > > > invalidate_on_bind,
> > > > +			       num_entries);
> > > >  	if (!err)
> > > >  		xe_tile_assert(tile, *num_entries);
> > > >
> > > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct
> xe_vm
> > > *vm,
> > > > struct xe_tile *tile,
> > > >  		return err;
> > > >
> > > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > > +				 pt_update_ops-
> > > > >invalidate_on_bind,
> > > >  				 &pt_op->num_entries);
> > > >  	if (!err) {
> > > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct
> xe_vm
> > > *vm,
> > > > struct xe_tile *tile,
> > > >  		 * it needs to be done here.
> > > >  		 */
> > > >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > > > pt_update_ops->invalidate_on_bind)
> > > >  			pt_update_ops->needs_invalidation =
> > > > true;
> > > >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> > > >  			/* We bump also if batch_invalidate_tlb
> > > > is
> > > > true */
> > > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm
> *vm,
> > > >
> > > >  	switch (op->base.op) {
> > > >  	case DRM_GPUVA_OP_MAP:
> > > > -		if (!op->map.immediate &&
> > > xe_vm_in_fault_mode(vm))
> > > > +		if (!op->map.immediate &&
> > > xe_vm_in_fault_mode(vm) &&
> > > > +		    !op->map.invalidate_on_bind)
> > > >  			break;
> > > >
> > > > +		if (op->map.invalidate_on_bind)
> > > > +			pt_update_ops->invalidate_on_bind =
> > > > true;
> > > > +
> > > >  		err = bind_op_prepare(vm, tile, pt_update_ops,
> > > > op-
> > > > > map.vma);
> > > >  		pt_update_ops->wait_vm_kernel = true;
> > > >  		break;
> > > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct
> xe_vm
> > > *vm,
> > > > struct xe_tile *tile,
> > > >  	}
> > > >  	vma->tile_present |= BIT(tile->id);
> > > >  	vma->tile_staged &= ~BIT(tile->id);
> > > > +	if (pt_update_ops->invalidate_on_bind)
> > > > +		vma->tile_invalidated |= BIT(tile->id);
> > > >  	if (xe_vma_is_userptr(vma)) {
> > > >  		lockdep_assert_held_read(&vm-
> > > > > userptr.notifier_lock);
> > > >  		to_userptr_vma(vma)->userptr.initial_bind =
> > > > true;
> > > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > index 384cc04de719..3d0aa2a5102e 100644
> > > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > > >  	bool needs_userptr_lock;
> > > >  	/** @needs_invalidation: Needs invalidation */
> > > >  	bool needs_invalidation;
> > > > +	/** @invalidate_on_bind: Invalidate the range before
> > > > bind */
> > > > +	bool invalidate_on_bind;
> > > >  	/**
> > > >  	 * @wait_vm_bookkeep: PT operations need to wait until
> > > > VM
> > > is
> > > > idle
> > > >  	 * (bookkeep dma-resv slots are idle) and stage all
> > > > future
> > > > VM activity
> > > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > b/drivers/gpu/drm/xe/xe_vm.c
> > > > index d664f2e418b2..813d893d9b63 100644
> > > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device
> *xe,
> > > > struct drm_gpuva_op *op)
> > > >  }
> > > >  #endif
> > > >
> > > > +static bool __xe_vm_needs_clear_scratch_pages(struct
> xe_vm
> > > *vm, u32
> > > > bind_flags)
> > > > +{
> > > > +	if (!xe_vm_in_fault_mode(vm))
> > > > +		return false;
> > > > +
> > > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > > +		return false;
> > > > +
> > > > +	if (!xe_vm_has_scratch(vm))
> > > > +		return false;
> > > > +
> > > > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > > +		return false;
> > > > +
> > > > +	return true;
> > > > +}
> > > > +
> > > >  /*
> > > >   * Create operations list from IOCTL arguments, setup
> operations
> > > > fields so parse
> > > >   * and commit steps are decoupled from IOCTL arguments. This
> > > step
> > > > can fail.
> > > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct
> xe_vm
> > > *vm,
> > > > struct xe_bo *bo,
> > > >  			op->map.is_null = flags &
> > > > DRM_XE_VM_BIND_FLAG_NULL;
> > > >  			op->map.dumpable = flags &
> > > > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > > >  			op->map.pat_index = pat_index;
> > > > +			op->map.invalidate_on_bind =
> > > > +
> > > 	__xe_vm_needs_clear_scratch_pages(vm
> > > > , flags);
> > > >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> > > >  			op->prefetch.region = prefetch_region;
> > > >  		}
> > > > @@ -2188,7 +2207,8 @@ static int
> vm_bind_ioctl_ops_parse(struct
> > > xe_vm
> > > > *vm, struct drm_gpuva_ops *ops,
> > > >  				return PTR_ERR(vma);
> > > >
> > > >  			op->map.vma = vma;
> > > > -			if (op->map.immediate ||
> > > > !xe_vm_in_fault_mode(vm))
> > > > +			if (op->map.immediate ||
> > > > !xe_vm_in_fault_mode(vm) ||
> > > > +			    op->map.invalidate_on_bind)
> > > >
> > > 	xe_vma_ops_incr_pt_update_ops(vops,
> > > >
> > > > op-
> > > > > tile_mask);
> > > >  			break;
> > > > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> > > drm_exec
> > > > *exec, struct xe_vm *vm,
> > > >
> > > >  	switch (op->base.op) {
> > > >  	case DRM_GPUVA_OP_MAP:
> > > > -		err = vma_lock_and_validate(exec, op->map.vma,
> > > > -
> > > > !xe_vm_in_fault_mode(vm)
> > > > > >
> > > > -					    op->map.immediate);
> > > > +		if (!op->map.invalidate_on_bind)
> > > > +			err = vma_lock_and_validate(exec, op-
> > > > > map.vma,
> > > > +
> > > > !xe_vm_in_fault_mode(vm) ||
> > > > +						    op-
> > > > > map.immediate);
> > > >  		break;
> > > >  	case DRM_GPUVA_OP_REMAP:
> > > >  		err = check_ufence(gpuva_to_vma(op-
> > > > > base.remap.unmap->va));
> > > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > index 52467b9b5348..dace04f4ea5e 100644
> > > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > > >  	bool is_null;
> > > >  	/** @dumpable: whether BO is dumped on GPU hang */
> > > >  	bool dumpable;
> > > > +	/** @invalidate: invalidate the VMA before bind */
> > > > +	bool invalidate_on_bind;
> > > >  	/** @pat_index: The pat index to use for this operation.
> > > > */
> > > >  	u16 pat_index;
> > > >  };
> >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 18:56         ` Zeng, Oak
@ 2025-02-06 20:11           ` Thomas Hellström
  2025-02-06 21:05             ` Zeng, Oak
  0 siblings, 1 reply; 38+ messages in thread
From: Thomas Hellström @ 2025-02-06 20:11 UTC (permalink / raw)
  To: Zeng, Oak, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan

Hi, Oak,

On Thu, 2025-02-06 at 18:56 +0000, Zeng, Oak wrote:
> Hi Thomas!
> 
> > -----Original Message-----
> > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > Sent: February 6, 2025 7:52 AM
> > To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > <jonathan.cavitt@intel.com>
> > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> > 
> > On Thu, 2025-02-06 at 01:52 +0000, Zeng, Oak wrote:
> > > 
> > > 
> > > > -----Original Message-----
> > > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > > Sent: February 5, 2025 9:36 AM
> > > > To: Zeng, Oak <oak.zeng@intel.com>; intel-
> > xe@lists.freedesktop.org
> > > > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > > > <jonathan.cavitt@intel.com>
> > > > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before
> > vm_bind
> > > > 
> > > > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > > > When a vm runs under fault mode, if scratch page is enabled,
> > we
> > > > need
> > > > > to clear the scratch page mapping before vm_bind for the
> > vm_bind
> > > > > address range. Under fault mode, we depend on recoverable
> > page
> > > > fault
> > > > > to establish mapping in page table. If scratch page is not
> > > > > cleared,
> > > > > GPU access of address won't cause page fault because it
> > > > > always
> > > > > hits
> > > > > the existing scratch page mapping.
> > > > 
> > > > I think we need to ephasize that we clear AT vm_bind time, not
> > > > before,
> > > > both in the commit description and the patch title.
> > > 
> > > Will fix.
> > > 
> > > > 
> > > > > 
> > > > > When vm_bind with IMMEDIATE flag, there is no need of
> > clearing as
> > > > > immediate bind can overwrite the scratch page mapping.
> > > > > 
> > > > > So far only is xe2 and xe3 products are allowed to enable
> > > > > scratch
> > > > > page
> > > > > under fault mode. On other platform we don't allow scratch
> > page
> > > > under
> > > > > fault mode, so no need of such clearing.
> > > > > 
> > > > > v2: Rework vm_bind pipeline to clear scratch page mapping.
> > > > > This
> > > > > is
> > > > > similar
> > > > > to a map operation, with the exception that PTEs are cleared
> > > > instead
> > > > > of
> > > > > pointing to valid physical pages. (Matt, Thomas)
> > > > > 
> > > > > TLB invalidation is needed after clear scratch page mapping
> > > > > as
> > > > > larger
> > > > > scratch page mapping could be backed by physical page and
> > cached
> > > > in
> > > > > TLB. (Matt, Thomas)
> > > > > 
> > > > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > > > ---
> > > > >  drivers/gpu/drm/xe/xe_pt.c       | 66
> > ++++++++++++++++++++++--
> > > > ------
> > > > > --
> > > > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > > > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > > > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > > > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > > > b/drivers/gpu/drm/xe/xe_pt.c
> > > > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > > > >  	 * granularity.
> > > > >  	 */
> > > > >  	bool needs_64K;
> > > > > +	/* @clear_pt: clear page table entries during the
> > > > > bind
> > > > > walk
> > > > > */
> > > > > +	bool clear_pt;
> > > > >  	/**
> > > > >  	 * @vma: VMA being mapped
> > > > >  	 */
> > > > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct
> > xe_ptw
> > > > *parent,
> > > > > pgoff_t offset,
> > > > > 
> > > > >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> > > > > 
> > > > > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0
> > > > > :
> > > > > -
> > > > > xe_res_dma(curs) +
> > > > > xe_walk->dma_offset,
> > > > > -						 xe_walk-
> > > > > >vma,
> > > > > pat_index, level);
> > > > > -		pte |= xe_walk->default_pte;
> > > > > +		if (xe_walk->clear_pt) {
> > > > > +			pte = 0;
> > > > > +		} else {
> > > > > +			pte = vm->pt_ops-
> > > > > >pte_encode_vma(is_null
> > > > > ?
> > > > 0
> > > > > :
> > > > > +					xe_res_dma(curs) +
> > > > > xe_walk-
> > > > > > dma_offset,
> > > > > +					xe_walk->vma,
> > > > > pat_index,
> > > > > level);
> > > > > +			pte |= xe_walk->default_pte;
> > > > > 
> > > > > -		/*
> > > > > -		 * Set the XE_PTE_PS64 hint if possible,
> > > > > otherwise
> > > > > if
> > > > > -		 * this device *requires* 64K PTE size for
> > > > > VRAM,
> > > > > fail.
> > > > > -		 */
> > > > > -		if (level == 0 && !xe_parent->is_compact) {
> > > > > -			if (xe_pt_is_pte_ps64K(addr, next,
> > > > > xe_walk))
> > > > > {
> > > > > -				xe_walk->vma->gpuva.flags |=
> > > > > XE_VMA_PTE_64K;
> > > > > -				pte |= XE_PTE_PS64;
> > > > > -			} else if (XE_WARN_ON(xe_walk-
> > > > > needs_64K)) {
> > > > > -				return -EINVAL;
> > > > > +			/*
> > > > > +			 * Set the XE_PTE_PS64 hint if
> > > > > possible,
> > > > > otherwise if
> > > > > +			 * this device *requires* 64K PTE
> > > > > size
> > > > > for
> > > > > VRAM, fail.
> > > > > +			 */
> > > > > +			if (level == 0 && !xe_parent-
> > > > > > is_compact) {
> > > > > +				if (xe_pt_is_pte_ps64K(addr,
> > > > > next,
> > > > > xe_walk)) {
> > > > > +					xe_walk->vma-
> > > > > > gpuva.flags
> > > > > =
> > > > > XE_VMA_PTE_64K;
> > > > > +					pte |= XE_PTE_PS64;
> > > > > +				} else if
> > > > > (XE_WARN_ON(xe_walk-
> > > > > > needs_64K)) {
> > > > > +					return -EINVAL;
> > > > > +				}
> > > > >  			}
> > > > >  		}
> > > > > 
> > > > 
> > > > I think there might be some things missing here.
> > > > The bind should look exactly as a is_null bind, with the
> > > > exception
> > > > that
> > > > the pte should be 0, so might be able to benefit from is_null,
> > > > for
> > > > example xe_pg_hugepte_possible() short-circuit the dma-
> > address
> > > > check.
> > > 
> > > You are right. Above code has a problem for huge pte cases: it
> > > wouldn't detect
> > > A huge pte for clear_pt case. I will fix it.
> > > 
> > > I will not make is_null be true for clear_pt case, but will make
> > > similar logic to
> > > Let clear_pt benefit from is_null logic.
> > > 
> > > > With clear_pt we don't want to use the XE_PTE_PS64K, though,
> > like
> > > > you
> > > > identified above.
> > > > 
> > > > Will get back with a look at the invalidation.
> > > 
> > > I don't quite get here. Please do let me know if you find
> > > anything.
> > 
> > Yes, I meant I hadn't had time to look at this part yet.
> > 
> > For binding with scratch, it looks like *all* binds need TLB
> > invalidation, not just clearing binds, So in the below code-path,
> > can't
> > we just replace the xe_vm_in_preempt_fence_mode() with
> > xe_vm_in_lr_mode()?
> 
> I think you are correct. 
> 
> For long run vm, regardless it is in fault mode or not, we need to
> invalidate
> TLB scratch page caching during bind. Original codes only did TLB inv
> on non-
> Fault mode. On fault mode + LR,  this runs into two sub-cases:
> 1.  immediate bind
> 2. bind triggered by page fault: note this can actually runs into the
> !rebind case
> When the fault address is first time accessed by gpu. But the TLB
> cache for
> Scratch page mapping should have been invalidated by the
> "invalidate_on_bind"
> Condition.
> 
> So this leaves case #1 problematic.
> 
> It would be better for others to comment also, as in the comment
> below, it explicitly
> Called out "non-faulting LR". My understanding is, people didn't
> think of case #1 above.
> 
> Since it is an existing issue, I will make it a separate patch for
> better tracking purpose.

Hmm. What is the existing issue, I don't quite follow?

/Thomas

> 
> Oak 
> 
> > 
> > 
> > 		/*
> > 		 * If rebind, we have to invalidate TLB on !LR vms
> > to
> > invalidate
> > 		 * cached PTEs point to freed memory. On LR vms
> > this
> > is done
> > 		 * automatically when the context is re-enabled by
> > the
> > rebind worker,
> > 		 * or in fault mode it was invalidated on PTE
> > zapping.
> > 		 *
> > 		 * If !rebind, and scratch enabled VMs, there is a
> > chance the scratch
> > 		 * PTE is already cached in the TLB so it needs to
> > be
> > invalidated.
> > 		 * On !LR VMs this is done in the ring ops
> > preceding a
> > batch, but on
> > 		 * non-faulting LR, in particular on user-space
> > batch
> > buffer chaining,
> > 		 * it needs to be done here.
> > 		 */
> > 		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > 		     xe_vm_in_preempt_fence_mode(vm)))
> > 
> > /Thomas
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > > 
> > > Oak
> > > > 
> > > > 
> > > > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > > > *parent,
> > > > > pgoff_t offset,
> > > > >  		if (unlikely(ret))
> > > > >  			return ret;
> > > > > 
> > > > > -		if (!is_null)
> > > > > +		if (!is_null && !xe_walk->clear_pt)
> > > > >  			xe_res_next(curs, next - addr);
> > > > >  		xe_walk->va_curs_start = next;
> > > > >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K
> > > > > <<
> > > > > level);
> > > > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > > > > xe_pt_stage_bind_ops = {
> > > > >   * @vma: The vma indicating the address range.
> > > > >   * @entries: Storage for the update entries used for
> > > > > connecting
> > > > > the
> > > > > tree to
> > > > >   * the main tree at commit time.
> > > > > + * @clear_pt: Clear the page table entries.
> > > > >   * @num_entries: On output contains the number of @entries
> > > > used.
> > > > >   *
> > > > >   * This function builds a disconnected page-table tree for a
> > > > > given
> > > > > address
> > > > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > > > > xe_pt_stage_bind_ops = {
> > > > >   */
> > > > >  static int
> > > > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > > > -		 struct xe_vm_pgtable_update *entries, u32
> > > > > *num_entries)
> > > > > +		 struct xe_vm_pgtable_update *entries,
> > > > > +		 bool clear_pt, u32 *num_entries)
> > > > >  {
> > > > >  	struct xe_device *xe = tile_to_xe(tile);
> > > > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> > > > struct
> > > > > xe_vma *vma,
> > > > >  		.vma = vma,
> > > > >  		.wupd.entries = entries,
> > > > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > > > > XE_VM_FLAG_64K) && is_devmem,
> > > > > +		.clear_pt = clear_pt,
> > > > >  	};
> > > > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile-
> > > > > >id];
> > > > >  	int ret;
> > > > > 
> > > > > +	if (clear_pt) {
> > > > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > > > > xe_vma_start(vma),
> > > > > +				       xe_vma_end(vma),
> > > > > &xe_walk.base);
> > > > > +
> > > > > +		*num_entries =
> > > > > xe_walk.wupd.num_used_entries;
> > > > > +		return ret;
> > > > > +	}
> > > > > +
> > > > >  	/**
> > > > >  	 * Default atomic expectations for different
> > > > > allocation
> > > > > scenarios are as follows:
> > > > >  	 *
> > > > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > > > > xe_vm_pgtable_update *entries,
> > > > > 
> > > > >  static int
> > > > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > > > -		   struct xe_vm_pgtable_update *entries, u32
> > > > > *num_entries)
> > > > > +		   struct xe_vm_pgtable_update *entries,
> > > > > +		   bool invalidate_on_bind, u32
> > > > > *num_entries)
> > > > >  {
> > > > >  	int err;
> > > > > 
> > > > >  	*num_entries = 0;
> > > > > -	err = xe_pt_stage_bind(tile, vma, entries,
> > > > > num_entries);
> > > > > +	err = xe_pt_stage_bind(tile, vma, entries,
> > > > > invalidate_on_bind,
> > > > > +			       num_entries);
> > > > >  	if (!err)
> > > > >  		xe_tile_assert(tile, *num_entries);
> > > > > 
> > > > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct
> > xe_vm
> > > > *vm,
> > > > > struct xe_tile *tile,
> > > > >  		return err;
> > > > > 
> > > > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > > > +				 pt_update_ops-
> > > > > > invalidate_on_bind,
> > > > >  				 &pt_op->num_entries);
> > > > >  	if (!err) {
> > > > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct
> > xe_vm
> > > > *vm,
> > > > > struct xe_tile *tile,
> > > > >  		 * it needs to be done here.
> > > > >  		 */
> > > > >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm)
> > > > > &&
> > > > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > > > > pt_update_ops->invalidate_on_bind)
> > > > >  			pt_update_ops->needs_invalidation =
> > > > > true;
> > > > >  		else if (pt_op->rebind &&
> > > > > !xe_vm_in_lr_mode(vm))
> > > > >  			/* We bump also if
> > > > > batch_invalidate_tlb
> > > > > is
> > > > > true */
> > > > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm
> > *vm,
> > > > > 
> > > > >  	switch (op->base.op) {
> > > > >  	case DRM_GPUVA_OP_MAP:
> > > > > -		if (!op->map.immediate &&
> > > > xe_vm_in_fault_mode(vm))
> > > > > +		if (!op->map.immediate &&
> > > > xe_vm_in_fault_mode(vm) &&
> > > > > +		    !op->map.invalidate_on_bind)
> > > > >  			break;
> > > > > 
> > > > > +		if (op->map.invalidate_on_bind)
> > > > > +			pt_update_ops->invalidate_on_bind =
> > > > > true;
> > > > > +
> > > > >  		err = bind_op_prepare(vm, tile,
> > > > > pt_update_ops,
> > > > > op-
> > > > > > map.vma);
> > > > >  		pt_update_ops->wait_vm_kernel = true;
> > > > >  		break;
> > > > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct
> > xe_vm
> > > > *vm,
> > > > > struct xe_tile *tile,
> > > > >  	}
> > > > >  	vma->tile_present |= BIT(tile->id);
> > > > >  	vma->tile_staged &= ~BIT(tile->id);
> > > > > +	if (pt_update_ops->invalidate_on_bind)
> > > > > +		vma->tile_invalidated |= BIT(tile->id);
> > > > >  	if (xe_vma_is_userptr(vma)) {
> > > > >  		lockdep_assert_held_read(&vm-
> > > > > > userptr.notifier_lock);
> > > > >  		to_userptr_vma(vma)->userptr.initial_bind =
> > > > > true;
> > > > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > index 384cc04de719..3d0aa2a5102e 100644
> > > > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > > > >  	bool needs_userptr_lock;
> > > > >  	/** @needs_invalidation: Needs invalidation */
> > > > >  	bool needs_invalidation;
> > > > > +	/** @invalidate_on_bind: Invalidate the range before
> > > > > bind */
> > > > > +	bool invalidate_on_bind;
> > > > >  	/**
> > > > >  	 * @wait_vm_bookkeep: PT operations need to wait
> > > > > until
> > > > > VM
> > > > is
> > > > > idle
> > > > >  	 * (bookkeep dma-resv slots are idle) and stage all
> > > > > future
> > > > > VM activity
> > > > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > > b/drivers/gpu/drm/xe/xe_vm.c
> > > > > index d664f2e418b2..813d893d9b63 100644
> > > > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > > > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device
> > *xe,
> > > > > struct drm_gpuva_op *op)
> > > > >  }
> > > > >  #endif
> > > > > 
> > > > > +static bool __xe_vm_needs_clear_scratch_pages(struct
> > xe_vm
> > > > *vm, u32
> > > > > bind_flags)
> > > > > +{
> > > > > +	if (!xe_vm_in_fault_mode(vm))
> > > > > +		return false;
> > > > > +
> > > > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > > > +		return false;
> > > > > +
> > > > > +	if (!xe_vm_has_scratch(vm))
> > > > > +		return false;
> > > > > +
> > > > > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > > > +		return false;
> > > > > +
> > > > > +	return true;
> > > > > +}
> > > > > +
> > > > >  /*
> > > > >   * Create operations list from IOCTL arguments, setup
> > operations
> > > > > fields so parse
> > > > >   * and commit steps are decoupled from IOCTL arguments. This
> > > > step
> > > > > can fail.
> > > > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct
> > xe_vm
> > > > *vm,
> > > > > struct xe_bo *bo,
> > > > >  			op->map.is_null = flags &
> > > > > DRM_XE_VM_BIND_FLAG_NULL;
> > > > >  			op->map.dumpable = flags &
> > > > > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > > > >  			op->map.pat_index = pat_index;
> > > > > +			op->map.invalidate_on_bind =
> > > > > +
> > > > 	__xe_vm_needs_clear_scratch_pages(vm
> > > > > , flags);
> > > > >  		} else if (__op->op ==
> > > > > DRM_GPUVA_OP_PREFETCH) {
> > > > >  			op->prefetch.region =
> > > > > prefetch_region;
> > > > >  		}
> > > > > @@ -2188,7 +2207,8 @@ static int
> > vm_bind_ioctl_ops_parse(struct
> > > > xe_vm
> > > > > *vm, struct drm_gpuva_ops *ops,
> > > > >  				return PTR_ERR(vma);
> > > > > 
> > > > >  			op->map.vma = vma;
> > > > > -			if (op->map.immediate ||
> > > > > !xe_vm_in_fault_mode(vm))
> > > > > +			if (op->map.immediate ||
> > > > > !xe_vm_in_fault_mode(vm) ||
> > > > > +			    op->map.invalidate_on_bind)
> > > > > 
> > > > 	xe_vma_ops_incr_pt_update_ops(vops,
> > > > > 
> > > > > op-
> > > > > > tile_mask);
> > > > >  			break;
> > > > > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> > > > drm_exec
> > > > > *exec, struct xe_vm *vm,
> > > > > 
> > > > >  	switch (op->base.op) {
> > > > >  	case DRM_GPUVA_OP_MAP:
> > > > > -		err = vma_lock_and_validate(exec, op-
> > > > > >map.vma,
> > > > > -
> > > > > !xe_vm_in_fault_mode(vm)
> > > > > > > 
> > > > > -					    op-
> > > > > >map.immediate);
> > > > > +		if (!op->map.invalidate_on_bind)
> > > > > +			err = vma_lock_and_validate(exec,
> > > > > op-
> > > > > > map.vma,
> > > > > +
> > > > > !xe_vm_in_fault_mode(vm) ||
> > > > > +						    op-
> > > > > > map.immediate);
> > > > >  		break;
> > > > >  	case DRM_GPUVA_OP_REMAP:
> > > > >  		err = check_ufence(gpuva_to_vma(op-
> > > > > > base.remap.unmap->va));
> > > > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > index 52467b9b5348..dace04f4ea5e 100644
> > > > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > > > >  	bool is_null;
> > > > >  	/** @dumpable: whether BO is dumped on GPU hang */
> > > > >  	bool dumpable;
> > > > > +	/** @invalidate: invalidate the VMA before bind */
> > > > > +	bool invalidate_on_bind;
> > > > >  	/** @pat_index: The pat index to use for this
> > > > > operation.
> > > > > */
> > > > >  	u16 pat_index;
> > > > >  };
> > > 
> 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 20:11           ` Thomas Hellström
@ 2025-02-06 21:05             ` Zeng, Oak
  0 siblings, 0 replies; 38+ messages in thread
From: Zeng, Oak @ 2025-02-06 21:05 UTC (permalink / raw)
  To: Thomas Hellström, intel-xe@lists.freedesktop.org
  Cc: Brost, Matthew, Cavitt, Jonathan



> -----Original Message-----
> From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Sent: February 6, 2025 3:12 PM
> To: Zeng, Oak <oak.zeng@intel.com>; intel-xe@lists.freedesktop.org
> Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> 
> Hi, Oak,
> 
> On Thu, 2025-02-06 at 18:56 +0000, Zeng, Oak wrote:
> > Hi Thomas!
> >
> > > -----Original Message-----
> > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > Sent: February 6, 2025 7:52 AM
> > > To: Zeng, Oak <oak.zeng@intel.com>; intel-
> xe@lists.freedesktop.org
> > > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt, Jonathan
> > > <jonathan.cavitt@intel.com>
> > > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before
> vm_bind
> > >
> > > On Thu, 2025-02-06 at 01:52 +0000, Zeng, Oak wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > > > Sent: February 5, 2025 9:36 AM
> > > > > To: Zeng, Oak <oak.zeng@intel.com>; intel-
> > > xe@lists.freedesktop.org
> > > > > Cc: Brost, Matthew <matthew.brost@intel.com>; Cavitt,
> Jonathan
> > > > > <jonathan.cavitt@intel.com>
> > > > > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before
> > > vm_bind
> > > > >
> > > > > On Tue, 2025-02-04 at 13:45 -0500, Oak Zeng wrote:
> > > > > > When a vm runs under fault mode, if scratch page is enabled,
> > > we
> > > > > need
> > > > > > to clear the scratch page mapping before vm_bind for the
> > > vm_bind
> > > > > > address range. Under fault mode, we depend on
> recoverable
> > > page
> > > > > fault
> > > > > > to establish mapping in page table. If scratch page is not
> > > > > > cleared,
> > > > > > GPU access of address won't cause page fault because it
> > > > > > always
> > > > > > hits
> > > > > > the existing scratch page mapping.
> > > > >
> > > > > I think we need to ephasize that we clear AT vm_bind time,
> not
> > > > > before,
> > > > > both in the commit description and the patch title.
> > > >
> > > > Will fix.
> > > >
> > > > >
> > > > > >
> > > > > > When vm_bind with IMMEDIATE flag, there is no need of
> > > clearing as
> > > > > > immediate bind can overwrite the scratch page mapping.
> > > > > >
> > > > > > So far only is xe2 and xe3 products are allowed to enable
> > > > > > scratch
> > > > > > page
> > > > > > under fault mode. On other platform we don't allow scratch
> > > page
> > > > > under
> > > > > > fault mode, so no need of such clearing.
> > > > > >
> > > > > > v2: Rework vm_bind pipeline to clear scratch page mapping.
> > > > > > This
> > > > > > is
> > > > > > similar
> > > > > > to a map operation, with the exception that PTEs are cleared
> > > > > instead
> > > > > > of
> > > > > > pointing to valid physical pages. (Matt, Thomas)
> > > > > >
> > > > > > TLB invalidation is needed after clear scratch page mapping
> > > > > > as
> > > > > > larger
> > > > > > scratch page mapping could be backed by physical page and
> > > cached
> > > > > in
> > > > > > TLB. (Matt, Thomas)
> > > > > >
> > > > > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > > > > ---
> > > > > >  drivers/gpu/drm/xe/xe_pt.c       | 66
> > > ++++++++++++++++++++++--
> > > > > ------
> > > > > > --
> > > > > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > > > > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > > > > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > > > > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > > > > b/drivers/gpu/drm/xe/xe_pt.c
> > > > > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > > > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > > > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > > > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > > > > >  	 * granularity.
> > > > > >  	 */
> > > > > >  	bool needs_64K;
> > > > > > +	/* @clear_pt: clear page table entries during the
> > > > > > bind
> > > > > > walk
> > > > > > */
> > > > > > +	bool clear_pt;
> > > > > >  	/**
> > > > > >  	 * @vma: VMA being mapped
> > > > > >  	 */
> > > > > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct
> > > xe_ptw
> > > > > *parent,
> > > > > > pgoff_t offset,
> > > > > >
> > > > > >  		XE_WARN_ON(xe_walk->va_curs_start !=
> addr);
> > > > > >
> > > > > > -		pte = vm->pt_ops->pte_encode_vma(is_null ?
> 0
> > > > > > :
> > > > > > -
> > > > > > xe_res_dma(curs) +
> > > > > > xe_walk->dma_offset,
> > > > > > -						 xe_walk-
> > > > > > >vma,
> > > > > > pat_index, level);
> > > > > > -		pte |= xe_walk->default_pte;
> > > > > > +		if (xe_walk->clear_pt) {
> > > > > > +			pte = 0;
> > > > > > +		} else {
> > > > > > +			pte = vm->pt_ops-
> > > > > > >pte_encode_vma(is_null
> > > > > > ?
> > > > > 0
> > > > > > :
> > > > > > +					xe_res_dma(curs) +
> > > > > > xe_walk-
> > > > > > > dma_offset,
> > > > > > +					xe_walk->vma,
> > > > > > pat_index,
> > > > > > level);
> > > > > > +			pte |= xe_walk->default_pte;
> > > > > >
> > > > > > -		/*
> > > > > > -		 * Set the XE_PTE_PS64 hint if possible,
> > > > > > otherwise
> > > > > > if
> > > > > > -		 * this device *requires* 64K PTE size for
> > > > > > VRAM,
> > > > > > fail.
> > > > > > -		 */
> > > > > > -		if (level == 0 && !xe_parent->is_compact) {
> > > > > > -			if (xe_pt_is_pte_ps64K(addr, next,
> > > > > > xe_walk))
> > > > > > {
> > > > > > -				xe_walk->vma->gpuva.flags
> |=
> > > > > > XE_VMA_PTE_64K;
> > > > > > -				pte |= XE_PTE_PS64;
> > > > > > -			} else if (XE_WARN_ON(xe_walk-
> > > > > > needs_64K)) {
> > > > > > -				return -EINVAL;
> > > > > > +			/*
> > > > > > +			 * Set the XE_PTE_PS64 hint if
> > > > > > possible,
> > > > > > otherwise if
> > > > > > +			 * this device *requires* 64K PTE
> > > > > > size
> > > > > > for
> > > > > > VRAM, fail.
> > > > > > +			 */
> > > > > > +			if (level == 0 && !xe_parent-
> > > > > > > is_compact) {
> > > > > > +				if (xe_pt_is_pte_ps64K(addr,
> > > > > > next,
> > > > > > xe_walk)) {
> > > > > > +					xe_walk->vma-
> > > > > > > gpuva.flags
> > > > > > =
> > > > > > XE_VMA_PTE_64K;
> > > > > > +					pte |= XE_PTE_PS64;
> > > > > > +				} else if
> > > > > > (XE_WARN_ON(xe_walk-
> > > > > > > needs_64K)) {
> > > > > > +					return -EINVAL;
> > > > > > +				}
> > > > > >  			}
> > > > > >  		}
> > > > > >
> > > > >
> > > > > I think there might be some things missing here.
> > > > > The bind should look exactly as a is_null bind, with the
> > > > > exception
> > > > > that
> > > > > the pte should be 0, so might be able to benefit from is_null,
> > > > > for
> > > > > example xe_pg_hugepte_possible() short-circuit the dma-
> > > address
> > > > > check.
> > > >
> > > > You are right. Above code has a problem for huge pte cases: it
> > > > wouldn't detect
> > > > A huge pte for clear_pt case. I will fix it.
> > > >
> > > > I will not make is_null be true for clear_pt case, but will make
> > > > similar logic to
> > > > Let clear_pt benefit from is_null logic.
> > > >
> > > > > With clear_pt we don't want to use the XE_PTE_PS64K, though,
> > > like
> > > > > you
> > > > > identified above.
> > > > >
> > > > > Will get back with a look at the invalidation.
> > > >
> > > > I don't quite get here. Please do let me know if you find
> > > > anything.
> > >
> > > Yes, I meant I hadn't had time to look at this part yet.
> > >
> > > For binding with scratch, it looks like *all* binds need TLB
> > > invalidation, not just clearing binds, So in the below code-path,
> > > can't
> > > we just replace the xe_vm_in_preempt_fence_mode() with
> > > xe_vm_in_lr_mode()?
> >
> > I think you are correct.
> >
> > For long run vm, regardless it is in fault mode or not, we need to
> > invalidate
> > TLB scratch page caching during bind. Original codes only did TLB inv
> > on non-
> > Fault mode. On fault mode + LR,  this runs into two sub-cases:
> > 1.  immediate bind
> > 2. bind triggered by page fault: note this can actually runs into the
> > !rebind case
> > When the fault address is first time accessed by gpu. But the TLB
> > cache for
> > Scratch page mapping should have been invalidated by the
> > "invalidate_on_bind"
> > Condition.
> >
> > So this leaves case #1 problematic.
> >
> > It would be better for others to comment also, as in the comment
> > below, it explicitly
> > Called out "non-faulting LR". My understanding is, people didn't
> > think of case #1 above.
> >
> > Since it is an existing issue, I will make it a separate patch for
> > better tracking purpose.
> 
> Hmm. What is the existing issue, I don't quite follow?

I just realized it is not an existing issue, as previously we didn't allow
Scratch on fault mode. 

I will still fix in the current patch.

Oak

> 
> /Thomas
> 
> >
> > Oak
> >
> > >
> > >
> > > 		/*
> > > 		 * If rebind, we have to invalidate TLB on !LR vms
> > > to
> > > invalidate
> > > 		 * cached PTEs point to freed memory. On LR vms
> > > this
> > > is done
> > > 		 * automatically when the context is re-enabled by
> > > the
> > > rebind worker,
> > > 		 * or in fault mode it was invalidated on PTE
> > > zapping.
> > > 		 *
> > > 		 * If !rebind, and scratch enabled VMs, there is a
> > > chance the scratch
> > > 		 * PTE is already cached in the TLB so it needs to
> > > be
> > > invalidated.
> > > 		 * On !LR VMs this is done in the ring ops
> > > preceding a
> > > batch, but on
> > > 		 * non-faulting LR, in particular on user-space
> > > batch
> > > buffer chaining,
> > > 		 * it needs to be done here.
> > > 		 */
> > > 		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > > 		     xe_vm_in_preempt_fence_mode(vm)))
> > >
> > > /Thomas
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > > > Oak
> > > > >
> > > > >
> > > > > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct
> xe_ptw
> > > > > *parent,
> > > > > > pgoff_t offset,
> > > > > >  		if (unlikely(ret))
> > > > > >  			return ret;
> > > > > >
> > > > > > -		if (!is_null)
> > > > > > +		if (!is_null && !xe_walk->clear_pt)
> > > > > >  			xe_res_next(curs, next - addr);
> > > > > >  		xe_walk->va_curs_start = next;
> > > > > >  		xe_walk->vma->gpuva.flags |=
> (XE_VMA_PTE_4K
> > > > > > <<
> > > > > > level);
> > > > > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > > > > > xe_pt_stage_bind_ops = {
> > > > > >   * @vma: The vma indicating the address range.
> > > > > >   * @entries: Storage for the update entries used for
> > > > > > connecting
> > > > > > the
> > > > > > tree to
> > > > > >   * the main tree at commit time.
> > > > > > + * @clear_pt: Clear the page table entries.
> > > > > >   * @num_entries: On output contains the number of
> @entries
> > > > > used.
> > > > > >   *
> > > > > >   * This function builds a disconnected page-table tree for a
> > > > > > given
> > > > > > address
> > > > > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > > > > > xe_pt_stage_bind_ops = {
> > > > > >   */
> > > > > >  static int
> > > > > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > > > > -		 struct xe_vm_pgtable_update *entries, u32
> > > > > > *num_entries)
> > > > > > +		 struct xe_vm_pgtable_update *entries,
> > > > > > +		 bool clear_pt, u32 *num_entries)
> > > > > >  {
> > > > > >  	struct xe_device *xe = tile_to_xe(tile);
> > > > > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > > > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile
> *tile,
> > > > > struct
> > > > > > xe_vma *vma,
> > > > > >  		.vma = vma,
> > > > > >  		.wupd.entries = entries,
> > > > > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > > > > > XE_VM_FLAG_64K) && is_devmem,
> > > > > > +		.clear_pt = clear_pt,
> > > > > >  	};
> > > > > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile-
> > > > > > >id];
> > > > > >  	int ret;
> > > > > >
> > > > > > +	if (clear_pt) {
> > > > > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > > > > > xe_vma_start(vma),
> > > > > > +				       xe_vma_end(vma),
> > > > > > &xe_walk.base);
> > > > > > +
> > > > > > +		*num_entries =
> > > > > > xe_walk.wupd.num_used_entries;
> > > > > > +		return ret;
> > > > > > +	}
> > > > > > +
> > > > > >  	/**
> > > > > >  	 * Default atomic expectations for different
> > > > > > allocation
> > > > > > scenarios are as follows:
> > > > > >  	 *
> > > > > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > > > > > xe_vm_pgtable_update *entries,
> > > > > >
> > > > > >  static int
> > > > > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma
> *vma,
> > > > > > -		   struct xe_vm_pgtable_update *entries, u32
> > > > > > *num_entries)
> > > > > > +		   struct xe_vm_pgtable_update *entries,
> > > > > > +		   bool invalidate_on_bind, u32
> > > > > > *num_entries)
> > > > > >  {
> > > > > >  	int err;
> > > > > >
> > > > > >  	*num_entries = 0;
> > > > > > -	err = xe_pt_stage_bind(tile, vma, entries,
> > > > > > num_entries);
> > > > > > +	err = xe_pt_stage_bind(tile, vma, entries,
> > > > > > invalidate_on_bind,
> > > > > > +			       num_entries);
> > > > > >  	if (!err)
> > > > > >  		xe_tile_assert(tile, *num_entries);
> > > > > >
> > > > > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct
> > > xe_vm
> > > > > *vm,
> > > > > > struct xe_tile *tile,
> > > > > >  		return err;
> > > > > >
> > > > > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > > > > +				 pt_update_ops-
> > > > > > > invalidate_on_bind,
> > > > > >  				 &pt_op->num_entries);
> > > > > >  	if (!err) {
> > > > > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > > > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct
> > > xe_vm
> > > > > *vm,
> > > > > > struct xe_tile *tile,
> > > > > >  		 * it needs to be done here.
> > > > > >  		 */
> > > > > >  		if ((!pt_op->rebind &&
> xe_vm_has_scratch(vm)
> > > > > > &&
> > > > > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > > > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > > > > > pt_update_ops->invalidate_on_bind)
> > > > > >  			pt_update_ops->needs_invalidation
> =
> > > > > > true;
> > > > > >  		else if (pt_op->rebind &&
> > > > > > !xe_vm_in_lr_mode(vm))
> > > > > >  			/* We bump also if
> > > > > > batch_invalidate_tlb
> > > > > > is
> > > > > > true */
> > > > > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct
> xe_vm
> > > *vm,
> > > > > >
> > > > > >  	switch (op->base.op) {
> > > > > >  	case DRM_GPUVA_OP_MAP:
> > > > > > -		if (!op->map.immediate &&
> > > > > xe_vm_in_fault_mode(vm))
> > > > > > +		if (!op->map.immediate &&
> > > > > xe_vm_in_fault_mode(vm) &&
> > > > > > +		    !op->map.invalidate_on_bind)
> > > > > >  			break;
> > > > > >
> > > > > > +		if (op->map.invalidate_on_bind)
> > > > > > +			pt_update_ops->invalidate_on_bind
> =
> > > > > > true;
> > > > > > +
> > > > > >  		err = bind_op_prepare(vm, tile,
> > > > > > pt_update_ops,
> > > > > > op-
> > > > > > > map.vma);
> > > > > >  		pt_update_ops->wait_vm_kernel = true;
> > > > > >  		break;
> > > > > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct
> > > xe_vm
> > > > > *vm,
> > > > > > struct xe_tile *tile,
> > > > > >  	}
> > > > > >  	vma->tile_present |= BIT(tile->id);
> > > > > >  	vma->tile_staged &= ~BIT(tile->id);
> > > > > > +	if (pt_update_ops->invalidate_on_bind)
> > > > > > +		vma->tile_invalidated |= BIT(tile->id);
> > > > > >  	if (xe_vma_is_userptr(vma)) {
> > > > > >  		lockdep_assert_held_read(&vm-
> > > > > > > userptr.notifier_lock);
> > > > > >  		to_userptr_vma(vma)->userptr.initial_bind =
> > > > > > true;
> > > > > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > > index 384cc04de719..3d0aa2a5102e 100644
> > > > > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > > > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > > > > >  	bool needs_userptr_lock;
> > > > > >  	/** @needs_invalidation: Needs invalidation */
> > > > > >  	bool needs_invalidation;
> > > > > > +	/** @invalidate_on_bind: Invalidate the range before
> > > > > > bind */
> > > > > > +	bool invalidate_on_bind;
> > > > > >  	/**
> > > > > >  	 * @wait_vm_bookkeep: PT operations need to wait
> > > > > > until
> > > > > > VM
> > > > > is
> > > > > > idle
> > > > > >  	 * (bookkeep dma-resv slots are idle) and stage all
> > > > > > future
> > > > > > VM activity
> > > > > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > > > b/drivers/gpu/drm/xe/xe_vm.c
> > > > > > index d664f2e418b2..813d893d9b63 100644
> > > > > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > > > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > > > > @@ -1921,6 +1921,23 @@ static void print_op(struct
> xe_device
> > > *xe,
> > > > > > struct drm_gpuva_op *op)
> > > > > >  }
> > > > > >  #endif
> > > > > >
> > > > > > +static bool __xe_vm_needs_clear_scratch_pages(struct
> > > xe_vm
> > > > > *vm, u32
> > > > > > bind_flags)
> > > > > > +{
> > > > > > +	if (!xe_vm_in_fault_mode(vm))
> > > > > > +		return false;
> > > > > > +
> > > > > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > > > > +		return false;
> > > > > > +
> > > > > > +	if (!xe_vm_has_scratch(vm))
> > > > > > +		return false;
> > > > > > +
> > > > > > +	if (bind_flags &
> DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > > > > +		return false;
> > > > > > +
> > > > > > +	return true;
> > > > > > +}
> > > > > > +
> > > > > >  /*
> > > > > >   * Create operations list from IOCTL arguments, setup
> > > operations
> > > > > > fields so parse
> > > > > >   * and commit steps are decoupled from IOCTL arguments.
> This
> > > > > step
> > > > > > can fail.
> > > > > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct
> > > xe_vm
> > > > > *vm,
> > > > > > struct xe_bo *bo,
> > > > > >  			op->map.is_null = flags &
> > > > > > DRM_XE_VM_BIND_FLAG_NULL;
> > > > > >  			op->map.dumpable = flags &
> > > > > > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > > > > >  			op->map.pat_index = pat_index;
> > > > > > +			op->map.invalidate_on_bind =
> > > > > > +
> > > > > 	__xe_vm_needs_clear_scratch_pages(vm
> > > > > > , flags);
> > > > > >  		} else if (__op->op ==
> > > > > > DRM_GPUVA_OP_PREFETCH) {
> > > > > >  			op->prefetch.region =
> > > > > > prefetch_region;
> > > > > >  		}
> > > > > > @@ -2188,7 +2207,8 @@ static int
> > > vm_bind_ioctl_ops_parse(struct
> > > > > xe_vm
> > > > > > *vm, struct drm_gpuva_ops *ops,
> > > > > >  				return PTR_ERR(vma);
> > > > > >
> > > > > >  			op->map.vma = vma;
> > > > > > -			if (op->map.immediate ||
> > > > > > !xe_vm_in_fault_mode(vm))
> > > > > > +			if (op->map.immediate ||
> > > > > > !xe_vm_in_fault_mode(vm) ||
> > > > > > +			    op->map.invalidate_on_bind)
> > > > > >
> > > > > 	xe_vma_ops_incr_pt_update_ops(vops,
> > > > > >
> > > > > > op-
> > > > > > > tile_mask);
> > > > > >  			break;
> > > > > > @@ -2416,9 +2436,10 @@ static int
> op_lock_and_prep(struct
> > > > > drm_exec
> > > > > > *exec, struct xe_vm *vm,
> > > > > >
> > > > > >  	switch (op->base.op) {
> > > > > >  	case DRM_GPUVA_OP_MAP:
> > > > > > -		err = vma_lock_and_validate(exec, op-
> > > > > > >map.vma,
> > > > > > -
> > > > > > !xe_vm_in_fault_mode(vm)
> > > > > > > >
> > > > > > -					    op-
> > > > > > >map.immediate);
> > > > > > +		if (!op->map.invalidate_on_bind)
> > > > > > +			err = vma_lock_and_validate(exec,
> > > > > > op-
> > > > > > > map.vma,
> > > > > > +
> > > > > > !xe_vm_in_fault_mode(vm) ||
> > > > > > +						    op-
> > > > > > > map.immediate);
> > > > > >  		break;
> > > > > >  	case DRM_GPUVA_OP_REMAP:
> > > > > >  		err = check_ufence(gpuva_to_vma(op-
> > > > > > > base.remap.unmap->va));
> > > > > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > > index 52467b9b5348..dace04f4ea5e 100644
> > > > > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > > > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > > > > >  	bool is_null;
> > > > > >  	/** @dumpable: whether BO is dumped on GPU
> hang */
> > > > > >  	bool dumpable;
> > > > > > +	/** @invalidate: invalidate the VMA before bind */
> > > > > > +	bool invalidate_on_bind;
> > > > > >  	/** @pat_index: The pat index to use for this
> > > > > > operation.
> > > > > > */
> > > > > >  	u16 pat_index;
> > > > > >  };
> > > >
> >


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-06 21:38 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
@ 2025-02-06 21:38 ` Oak Zeng
  0 siblings, 0 replies; 38+ messages in thread
From: Oak Zeng @ 2025-02-06 21:38 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

Normally scratch page is not allowed when a vm is operate under page
fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
is fault mode relies on recoverable page to work, while scratch page
can mute recoverable page fault.

On xe2 and xe3, out of bound prefetch can cause page fault and further
system hang because xekmd can't resolve such page fault. SYCL and OCL
language runtime requires out of bound prefetch to be silently dropped
without causing any functional problem, thus the existing behavior
doesn't meet language runtime requirement.

At the same time, HW prefetching can cause page fault interrupt. Due to
page fault interrupt overhead (i.e., need Guc and KMD involved to fix
the page fault), HW prefetching can be slowed by many orders of magnitude.

Fix those problems by allowing scratch page under fault mode for xe2 and
xe3. With scratch page in place, HW prefetching could always hit scratch
page instead of causing interrupt.

A side effect is, scratch page could hide application program error.
Application out of bound accesses are hided by scratch page mapping,
instead of get reported to user.

igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
BMG:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738684805 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
Subtest scratch-fault: SUCCESS (0.080s)

Without this series, the test result is:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738686046 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
(xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
(xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
(xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
(xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
Stack trace:
  #0 ../lib/igt_core.c:2266 __igt_fail_assert()
  #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
  #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
  #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
  #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
  #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
  #6 [_start+0x2e]
Subtest scratch-fault failed.

v2: Refine commit message (Thomas)

v3: Move the scratch page flag check to after scratch page wa (Thomas)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 813d893d9b63..c2dfd0ade403 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1766,7 +1766,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 		return -EINVAL;
 
 	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
-			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
+			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
+			 !(NEEDS_SCRATCH(xe))))
 		return -EINVAL;
 
 	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 10:43     ` Thomas Hellström
@ 2025-02-10 17:08       ` Matthew Brost
  0 siblings, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-10 17:08 UTC (permalink / raw)
  To: Thomas Hellström; +Cc: Oak Zeng, intel-xe, jonathan.cavitt

On Thu, Feb 06, 2025 at 11:43:10AM +0100, Thomas Hellström wrote:
> Hi, Matt,
> 
> On Thu, 2025-02-06 at 02:34 -0800, Matthew Brost wrote:
> > On Tue, Feb 04, 2025 at 01:45:57PM -0500, Oak Zeng wrote:
> > > When a vm runs under fault mode, if scratch page is enabled, we
> > > need
> > > to clear the scratch page mapping before vm_bind for the vm_bind
> > > address range. Under fault mode, we depend on recoverable page
> > > fault
> > > to establish mapping in page table. If scratch page is not cleared,
> > > GPU access of address won't cause page fault because it always hits
> > > the existing scratch page mapping.
> > > 
> > > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > > immediate bind can overwrite the scratch page mapping.
> > > 
> > > So far only is xe2 and xe3 products are allowed to enable scratch
> > > page
> > > under fault mode. On other platform we don't allow scratch page
> > > under
> > > fault mode, so no need of such clearing.
> > > 
> > > v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> > > similar
> > > to a map operation, with the exception that PTEs are cleared
> > > instead of
> > > pointing to valid physical pages. (Matt, Thomas)
> > > 
> > > TLB invalidation is needed after clear scratch page mapping as
> > > larger
> > > scratch page mapping could be backed by physical page and cached in
> > > TLB. (Matt, Thomas)
> > > 
> > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > 
> > Given the complexity of the VM bind path, I think we need an IGT
> > posted
> > with this series before merging as I suggested in v1. Without it, it
> > will be fairly difficult to ensure correctness by reviews only.
> 
> There is an igt posted that exercises xe_exec_fault_mode + scratch
> pages. Although you might have had something more elaborate in mind?
> 

Ah, missed this as generally look at cover letter to find associated
IGTs.

A quick look at IGT and thinking we need to do slightly better. Off the
top of my head something like:

- Run a batch which does a DW write to scratch address
- Bind scratch address /w IMMEDIATE cleared
- Run a batch which does a DW write to newly bound address, verfiy
  result

Matt

> /Thomas
> 
> 
> > 
> > Matt
> > 
> > > ---
> > >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++------
> > > ----
> > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > > b/drivers/gpu/drm/xe/xe_pt.c
> > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > >  	 * granularity.
> > >  	 */
> > >  	bool needs_64K;
> > > +	/* @clear_pt: clear page table entries during the bind
> > > walk */
> > > +	bool clear_pt;
> > >  	/**
> > >  	 * @vma: VMA being mapped
> > >  	 */
> > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > > pgoff_t offset,
> > >  
> > >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> > >  
> > > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > > -						 xe_res_dma(curs)
> > > + xe_walk->dma_offset,
> > > -						 xe_walk->vma,
> > > pat_index, level);
> > > -		pte |= xe_walk->default_pte;
> > > +		if (xe_walk->clear_pt) {
> > > +			pte = 0;
> > > +		} else {
> > > +			pte = vm->pt_ops->pte_encode_vma(is_null ?
> > > 0 :
> > > +					xe_res_dma(curs) +
> > > xe_walk->dma_offset,
> > > +					xe_walk->vma, pat_index,
> > > level);
> > > +			pte |= xe_walk->default_pte;
> > >  
> > > -		/*
> > > -		 * Set the XE_PTE_PS64 hint if possible, otherwise
> > > if
> > > -		 * this device *requires* 64K PTE size for VRAM,
> > > fail.
> > > -		 */
> > > -		if (level == 0 && !xe_parent->is_compact) {
> > > -			if (xe_pt_is_pte_ps64K(addr, next,
> > > xe_walk)) {
> > > -				xe_walk->vma->gpuva.flags |=
> > > XE_VMA_PTE_64K;
> > > -				pte |= XE_PTE_PS64;
> > > -			} else if (XE_WARN_ON(xe_walk->needs_64K))
> > > {
> > > -				return -EINVAL;
> > > +			/*
> > > +			 * Set the XE_PTE_PS64 hint if possible,
> > > otherwise if
> > > +			 * this device *requires* 64K PTE size for
> > > VRAM, fail.
> > > +			 */
> > > +			if (level == 0 && !xe_parent->is_compact)
> > > {
> > > +				if (xe_pt_is_pte_ps64K(addr, next,
> > > xe_walk)) {
> > > +					xe_walk->vma->gpuva.flags
> > > |= XE_VMA_PTE_64K;
> > > +					pte |= XE_PTE_PS64;
> > > +				} else if (XE_WARN_ON(xe_walk-
> > > >needs_64K)) {
> > > +					return -EINVAL;
> > > +				}
> > >  			}
> > >  		}
> > >  
> > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > > pgoff_t offset,
> > >  		if (unlikely(ret))
> > >  			return ret;
> > >  
> > > -		if (!is_null)
> > > +		if (!is_null && !xe_walk->clear_pt)
> > >  			xe_res_next(curs, next - addr);
> > >  		xe_walk->va_curs_start = next;
> > >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > > level);
> > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > > xe_pt_stage_bind_ops = {
> > >   * @vma: The vma indicating the address range.
> > >   * @entries: Storage for the update entries used for connecting
> > > the tree to
> > >   * the main tree at commit time.
> > > + * @clear_pt: Clear the page table entries.
> > >   * @num_entries: On output contains the number of @entries used.
> > >   *
> > >   * This function builds a disconnected page-table tree for a given
> > > address
> > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > > xe_pt_stage_bind_ops = {
> > >   */
> > >  static int
> > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		 struct xe_vm_pgtable_update *entries, u32
> > > *num_entries)
> > > +		 struct xe_vm_pgtable_update *entries,
> > > +		 bool clear_pt, u32 *num_entries)
> > >  {
> > >  	struct xe_device *xe = tile_to_xe(tile);
> > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile, struct
> > > xe_vma *vma,
> > >  		.vma = vma,
> > >  		.wupd.entries = entries,
> > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > > XE_VM_FLAG_64K) && is_devmem,
> > > +		.clear_pt = clear_pt,
> > >  	};
> > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> > >  	int ret;
> > >  
> > > +	if (clear_pt) {
> > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > > xe_vma_start(vma),
> > > +				       xe_vma_end(vma),
> > > &xe_walk.base);
> > > +
> > > +		*num_entries = xe_walk.wupd.num_used_entries;
> > > +		return ret;
> > > +	}
> > > +
> > >  	/**
> > >  	 * Default atomic expectations for different allocation
> > > scenarios are as follows:
> > >  	 *
> > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > > xe_vm_pgtable_update *entries,
> > >  
> > >  static int
> > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		   struct xe_vm_pgtable_update *entries, u32
> > > *num_entries)
> > > +		   struct xe_vm_pgtable_update *entries,
> > > +		   bool invalidate_on_bind, u32 *num_entries)
> > >  {
> > >  	int err;
> > >  
> > >  	*num_entries = 0;
> > > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > > +	err = xe_pt_stage_bind(tile, vma, entries,
> > > invalidate_on_bind,
> > > +			       num_entries);
> > >  	if (!err)
> > >  		xe_tile_assert(tile, *num_entries);
> > >  
> > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> > > struct xe_tile *tile,
> > >  		return err;
> > >  
> > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > +				 pt_update_ops-
> > > >invalidate_on_bind,
> > >  				 &pt_op->num_entries);
> > >  	if (!err) {
> > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm *vm,
> > > struct xe_tile *tile,
> > >  		 * it needs to be done here.
> > >  		 */
> > >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > > pt_update_ops->invalidate_on_bind)
> > >  			pt_update_ops->needs_invalidation = true;
> > >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> > >  			/* We bump also if batch_invalidate_tlb is
> > > true */
> > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> > >  
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		if (!op->map.immediate && xe_vm_in_fault_mode(vm))
> > > +		if (!op->map.immediate && xe_vm_in_fault_mode(vm)
> > > &&
> > > +		    !op->map.invalidate_on_bind)
> > >  			break;
> > >  
> > > +		if (op->map.invalidate_on_bind)
> > > +			pt_update_ops->invalidate_on_bind = true;
> > > +
> > >  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> > > >map.vma);
> > >  		pt_update_ops->wait_vm_kernel = true;
> > >  		break;
> > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm *vm,
> > > struct xe_tile *tile,
> > >  	}
> > >  	vma->tile_present |= BIT(tile->id);
> > >  	vma->tile_staged &= ~BIT(tile->id);
> > > +	if (pt_update_ops->invalidate_on_bind)
> > > +		vma->tile_invalidated |= BIT(tile->id);
> > >  	if (xe_vma_is_userptr(vma)) {
> > >  		lockdep_assert_held_read(&vm-
> > > >userptr.notifier_lock);
> > >  		to_userptr_vma(vma)->userptr.initial_bind = true;
> > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > index 384cc04de719..3d0aa2a5102e 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > >  	bool needs_userptr_lock;
> > >  	/** @needs_invalidation: Needs invalidation */
> > >  	bool needs_invalidation;
> > > +	/** @invalidate_on_bind: Invalidate the range before bind
> > > */
> > > +	bool invalidate_on_bind;
> > >  	/**
> > >  	 * @wait_vm_bookkeep: PT operations need to wait until VM
> > > is idle
> > >  	 * (bookkeep dma-resv slots are idle) and stage all future
> > > VM activity
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > b/drivers/gpu/drm/xe/xe_vm.c
> > > index d664f2e418b2..813d893d9b63 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> > > struct drm_gpuva_op *op)
> > >  }
> > >  #endif
> > >  
> > > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm,
> > > u32 bind_flags)
> > > +{
> > > +	if (!xe_vm_in_fault_mode(vm))
> > > +		return false;
> > > +
> > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > +		return false;
> > > +
> > > +	if (!xe_vm_has_scratch(vm))
> > > +		return false;
> > > +
> > > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > +		return false;
> > > +
> > > +	return true;
> > > +}
> > > +
> > >  /*
> > >   * Create operations list from IOCTL arguments, setup operations
> > > fields so parse
> > >   * and commit steps are decoupled from IOCTL arguments. This step
> > > can fail.
> > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm,
> > > struct xe_bo *bo,
> > >  			op->map.is_null = flags &
> > > DRM_XE_VM_BIND_FLAG_NULL;
> > >  			op->map.dumpable = flags &
> > > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > >  			op->map.pat_index = pat_index;
> > > +			op->map.invalidate_on_bind =
> > > +				__xe_vm_needs_clear_scratch_pages(
> > > vm, flags);
> > >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> > >  			op->prefetch.region = prefetch_region;
> > >  		}
> > > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> > > xe_vm *vm, struct drm_gpuva_ops *ops,
> > >  				return PTR_ERR(vma);
> > >  
> > >  			op->map.vma = vma;
> > > -			if (op->map.immediate ||
> > > !xe_vm_in_fault_mode(vm))
> > > +			if (op->map.immediate ||
> > > !xe_vm_in_fault_mode(vm) ||
> > > +			    op->map.invalidate_on_bind)
> > >  				xe_vma_ops_incr_pt_update_ops(vops
> > > ,
> > >  							      op-
> > > >tile_mask);
> > >  			break;
> > > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct drm_exec
> > > *exec, struct xe_vm *vm,
> > >  
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		err = vma_lock_and_validate(exec, op->map.vma,
> > > -					   
> > > !xe_vm_in_fault_mode(vm) ||
> > > -					    op->map.immediate);
> > > +		if (!op->map.invalidate_on_bind)
> > > +			err = vma_lock_and_validate(exec, op-
> > > >map.vma,
> > > +						   
> > > !xe_vm_in_fault_mode(vm) ||
> > > +						    op-
> > > >map.immediate);
> > >  		break;
> > >  	case DRM_GPUVA_OP_REMAP:
> > >  		err = check_ufence(gpuva_to_vma(op-
> > > >base.remap.unmap->va));
> > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > index 52467b9b5348..dace04f4ea5e 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > >  	bool is_null;
> > >  	/** @dumpable: whether BO is dumped on GPU hang */
> > >  	bool dumpable;
> > > +	/** @invalidate: invalidate the VMA before bind */
> > > +	bool invalidate_on_bind;
> > >  	/** @pat_index: The pat index to use for this operation.
> > > */
> > >  	u16 pat_index;
> > >  };
> > > -- 
> > > 2.26.3
> > > 
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
  2025-02-06 15:16     ` Zeng, Oak
@ 2025-02-10 17:09       ` Matthew Brost
  0 siblings, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-10 17:09 UTC (permalink / raw)
  To: Zeng, Oak
  Cc: intel-xe@lists.freedesktop.org, Thomas.Hellstrom@linux.intel.com,
	Cavitt, Jonathan

On Thu, Feb 06, 2025 at 08:16:36AM -0700, Zeng, Oak wrote:
> 
> 
> > -----Original Message-----
> > From: Brost, Matthew <matthew.brost@intel.com>
> > Sent: February 6, 2025 5:35 AM
> > To: Zeng, Oak <oak.zeng@intel.com>
> > Cc: intel-xe@lists.freedesktop.org;
> > Thomas.Hellstrom@linux.intel.com; Cavitt, Jonathan
> > <jonathan.cavitt@intel.com>
> > Subject: Re: [PATCH 2/3] drm/xe: Clear scratch page before vm_bind
> > 
> > On Tue, Feb 04, 2025 at 01:45:57PM -0500, Oak Zeng wrote:
> > > When a vm runs under fault mode, if scratch page is enabled, we
> > need
> > > to clear the scratch page mapping before vm_bind for the vm_bind
> > > address range. Under fault mode, we depend on recoverable page
> > fault
> > > to establish mapping in page table. If scratch page is not cleared,
> > > GPU access of address won't cause page fault because it always hits
> > > the existing scratch page mapping.
> > >
> > > When vm_bind with IMMEDIATE flag, there is no need of clearing as
> > > immediate bind can overwrite the scratch page mapping.
> > >
> > > So far only is xe2 and xe3 products are allowed to enable scratch
> > page
> > > under fault mode. On other platform we don't allow scratch page
> > under
> > > fault mode, so no need of such clearing.
> > >
> > > v2: Rework vm_bind pipeline to clear scratch page mapping. This is
> > similar
> > > to a map operation, with the exception that PTEs are cleared
> > instead of
> > > pointing to valid physical pages. (Matt, Thomas)
> > >
> > > TLB invalidation is needed after clear scratch page mapping as larger
> > > scratch page mapping could be backed by physical page and cached
> > in
> > > TLB. (Matt, Thomas)
> > >
> > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > 
> > Given the complexity of the VM bind path, I think we need an IGT
> > posted
> > with this series before merging as I suggested in v1. Without it, it
> > will be fairly difficult to ensure correctness by reviews only.
> 
> I did have an igt posted in patch 3 of this series.
> 
> The reason I posted on patch 3 is, without patch 3 this "feature"
> Is not enabled. 
> 

Just replied to Thomas, see my reply there - I did miss an IGT was
included but I think we need to do a bit better, more details in my
reply to Thomas.

Matt

> Oak
> 
> > 
> > Matt
> > 
> > > ---
> > >  drivers/gpu/drm/xe/xe_pt.c       | 66 ++++++++++++++++++++++--
> > --------
> > >  drivers/gpu/drm/xe/xe_pt_types.h |  2 +
> > >  drivers/gpu/drm/xe/xe_vm.c       | 29 ++++++++++++--
> > >  drivers/gpu/drm/xe/xe_vm_types.h |  2 +
> > >  4 files changed, 75 insertions(+), 24 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > b/drivers/gpu/drm/xe/xe_pt.c
> > > index 1ddcc7e79a93..3fd0ae2dbe7d 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt.c
> > > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > > @@ -268,6 +268,8 @@ struct xe_pt_stage_bind_walk {
> > >  	 * granularity.
> > >  	 */
> > >  	bool needs_64K;
> > > +	/* @clear_pt: clear page table entries during the bind walk */
> > > +	bool clear_pt;
> > >  	/**
> > >  	 * @vma: VMA being mapped
> > >  	 */
> > > @@ -497,21 +499,25 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > *parent, pgoff_t offset,
> > >
> > >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> > >
> > > -		pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
> > > -						 xe_res_dma(curs) +
> > xe_walk->dma_offset,
> > > -						 xe_walk->vma,
> > pat_index, level);
> > > -		pte |= xe_walk->default_pte;
> > > +		if (xe_walk->clear_pt) {
> > > +			pte = 0;
> > > +		} else {
> > > +			pte = vm->pt_ops->pte_encode_vma(is_null ?
> > 0 :
> > > +					xe_res_dma(curs) + xe_walk-
> > >dma_offset,
> > > +					xe_walk->vma, pat_index,
> > level);
> > > +			pte |= xe_walk->default_pte;
> > >
> > > -		/*
> > > -		 * Set the XE_PTE_PS64 hint if possible, otherwise if
> > > -		 * this device *requires* 64K PTE size for VRAM, fail.
> > > -		 */
> > > -		if (level == 0 && !xe_parent->is_compact) {
> > > -			if (xe_pt_is_pte_ps64K(addr, next, xe_walk))
> > {
> > > -				xe_walk->vma->gpuva.flags |=
> > XE_VMA_PTE_64K;
> > > -				pte |= XE_PTE_PS64;
> > > -			} else if (XE_WARN_ON(xe_walk-
> > >needs_64K)) {
> > > -				return -EINVAL;
> > > +			/*
> > > +			 * Set the XE_PTE_PS64 hint if possible,
> > otherwise if
> > > +			 * this device *requires* 64K PTE size for
> > VRAM, fail.
> > > +			 */
> > > +			if (level == 0 && !xe_parent->is_compact) {
> > > +				if (xe_pt_is_pte_ps64K(addr, next,
> > xe_walk)) {
> > > +					xe_walk->vma->gpuva.flags
> > |= XE_VMA_PTE_64K;
> > > +					pte |= XE_PTE_PS64;
> > > +				} else if (XE_WARN_ON(xe_walk-
> > >needs_64K)) {
> > > +					return -EINVAL;
> > > +				}
> > >  			}
> > >  		}
> > >
> > > @@ -519,7 +525,7 @@ xe_pt_stage_bind_entry(struct xe_ptw
> > *parent, pgoff_t offset,
> > >  		if (unlikely(ret))
> > >  			return ret;
> > >
> > > -		if (!is_null)
> > > +		if (!is_null && !xe_walk->clear_pt)
> > >  			xe_res_next(curs, next - addr);
> > >  		xe_walk->va_curs_start = next;
> > >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > level);
> > > @@ -589,6 +595,7 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> > >   * @vma: The vma indicating the address range.
> > >   * @entries: Storage for the update entries used for connecting the
> > tree to
> > >   * the main tree at commit time.
> > > + * @clear_pt: Clear the page table entries.
> > >   * @num_entries: On output contains the number of @entries
> > used.
> > >   *
> > >   * This function builds a disconnected page-table tree for a given
> > address
> > > @@ -602,7 +609,8 @@ static const struct xe_pt_walk_ops
> > xe_pt_stage_bind_ops = {
> > >   */
> > >  static int
> > >  xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		 struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > > +		 struct xe_vm_pgtable_update *entries,
> > > +		 bool clear_pt, u32 *num_entries)
> > >  {
> > >  	struct xe_device *xe = tile_to_xe(tile);
> > >  	struct xe_bo *bo = xe_vma_bo(vma);
> > > @@ -622,10 +630,19 @@ xe_pt_stage_bind(struct xe_tile *tile,
> > struct xe_vma *vma,
> > >  		.vma = vma,
> > >  		.wupd.entries = entries,
> > >  		.needs_64K = (xe_vma_vm(vma)->flags &
> > XE_VM_FLAG_64K) && is_devmem,
> > > +		.clear_pt = clear_pt,
> > >  	};
> > >  	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
> > >  	int ret;
> > >
> > > +	if (clear_pt) {
> > > +		ret = xe_pt_walk_range(&pt->base, pt->level,
> > xe_vma_start(vma),
> > > +				       xe_vma_end(vma),
> > &xe_walk.base);
> > > +
> > > +		*num_entries = xe_walk.wupd.num_used_entries;
> > > +		return ret;
> > > +	}
> > > +
> > >  	/**
> > >  	 * Default atomic expectations for different allocation
> > scenarios are as follows:
> > >  	 *
> > > @@ -981,12 +998,14 @@ static void xe_pt_free_bind(struct
> > xe_vm_pgtable_update *entries,
> > >
> > >  static int
> > >  xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
> > > -		   struct xe_vm_pgtable_update *entries, u32
> > *num_entries)
> > > +		   struct xe_vm_pgtable_update *entries,
> > > +		   bool invalidate_on_bind, u32 *num_entries)
> > >  {
> > >  	int err;
> > >
> > >  	*num_entries = 0;
> > > -	err = xe_pt_stage_bind(tile, vma, entries, num_entries);
> > > +	err = xe_pt_stage_bind(tile, vma, entries, invalidate_on_bind,
> > > +			       num_entries);
> > >  	if (!err)
> > >  		xe_tile_assert(tile, *num_entries);
> > >
> > > @@ -1661,6 +1680,7 @@ static int bind_op_prepare(struct xe_vm
> > *vm, struct xe_tile *tile,
> > >  		return err;
> > >
> > >  	err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
> > > +				 pt_update_ops->invalidate_on_bind,
> > >  				 &pt_op->num_entries);
> > >  	if (!err) {
> > >  		xe_tile_assert(tile, pt_op->num_entries <=
> > > @@ -1685,7 +1705,7 @@ static int bind_op_prepare(struct xe_vm
> > *vm, struct xe_tile *tile,
> > >  		 * it needs to be done here.
> > >  		 */
> > >  		if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
> > > -		     xe_vm_in_preempt_fence_mode(vm)))
> > > +		     xe_vm_in_preempt_fence_mode(vm)) ||
> > pt_update_ops->invalidate_on_bind)
> > >  			pt_update_ops->needs_invalidation = true;
> > >  		else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
> > >  			/* We bump also if batch_invalidate_tlb is
> > true */
> > > @@ -1759,9 +1779,13 @@ static int op_prepare(struct xe_vm *vm,
> > >
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		if (!op->map.immediate &&
> > xe_vm_in_fault_mode(vm))
> > > +		if (!op->map.immediate &&
> > xe_vm_in_fault_mode(vm) &&
> > > +		    !op->map.invalidate_on_bind)
> > >  			break;
> > >
> > > +		if (op->map.invalidate_on_bind)
> > > +			pt_update_ops->invalidate_on_bind = true;
> > > +
> > >  		err = bind_op_prepare(vm, tile, pt_update_ops, op-
> > >map.vma);
> > >  		pt_update_ops->wait_vm_kernel = true;
> > >  		break;
> > > @@ -1871,6 +1895,8 @@ static void bind_op_commit(struct xe_vm
> > *vm, struct xe_tile *tile,
> > >  	}
> > >  	vma->tile_present |= BIT(tile->id);
> > >  	vma->tile_staged &= ~BIT(tile->id);
> > > +	if (pt_update_ops->invalidate_on_bind)
> > > +		vma->tile_invalidated |= BIT(tile->id);
> > >  	if (xe_vma_is_userptr(vma)) {
> > >  		lockdep_assert_held_read(&vm-
> > >userptr.notifier_lock);
> > >  		to_userptr_vma(vma)->userptr.initial_bind = true;
> > > diff --git a/drivers/gpu/drm/xe/xe_pt_types.h
> > b/drivers/gpu/drm/xe/xe_pt_types.h
> > > index 384cc04de719..3d0aa2a5102e 100644
> > > --- a/drivers/gpu/drm/xe/xe_pt_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_pt_types.h
> > > @@ -108,6 +108,8 @@ struct xe_vm_pgtable_update_ops {
> > >  	bool needs_userptr_lock;
> > >  	/** @needs_invalidation: Needs invalidation */
> > >  	bool needs_invalidation;
> > > +	/** @invalidate_on_bind: Invalidate the range before bind */
> > > +	bool invalidate_on_bind;
> > >  	/**
> > >  	 * @wait_vm_bookkeep: PT operations need to wait until VM
> > is idle
> > >  	 * (bookkeep dma-resv slots are idle) and stage all future VM
> > activity
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > > index d664f2e418b2..813d893d9b63 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -1921,6 +1921,23 @@ static void print_op(struct xe_device *xe,
> > struct drm_gpuva_op *op)
> > >  }
> > >  #endif
> > >
> > > +static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm
> > *vm, u32 bind_flags)
> > > +{
> > > +	if (!xe_vm_in_fault_mode(vm))
> > > +		return false;
> > > +
> > > +	if (!NEEDS_SCRATCH(vm->xe))
> > > +		return false;
> > > +
> > > +	if (!xe_vm_has_scratch(vm))
> > > +		return false;
> > > +
> > > +	if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
> > > +		return false;
> > > +
> > > +	return true;
> > > +}
> > > +
> > >  /*
> > >   * Create operations list from IOCTL arguments, setup operations
> > fields so parse
> > >   * and commit steps are decoupled from IOCTL arguments. This
> > step can fail.
> > > @@ -1991,6 +2008,8 @@ vm_bind_ioctl_ops_create(struct xe_vm
> > *vm, struct xe_bo *bo,
> > >  			op->map.is_null = flags &
> > DRM_XE_VM_BIND_FLAG_NULL;
> > >  			op->map.dumpable = flags &
> > DRM_XE_VM_BIND_FLAG_DUMPABLE;
> > >  			op->map.pat_index = pat_index;
> > > +			op->map.invalidate_on_bind =
> > > +
> > 	__xe_vm_needs_clear_scratch_pages(vm, flags);
> > >  		} else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
> > >  			op->prefetch.region = prefetch_region;
> > >  		}
> > > @@ -2188,7 +2207,8 @@ static int vm_bind_ioctl_ops_parse(struct
> > xe_vm *vm, struct drm_gpuva_ops *ops,
> > >  				return PTR_ERR(vma);
> > >
> > >  			op->map.vma = vma;
> > > -			if (op->map.immediate
> > || !xe_vm_in_fault_mode(vm))
> > > +			if (op->map.immediate
> > || !xe_vm_in_fault_mode(vm) ||
> > > +			    op->map.invalidate_on_bind)
> > >
> > 	xe_vma_ops_incr_pt_update_ops(vops,
> > >  							      op-
> > >tile_mask);
> > >  			break;
> > > @@ -2416,9 +2436,10 @@ static int op_lock_and_prep(struct
> > drm_exec *exec, struct xe_vm *vm,
> > >
> > >  	switch (op->base.op) {
> > >  	case DRM_GPUVA_OP_MAP:
> > > -		err = vma_lock_and_validate(exec, op->map.vma,
> > > -					    !xe_vm_in_fault_mode(vm)
> > ||
> > > -					    op->map.immediate);
> > > +		if (!op->map.invalidate_on_bind)
> > > +			err = vma_lock_and_validate(exec, op-
> > >map.vma,
> > > +
> > 	    !xe_vm_in_fault_mode(vm) ||
> > > +						    op-
> > >map.immediate);
> > >  		break;
> > >  	case DRM_GPUVA_OP_REMAP:
> > >  		err = check_ufence(gpuva_to_vma(op-
> > >base.remap.unmap->va));
> > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > index 52467b9b5348..dace04f4ea5e 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > @@ -297,6 +297,8 @@ struct xe_vma_op_map {
> > >  	bool is_null;
> > >  	/** @dumpable: whether BO is dumped on GPU hang */
> > >  	bool dumpable;
> > > +	/** @invalidate: invalidate the VMA before bind */
> > > +	bool invalidate_on_bind;
> > >  	/** @pat_index: The pat index to use for this operation. */
> > >  	u16 pat_index;
> > >  };
> > > --
> > > 2.26.3
> > >

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-13  2:23 Oak Zeng
@ 2025-02-13  2:23 ` Oak Zeng
  2025-02-25 22:10   ` Matthew Brost
  0 siblings, 1 reply; 38+ messages in thread
From: Oak Zeng @ 2025-02-13  2:23 UTC (permalink / raw)
  To: intel-xe; +Cc: Thomas.Hellstrom, matthew.brost, jonathan.cavitt

Normally scratch page is not allowed when a vm is operate under page
fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
is fault mode relies on recoverable page to work, while scratch page
can mute recoverable page fault.

On xe2 and xe3, out of bound prefetch can cause page fault and further
system hang because xekmd can't resolve such page fault. SYCL and OCL
language runtime requires out of bound prefetch to be silently dropped
without causing any functional problem, thus the existing behavior
doesn't meet language runtime requirement.

At the same time, HW prefetching can cause page fault interrupt. Due to
page fault interrupt overhead (i.e., need Guc and KMD involved to fix
the page fault), HW prefetching can be slowed by many orders of magnitude.

Fix those problems by allowing scratch page under fault mode for xe2 and
xe3. With scratch page in place, HW prefetching could always hit scratch
page instead of causing interrupt.

A side effect is, scratch page could hide application program error.
Application out of bound accesses are hided by scratch page mapping,
instead of get reported to user.

igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
BMG:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738684805 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
Subtest scratch-fault: SUCCESS (0.080s)

Without this series, the test result is:

root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
Using IGT_SRANDOM=1738686046 for randomisation
Opened device: /dev/dri/card0
Starting subtest: scratch-fault
(xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
(xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
(xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
(xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
Stack trace:
  #0 ../lib/igt_core.c:2266 __igt_fail_assert()
  #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
  #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
  #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
  #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
  #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
  #6 [_start+0x2e]
Subtest scratch-fault failed.

v2: Refine commit message (Thomas)

v3: Move the scratch page flag check to after scratch page wa (Thomas)

Signed-off-by: Oak Zeng <oak.zeng@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 813d893d9b63..c2dfd0ade403 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1766,7 +1766,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 		return -EINVAL;
 
 	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
-			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
+			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
+			 !(NEEDS_SCRATCH(xe))))
 		return -EINVAL;
 
 	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
-- 
2.26.3


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor
  2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
                   ` (9 preceding siblings ...)
  2025-02-05 13:16 ` [PATCH 1/3] " Thomas Hellström
@ 2025-02-25 21:37 ` Matthew Brost
  10 siblings, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-25 21:37 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe, Thomas.Hellstrom, jonathan.cavitt

On Tue, Feb 04, 2025 at 01:45:56PM -0500, Oak Zeng wrote:
> On some platform, scratch page is needed for out of bound prefetch
> to work. Introduce a bit in device descriptor to specify whether
> this device needs scratch page to work.
> 
> v2: introduce a needs_scratch bit in device info (Thomas, Jonathan)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_device_types.h | 3 +++
>  drivers/gpu/drm/xe/xe_pci.c          | 5 +++++
>  2 files changed, 8 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> index c0e886bac183..1197ab31e528 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -45,6 +45,7 @@ struct xe_pxp;
>  #define GRAPHICS_VERx100(xe) ((xe)->info.graphics_verx100)
>  #define MEDIA_VERx100(xe) ((xe)->info.media_verx100)
>  #define IS_DGFX(xe) ((xe)->info.is_dgfx)
> +#define NEEDS_SCRATCH(xe) ((xe)->info.needs_scratch)

I'd drop this macro. These are leftovers from the i915 code and I see
like value in a macro looking directly at a field. In the existing cases
these macros are used all over the code so they make a little more sense
so if change the variable we only have change this macro, but in this
new case the macro is used 2 places in the code so I'd say just look at
the variable.

Matt
 
>  
>  #define XE_VRAM_FLAGS_NEED64K		BIT(0)
>  
> @@ -318,6 +319,8 @@ struct xe_device {
>  		u8 has_usm:1;
>  		/** @info.is_dgfx: is discrete device */
>  		u8 is_dgfx:1;
> +		/** @info.needs_scratch: needs scratch page for oob prefetch to work */
> +		u8 needs_scratch:1;
>  		/**
>  		 * @info.probe_display: Probe display hardware.  If set to
>  		 * false, the driver will behave as if there is no display
> diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> index e8ef7d6b4db8..09736674a77e 100644
> --- a/drivers/gpu/drm/xe/xe_pci.c
> +++ b/drivers/gpu/drm/xe/xe_pci.c
> @@ -67,6 +67,7 @@ struct xe_device_desc {
>  	u8 has_llc:1;
>  	u8 has_pxp:1;
>  	u8 has_sriov:1;
> +	u8 needs_scratch:1;
>  	u8 skip_guc_pc:1;
>  	u8 skip_mtcfg:1;
>  	u8 skip_pcode:1;
> @@ -353,6 +354,7 @@ static const struct xe_device_desc lnl_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.has_pxp = true,
> +	.needs_scratch = true,
>  };
>  
>  static const struct xe_device_desc bmg_desc = {
> @@ -361,6 +363,7 @@ static const struct xe_device_desc bmg_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.has_heci_cscfi = 1,
> +	.needs_scratch = true,
>  };
>  
>  static const struct xe_device_desc ptl_desc = {
> @@ -368,6 +371,7 @@ static const struct xe_device_desc ptl_desc = {
>  	.dma_mask_size = 46,
>  	.has_display = true,
>  	.require_force_probe = true,
> +	.needs_scratch = true,
>  };
>  
>  #undef PLATFORM
> @@ -643,6 +647,7 @@ static int xe_info_init_early(struct xe_device *xe,
>  	xe->info.skip_guc_pc = desc->skip_guc_pc;
>  	xe->info.skip_mtcfg = desc->skip_mtcfg;
>  	xe->info.skip_pcode = desc->skip_pcode;
> +	xe->info.needs_scratch = desc->needs_scratch;
>  
>  	xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) &&
>  				 xe_modparam.probe_display &&
> -- 
> 2.26.3
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-04 18:45 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
  2025-02-05 13:14   ` Thomas Hellström
@ 2025-02-25 22:05   ` Matthew Brost
  1 sibling, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-25 22:05 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe, Thomas.Hellstrom, jonathan.cavitt

On Tue, Feb 04, 2025 at 01:45:58PM -0500, Oak Zeng wrote:
> Normally scratch page is not allowed when a vm is operate under page
> fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> is fault mode relies on recoverable page to work, while scratch page
> can mute recoverable page fault.
> 
> On xe2 and xe3, out of bound prefetch can cause page fault and further
> system hang because xekmd can't resolve such page fault. SYCL and OCL
> language runtime requires out of bound prefetch to be silently dropped
> without causing any functional problem, thus the existing behavior
> doesn't meet language runtime requirement.
> 
> At the same time, HW prefetching can cause page fault interrupt. Due to
> page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> the page fault), HW prefetching can be slowed by many orders of magnitude.
> 
> Fix those problems by allowing scratch page under fault mode for xe2 and
> xe3. With scratch page in place, HW prefetching could always hit scratch
> page instead of causing interrupt.
> 
> A side effect is, scratch page could hide application program error.
> Application out of bound accesses are hided by scratch page mapping,
> instead of get reported to user.
> 
> igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
> BMG:
> 

I'd include the IGT information in the cover letter, not the patch.

> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738684805 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> Subtest scratch-fault: SUCCESS (0.080s)
> 
> Without this series, the test result is:
> 
> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738686046 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> (xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
> (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> Stack trace:
>   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
>   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
>   #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
>   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
>   #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
>   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
>   #6 [_start+0x2e]
> Subtest scratch-fault failed.
> 
> v2: Refine commit message (Thomas)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_vm.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 813d893d9b63..c0372f083d42 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1752,6 +1752,11 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>  	if (XE_IOCTL_DBG(xe, args->extensions))
>  		return -EINVAL;
>  
> +	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> +			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> +			 !(NEEDS_SCRATCH(xe))))

Same comment as patch #1, I'd drop this macro.

Do we need a query uAPI so the UMD can test upon process open if the VM
supports scratch page + faults? Or should we just not restrict VM
scratch page + faults ever and have it choose based on platform
recommnedation?

Matt

> +		return -EINVAL;
> +
>  	if (XE_WA(xe_root_mmio_gt(xe), 14016763929))
>  		args->flags |= DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE;
>  
> @@ -1765,10 +1770,6 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>  	if (XE_IOCTL_DBG(xe, args->flags & ~ALL_DRM_XE_VM_CREATE_FLAGS))
>  		return -EINVAL;
>  
> -	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> -			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> -		return -EINVAL;
> -
>  	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
>  			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
>  		return -EINVAL;
> -- 
> 2.26.3
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-13  2:23 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
@ 2025-02-25 22:10   ` Matthew Brost
  2025-02-26 22:12     ` Zeng, Oak
  0 siblings, 1 reply; 38+ messages in thread
From: Matthew Brost @ 2025-02-25 22:10 UTC (permalink / raw)
  To: Oak Zeng; +Cc: intel-xe, Thomas.Hellstrom, jonathan.cavitt

On Wed, Feb 12, 2025 at 09:23:31PM -0500, Oak Zeng wrote:

I replied to the wrong versions... Please generate the patches with:

git format-patch -v<n>

Where <n> is the version number. This will help avoiding replying to the
wrong patch.

Copy / pasting my reply here...

> Normally scratch page is not allowed when a vm is operate under page
> fault mode, i.e., in the existing codes, DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual exclusive. The reason
> is fault mode relies on recoverable page to work, while scratch page
> can mute recoverable page fault.
> 
> On xe2 and xe3, out of bound prefetch can cause page fault and further
> system hang because xekmd can't resolve such page fault. SYCL and OCL
> language runtime requires out of bound prefetch to be silently dropped
> without causing any functional problem, thus the existing behavior
> doesn't meet language runtime requirement.
> 
> At the same time, HW prefetching can cause page fault interrupt. Due to
> page fault interrupt overhead (i.e., need Guc and KMD involved to fix
> the page fault), HW prefetching can be slowed by many orders of magnitude.
> 
> Fix those problems by allowing scratch page under fault mode for xe2 and
> xe3. With scratch page in place, HW prefetching could always hit scratch
> page instead of causing interrupt.
> 
> A side effect is, scratch page could hide application program error.
> Application out of bound accesses are hided by scratch page mapping,
> instead of get reported to user.
>

I'd include the IGT information in the cover letter, not the patch.
 
> igt test: https://patchwork.freedesktop.org/series/144334/. Test result on
> BMG:
> 
> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738684805 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> Subtest scratch-fault: SUCCESS (0.080s)
> 
> Without this series, the test result is:
> 
> root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> Using IGT_SRANDOM=1738686046 for randomisation
> Opened device: /dev/dri/card0
> Starting subtest: scratch-fault
> (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> (xe_exec_fault_mode:5047) CRITICAL: Failed assertion: __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull, exec_queues[i % n_exec_queues], &timeout) == 0
> (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> Stack trace:
>   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
>   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
>   #2 ../tests/intel/xe_exec_fault_mode.c:537 __igt_unique____real_main407()
>   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
>   #4 ../sysdeps/nptl/libc_start_call_main.h:74 __libc_start_call_main()
>   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
>   #6 [_start+0x2e]
> Subtest scratch-fault failed.
> 
> v2: Refine commit message (Thomas)
> 
> v3: Move the scratch page flag check to after scratch page wa (Thomas)
> 
> Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_vm.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 813d893d9b63..c2dfd0ade403 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -1766,7 +1766,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
>  		return -EINVAL;
>  
>  	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> -			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> +			 args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> +			 !(NEEDS_SCRATCH(xe))))

Same comment as patch #1, I'd drop this macro.

Do we need a query uAPI so the UMD can test upon process open if the VM
supports scratch page + faults? Or should we just not restrict VM
scratch page + faults ever and have it choose based on platform
recommnedation?

Matt

>  		return -EINVAL;
>  
>  	if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> -- 
> 2.26.3
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* RE: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-25 22:10   ` Matthew Brost
@ 2025-02-26 22:12     ` Zeng, Oak
  2025-02-27  0:22       ` Matthew Brost
  0 siblings, 1 reply; 38+ messages in thread
From: Zeng, Oak @ 2025-02-26 22:12 UTC (permalink / raw)
  To: Brost, Matthew
  Cc: intel-xe@lists.freedesktop.org, Thomas.Hellstrom@linux.intel.com,
	Cavitt, Jonathan



> -----Original Message-----
> From: Brost, Matthew <matthew.brost@intel.com>
> Sent: February 25, 2025 5:11 PM
> To: Zeng, Oak <oak.zeng@intel.com>
> Cc: intel-xe@lists.freedesktop.org;
> Thomas.Hellstrom@linux.intel.com; Cavitt, Jonathan
> <jonathan.cavitt@intel.com>
> Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> mode for certain platform
> 
> On Wed, Feb 12, 2025 at 09:23:31PM -0500, Oak Zeng wrote:
> 
> I replied to the wrong versions... Please generate the patches with:
> 
> git format-patch -v<n>
> 
> Where <n> is the version number. This will help avoiding replying to
> the
> wrong patch.
> 
> Copy / pasting my reply here...
> 
> > Normally scratch page is not allowed when a vm is operate under
> page
> > fault mode, i.e., in the existing codes,
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual
> exclusive. The reason
> > is fault mode relies on recoverable page to work, while scratch page
> > can mute recoverable page fault.
> >
> > On xe2 and xe3, out of bound prefetch can cause page fault and
> further
> > system hang because xekmd can't resolve such page fault. SYCL and
> OCL
> > language runtime requires out of bound prefetch to be silently
> dropped
> > without causing any functional problem, thus the existing behavior
> > doesn't meet language runtime requirement.
> >
> > At the same time, HW prefetching can cause page fault interrupt.
> Due to
> > page fault interrupt overhead (i.e., need Guc and KMD involved to
> fix
> > the page fault), HW prefetching can be slowed by many orders of
> magnitude.
> >
> > Fix those problems by allowing scratch page under fault mode for
> xe2 and
> > xe3. With scratch page in place, HW prefetching could always hit
> scratch
> > page instead of causing interrupt.
> >
> > A side effect is, scratch page could hide application program error.
> > Application out of bound accesses are hided by scratch page
> mapping,
> > instead of get reported to user.
> >
> 
> I'd include the IGT information in the cover letter, not the patch.
> 
> > igt test: https://patchwork.freedesktop.org/series/144334/. Test
> result on
> > BMG:
> >
> > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > Using IGT_SRANDOM=1738684805 for randomisation
> > Opened device: /dev/dri/card0
> > Starting subtest: scratch-fault
> > Subtest scratch-fault: SUCCESS (0.080s)
> >
> > Without this series, the test result is:
> >
> > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > Using IGT_SRANDOM=1738686046 for randomisation
> > Opened device: /dev/dri/card0
> > Starting subtest: scratch-fault
> > (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure
> function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> > (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> exec_queues[i % n_exec_queues], &timeout) == 0
> > (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> > (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> > Stack trace:
> >   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
> >   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
> >   #2 ../tests/intel/xe_exec_fault_mode.c:537
> __igt_unique____real_main407()
> >   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
> >   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> __libc_start_call_main()
> >   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
> >   #6 [_start+0x2e]
> > Subtest scratch-fault failed.
> >
> > v2: Refine commit message (Thomas)
> >
> > v3: Move the scratch page flag check to after scratch page wa
> (Thomas)
> >
> > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_vm.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> b/drivers/gpu/drm/xe/xe_vm.c
> > index 813d893d9b63..c2dfd0ade403 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1766,7 +1766,8 @@ int xe_vm_create_ioctl(struct drm_device
> *dev, void *data,
> >  		return -EINVAL;
> >
> >  	if (XE_IOCTL_DBG(xe, args->flags &
> DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > -			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > +			 args->flags &
> DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > +			 !(NEEDS_SCRATCH(xe))))
> 
> Same comment as patch #1, I'd drop this macro.
> 
> Do we need a query uAPI so the UMD can test upon process open if
> the VM
> supports scratch page + faults?

Introducing a query API seems a little overkill to me. 

Maybe I can add a little comment in the xe_drm.h, saying scratch and fault
Flags are normally mutual exclusive except on xe2/3?


Or should we just not restrict VM
> scratch page + faults ever and have it choose based on platform
> recommnedation?
> 

I think having above checking/restriction is still helpful.

Oak

> Matt
> 
> >  		return -EINVAL;
> >
> >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> > --
> > 2.26.3
> >

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform
  2025-02-26 22:12     ` Zeng, Oak
@ 2025-02-27  0:22       ` Matthew Brost
  0 siblings, 0 replies; 38+ messages in thread
From: Matthew Brost @ 2025-02-27  0:22 UTC (permalink / raw)
  To: Zeng, Oak
  Cc: intel-xe@lists.freedesktop.org, Thomas.Hellstrom@linux.intel.com,
	Cavitt, Jonathan

On Wed, Feb 26, 2025 at 03:12:14PM -0700, Zeng, Oak wrote:
> 
> 
> > -----Original Message-----
> > From: Brost, Matthew <matthew.brost@intel.com>
> > Sent: February 25, 2025 5:11 PM
> > To: Zeng, Oak <oak.zeng@intel.com>
> > Cc: intel-xe@lists.freedesktop.org;
> > Thomas.Hellstrom@linux.intel.com; Cavitt, Jonathan
> > <jonathan.cavitt@intel.com>
> > Subject: Re: [PATCH 3/3] drm/xe: Allow scratch page under fault
> > mode for certain platform
> > 
> > On Wed, Feb 12, 2025 at 09:23:31PM -0500, Oak Zeng wrote:
> > 
> > I replied to the wrong versions... Please generate the patches with:
> > 
> > git format-patch -v<n>
> > 
> > Where <n> is the version number. This will help avoiding replying to
> > the
> > wrong patch.
> > 
> > Copy / pasting my reply here...
> > 
> > > Normally scratch page is not allowed when a vm is operate under
> > page
> > > fault mode, i.e., in the existing codes,
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
> > > and DRM_XE_VM_CREATE_FLAG_FAULT_MODE are mutual
> > exclusive. The reason
> > > is fault mode relies on recoverable page to work, while scratch page
> > > can mute recoverable page fault.
> > >
> > > On xe2 and xe3, out of bound prefetch can cause page fault and
> > further
> > > system hang because xekmd can't resolve such page fault. SYCL and
> > OCL
> > > language runtime requires out of bound prefetch to be silently
> > dropped
> > > without causing any functional problem, thus the existing behavior
> > > doesn't meet language runtime requirement.
> > >
> > > At the same time, HW prefetching can cause page fault interrupt.
> > Due to
> > > page fault interrupt overhead (i.e., need Guc and KMD involved to
> > fix
> > > the page fault), HW prefetching can be slowed by many orders of
> > magnitude.
> > >
> > > Fix those problems by allowing scratch page under fault mode for
> > xe2 and
> > > xe3. With scratch page in place, HW prefetching could always hit
> > scratch
> > > page instead of causing interrupt.
> > >
> > > A side effect is, scratch page could hide application program error.
> > > Application out of bound accesses are hided by scratch page
> > mapping,
> > > instead of get reported to user.
> > >
> > 
> > I'd include the IGT information in the cover letter, not the patch.
> > 
> > > igt test: https://patchwork.freedesktop.org/series/144334/. Test
> > result on
> > > BMG:
> > >
> > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > Using IGT_SRANDOM=1738684805 for randomisation
> > > Opened device: /dev/dri/card0
> > > Starting subtest: scratch-fault
> > > Subtest scratch-fault: SUCCESS (0.080s)
> > >
> > > Without this series, the test result is:
> > >
> > > root@DUT1130BMGFRD:/home/szeng/dii-tools/igt-
> > public/build/tests# ./xe_exec_fault_mode --run-subtest scratch-fault
> > > IGT-Version: 1.30-gde1a3cb42 (x86_64) (Linux: 6.13.0-xe x86_64)
> > > Using IGT_SRANDOM=1738686046 for randomisation
> > > Opened device: /dev/dri/card0
> > > Starting subtest: scratch-fault
> > > (xe_exec_fault_mode:5047) CRITICAL: Test assertion failure
> > function test_exec, file ../tests/intel/xe_exec_fault_mode.c:349:
> > > (xe_exec_fault_mode:5047) CRITICAL: Failed assertion:
> > __xe_wait_ufence(fd, &exec_sync[i], 0xdeadbeefdeadbeefull,
> > exec_queues[i % n_exec_queues], &timeout) == 0
> > > (xe_exec_fault_mode:5047) CRITICAL: Last errno: 62, Timer expired
> > > (xe_exec_fault_mode:5047) CRITICAL: error: -62 != 0
> > > Stack trace:
> > >   #0 ../lib/igt_core.c:2266 __igt_fail_assert()
> > >   #1 ../tests/intel/xe_exec_fault_mode.c:346 test_exec()
> > >   #2 ../tests/intel/xe_exec_fault_mode.c:537
> > __igt_unique____real_main407()
> > >   #3 ../tests/intel/xe_exec_fault_mode.c:407 main()
> > >   #4 ../sysdeps/nptl/libc_start_call_main.h:74
> > __libc_start_call_main()
> > >   #5 ../csu/libc-start.c:128 __libc_start_main@@GLIBC_2.34()
> > >   #6 [_start+0x2e]
> > > Subtest scratch-fault failed.
> > >
> > > v2: Refine commit message (Thomas)
> > >
> > > v3: Move the scratch page flag check to after scratch page wa
> > (Thomas)
> > >
> > > Signed-off-by: Oak Zeng <oak.zeng@intel.com>
> > > ---
> > >  drivers/gpu/drm/xe/xe_vm.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > > index 813d893d9b63..c2dfd0ade403 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -1766,7 +1766,8 @@ int xe_vm_create_ioctl(struct drm_device
> > *dev, void *data,
> > >  		return -EINVAL;
> > >
> > >  	if (XE_IOCTL_DBG(xe, args->flags &
> > DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
> > > -			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
> > > +			 args->flags &
> > DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
> > > +			 !(NEEDS_SCRATCH(xe))))
> > 
> > Same comment as patch #1, I'd drop this macro.
> > 
> > Do we need a query uAPI so the UMD can test upon process open if
> > the VM
> > supports scratch page + faults?
> 
> Introducing a query API seems a little overkill to me. 
> 

This is really a question for the level0 UMD I suppose, I'd follow up
there. They have asked for a query uAPI for other seemly simple things
before.

Matt

> Maybe I can add a little comment in the xe_drm.h, saying scratch and fault
> Flags are normally mutual exclusive except on xe2/3?
> 
> 
> Or should we just not restrict VM
> > scratch page + faults ever and have it choose based on platform
> > recommnedation?
> > 
> 
> I think having above checking/restriction is still helpful.
> 
> Oak
> 
> > Matt
> > 
> > >  		return -EINVAL;
> > >
> > >  	if (XE_IOCTL_DBG(xe, !(args->flags &
> > DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
> > > --
> > > 2.26.3
> > >

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2025-02-27  0:22 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-04 18:45 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
2025-02-04 18:45 ` [PATCH 2/3] drm/xe: Clear scratch page before vm_bind Oak Zeng
2025-02-05 14:35   ` Thomas Hellström
2025-02-06  1:52     ` Zeng, Oak
2025-02-06 12:51       ` Thomas Hellström
2025-02-06 18:56         ` Zeng, Oak
2025-02-06 20:11           ` Thomas Hellström
2025-02-06 21:05             ` Zeng, Oak
2025-02-06 10:34   ` Matthew Brost
2025-02-06 10:43     ` Thomas Hellström
2025-02-10 17:08       ` Matthew Brost
2025-02-06 15:16     ` Zeng, Oak
2025-02-10 17:09       ` Matthew Brost
2025-02-04 18:45 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
2025-02-05 13:14   ` Thomas Hellström
2025-02-06  1:54     ` Zeng, Oak
2025-02-06  9:29       ` Thomas Hellström
2025-02-06 15:14         ` Zeng, Oak
2025-02-25 22:05   ` Matthew Brost
2025-02-04 23:27 ` ✓ CI.Patch_applied: success for series starting with [1/3] drm/xe: Introduced needs_scratch bit in device descriptor Patchwork
2025-02-04 23:27 ` ✗ CI.checkpatch: warning " Patchwork
2025-02-04 23:28 ` ✓ CI.KUnit: success " Patchwork
2025-02-04 23:45 ` ✓ CI.Build: " Patchwork
2025-02-04 23:47 ` ✓ CI.Hooks: " Patchwork
2025-02-04 23:48 ` ✓ CI.checksparse: " Patchwork
2025-02-05  6:05 ` ✓ Xe.CI.BAT: " Patchwork
2025-02-05 13:16 ` [PATCH 1/3] " Thomas Hellström
2025-02-25 21:37 ` Matthew Brost
  -- strict thread matches above, loose matches on Subject: below --
2025-02-13  2:23 Oak Zeng
2025-02-13  2:23 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
2025-02-25 22:10   ` Matthew Brost
2025-02-26 22:12     ` Zeng, Oak
2025-02-27  0:22       ` Matthew Brost
2025-02-06 21:38 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
2025-02-06 21:38 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
2025-02-06  2:11 [PATCH 1/3] drm/xe: Introduced needs_scratch bit in device descriptor Oak Zeng
2025-02-06  2:11 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
2025-01-28 22:21 [PATCH 1/3] drm/xe: Add a function to zap page table by address range Oak Zeng
2025-01-28 22:21 ` [PATCH 3/3] drm/xe: Allow scratch page under fault mode for certain platform Oak Zeng
2025-01-28 23:05   ` Cavitt, Jonathan
2025-01-29  8:52   ` Thomas Hellström
2025-01-29 16:41     ` Matthew Brost

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox