* [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level
@ 2023-05-08 23:48 fei.yang
2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 1/2] drm/i915: preparation for using PAT index fei.yang
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: fei.yang @ 2023-05-08 23:48 UTC (permalink / raw)
To: intel-gfx; +Cc: dri-devel
From: Fei Yang <fei.yang@intel.com>
This patch set was posted at
https://patchwork.freedesktop.org/series/116868/
Change title since the PTE patch was merged separately.
These patches are extracted from series
https://patchwork.freedesktop.org/series/115980/
This series refactor the cache policy programming so that the PTE
encode functions can be unified across all GEN12 platforms. This
refactor is also important in implementing the design which allows
uerspace to directly set cache policy for each Buffer Object.
v2: drop one patch that was merged separately
341ad0e8e254 drm/i915/mtl: Add PTE encode function
v3: disable {get, set}_caching ioctl
v4: fix missing unlock introduced in v3, and
solve a rebase conflict
v5: replace obj->cache_level with pat_set_by_user,
fix i915_cache_level_str() for legacy platforms.
v6: squash the pte_encode patch because separating them causes
bisect probelm. Also addressing some review comments from
Tvrtko and Matt.
v7: fix checkpatch errors and warnings.
Fei Yang (2):
drm/i915: preparation for using PAT index
drm/i915: use pat_index instead of cache_level
drivers/gpu/drm/i915/display/intel_dpt.c | 12 +--
drivers/gpu/drm/i915/gem/i915_gem_domain.c | 58 +++++++++-----
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 15 +++-
drivers/gpu/drm/i915/gem/i915_gem_mman.c | 11 ++-
drivers/gpu/drm/i915/gem/i915_gem_object.c | 60 +++++++++++++-
drivers/gpu/drm/i915/gem/i915_gem_object.h | 8 ++
.../gpu/drm/i915/gem/i915_gem_object_types.h | 51 +++++++++++-
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c | 2 -
drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 4 +-
drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 8 +-
.../gpu/drm/i915/gem/selftests/huge_pages.c | 2 +-
.../drm/i915/gem/selftests/i915_gem_migrate.c | 2 +-
.../drm/i915/gem/selftests/i915_gem_mman.c | 2 +-
drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 10 ++-
drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 78 +++++++++---------
drivers/gpu/drm/i915/gt/gen8_ppgtt.h | 3 +-
drivers/gpu/drm/i915/gt/intel_ggtt.c | 76 +++++++++---------
drivers/gpu/drm/i915/gt/intel_gtt.h | 18 ++---
drivers/gpu/drm/i915/gt/intel_migrate.c | 47 ++++++-----
drivers/gpu/drm/i915/gt/intel_migrate.h | 13 ++-
drivers/gpu/drm/i915/gt/intel_ppgtt.c | 4 +-
drivers/gpu/drm/i915/gt/selftest_migrate.c | 47 +++++------
drivers/gpu/drm/i915/gt/selftest_reset.c | 8 +-
drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +-
drivers/gpu/drm/i915/gt/selftest_tlb.c | 4 +-
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 10 ++-
drivers/gpu/drm/i915/i915_debugfs.c | 53 ++++++++++---
drivers/gpu/drm/i915/i915_gem.c | 27 ++++++-
drivers/gpu/drm/i915/i915_gpu_error.c | 8 +-
drivers/gpu/drm/i915/i915_pci.c | 79 ++++++++++++++++---
drivers/gpu/drm/i915/i915_vma.c | 16 ++--
drivers/gpu/drm/i915/i915_vma.h | 2 +-
drivers/gpu/drm/i915/i915_vma_types.h | 2 -
drivers/gpu/drm/i915/intel_device_info.h | 5 ++
drivers/gpu/drm/i915/selftests/i915_gem.c | 5 +-
.../gpu/drm/i915/selftests/i915_gem_evict.c | 4 +-
drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 15 ++--
.../drm/i915/selftests/intel_memory_region.c | 4 +-
.../gpu/drm/i915/selftests/mock_gem_device.c | 9 +++
drivers/gpu/drm/i915/selftests/mock_gtt.c | 8 +-
40 files changed, 555 insertions(+), 237 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 10+ messages in thread* [Intel-gfx] [PATCH v7 1/2] drm/i915: preparation for using PAT index 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang @ 2023-05-08 23:48 ` fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level fei.yang ` (3 subsequent siblings) 4 siblings, 0 replies; 10+ messages in thread From: fei.yang @ 2023-05-08 23:48 UTC (permalink / raw) To: intel-gfx; +Cc: Chris Wilson, dri-devel, Andrzej Hajda, Matt Roper From: Fei Yang <fei.yang@intel.com> This patch is a preparation for replacing enum i915_cache_level with PAT index. Caching policy for buffer objects is set through the PAT index in PTE, the old i915_cache_level is not sufficient to represent all caching modes supported by the hardware. Preparing the transition by adding some platform dependent data structures and helper functions to translate the cache_level to pat_index. cachelevel_to_pat: a platform dependent array mapping cache_level to pat_index. max_pat_index: the maximum PAT index recommended in hardware specification Needed for validating the PAT index passed in from user space. i915_gem_get_pat_index: function to convert cache_level to PAT index. obj_to_i915(obj): macro moved to header file for wider usage. I915_MAX_CACHE_LEVEL: upper bound of i915_cache_level for the convenience of coding. Cc: Chris Wilson <chris.p.wilson@linux.intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Fei Yang <fei.yang@intel.com> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> --- drivers/gpu/drm/i915/gem/i915_gem_object.c | 9 +++ drivers/gpu/drm/i915/gem/i915_gem_object.h | 4 + .../gpu/drm/i915/gem/i915_gem_object_types.h | 7 ++ drivers/gpu/drm/i915/gem/i915_gem_shrinker.c | 2 - drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 6 ++ drivers/gpu/drm/i915/gt/intel_ggtt.c | 6 ++ drivers/gpu/drm/i915/i915_pci.c | 79 ++++++++++++++++--- drivers/gpu/drm/i915/intel_device_info.h | 5 ++ .../gpu/drm/i915/selftests/mock_gem_device.c | 9 +++ 9 files changed, 116 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c index 4666bb82f312..8c70a0ec7d2f 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c @@ -45,6 +45,15 @@ static struct kmem_cache *slab_objects; static const struct drm_gem_object_funcs i915_gem_object_funcs; +unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, + enum i915_cache_level level) +{ + if (drm_WARN_ON(&i915->drm, level >= I915_MAX_CACHE_LEVEL)) + return 0; + + return INTEL_INFO(i915)->cachelevel_to_pat[level]; +} + struct drm_i915_gem_object *i915_gem_object_alloc(void) { struct drm_i915_gem_object *obj; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h index bc1291887d4f..284e1aa396cd 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h @@ -20,6 +20,8 @@ enum intel_region_id; +#define obj_to_i915(obj__) to_i915((obj__)->base.dev) + static inline bool i915_gem_object_size_2big(u64 size) { struct drm_i915_gem_object *obj; @@ -30,6 +32,8 @@ static inline bool i915_gem_object_size_2big(u64 size) return false; } +unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, + enum i915_cache_level level); void i915_gem_init__objects(struct drm_i915_private *i915); void i915_objects_module_exit(void); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index 830c11431ee8..bf0bd8e11355 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -194,6 +194,13 @@ enum i915_cache_level { * engine. */ I915_CACHE_WT, + /** + * @I915_MAX_CACHE_LEVEL: + * + * Mark the last entry in the enum. Used for defining cachelevel_to_pat + * array for cache_level to pat translation table. + */ + I915_MAX_CACHE_LEVEL, }; enum i915_map_type { diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c b/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c index b1672e054b21..214763942aa2 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shrinker.c @@ -460,8 +460,6 @@ void i915_gem_shrinker_taints_mutex(struct drm_i915_private *i915, fs_reclaim_release(GFP_KERNEL); } -#define obj_to_i915(obj__) to_i915((obj__)->base.dev) - /** * i915_gem_object_make_unshrinkable - Hide the object from the shrinker. By * default all object types that support shrinking(see IS_SHRINKABLE), will also diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c index 22ec1566d2a7..bb6998d67133 100644 --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c @@ -78,6 +78,12 @@ static u64 mtl_pte_encode(dma_addr_t addr, case I915_CACHE_WT: pte |= GEN12_PPGTT_PTE_PAT0; break; + default: + /* This should never happen. Added to deal with the compile + * error due to the addition of I915_MAX_CACHE_LEVEL. Will + * be removed by the pat_index patch. + */ + break; } return pte; diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c index 20915edc8bd9..c8390d03fce2 100644 --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c @@ -242,6 +242,12 @@ static u64 mtl_ggtt_pte_encode(dma_addr_t addr, case I915_CACHE_WT: pte |= MTL_GGTT_PTE_PAT0; break; + default: + /* This should never happen. Added to deal with the compile + * error due to the addition of I915_MAX_CACHE_LEVEL. Will + * be removed by the pat_index patch. + */ + break; } return pte; diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c index 5fcc9cfed671..f7677cc95435 100644 --- a/drivers/gpu/drm/i915/i915_pci.c +++ b/drivers/gpu/drm/i915/i915_pci.c @@ -30,6 +30,7 @@ #include "display/intel_display_driver.h" #include "gt/intel_gt_regs.h" #include "gt/intel_sa_media.h" +#include "gem/i915_gem_object_types.h" #include "i915_driver.h" #include "i915_drv.h" @@ -164,6 +165,38 @@ .gamma_lut_tests = DRM_COLOR_LUT_NON_DECREASING, \ } +#define LEGACY_CACHELEVEL \ + .cachelevel_to_pat = { \ + [I915_CACHE_NONE] = 0, \ + [I915_CACHE_LLC] = 1, \ + [I915_CACHE_L3_LLC] = 2, \ + [I915_CACHE_WT] = 3, \ + } + +#define TGL_CACHELEVEL \ + .cachelevel_to_pat = { \ + [I915_CACHE_NONE] = 3, \ + [I915_CACHE_LLC] = 0, \ + [I915_CACHE_L3_LLC] = 0, \ + [I915_CACHE_WT] = 2, \ + } + +#define PVC_CACHELEVEL \ + .cachelevel_to_pat = { \ + [I915_CACHE_NONE] = 0, \ + [I915_CACHE_LLC] = 3, \ + [I915_CACHE_L3_LLC] = 3, \ + [I915_CACHE_WT] = 2, \ + } + +#define MTL_CACHELEVEL \ + .cachelevel_to_pat = { \ + [I915_CACHE_NONE] = 2, \ + [I915_CACHE_LLC] = 3, \ + [I915_CACHE_L3_LLC] = 3, \ + [I915_CACHE_WT] = 1, \ + } + /* Keep in gen based order, and chronological order within a gen */ #define GEN_DEFAULT_PAGE_SIZES \ @@ -189,11 +222,13 @@ .has_snoop = true, \ .has_coherent_ggtt = false, \ .dma_mask_size = 32, \ + .max_pat_index = 3, \ I9XX_PIPE_OFFSETS, \ I9XX_CURSOR_OFFSETS, \ I9XX_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL #define I845_FEATURES \ GEN(2), \ @@ -210,11 +245,13 @@ .has_snoop = true, \ .has_coherent_ggtt = false, \ .dma_mask_size = 32, \ + .max_pat_index = 3, \ I845_PIPE_OFFSETS, \ I845_CURSOR_OFFSETS, \ I845_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL static const struct intel_device_info i830_info = { I830_FEATURES, @@ -249,11 +286,13 @@ static const struct intel_device_info i865g_info = { .has_snoop = true, \ .has_coherent_ggtt = true, \ .dma_mask_size = 32, \ + .max_pat_index = 3, \ I9XX_PIPE_OFFSETS, \ I9XX_CURSOR_OFFSETS, \ I9XX_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL static const struct intel_device_info i915g_info = { GEN3_FEATURES, @@ -341,11 +380,13 @@ static const struct intel_device_info pnv_m_info = { .has_snoop = true, \ .has_coherent_ggtt = true, \ .dma_mask_size = 36, \ + .max_pat_index = 3, \ I9XX_PIPE_OFFSETS, \ I9XX_CURSOR_OFFSETS, \ I9XX_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL static const struct intel_device_info i965g_info = { GEN4_FEATURES, @@ -395,11 +436,13 @@ static const struct intel_device_info gm45_info = { /* ilk does support rc6, but we do not implement [power] contexts */ \ .has_rc6 = 0, \ .dma_mask_size = 36, \ + .max_pat_index = 3, \ I9XX_PIPE_OFFSETS, \ I9XX_CURSOR_OFFSETS, \ ILK_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL static const struct intel_device_info ilk_d_info = { GEN5_FEATURES, @@ -429,13 +472,15 @@ static const struct intel_device_info ilk_m_info = { .has_rc6p = 0, \ .has_rps = true, \ .dma_mask_size = 40, \ + .max_pat_index = 3, \ .__runtime.ppgtt_type = INTEL_PPGTT_ALIASING, \ .__runtime.ppgtt_size = 31, \ I9XX_PIPE_OFFSETS, \ I9XX_CURSOR_OFFSETS, \ ILK_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL #define SNB_D_PLATFORM \ GEN6_FEATURES, \ @@ -482,13 +527,15 @@ static const struct intel_device_info snb_m_gt2_info = { .has_reset_engine = true, \ .has_rps = true, \ .dma_mask_size = 40, \ + .max_pat_index = 3, \ .__runtime.ppgtt_type = INTEL_PPGTT_ALIASING, \ .__runtime.ppgtt_size = 31, \ IVB_PIPE_OFFSETS, \ IVB_CURSOR_OFFSETS, \ IVB_COLORS, \ GEN_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL #define IVB_D_PLATFORM \ GEN7_FEATURES, \ @@ -542,6 +589,7 @@ static const struct intel_device_info vlv_info = { .display.has_gmch = 1, .display.has_hotplug = 1, .dma_mask_size = 40, + .max_pat_index = 3, .__runtime.ppgtt_type = INTEL_PPGTT_ALIASING, .__runtime.ppgtt_size = 31, .has_snoop = true, @@ -553,6 +601,7 @@ static const struct intel_device_info vlv_info = { I9XX_COLORS, GEN_DEFAULT_PAGE_SIZES, GEN_DEFAULT_REGIONS, + LEGACY_CACHELEVEL, }; #define G75_FEATURES \ @@ -640,6 +689,7 @@ static const struct intel_device_info chv_info = { .has_logical_ring_contexts = 1, .display.has_gmch = 1, .dma_mask_size = 39, + .max_pat_index = 3, .__runtime.ppgtt_type = INTEL_PPGTT_FULL, .__runtime.ppgtt_size = 32, .has_reset_engine = 1, @@ -651,6 +701,7 @@ static const struct intel_device_info chv_info = { CHV_COLORS, GEN_DEFAULT_PAGE_SIZES, GEN_DEFAULT_REGIONS, + LEGACY_CACHELEVEL, }; #define GEN9_DEFAULT_PAGE_SIZES \ @@ -732,11 +783,13 @@ static const struct intel_device_info skl_gt4_info = { .has_snoop = true, \ .has_coherent_ggtt = false, \ .display.has_ipc = 1, \ + .max_pat_index = 3, \ HSW_PIPE_OFFSETS, \ IVB_CURSOR_OFFSETS, \ IVB_COLORS, \ GEN9_DEFAULT_PAGE_SIZES, \ - GEN_DEFAULT_REGIONS + GEN_DEFAULT_REGIONS, \ + LEGACY_CACHELEVEL static const struct intel_device_info bxt_info = { GEN9_LP_FEATURES, @@ -890,9 +943,11 @@ static const struct intel_device_info jsl_info = { [TRANSCODER_DSI_1] = TRANSCODER_DSI1_OFFSET, \ }, \ TGL_CURSOR_OFFSETS, \ + TGL_CACHELEVEL, \ .has_global_mocs = 1, \ .has_pxp = 1, \ - .display.has_dsb = 1 + .display.has_dsb = 1, \ + .max_pat_index = 3 static const struct intel_device_info tgl_info = { GEN12_FEATURES, @@ -1014,6 +1069,7 @@ static const struct intel_device_info adl_p_info = { .__runtime.graphics.ip.ver = 12, \ .__runtime.graphics.ip.rel = 50, \ XE_HP_PAGE_SIZES, \ + TGL_CACHELEVEL, \ .dma_mask_size = 46, \ .has_3d_pipeline = 1, \ .has_64bit_reloc = 1, \ @@ -1032,6 +1088,7 @@ static const struct intel_device_info adl_p_info = { .has_reset_engine = 1, \ .has_rps = 1, \ .has_runtime_pm = 1, \ + .max_pat_index = 3, \ .__runtime.ppgtt_size = 48, \ .__runtime.ppgtt_type = INTEL_PPGTT_FULL @@ -1108,11 +1165,13 @@ static const struct intel_device_info pvc_info = { PLATFORM(INTEL_PONTEVECCHIO), NO_DISPLAY, .has_flat_ccs = 0, + .max_pat_index = 7, .__runtime.platform_engine_mask = BIT(BCS0) | BIT(VCS0) | BIT(CCS0) | BIT(CCS1) | BIT(CCS2) | BIT(CCS3), .require_force_probe = 1, + PVC_CACHELEVEL, }; #define XE_LPDP_FEATURES \ @@ -1152,9 +1211,11 @@ static const struct intel_device_info mtl_info = { .has_llc = 0, .has_mslice_steering = 0, .has_snoop = 1, + .max_pat_index = 4, .__runtime.memory_regions = REGION_SMEM | REGION_STOLEN_LMEM, .__runtime.platform_engine_mask = BIT(RCS0) | BIT(BCS0) | BIT(CCS0), .require_force_probe = 1, + MTL_CACHELEVEL, }; #undef PLATFORM diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h index f032f2500f50..959a4080840c 100644 --- a/drivers/gpu/drm/i915/intel_device_info.h +++ b/drivers/gpu/drm/i915/intel_device_info.h @@ -35,6 +35,8 @@ #include "gt/intel_context_types.h" #include "gt/intel_sseu.h" +#include "gem/i915_gem_object_types.h" + struct drm_printer; struct drm_i915_private; struct intel_gt_definition; @@ -308,6 +310,9 @@ struct intel_device_info { * Initial runtime info. Do not access outside of i915_driver_create(). */ const struct intel_runtime_info __runtime; + + u32 cachelevel_to_pat[I915_MAX_CACHE_LEVEL]; + u32 max_pat_index; }; struct intel_driver_caps { diff --git a/drivers/gpu/drm/i915/selftests/mock_gem_device.c b/drivers/gpu/drm/i915/selftests/mock_gem_device.c index f6a7c0bd2955..0eda8b4ee17f 100644 --- a/drivers/gpu/drm/i915/selftests/mock_gem_device.c +++ b/drivers/gpu/drm/i915/selftests/mock_gem_device.c @@ -123,7 +123,9 @@ struct drm_i915_private *mock_gem_device(void) static struct dev_iommu fake_iommu = { .priv = (void *)-1 }; #endif struct drm_i915_private *i915; + struct intel_device_info *i915_info; struct pci_dev *pdev; + unsigned int i; int ret; pdev = kzalloc(sizeof(*pdev), GFP_KERNEL); @@ -180,6 +182,13 @@ struct drm_i915_private *mock_gem_device(void) I915_GTT_PAGE_SIZE_2M; RUNTIME_INFO(i915)->memory_regions = REGION_SMEM; + + /* simply use legacy cache level for mock device */ + i915_info = (struct intel_device_info *)INTEL_INFO(i915); + i915_info->max_pat_index = 3; + for (i = 0; i < I915_MAX_CACHE_LEVEL; i++) + i915_info->cachelevel_to_pat[i] = i; + intel_memory_regions_hw_probe(i915); spin_lock_init(&i915->gpu_error.lock); -- 2.25.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 1/2] drm/i915: preparation for using PAT index fei.yang @ 2023-05-08 23:48 ` fei.yang 2023-05-09 11:08 ` Andi Shyti 2023-05-09 12:15 ` Tvrtko Ursulin 2023-05-09 1:18 ` [Intel-gfx] ✗ Fi.CI.SPARSE: warning for " Patchwork ` (2 subsequent siblings) 4 siblings, 2 replies; 10+ messages in thread From: fei.yang @ 2023-05-08 23:48 UTC (permalink / raw) To: intel-gfx; +Cc: Matt Roper, Chris Wilson, dri-devel From: Fei Yang <fei.yang@intel.com> Currently the KMD is using enum i915_cache_level to set caching policy for buffer objects. This is flaky because the PAT index which really controls the caching behavior in PTE has far more levels than what's defined in the enum. In addition, the PAT index is platform dependent, having to translate between i915_cache_level and PAT index is not reliable, and makes the code more complicated. From UMD's perspective there is also a necessity to set caching policy for performance fine tuning. It's much easier for the UMD to directly use PAT index because the behavior of each PAT index is clearly defined in Bspec. Having the abstracted i915_cache_level sitting in between would only cause more ambiguity. PAT is expected to work much like MOCS already works today, and by design userspace is expected to select the index that exactly matches the desired behavior described in the hardware specification. For these reasons this patch replaces i915_cache_level with PAT index. Also note, the cache_level is not completely removed yet, because the KMD still has the need of creating buffer objects with simple cache settings such as cached, uncached, or writethrough. For kernel objects, cache_level is used for simplicity and backward compatibility. For Pre-gen12 platforms PAT can have 1:1 mapping to i915_cache_level, so these two are interchangeable. see the use of LEGACY_CACHELEVEL. One consequence of this change is that gen8_pte_encode is no longer working for gen12 platforms due to the fact that gen12 platforms has different PAT definitions. In the meantime the mtl_pte_encode introduced specfically for MTL becomes generic for all gen12 platforms. This patch renames the MTL PTE encode function into gen12_pte_encode and apply it to all gen12. Even though this change looks unrelated, but separating them would temporarily break gen12 PTE encoding, thus squash them in one patch. Special note: this patch changes the way caching behavior is controlled in the sense that some objects are left to be managed by userspace. For such objects we need to be careful not to change the userspace settings.There are kerneldoc and comments added around obj->cache_coherent, cache_dirty, and how to bypass the checkings by i915_gem_object_has_cache_level. For full understanding, these changes need to be looked at together with the two follow-up patches, one disables the {set|get}_caching ioctl's and the other adds set_pat extension to the GEM_CREATE uAPI. Bspec: 63019 Cc: Chris Wilson <chris.p.wilson@linux.intel.com> Signed-off-by: Fei Yang <fei.yang@intel.com> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> To be squashed --- drivers/gpu/drm/i915/display/intel_dpt.c | 12 +-- drivers/gpu/drm/i915/gem/i915_gem_domain.c | 58 +++++++++---- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 15 +++- drivers/gpu/drm/i915/gem/i915_gem_mman.c | 11 ++- drivers/gpu/drm/i915/gem/i915_gem_object.c | 51 ++++++++++- drivers/gpu/drm/i915/gem/i915_gem_object.h | 4 + .../gpu/drm/i915/gem/i915_gem_object_types.h | 44 +++++++++- drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 4 +- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 8 +- .../gpu/drm/i915/gem/selftests/huge_pages.c | 2 +- .../drm/i915/gem/selftests/i915_gem_migrate.c | 2 +- .../drm/i915/gem/selftests/i915_gem_mman.c | 2 +- drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 10 ++- drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 84 +++++++++---------- drivers/gpu/drm/i915/gt/gen8_ppgtt.h | 3 +- drivers/gpu/drm/i915/gt/intel_ggtt.c | 82 +++++++++--------- drivers/gpu/drm/i915/gt/intel_gtt.h | 18 ++-- drivers/gpu/drm/i915/gt/intel_migrate.c | 47 ++++++----- drivers/gpu/drm/i915/gt/intel_migrate.h | 13 ++- drivers/gpu/drm/i915/gt/intel_ppgtt.c | 4 +- drivers/gpu/drm/i915/gt/selftest_migrate.c | 47 ++++++----- drivers/gpu/drm/i915/gt/selftest_reset.c | 8 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/selftest_tlb.c | 4 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 10 ++- drivers/gpu/drm/i915/i915_debugfs.c | 53 +++++++++--- drivers/gpu/drm/i915/i915_gem.c | 27 +++++- drivers/gpu/drm/i915/i915_gpu_error.c | 8 +- drivers/gpu/drm/i915/i915_vma.c | 16 ++-- drivers/gpu/drm/i915/i915_vma.h | 2 +- drivers/gpu/drm/i915/i915_vma_types.h | 2 - drivers/gpu/drm/i915/selftests/i915_gem.c | 5 +- .../gpu/drm/i915/selftests/i915_gem_evict.c | 4 +- drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 15 ++-- .../drm/i915/selftests/intel_memory_region.c | 4 +- drivers/gpu/drm/i915/selftests/mock_gtt.c | 8 +- 36 files changed, 451 insertions(+), 238 deletions(-) diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c index c5eacfdba1a5..7c5fddb203ba 100644 --- a/drivers/gpu/drm/i915/display/intel_dpt.c +++ b/drivers/gpu/drm/i915/display/intel_dpt.c @@ -43,24 +43,24 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) static void dpt_insert_page(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_dpt *dpt = i915_vm_to_dpt(vm); gen8_pte_t __iomem *base = dpt->iomem; gen8_set_pte(base + offset / I915_GTT_PAGE_SIZE, - vm->pte_encode(addr, level, flags)); + vm->pte_encode(addr, pat_index, flags)); } static void dpt_insert_entries(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_dpt *dpt = i915_vm_to_dpt(vm); gen8_pte_t __iomem *base = dpt->iomem; - const gen8_pte_t pte_encode = vm->pte_encode(0, level, flags); + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); struct sgt_iter sgt_iter; dma_addr_t addr; int i; @@ -83,7 +83,7 @@ static void dpt_clear_range(struct i915_address_space *vm, static void dpt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { u32 pte_flags; @@ -98,7 +98,7 @@ static void dpt_bind_vma(struct i915_address_space *vm, if (vma_res->bi.lmem) pte_flags |= PTE_LM; - vm->insert_entries(vm, vma_res, cache_level, pte_flags); + vm->insert_entries(vm, vma_res, pat_index, pte_flags); vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c index d2d5a24301b2..05107a6efe45 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c @@ -27,8 +27,15 @@ static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj) if (IS_DGFX(i915)) return false; - return !(obj->cache_level == I915_CACHE_NONE || - obj->cache_level == I915_CACHE_WT); + /* + * For objects created by userspace through GEM_CREATE with pat_index + * set by set_pat extension, i915_gem_object_has_cache_level() will + * always return true, because the coherency of such object is managed + * by userspace. Othereise the call here would fall back to checking + * whether the object is un-cached or write-through. + */ + return !(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || + i915_gem_object_has_cache_level(obj, I915_CACHE_WT)); } bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj) @@ -267,7 +274,13 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, { int ret; - if (obj->cache_level == cache_level) + /* + * For objects created by userspace through GEM_CREATE with pat_index + * set by set_pat extension, simply return 0 here without touching + * the cache setting, because such objects should have an immutable + * cache setting by desgin and always managed by userspace. + */ + if (i915_gem_object_has_cache_level(obj, cache_level)) return 0; ret = i915_gem_object_wait(obj, @@ -278,10 +291,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, return ret; /* Always invalidate stale cachelines */ - if (obj->cache_level != cache_level) { - i915_gem_object_set_cache_coherency(obj, cache_level); - obj->cache_dirty = true; - } + i915_gem_object_set_cache_coherency(obj, cache_level); + obj->cache_dirty = true; /* The cache-level will be applied when each vma is rebound. */ return i915_gem_object_unbind(obj, @@ -306,20 +317,22 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data, goto out; } - switch (obj->cache_level) { - case I915_CACHE_LLC: - case I915_CACHE_L3_LLC: - args->caching = I915_CACHING_CACHED; - break; + /* + * This ioctl should be disabled for the objects with pat_index + * set by user space. + */ + if (obj->pat_set_by_user) { + err = -EOPNOTSUPP; + goto out; + } - case I915_CACHE_WT: + if (i915_gem_object_has_cache_level(obj, I915_CACHE_LLC) || + i915_gem_object_has_cache_level(obj, I915_CACHE_L3_LLC)) + args->caching = I915_CACHING_CACHED; + else if (i915_gem_object_has_cache_level(obj, I915_CACHE_WT)) args->caching = I915_CACHING_DISPLAY; - break; - - default: + else args->caching = I915_CACHING_NONE; - break; - } out: rcu_read_unlock(); return err; @@ -364,6 +377,15 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data, if (!obj) return -ENOENT; + /* + * This ioctl should be disabled for the objects with pat_index + * set by user space. + */ + if (obj->pat_set_by_user) { + ret = -EOPNOTSUPP; + goto out; + } + /* * The caching mode of proxy object is handled by its generator, and * not allowed to be changed by userspace. diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 3aeede6aee4d..5fb459ea4294 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -640,9 +640,15 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache, if (DBG_FORCE_RELOC == FORCE_GTT_RELOC) return false; + /* + * For objects created by userspace through GEM_CREATE with pat_index + * set by set_pat extension, i915_gem_object_has_cache_level() always + * return true, otherwise the call would fall back to checking whether + * the object is un-cached. + */ return (cache->has_llc || obj->cache_dirty || - obj->cache_level != I915_CACHE_NONE); + !i915_gem_object_has_cache_level(obj, I915_CACHE_NONE)); } static int eb_reserve_vma(struct i915_execbuffer *eb, @@ -1324,7 +1330,10 @@ static void *reloc_iomap(struct i915_vma *batch, if (drm_mm_node_allocated(&cache->node)) { ggtt->vm.insert_page(&ggtt->vm, i915_gem_object_get_dma_address(obj, page), - offset, I915_CACHE_NONE, 0); + offset, + i915_gem_get_pat_index(ggtt->vm.i915, + I915_CACHE_NONE), + 0); } else { offset += page << PAGE_SHIFT; } @@ -1464,7 +1473,7 @@ eb_relocate_entry(struct i915_execbuffer *eb, reloc_cache_unmap(&eb->reloc_cache); mutex_lock(&vma->vm->mutex); err = i915_vma_bind(target->vma, - target->vma->obj->cache_level, + target->vma->obj->pat_index, PIN_GLOBAL, NULL, NULL); mutex_unlock(&vma->vm->mutex); reloc_cache_remap(&eb->reloc_cache, ev->vma->obj); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c index 3dbacdf0911a..4e7a838ab7bd 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c @@ -383,7 +383,16 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf) } /* Access to snoopable pages through the GTT is incoherent. */ - if (obj->cache_level != I915_CACHE_NONE && !HAS_LLC(i915)) { + /* + * For objects created by userspace through GEM_CREATE with pat_index + * set by set_pat extension, coherency is managed by userspace, make + * sure we don't fail handling the vm fault by calling + * i915_gem_object_has_cache_level() which always return true for such + * objects. Otherwise this helper function would fall back to checking + * whether the object is un-cached. + */ + if (!(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || + HAS_LLC(i915))) { ret = -EFAULT; goto err_unpin; } diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c index 8c70a0ec7d2f..46a19b099ec8 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c @@ -54,6 +54,24 @@ unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, return INTEL_INFO(i915)->cachelevel_to_pat[level]; } +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, + enum i915_cache_level lvl) +{ + /* + * In case the pat_index is set by user space, this kernel mode + * driver should leave the coherency to be managed by user space, + * simply return true here. + */ + if (obj->pat_set_by_user) + return true; + + /* + * Otherwise the pat_index should have been converted from cache_level + * so that the following comparison is valid. + */ + return obj->pat_index == i915_gem_get_pat_index(obj_to_i915(obj), lvl); +} + struct drm_i915_gem_object *i915_gem_object_alloc(void) { struct drm_i915_gem_object *obj; @@ -133,7 +151,7 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, { struct drm_i915_private *i915 = to_i915(obj->base.dev); - obj->cache_level = cache_level; + obj->pat_index = i915_gem_get_pat_index(i915, cache_level); if (cache_level != I915_CACHE_NONE) obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | @@ -148,6 +166,37 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, !IS_DGFX(i915); } +/** + * i915_gem_object_set_pat_index - set PAT index to be used in PTE encode + * @obj: #drm_i915_gem_object + * @pat_index: PAT index + * + * This is a clone of i915_gem_object_set_cache_coherency taking pat index + * instead of cache_level as its second argument. + */ +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, + unsigned int pat_index) +{ + struct drm_i915_private *i915 = to_i915(obj->base.dev); + + if (obj->pat_index == pat_index) + return; + + obj->pat_index = pat_index; + + if (pat_index != i915_gem_get_pat_index(i915, I915_CACHE_NONE)) + obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | + I915_BO_CACHE_COHERENT_FOR_WRITE); + else if (HAS_LLC(i915)) + obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ; + else + obj->cache_coherent = 0; + + obj->cache_dirty = + !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) && + !IS_DGFX(i915); +} + bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj) { struct drm_i915_private *i915 = to_i915(obj->base.dev); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h index 284e1aa396cd..884a17275b3a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h @@ -34,6 +34,8 @@ static inline bool i915_gem_object_size_2big(u64 size) unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, enum i915_cache_level level); +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, + enum i915_cache_level lvl); void i915_gem_init__objects(struct drm_i915_private *i915); void i915_objects_module_exit(void); @@ -764,6 +766,8 @@ bool i915_gem_object_has_unknown_state(struct drm_i915_gem_object *obj); void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, unsigned int cache_level); +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, + unsigned int pat_index); bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj); void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj); void i915_gem_object_flush_if_display_locked(struct drm_i915_gem_object *obj); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index bf0bd8e11355..adf4c2b28135 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -364,15 +364,42 @@ struct drm_i915_gem_object { #define I915_BO_FLAG_STRUCT_PAGE BIT(0) /* Object backed by struct pages */ #define I915_BO_FLAG_IOMEM BIT(1) /* Object backed by IO memory */ /** - * @cache_level: The desired GTT caching level. + * @pat_index: The desired PAT index. + * + * See hardware specification for valid PAT indices for each platform. + * This field used to contain a value of enum i915_cache_level. It's + * changed to an unsigned int because PAT indices are being used by + * both UMD and KMD for caching policy control after GEN12. + * For backward compatibility, this field will continue to contain + * value of i915_cache_level for pre-GEN12 platforms so that the PTE + * encode functions for these legacy platforms can stay the same. + * In the meantime platform specific tables are created to translate + * i915_cache_level into pat index, for more details check the macros + * defined i915/i915_pci.c, e.g. PVC_CACHELEVEL. + */ + unsigned int pat_index:6; + /** + * @pat_set_by_user: Indicate whether pat_index is set by user space * - * See enum i915_cache_level for possible values, along with what - * each does. + * This field is set to false by default, only set to true if the + * pat_index is set by user space. By design, user space is capable of + * managing caching behavior by setting pat_index, in which case this + * kernel mode driver should never touch the pat_index. */ - unsigned int cache_level:3; + unsigned int pat_set_by_user; /** * @cache_coherent: * + * Note: with the change above which replaced cache_level with + * pat_index, the use of cache_coherent is limited to objects created + * by kernel. Check for @pat_set_by_user to find out if an object is + * created by userspace or kernel. A later patch will disable the + * ioctl's to change cache settings for objects with pat_index set by + * userspace. Please don't assume @cache_coherent having the flags set + * as describe here for these objects. Also see the helper function + * i915_gem_object_has_cache_level() for possible ways to bypass the + * use of this field. + * * Track whether the pages are coherent with the GPU if reading or * writing through the CPU caches. The largely depends on the * @cache_level setting. @@ -446,6 +473,15 @@ struct drm_i915_gem_object { /** * @cache_dirty: * + * Note: with the change above which replaced cache_level with + * pat_index, the use of cache_dirty is limited to objects created + * by kernel. Check for @pat_set_by_user to find out if an object is + * created by userspace or kernel. A later patch will disable the + * ioctl's to change cache settings for objects with pat_index set by + * userspace. Please don't assume @cache_dirty is set as describe here + * for these objects. Also see i915_gem_object_has_cache_level() for + * possible ways to bypass the use of this field. + * * Track if we are we dirty with writes through the CPU cache for this * object. As a result reading directly from main memory might yield * stale data. diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c index ee492d823f1b..3b094d36a0b0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c @@ -565,7 +565,9 @@ static void dbg_poison(struct i915_ggtt *ggtt, ggtt->vm.insert_page(&ggtt->vm, addr, ggtt->error_capture.start, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(ggtt->vm.i915, + I915_CACHE_NONE), + 0); mb(); s = io_mapping_map_wc(&ggtt->iomap, diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index 69eb20ed4d47..7078af2f8f79 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -214,7 +214,8 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, intel_engine_pm_get(to_gt(i915)->migrate.context->engine); ret = intel_context_migrate_clear(to_gt(i915)->migrate.context, deps, - dst_st->sgl, dst_level, + dst_st->sgl, + i915_gem_get_pat_index(i915, dst_level), i915_ttm_gtt_binds_lmem(dst_mem), 0, &rq); } else { @@ -228,9 +229,10 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, intel_engine_pm_get(to_gt(i915)->migrate.context->engine); ret = intel_context_migrate_copy(to_gt(i915)->migrate.context, deps, src_rsgt->table.sgl, - src_level, + i915_gem_get_pat_index(i915, src_level), i915_ttm_gtt_binds_lmem(bo->resource), - dst_st->sgl, dst_level, + dst_st->sgl, + i915_gem_get_pat_index(i915, dst_level), i915_ttm_gtt_binds_lmem(dst_mem), &rq); diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c index d1057574e966..df6c9a84252c 100644 --- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c +++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c @@ -354,7 +354,7 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single) obj->write_domain = I915_GEM_DOMAIN_CPU; obj->read_domains = I915_GEM_DOMAIN_CPU; - obj->cache_level = I915_CACHE_NONE; + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); return obj; } diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c index fe6c37fd7859..a93a90b15907 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c @@ -219,7 +219,7 @@ static int __igt_lmem_pages_migrate(struct intel_gt *gt, continue; err = intel_migrate_clear(>->migrate, &ww, deps, - obj->mm.pages->sgl, obj->cache_level, + obj->mm.pages->sgl, obj->pat_index, i915_gem_object_is_lmem(obj), 0xdeadbeaf, &rq); if (rq) { diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c index 56279908ed30..a93d8f9f8bc1 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c @@ -1222,7 +1222,7 @@ static int __igt_mmap_migrate(struct intel_memory_region **placements, } err = intel_context_migrate_clear(to_gt(i915)->migrate.context, NULL, - obj->mm.pages->sgl, obj->cache_level, + obj->mm.pages->sgl, obj->pat_index, i915_gem_object_is_lmem(obj), expand32(POISON_INUSE), &rq); i915_gem_object_unpin_pages(obj); diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c index 5aaacc53fa4c..c2bdc133c89a 100644 --- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c @@ -109,7 +109,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm, static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); @@ -117,7 +117,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, unsigned int first_entry = vma_res->start / I915_GTT_PAGE_SIZE; unsigned int act_pt = first_entry / GEN6_PTES; unsigned int act_pte = first_entry % GEN6_PTES; - const u32 pte_encode = vm->pte_encode(0, cache_level, flags); + const u32 pte_encode = vm->pte_encode(0, pat_index, flags); struct sgt_dma iter = sgt_dma(vma_res); gen6_pte_t *vaddr; @@ -227,7 +227,9 @@ static int gen6_ppgtt_init_scratch(struct gen6_ppgtt *ppgtt) vm->scratch[0]->encode = vm->pte_encode(px_dma(vm->scratch[0]), - I915_CACHE_NONE, PTE_READ_ONLY); + i915_gem_get_pat_index(vm->i915, + I915_CACHE_NONE), + PTE_READ_ONLY); vm->scratch[1] = vm->alloc_pt_dma(vm, I915_GTT_PAGE_SIZE_4K); if (IS_ERR(vm->scratch[1])) { @@ -278,7 +280,7 @@ static void gen6_ppgtt_cleanup(struct i915_address_space *vm) static void pd_vma_bind(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 unused) { struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c index bb6998d67133..f948d33e5ec5 100644 --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c @@ -29,7 +29,7 @@ static u64 gen8_pde_encode(const dma_addr_t addr, } static u64 gen8_pte_encode(dma_addr_t addr, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; @@ -40,7 +40,12 @@ static u64 gen8_pte_encode(dma_addr_t addr, if (flags & PTE_LM) pte |= GEN12_PPGTT_PTE_LM; - switch (level) { + /* + * For pre-gen12 platforms pat_index is the same as enum + * i915_cache_level, so the switch-case here is still valid. + * See translation table defined by LEGACY_CACHELEVEL. + */ + switch (pat_index) { case I915_CACHE_NONE: pte |= PPAT_UNCACHED; break; @@ -55,9 +60,9 @@ static u64 gen8_pte_encode(dma_addr_t addr, return pte; } -static u64 mtl_pte_encode(dma_addr_t addr, - enum i915_cache_level level, - u32 flags) +static u64 gen12_pte_encode(dma_addr_t addr, + unsigned int pat_index, + u32 flags) { gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; @@ -67,24 +72,17 @@ static u64 mtl_pte_encode(dma_addr_t addr, if (flags & PTE_LM) pte |= GEN12_PPGTT_PTE_LM; - switch (level) { - case I915_CACHE_NONE: - pte |= GEN12_PPGTT_PTE_PAT1; - break; - case I915_CACHE_LLC: - case I915_CACHE_L3_LLC: - pte |= GEN12_PPGTT_PTE_PAT0 | GEN12_PPGTT_PTE_PAT1; - break; - case I915_CACHE_WT: + if (pat_index & BIT(0)) pte |= GEN12_PPGTT_PTE_PAT0; - break; - default: - /* This should never happen. Added to deal with the compile - * error due to the addition of I915_MAX_CACHE_LEVEL. Will - * be removed by the pat_index patch. - */ - break; - } + + if (pat_index & BIT(1)) + pte |= GEN12_PPGTT_PTE_PAT1; + + if (pat_index & BIT(2)) + pte |= GEN12_PPGTT_PTE_PAT2; + + if (pat_index & BIT(3)) + pte |= MTL_PPGTT_PTE_PAT3; return pte; } @@ -457,11 +455,11 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt, struct i915_page_directory *pdp, struct sgt_dma *iter, u64 idx, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { struct i915_page_directory *pd; - const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, cache_level, flags); + const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, pat_index, flags); gen8_pte_t *vaddr; pd = i915_pd_entry(pdp, gen8_pd_index(idx, 2)); @@ -504,10 +502,10 @@ static void xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, struct i915_vma_resource *vma_res, struct sgt_dma *iter, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); unsigned int rem = sg_dma_len(iter->sg); u64 start = vma_res->start; u64 end = start + vma_res->vma_size; @@ -612,10 +610,10 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, struct i915_vma_resource *vma_res, struct sgt_dma *iter, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); unsigned int rem = sg_dma_len(iter->sg); u64 start = vma_res->start; @@ -735,7 +733,7 @@ static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, static void gen8_ppgtt_insert(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(vm); @@ -743,9 +741,9 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, if (vma_res->bi.page_sizes.sg > I915_GTT_PAGE_SIZE) { if (GRAPHICS_VER_FULL(vm->i915) >= IP_VER(12, 50)) - xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); + xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); else - gen8_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); + gen8_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); } else { u64 idx = vma_res->start >> GEN8_PTE_SHIFT; @@ -754,7 +752,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, gen8_pdp_for_page_index(vm, idx); idx = gen8_ppgtt_insert_pte(ppgtt, pdp, &iter, idx, - cache_level, flags); + pat_index, flags); } while (idx); vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; @@ -764,7 +762,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { u64 idx = offset >> GEN8_PTE_SHIFT; @@ -778,14 +776,14 @@ static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, GEM_BUG_ON(pt->is_compact); vaddr = px_vaddr(pt); - vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, level, flags); + vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, pat_index, flags); drm_clflush_virt_range(&vaddr[gen8_pd_index(idx, 0)], sizeof(*vaddr)); } static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { u64 idx = offset >> GEN8_PTE_SHIFT; @@ -808,20 +806,20 @@ static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, } vaddr = px_vaddr(pt); - vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, level, flags); + vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, pat_index, flags); } static void xehpsdv_ppgtt_insert_entry(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { if (flags & PTE_LM) return __xehpsdv_ppgtt_insert_entry_lm(vm, addr, offset, - level, flags); + pat_index, flags); - return gen8_ppgtt_insert_entry(vm, addr, offset, level, flags); + return gen8_ppgtt_insert_entry(vm, addr, offset, pat_index, flags); } static int gen8_init_scratch(struct i915_address_space *vm) @@ -856,7 +854,9 @@ static int gen8_init_scratch(struct i915_address_space *vm) vm->scratch[0]->encode = vm->pte_encode(px_dma(vm->scratch[0]), - I915_CACHE_NONE, pte_flags); + i915_gem_get_pat_index(vm->i915, + I915_CACHE_NONE), + pte_flags); for (i = 1; i <= vm->top; i++) { struct drm_i915_gem_object *obj; @@ -998,8 +998,8 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, */ ppgtt->vm.alloc_scratch_dma = alloc_pt_dma; - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70)) - ppgtt->vm.pte_encode = mtl_pte_encode; + if (GRAPHICS_VER(gt->i915) >= 12) + ppgtt->vm.pte_encode = gen12_pte_encode; else ppgtt->vm.pte_encode = gen8_pte_encode; diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h index f541d19264b4..19c635441642 100644 --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h @@ -10,13 +10,12 @@ struct i915_address_space; struct intel_gt; -enum i915_cache_level; struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, unsigned long lmem_pt_obj_flags); u64 gen8_ggtt_pte_encode(dma_addr_t addr, - enum i915_cache_level level, + unsigned int pat_index, u32 flags); #endif diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c index c8390d03fce2..2a7942fac798 100644 --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c @@ -221,7 +221,7 @@ static void guc_ggtt_invalidate(struct i915_ggtt *ggtt) } static u64 mtl_ggtt_pte_encode(dma_addr_t addr, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; @@ -231,30 +231,17 @@ static u64 mtl_ggtt_pte_encode(dma_addr_t addr, if (flags & PTE_LM) pte |= GEN12_GGTT_PTE_LM; - switch (level) { - case I915_CACHE_NONE: - pte |= MTL_GGTT_PTE_PAT1; - break; - case I915_CACHE_LLC: - case I915_CACHE_L3_LLC: - pte |= MTL_GGTT_PTE_PAT0 | MTL_GGTT_PTE_PAT1; - break; - case I915_CACHE_WT: + if (pat_index & BIT(0)) pte |= MTL_GGTT_PTE_PAT0; - break; - default: - /* This should never happen. Added to deal with the compile - * error due to the addition of I915_MAX_CACHE_LEVEL. Will - * be removed by the pat_index patch. - */ - break; - } + + if (pat_index & BIT(1)) + pte |= MTL_GGTT_PTE_PAT1; return pte; } u64 gen8_ggtt_pte_encode(dma_addr_t addr, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; @@ -273,25 +260,25 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) static void gen8_ggtt_insert_page(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); gen8_pte_t __iomem *pte = (gen8_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; - gen8_set_pte(pte, ggtt->vm.pte_encode(addr, level, flags)); + gen8_set_pte(pte, ggtt->vm.pte_encode(addr, pat_index, flags)); ggtt->invalidate(ggtt); } static void gen8_ggtt_insert_entries(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); - const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, level, flags); + const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, pat_index, flags); gen8_pte_t __iomem *gte; gen8_pte_t __iomem *end; struct sgt_iter iter; @@ -348,14 +335,14 @@ static void gen8_ggtt_clear_range(struct i915_address_space *vm, static void gen6_ggtt_insert_page(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); gen6_pte_t __iomem *pte = (gen6_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; - iowrite32(vm->pte_encode(addr, level, flags), pte); + iowrite32(vm->pte_encode(addr, pat_index, flags), pte); ggtt->invalidate(ggtt); } @@ -368,7 +355,7 @@ static void gen6_ggtt_insert_page(struct i915_address_space *vm, */ static void gen6_ggtt_insert_entries(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); @@ -385,7 +372,7 @@ static void gen6_ggtt_insert_entries(struct i915_address_space *vm, iowrite32(vm->scratch[0]->encode, gte++); end += (vma_res->node_size + vma_res->guard) / I915_GTT_PAGE_SIZE; for_each_sgt_daddr(addr, iter, vma_res->bi.pages) - iowrite32(vm->pte_encode(addr, level, flags), gte++); + iowrite32(vm->pte_encode(addr, pat_index, flags), gte++); GEM_BUG_ON(gte > end); /* Fill the allocated but "unused" space beyond the end of the buffer */ @@ -420,14 +407,15 @@ struct insert_page { struct i915_address_space *vm; dma_addr_t addr; u64 offset; - enum i915_cache_level level; + unsigned int pat_index; }; static int bxt_vtd_ggtt_insert_page__cb(void *_arg) { struct insert_page *arg = _arg; - gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, arg->level, 0); + gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, + arg->pat_index, 0); bxt_vtd_ggtt_wa(arg->vm); return 0; @@ -436,10 +424,10 @@ static int bxt_vtd_ggtt_insert_page__cb(void *_arg) static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 unused) { - struct insert_page arg = { vm, addr, offset, level }; + struct insert_page arg = { vm, addr, offset, pat_index }; stop_machine(bxt_vtd_ggtt_insert_page__cb, &arg, NULL); } @@ -447,7 +435,7 @@ static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, struct insert_entries { struct i915_address_space *vm; struct i915_vma_resource *vma_res; - enum i915_cache_level level; + unsigned int pat_index; u32 flags; }; @@ -455,7 +443,8 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) { struct insert_entries *arg = _arg; - gen8_ggtt_insert_entries(arg->vm, arg->vma_res, arg->level, arg->flags); + gen8_ggtt_insert_entries(arg->vm, arg->vma_res, + arg->pat_index, arg->flags); bxt_vtd_ggtt_wa(arg->vm); return 0; @@ -463,10 +452,10 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) static void bxt_vtd_ggtt_insert_entries__BKL(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { - struct insert_entries arg = { vm, vma_res, level, flags }; + struct insert_entries arg = { vm, vma_res, pat_index, flags }; stop_machine(bxt_vtd_ggtt_insert_entries__cb, &arg, NULL); } @@ -495,7 +484,7 @@ static void gen6_ggtt_clear_range(struct i915_address_space *vm, void intel_ggtt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { u32 pte_flags; @@ -512,7 +501,7 @@ void intel_ggtt_bind_vma(struct i915_address_space *vm, if (vma_res->bi.lmem) pte_flags |= PTE_LM; - vm->insert_entries(vm, vma_res, cache_level, pte_flags); + vm->insert_entries(vm, vma_res, pat_index, pte_flags); vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; } @@ -661,7 +650,7 @@ static int init_ggtt(struct i915_ggtt *ggtt) static void aliasing_gtt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { u32 pte_flags; @@ -673,10 +662,10 @@ static void aliasing_gtt_bind_vma(struct i915_address_space *vm, if (flags & I915_VMA_LOCAL_BIND) ppgtt_bind_vma(&i915_vm_to_ggtt(vm)->alias->vm, - stash, vma_res, cache_level, flags); + stash, vma_res, pat_index, flags); if (flags & I915_VMA_GLOBAL_BIND) - vm->insert_entries(vm, vma_res, cache_level, pte_flags); + vm->insert_entries(vm, vma_res, pat_index, pte_flags); vma_res->bound_flags |= flags; } @@ -933,7 +922,9 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size) ggtt->vm.scratch[0]->encode = ggtt->vm.pte_encode(px_dma(ggtt->vm.scratch[0]), - I915_CACHE_NONE, pte_flags); + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), + pte_flags); return 0; } @@ -1022,6 +1013,11 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt) return ggtt_probe_common(ggtt, size); } +/* + * For pre-gen8 platforms pat_index is the same as enum i915_cache_level, + * so these PTE encode functions are left with using cache_level. + * See translation table LEGACY_CACHELEVEL. + */ static u64 snb_pte_encode(dma_addr_t addr, enum i915_cache_level level, u32 flags) @@ -1302,7 +1298,9 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) */ vma->resource->bound_flags = 0; vma->ops->bind_vma(vm, NULL, vma->resource, - obj ? obj->cache_level : 0, + obj ? obj->pat_index : + i915_gem_get_pat_index(vm->i915, + I915_CACHE_NONE), was_bound); if (obj) { /* only used during resume => exclusive access */ diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h index 9aff343beaa8..4d6296cdbcfd 100644 --- a/drivers/gpu/drm/i915/gt/intel_gtt.h +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h @@ -163,8 +163,6 @@ typedef u64 gen8_pte_t; #define MTL_3_COH_2W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 3) #define MTL_2_COH_1W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 2) -enum i915_cache_level; - struct drm_i915_gem_object; struct i915_fence_reg; struct i915_vma; @@ -232,7 +230,7 @@ struct i915_vma_ops { void (*bind_vma)(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); /* * Unmap an object from an address space. This usually consists of @@ -304,7 +302,7 @@ struct i915_address_space { (*alloc_scratch_dma)(struct i915_address_space *vm, int sz); u64 (*pte_encode)(dma_addr_t addr, - enum i915_cache_level level, + unsigned int pat_index, u32 flags); /* Create a valid PTE */ #define PTE_READ_ONLY BIT(0) #define PTE_LM BIT(1) @@ -319,20 +317,20 @@ struct i915_address_space { void (*insert_page)(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void (*insert_entries)(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void (*raw_insert_page)(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void (*raw_insert_entries)(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void (*cleanup)(struct i915_address_space *vm); @@ -579,7 +577,7 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt, void intel_ggtt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void intel_ggtt_unbind_vma(struct i915_address_space *vm, struct i915_vma_resource *vma_res); @@ -657,7 +655,7 @@ void gen6_ggtt_invalidate(struct i915_ggtt *ggtt); void ppgtt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags); void ppgtt_unbind_vma(struct i915_address_space *vm, struct i915_vma_resource *vma_res); diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c index e0998879a0e1..6023288b0e2d 100644 --- a/drivers/gpu/drm/i915/gt/intel_migrate.c +++ b/drivers/gpu/drm/i915/gt/intel_migrate.c @@ -45,7 +45,9 @@ static void xehpsdv_toggle_pdes(struct i915_address_space *vm, * Insert a dummy PTE into every PT that will map to LMEM to ensure * we have a correctly setup PDE structure for later use. */ - vm->insert_page(vm, 0, d->offset, I915_CACHE_NONE, PTE_LM); + vm->insert_page(vm, 0, d->offset, + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), + PTE_LM); GEM_BUG_ON(!pt->is_compact); d->offset += SZ_2M; } @@ -63,7 +65,9 @@ static void xehpsdv_insert_pte(struct i915_address_space *vm, * alignment is 64K underneath for the pt, and we are careful * not to access the space in the void. */ - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, PTE_LM); + vm->insert_page(vm, px_dma(pt), d->offset, + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), + PTE_LM); d->offset += SZ_64K; } @@ -73,7 +77,8 @@ static void insert_pte(struct i915_address_space *vm, { struct insert_pte_data *d = data; - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, + vm->insert_page(vm, px_dma(pt), d->offset, + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), i915_gem_object_is_lmem(pt->base) ? PTE_LM : 0); d->offset += PAGE_SIZE; } @@ -356,13 +361,13 @@ static int max_pte_pkt_size(struct i915_request *rq, int pkt) static int emit_pte(struct i915_request *rq, struct sgt_dma *it, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, u64 offset, int length) { bool has_64K_pages = HAS_64K_PAGES(rq->engine->i915); - const u64 encode = rq->context->vm->pte_encode(0, cache_level, + const u64 encode = rq->context->vm->pte_encode(0, pat_index, is_lmem ? PTE_LM : 0); struct intel_ring *ring = rq->ring; int pkt, dword_length; @@ -673,17 +678,17 @@ int intel_context_migrate_copy(struct intel_context *ce, const struct i915_deps *deps, struct scatterlist *src, - enum i915_cache_level src_cache_level, + unsigned int src_pat_index, bool src_is_lmem, struct scatterlist *dst, - enum i915_cache_level dst_cache_level, + unsigned int dst_pat_index, bool dst_is_lmem, struct i915_request **out) { struct sgt_dma it_src = sg_sgt(src), it_dst = sg_sgt(dst), it_ccs; struct drm_i915_private *i915 = ce->engine->i915; u64 ccs_bytes_to_cpy = 0, bytes_to_cpy; - enum i915_cache_level ccs_cache_level; + unsigned int ccs_pat_index; u32 src_offset, dst_offset; u8 src_access, dst_access; struct i915_request *rq; @@ -707,12 +712,12 @@ intel_context_migrate_copy(struct intel_context *ce, dst_sz = scatter_list_length(dst); if (src_is_lmem) { it_ccs = it_dst; - ccs_cache_level = dst_cache_level; + ccs_pat_index = dst_pat_index; ccs_is_src = false; } else if (dst_is_lmem) { bytes_to_cpy = dst_sz; it_ccs = it_src; - ccs_cache_level = src_cache_level; + ccs_pat_index = src_pat_index; ccs_is_src = true; } @@ -773,7 +778,7 @@ intel_context_migrate_copy(struct intel_context *ce, src_sz = calculate_chunk_sz(i915, src_is_lmem, bytes_to_cpy, ccs_bytes_to_cpy); - len = emit_pte(rq, &it_src, src_cache_level, src_is_lmem, + len = emit_pte(rq, &it_src, src_pat_index, src_is_lmem, src_offset, src_sz); if (!len) { err = -EINVAL; @@ -784,7 +789,7 @@ intel_context_migrate_copy(struct intel_context *ce, goto out_rq; } - err = emit_pte(rq, &it_dst, dst_cache_level, dst_is_lmem, + err = emit_pte(rq, &it_dst, dst_pat_index, dst_is_lmem, dst_offset, len); if (err < 0) goto out_rq; @@ -811,7 +816,7 @@ intel_context_migrate_copy(struct intel_context *ce, goto out_rq; ccs_sz = GET_CCS_BYTES(i915, len); - err = emit_pte(rq, &it_ccs, ccs_cache_level, false, + err = emit_pte(rq, &it_ccs, ccs_pat_index, false, ccs_is_src ? src_offset : dst_offset, ccs_sz); if (err < 0) @@ -979,7 +984,7 @@ int intel_context_migrate_clear(struct intel_context *ce, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, u32 value, struct i915_request **out) @@ -1027,7 +1032,7 @@ intel_context_migrate_clear(struct intel_context *ce, if (err) goto out_rq; - len = emit_pte(rq, &it, cache_level, is_lmem, offset, CHUNK_SZ); + len = emit_pte(rq, &it, pat_index, is_lmem, offset, CHUNK_SZ); if (len <= 0) { err = len; goto out_rq; @@ -1074,10 +1079,10 @@ int intel_migrate_copy(struct intel_migrate *m, struct i915_gem_ww_ctx *ww, const struct i915_deps *deps, struct scatterlist *src, - enum i915_cache_level src_cache_level, + unsigned int src_pat_index, bool src_is_lmem, struct scatterlist *dst, - enum i915_cache_level dst_cache_level, + unsigned int dst_pat_index, bool dst_is_lmem, struct i915_request **out) { @@ -1098,8 +1103,8 @@ int intel_migrate_copy(struct intel_migrate *m, goto out; err = intel_context_migrate_copy(ce, deps, - src, src_cache_level, src_is_lmem, - dst, dst_cache_level, dst_is_lmem, + src, src_pat_index, src_is_lmem, + dst, dst_pat_index, dst_is_lmem, out); intel_context_unpin(ce); @@ -1113,7 +1118,7 @@ intel_migrate_clear(struct intel_migrate *m, struct i915_gem_ww_ctx *ww, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, u32 value, struct i915_request **out) @@ -1134,7 +1139,7 @@ intel_migrate_clear(struct intel_migrate *m, if (err) goto out; - err = intel_context_migrate_clear(ce, deps, sg, cache_level, + err = intel_context_migrate_clear(ce, deps, sg, pat_index, is_lmem, value, out); intel_context_unpin(ce); diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.h b/drivers/gpu/drm/i915/gt/intel_migrate.h index ccc677ec4aa3..11fc09a00c4b 100644 --- a/drivers/gpu/drm/i915/gt/intel_migrate.h +++ b/drivers/gpu/drm/i915/gt/intel_migrate.h @@ -16,7 +16,6 @@ struct i915_request; struct i915_gem_ww_ctx; struct intel_gt; struct scatterlist; -enum i915_cache_level; int intel_migrate_init(struct intel_migrate *m, struct intel_gt *gt); @@ -26,20 +25,20 @@ int intel_migrate_copy(struct intel_migrate *m, struct i915_gem_ww_ctx *ww, const struct i915_deps *deps, struct scatterlist *src, - enum i915_cache_level src_cache_level, + unsigned int src_pat_index, bool src_is_lmem, struct scatterlist *dst, - enum i915_cache_level dst_cache_level, + unsigned int dst_pat_index, bool dst_is_lmem, struct i915_request **out); int intel_context_migrate_copy(struct intel_context *ce, const struct i915_deps *deps, struct scatterlist *src, - enum i915_cache_level src_cache_level, + unsigned int src_pat_index, bool src_is_lmem, struct scatterlist *dst, - enum i915_cache_level dst_cache_level, + unsigned int dst_pat_index, bool dst_is_lmem, struct i915_request **out); @@ -48,7 +47,7 @@ intel_migrate_clear(struct intel_migrate *m, struct i915_gem_ww_ctx *ww, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, u32 value, struct i915_request **out); @@ -56,7 +55,7 @@ int intel_context_migrate_clear(struct intel_context *ce, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, u32 value, struct i915_request **out); diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/gt/intel_ppgtt.c index 7ecfa672f738..436756bfbb1a 100644 --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c @@ -181,7 +181,7 @@ struct i915_ppgtt *i915_ppgtt_create(struct intel_gt *gt, void ppgtt_bind_vma(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { u32 pte_flags; @@ -199,7 +199,7 @@ void ppgtt_bind_vma(struct i915_address_space *vm, if (vma_res->bi.lmem) pte_flags |= PTE_LM; - vm->insert_entries(vm, vma_res, cache_level, pte_flags); + vm->insert_entries(vm, vma_res, pat_index, pte_flags); wmb(); } diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c index e677f2da093d..3def5ca72dec 100644 --- a/drivers/gpu/drm/i915/gt/selftest_migrate.c +++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c @@ -137,7 +137,7 @@ static int copy(struct intel_migrate *migrate, static int intel_context_copy_ccs(struct intel_context *ce, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool write_to_ccs, struct i915_request **out) { @@ -185,7 +185,7 @@ static int intel_context_copy_ccs(struct intel_context *ce, if (err) goto out_rq; - len = emit_pte(rq, &it, cache_level, true, offset, CHUNK_SZ); + len = emit_pte(rq, &it, pat_index, true, offset, CHUNK_SZ); if (len <= 0) { err = len; goto out_rq; @@ -223,7 +223,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, struct i915_gem_ww_ctx *ww, const struct i915_deps *deps, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool write_to_ccs, struct i915_request **out) { @@ -243,7 +243,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, if (err) goto out; - err = intel_context_copy_ccs(ce, deps, sg, cache_level, + err = intel_context_copy_ccs(ce, deps, sg, pat_index, write_to_ccs, out); intel_context_unpin(ce); @@ -300,7 +300,7 @@ static int clear(struct intel_migrate *migrate, /* Write the obj data into ccs surface */ err = intel_migrate_ccs_copy(migrate, &ww, NULL, obj->mm.pages->sgl, - obj->cache_level, + obj->pat_index, true, &rq); if (rq && !err) { if (i915_request_wait(rq, 0, HZ) < 0) { @@ -351,7 +351,7 @@ static int clear(struct intel_migrate *migrate, err = intel_migrate_ccs_copy(migrate, &ww, NULL, obj->mm.pages->sgl, - obj->cache_level, + obj->pat_index, false, &rq); if (rq && !err) { if (i915_request_wait(rq, 0, HZ) < 0) { @@ -414,9 +414,9 @@ static int __migrate_copy(struct intel_migrate *migrate, struct i915_request **out) { return intel_migrate_copy(migrate, ww, NULL, - src->mm.pages->sgl, src->cache_level, + src->mm.pages->sgl, src->pat_index, i915_gem_object_is_lmem(src), - dst->mm.pages->sgl, dst->cache_level, + dst->mm.pages->sgl, dst->pat_index, i915_gem_object_is_lmem(dst), out); } @@ -428,9 +428,9 @@ static int __global_copy(struct intel_migrate *migrate, struct i915_request **out) { return intel_context_migrate_copy(migrate->context, NULL, - src->mm.pages->sgl, src->cache_level, + src->mm.pages->sgl, src->pat_index, i915_gem_object_is_lmem(src), - dst->mm.pages->sgl, dst->cache_level, + dst->mm.pages->sgl, dst->pat_index, i915_gem_object_is_lmem(dst), out); } @@ -455,7 +455,7 @@ static int __migrate_clear(struct intel_migrate *migrate, { return intel_migrate_clear(migrate, ww, NULL, obj->mm.pages->sgl, - obj->cache_level, + obj->pat_index, i915_gem_object_is_lmem(obj), value, out); } @@ -468,7 +468,7 @@ static int __global_clear(struct intel_migrate *migrate, { return intel_context_migrate_clear(migrate->context, NULL, obj->mm.pages->sgl, - obj->cache_level, + obj->pat_index, i915_gem_object_is_lmem(obj), value, out); } @@ -648,7 +648,7 @@ static int live_emit_pte_full_ring(void *arg) */ pr_info("%s emite_pte ring space=%u\n", __func__, rq->ring->space); it = sg_sgt(obj->mm.pages->sgl); - len = emit_pte(rq, &it, obj->cache_level, false, 0, CHUNK_SZ); + len = emit_pte(rq, &it, obj->pat_index, false, 0, CHUNK_SZ); if (!len) { err = -EINVAL; goto out_rq; @@ -844,7 +844,7 @@ static int wrap_ktime_compare(const void *A, const void *B) static int __perf_clear_blt(struct intel_context *ce, struct scatterlist *sg, - enum i915_cache_level cache_level, + unsigned int pat_index, bool is_lmem, size_t sz) { @@ -858,7 +858,7 @@ static int __perf_clear_blt(struct intel_context *ce, t0 = ktime_get(); - err = intel_context_migrate_clear(ce, NULL, sg, cache_level, + err = intel_context_migrate_clear(ce, NULL, sg, pat_index, is_lmem, 0, &rq); if (rq) { if (i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT) < 0) @@ -904,7 +904,8 @@ static int perf_clear_blt(void *arg) err = __perf_clear_blt(gt->migrate.context, dst->mm.pages->sgl, - I915_CACHE_NONE, + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), i915_gem_object_is_lmem(dst), sizes[i]); @@ -919,10 +920,10 @@ static int perf_clear_blt(void *arg) static int __perf_copy_blt(struct intel_context *ce, struct scatterlist *src, - enum i915_cache_level src_cache_level, + unsigned int src_pat_index, bool src_is_lmem, struct scatterlist *dst, - enum i915_cache_level dst_cache_level, + unsigned int dst_pat_index, bool dst_is_lmem, size_t sz) { @@ -937,9 +938,9 @@ static int __perf_copy_blt(struct intel_context *ce, t0 = ktime_get(); err = intel_context_migrate_copy(ce, NULL, - src, src_cache_level, + src, src_pat_index, src_is_lmem, - dst, dst_cache_level, + dst, dst_pat_index, dst_is_lmem, &rq); if (rq) { @@ -994,10 +995,12 @@ static int perf_copy_blt(void *arg) err = __perf_copy_blt(gt->migrate.context, src->mm.pages->sgl, - I915_CACHE_NONE, + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), i915_gem_object_is_lmem(src), dst->mm.pages->sgl, - I915_CACHE_NONE, + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), i915_gem_object_is_lmem(dst), sz); diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c index a9e0a91bc0e0..79aa6ac66ad2 100644 --- a/drivers/gpu/drm/i915/gt/selftest_reset.c +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c @@ -86,7 +86,9 @@ __igt_reset_stolen(struct intel_gt *gt, ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), + 0); mb(); s = io_mapping_map_wc(&ggtt->iomap, @@ -127,7 +129,9 @@ __igt_reset_stolen(struct intel_gt *gt, ggtt->vm.insert_page(&ggtt->vm, dma, ggtt->error_capture.start, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), + 0); mb(); s = io_mapping_map_wc(&ggtt->iomap, diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c index 9f536c251179..39c3ec12df1a 100644 --- a/drivers/gpu/drm/i915/gt/selftest_timeline.c +++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c @@ -836,7 +836,7 @@ static int setup_watcher(struct hwsp_watcher *w, struct intel_gt *gt, return PTR_ERR(obj); /* keep the same cache settings as timeline */ - i915_gem_object_set_cache_coherency(obj, tl->hwsp_ggtt->obj->cache_level); + i915_gem_object_set_pat_index(obj, tl->hwsp_ggtt->obj->pat_index); w->map = i915_gem_object_pin_map_unlocked(obj, page_unmask_bits(tl->hwsp_ggtt->obj->mm.mapping)); if (IS_ERR(w->map)) { diff --git a/drivers/gpu/drm/i915/gt/selftest_tlb.c b/drivers/gpu/drm/i915/gt/selftest_tlb.c index e6cac1f15d6e..4493c8518e91 100644 --- a/drivers/gpu/drm/i915/gt/selftest_tlb.c +++ b/drivers/gpu/drm/i915/gt/selftest_tlb.c @@ -36,6 +36,8 @@ pte_tlbinv(struct intel_context *ce, u64 length, struct rnd_state *prng) { + const unsigned int pat_index = + i915_gem_get_pat_index(ce->vm->i915, I915_CACHE_NONE); struct drm_i915_gem_object *batch; struct drm_mm_node vb_node; struct i915_request *rq; @@ -155,7 +157,7 @@ pte_tlbinv(struct intel_context *ce, /* Flip the PTE between A and B */ if (i915_gem_object_is_lmem(vb->obj)) pte_flags |= PTE_LM; - ce->vm->insert_entries(ce->vm, &vb_res, 0, pte_flags); + ce->vm->insert_entries(ce->vm, &vb_res, pat_index, pte_flags); /* Flush the PTE update to concurrent HW */ tlbinv(ce->vm, addr & -length, length); diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c index 4ec7df9ed5ff..e34c79120090 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c @@ -957,9 +957,15 @@ static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw) pte_flags |= PTE_LM; if (ggtt->vm.raw_insert_entries) - ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); + ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, + i915_gem_get_pat_index(ggtt->vm.i915, + I915_CACHE_NONE), + pte_flags); else - ggtt->vm.insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); + ggtt->vm.insert_entries(&ggtt->vm, dummy, + i915_gem_get_pat_index(ggtt->vm.i915, + I915_CACHE_NONE), + pte_flags); } static void uc_fw_unbind_ggtt(struct intel_uc_fw *uc_fw) diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 41389a32e998..76ccd4e03e31 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -139,21 +139,54 @@ static const char *stringify_vma_type(const struct i915_vma *vma) return "ppgtt"; } -static const char *i915_cache_level_str(struct drm_i915_private *i915, int type) -{ - switch (type) { - case I915_CACHE_NONE: return " uncached"; - case I915_CACHE_LLC: return HAS_LLC(i915) ? " LLC" : " snooped"; - case I915_CACHE_L3_LLC: return " L3+LLC"; - case I915_CACHE_WT: return " WT"; - default: return ""; +static const char *i915_cache_level_str(struct drm_i915_gem_object *obj) +{ + struct drm_i915_private *i915 = obj_to_i915(obj); + + if (IS_METEORLAKE(i915)) { + switch (obj->pat_index) { + case 0: return " WB"; + case 1: return " WT"; + case 2: return " UC"; + case 3: return " WB (1-Way Coh)"; + case 4: return " WB (2-Way Coh)"; + default: return " not defined"; + } + } else if (IS_PONTEVECCHIO(i915)) { + switch (obj->pat_index) { + case 0: return " UC"; + case 1: return " WC"; + case 2: return " WT"; + case 3: return " WB"; + case 4: return " WT (CLOS1)"; + case 5: return " WB (CLOS1)"; + case 6: return " WT (CLOS2)"; + case 7: return " WT (CLOS2)"; + default: return " not defined"; + } + } else if (GRAPHICS_VER(i915) >= 12) { + switch (obj->pat_index) { + case 0: return " WB"; + case 1: return " WC"; + case 2: return " WT"; + case 3: return " UC"; + default: return " not defined"; + } + } else { + switch (obj->pat_index) { + case 0: return " UC"; + case 1: return HAS_LLC(i915) ? + " LLC" : " snooped"; + case 2: return " L3+LLC"; + case 3: return " WT"; + default: return " not defined"; + } } } void i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) { - struct drm_i915_private *dev_priv = to_i915(obj->base.dev); struct i915_vma *vma; int pin_count = 0; @@ -165,7 +198,7 @@ i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) obj->base.size / 1024, obj->read_domains, obj->write_domain, - i915_cache_level_str(dev_priv, obj->cache_level), + i915_cache_level_str(obj), obj->mm.dirty ? " dirty" : "", obj->mm.madv == I915_MADV_DONTNEED ? " purgeable" : ""); if (obj->base.name) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 0a78bdbd36b1..38933c00f705 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -420,8 +420,11 @@ i915_gem_gtt_pread(struct drm_i915_gem_object *obj, page_length = remain < page_length ? remain : page_length; if (drm_mm_node_allocated(&node)) { ggtt->vm.insert_page(&ggtt->vm, - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), - node.start, I915_CACHE_NONE, 0); + i915_gem_object_get_dma_address(obj, + offset >> PAGE_SHIFT), + node.start, + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), 0); } else { page_base += offset & PAGE_MASK; } @@ -598,8 +601,11 @@ i915_gem_gtt_pwrite_fast(struct drm_i915_gem_object *obj, /* flush the write before we modify the GGTT */ intel_gt_flush_ggtt_writes(ggtt->vm.gt); ggtt->vm.insert_page(&ggtt->vm, - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), - node.start, I915_CACHE_NONE, 0); + i915_gem_object_get_dma_address(obj, + offset >> PAGE_SHIFT), + node.start, + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), 0); wmb(); /* flush modifications to the GGTT (insert_page) */ } else { page_base += offset & PAGE_MASK; @@ -1142,6 +1148,19 @@ int i915_gem_init(struct drm_i915_private *dev_priv) unsigned int i; int ret; + /* + * In the proccess of replacing cache_level with pat_index a tricky + * dependency is created on the definition of the enum i915_cache_level. + * in case this enum is changed, PTE encode would be broken. + * Add a WARNING here. And remove when we completely quit using this + * enum + */ + WARN_ON_ONCE(I915_CACHE_NONE != 0 || + I915_CACHE_LLC != 1 || + I915_CACHE_L3_LLC != 2 || + I915_CACHE_WT != 3 || + I915_MAX_CACHE_LEVEL != 4); + /* We need to fallback to 4K pages if host doesn't support huge gtt. */ if (intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv)) RUNTIME_INFO(dev_priv)->page_sizes = I915_GTT_PAGE_SIZE_4K; diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c index 8ffdd7f47e93..566536499954 100644 --- a/drivers/gpu/drm/i915/i915_gpu_error.c +++ b/drivers/gpu/drm/i915/i915_gpu_error.c @@ -1122,10 +1122,14 @@ i915_vma_coredump_create(const struct intel_gt *gt, mutex_lock(&ggtt->error_mutex); if (ggtt->vm.raw_insert_page) ggtt->vm.raw_insert_page(&ggtt->vm, dma, slot, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), + 0); else ggtt->vm.insert_page(&ggtt->vm, dma, slot, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(gt->i915, + I915_CACHE_NONE), + 0); mb(); s = io_mapping_map_wc(&ggtt->iomap, slot, PAGE_SIZE); diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c index 20a44788999e..a814775a363d 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -315,7 +315,7 @@ struct i915_vma_work { struct i915_vma_resource *vma_res; struct drm_i915_gem_object *obj; struct i915_sw_dma_fence_cb cb; - enum i915_cache_level cache_level; + unsigned int pat_index; unsigned int flags; }; @@ -334,7 +334,7 @@ static void __vma_bind(struct dma_fence_work *work) return; vma_res->ops->bind_vma(vma_res->vm, &vw->stash, - vma_res, vw->cache_level, vw->flags); + vma_res, vw->pat_index, vw->flags); } static void __vma_release(struct dma_fence_work *work) @@ -426,7 +426,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, /** * i915_vma_bind - Sets up PTEs for an VMA in it's corresponding address space. * @vma: VMA to map - * @cache_level: mapping cache level + * @pat_index: PAT index to set in PTE * @flags: flags like global or local mapping * @work: preallocated worker for allocating and binding the PTE * @vma_res: pointer to a preallocated vma resource. The resource is either @@ -437,7 +437,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, * Note that DMA addresses are also the only part of the SG table we care about. */ int i915_vma_bind(struct i915_vma *vma, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags, struct i915_vma_work *work, struct i915_vma_resource *vma_res) @@ -507,7 +507,7 @@ int i915_vma_bind(struct i915_vma *vma, struct dma_fence *prev; work->vma_res = i915_vma_resource_get(vma->resource); - work->cache_level = cache_level; + work->pat_index = pat_index; work->flags = bind_flags; /* @@ -537,7 +537,7 @@ int i915_vma_bind(struct i915_vma *vma, return ret; } - vma->ops->bind_vma(vma->vm, NULL, vma->resource, cache_level, + vma->ops->bind_vma(vma->vm, NULL, vma->resource, pat_index, bind_flags); } @@ -814,7 +814,7 @@ i915_vma_insert(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, color = 0; if (i915_vm_has_cache_coloring(vma->vm)) - color = vma->obj->cache_level; + color = vma->obj->pat_index; if (flags & PIN_OFFSET_FIXED) { u64 offset = flags & PIN_OFFSET_MASK; @@ -1518,7 +1518,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, GEM_BUG_ON(!vma->pages); err = i915_vma_bind(vma, - vma->obj->cache_level, + vma->obj->pat_index, flags, work, vma_res); vma_res = NULL; if (err) diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h index 38c8c66ed724..9a9729205d5b 100644 --- a/drivers/gpu/drm/i915/i915_vma.h +++ b/drivers/gpu/drm/i915/i915_vma.h @@ -250,7 +250,7 @@ i915_vma_compare(struct i915_vma *vma, struct i915_vma_work *i915_vma_work(void); int i915_vma_bind(struct i915_vma *vma, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags, struct i915_vma_work *work, struct i915_vma_resource *vma_res); diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h index 77fda2244d16..64472b7f0e77 100644 --- a/drivers/gpu/drm/i915/i915_vma_types.h +++ b/drivers/gpu/drm/i915/i915_vma_types.h @@ -32,8 +32,6 @@ #include "gem/i915_gem_object_types.h" -enum i915_cache_level; - /** * DOC: Global GTT views * diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c index d91d0ade8abd..61da4ed9d521 100644 --- a/drivers/gpu/drm/i915/selftests/i915_gem.c +++ b/drivers/gpu/drm/i915/selftests/i915_gem.c @@ -57,7 +57,10 @@ static void trash_stolen(struct drm_i915_private *i915) u32 __iomem *s; int x; - ggtt->vm.insert_page(&ggtt->vm, dma, slot, I915_CACHE_NONE, 0); + ggtt->vm.insert_page(&ggtt->vm, dma, slot, + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), + 0); s = io_mapping_map_atomic_wc(&ggtt->iomap, slot); for (x = 0; x < PAGE_SIZE / sizeof(u32); x++) { diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c index f68ef4074088..f8fe3681c3dc 100644 --- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c +++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c @@ -246,7 +246,7 @@ static int igt_evict_for_cache_color(void *arg) struct drm_mm_node target = { .start = I915_GTT_PAGE_SIZE * 2, .size = I915_GTT_PAGE_SIZE, - .color = I915_CACHE_LLC, + .color = i915_gem_get_pat_index(gt->i915, I915_CACHE_LLC), }; struct drm_i915_gem_object *obj; struct i915_vma *vma; @@ -309,7 +309,7 @@ static int igt_evict_for_cache_color(void *arg) /* Attempt to remove the first *pinned* vma, by removing the (empty) * neighbour -- this should fail. */ - target.color = I915_CACHE_L3_LLC; + target.color = i915_gem_get_pat_index(gt->i915, I915_CACHE_L3_LLC); mutex_lock(&ggtt->vm.mutex); err = i915_gem_evict_for_node(&ggtt->vm, NULL, &target, 0); diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c index 154801f1c468..36940ef10108 100644 --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c @@ -135,7 +135,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size) obj->write_domain = I915_GEM_DOMAIN_CPU; obj->read_domains = I915_GEM_DOMAIN_CPU; - obj->cache_level = I915_CACHE_NONE; + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); /* Preallocate the "backing storage" */ if (i915_gem_object_pin_pages_unlocked(obj)) @@ -359,7 +359,9 @@ static int lowlevel_hole(struct i915_address_space *vm, with_intel_runtime_pm(vm->gt->uncore->rpm, wakeref) vm->insert_entries(vm, mock_vma_res, - I915_CACHE_NONE, 0); + i915_gem_get_pat_index(vm->i915, + I915_CACHE_NONE), + 0); } count = n; @@ -1377,7 +1379,10 @@ static int igt_ggtt_page(void *arg) ggtt->vm.insert_page(&ggtt->vm, i915_gem_object_get_dma_address(obj, 0), - offset, I915_CACHE_NONE, 0); + offset, + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), + 0); } order = i915_random_order(count, &prng); @@ -1510,7 +1515,7 @@ static int reserve_gtt_with_resource(struct i915_vma *vma, u64 offset) mutex_lock(&vm->mutex); err = i915_gem_gtt_reserve(vm, NULL, &vma->node, obj->base.size, offset, - obj->cache_level, + obj->pat_index, 0); if (!err) { i915_vma_resource_init_from_vma(vma_res, vma); @@ -1690,7 +1695,7 @@ static int insert_gtt_with_resource(struct i915_vma *vma) mutex_lock(&vm->mutex); err = i915_gem_gtt_insert(vm, NULL, &vma->node, obj->base.size, 0, - obj->cache_level, 0, vm->total, 0); + obj->pat_index, 0, vm->total, 0); if (!err) { i915_vma_resource_init_from_vma(vma_res, vma); vma->resource = vma_res; diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c index 3b18e5905c86..d985d9bae2e8 100644 --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c @@ -1070,7 +1070,9 @@ static int igt_lmem_write_cpu(void *arg) /* Put the pages into a known state -- from the gpu for added fun */ intel_engine_pm_get(engine); err = intel_context_migrate_clear(engine->gt->migrate.context, NULL, - obj->mm.pages->sgl, I915_CACHE_NONE, + obj->mm.pages->sgl, + i915_gem_get_pat_index(i915, + I915_CACHE_NONE), true, 0xdeadbeaf, &rq); if (rq) { dma_resv_add_fence(obj->base.resv, &rq->fence, diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c index ece97e4faacb..a516c0aa88fd 100644 --- a/drivers/gpu/drm/i915/selftests/mock_gtt.c +++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c @@ -27,21 +27,21 @@ static void mock_insert_page(struct i915_address_space *vm, dma_addr_t addr, u64 offset, - enum i915_cache_level level, + unsigned int pat_index, u32 flags) { } static void mock_insert_entries(struct i915_address_space *vm, struct i915_vma_resource *vma_res, - enum i915_cache_level level, u32 flags) + unsigned int pat_index, u32 flags) { } static void mock_bind_ppgtt(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { GEM_BUG_ON(flags & I915_VMA_GLOBAL_BIND); @@ -94,7 +94,7 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name) static void mock_bind_ggtt(struct i915_address_space *vm, struct i915_vm_pt_stash *stash, struct i915_vma_resource *vma_res, - enum i915_cache_level cache_level, + unsigned int pat_index, u32 flags) { } -- 2.25.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level fei.yang @ 2023-05-09 11:08 ` Andi Shyti 2023-05-09 12:15 ` Tvrtko Ursulin 1 sibling, 0 replies; 10+ messages in thread From: Andi Shyti @ 2023-05-09 11:08 UTC (permalink / raw) To: fei.yang; +Cc: Chris Wilson, intel-gfx, dri-devel, Matt Roper Hi Fei, On Mon, May 08, 2023 at 04:48:54PM -0700, fei.yang@intel.com wrote: > From: Fei Yang <fei.yang@intel.com> > > Currently the KMD is using enum i915_cache_level to set caching policy for > buffer objects. This is flaky because the PAT index which really controls > the caching behavior in PTE has far more levels than what's defined in the > enum. In addition, the PAT index is platform dependent, having to translate > between i915_cache_level and PAT index is not reliable, and makes the code > more complicated. > > >From UMD's perspective there is also a necessity to set caching policy for > performance fine tuning. It's much easier for the UMD to directly use PAT > index because the behavior of each PAT index is clearly defined in Bspec. > Having the abstracted i915_cache_level sitting in between would only cause > more ambiguity. PAT is expected to work much like MOCS already works today, > and by design userspace is expected to select the index that exactly > matches the desired behavior described in the hardware specification. > > For these reasons this patch replaces i915_cache_level with PAT index. Also > note, the cache_level is not completely removed yet, because the KMD still > has the need of creating buffer objects with simple cache settings such as > cached, uncached, or writethrough. For kernel objects, cache_level is used > for simplicity and backward compatibility. For Pre-gen12 platforms PAT can > have 1:1 mapping to i915_cache_level, so these two are interchangeable. see > the use of LEGACY_CACHELEVEL. > > One consequence of this change is that gen8_pte_encode is no longer working > for gen12 platforms due to the fact that gen12 platforms has different PAT > definitions. In the meantime the mtl_pte_encode introduced specfically for > MTL becomes generic for all gen12 platforms. This patch renames the MTL > PTE encode function into gen12_pte_encode and apply it to all gen12. Even > though this change looks unrelated, but separating them would temporarily > break gen12 PTE encoding, thus squash them in one patch. > > Special note: this patch changes the way caching behavior is controlled in > the sense that some objects are left to be managed by userspace. For such > objects we need to be careful not to change the userspace settings.There > are kerneldoc and comments added around obj->cache_coherent, cache_dirty, > and how to bypass the checkings by i915_gem_object_has_cache_level. For > full understanding, these changes need to be looked at together with the > two follow-up patches, one disables the {set|get}_caching ioctl's and the > other adds set_pat extension to the GEM_CREATE uAPI. > > Bspec: 63019 > > Cc: Chris Wilson <chris.p.wilson@linux.intel.com> > Signed-off-by: Fei Yang <fei.yang@intel.com> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > Reviewed-by: Matt Roper <matthew.d.roper@intel.com> > > To be squashed Ha! you forgot to remove this... I also do the same :) No worries, if the patch is right I'll fix it before pushing it. Tvrtko? Any opinion? Andi ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level fei.yang 2023-05-09 11:08 ` Andi Shyti @ 2023-05-09 12:15 ` Tvrtko Ursulin 2023-05-09 17:12 ` Yang, Fei 1 sibling, 1 reply; 10+ messages in thread From: Tvrtko Ursulin @ 2023-05-09 12:15 UTC (permalink / raw) To: fei.yang, intel-gfx; +Cc: Chris Wilson, Matt Roper, dri-devel On 09/05/2023 00:48, fei.yang@intel.com wrote: > From: Fei Yang <fei.yang@intel.com> > > Currently the KMD is using enum i915_cache_level to set caching policy for > buffer objects. This is flaky because the PAT index which really controls > the caching behavior in PTE has far more levels than what's defined in the > enum. In addition, the PAT index is platform dependent, having to translate > between i915_cache_level and PAT index is not reliable, and makes the code > more complicated. > >>From UMD's perspective there is also a necessity to set caching policy for > performance fine tuning. It's much easier for the UMD to directly use PAT > index because the behavior of each PAT index is clearly defined in Bspec. > Having the abstracted i915_cache_level sitting in between would only cause > more ambiguity. PAT is expected to work much like MOCS already works today, > and by design userspace is expected to select the index that exactly > matches the desired behavior described in the hardware specification. > > For these reasons this patch replaces i915_cache_level with PAT index. Also > note, the cache_level is not completely removed yet, because the KMD still > has the need of creating buffer objects with simple cache settings such as > cached, uncached, or writethrough. For kernel objects, cache_level is used > for simplicity and backward compatibility. For Pre-gen12 platforms PAT can > have 1:1 mapping to i915_cache_level, so these two are interchangeable. see > the use of LEGACY_CACHELEVEL. > > One consequence of this change is that gen8_pte_encode is no longer working > for gen12 platforms due to the fact that gen12 platforms has different PAT > definitions. In the meantime the mtl_pte_encode introduced specfically for > MTL becomes generic for all gen12 platforms. This patch renames the MTL > PTE encode function into gen12_pte_encode and apply it to all gen12. Even > though this change looks unrelated, but separating them would temporarily > break gen12 PTE encoding, thus squash them in one patch. > > Special note: this patch changes the way caching behavior is controlled in > the sense that some objects are left to be managed by userspace. For such > objects we need to be careful not to change the userspace settings.There > are kerneldoc and comments added around obj->cache_coherent, cache_dirty, > and how to bypass the checkings by i915_gem_object_has_cache_level. For > full understanding, these changes need to be looked at together with the > two follow-up patches, one disables the {set|get}_caching ioctl's and the > other adds set_pat extension to the GEM_CREATE uAPI. > > Bspec: 63019 > > Cc: Chris Wilson <chris.p.wilson@linux.intel.com> > Signed-off-by: Fei Yang <fei.yang@intel.com> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > Reviewed-by: Matt Roper <matthew.d.roper@intel.com> > > To be squashed > --- > drivers/gpu/drm/i915/display/intel_dpt.c | 12 +-- > drivers/gpu/drm/i915/gem/i915_gem_domain.c | 58 +++++++++---- > .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 15 +++- > drivers/gpu/drm/i915/gem/i915_gem_mman.c | 11 ++- > drivers/gpu/drm/i915/gem/i915_gem_object.c | 51 ++++++++++- > drivers/gpu/drm/i915/gem/i915_gem_object.h | 4 + > .../gpu/drm/i915/gem/i915_gem_object_types.h | 44 +++++++++- > drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 4 +- > drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 8 +- > .../gpu/drm/i915/gem/selftests/huge_pages.c | 2 +- > .../drm/i915/gem/selftests/i915_gem_migrate.c | 2 +- > .../drm/i915/gem/selftests/i915_gem_mman.c | 2 +- > drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 10 ++- > drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 84 +++++++++---------- > drivers/gpu/drm/i915/gt/gen8_ppgtt.h | 3 +- > drivers/gpu/drm/i915/gt/intel_ggtt.c | 82 +++++++++--------- > drivers/gpu/drm/i915/gt/intel_gtt.h | 18 ++-- > drivers/gpu/drm/i915/gt/intel_migrate.c | 47 ++++++----- > drivers/gpu/drm/i915/gt/intel_migrate.h | 13 ++- > drivers/gpu/drm/i915/gt/intel_ppgtt.c | 4 +- > drivers/gpu/drm/i915/gt/selftest_migrate.c | 47 ++++++----- > drivers/gpu/drm/i915/gt/selftest_reset.c | 8 +- > drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- > drivers/gpu/drm/i915/gt/selftest_tlb.c | 4 +- > drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 10 ++- > drivers/gpu/drm/i915/i915_debugfs.c | 53 +++++++++--- > drivers/gpu/drm/i915/i915_gem.c | 27 +++++- > drivers/gpu/drm/i915/i915_gpu_error.c | 8 +- > drivers/gpu/drm/i915/i915_vma.c | 16 ++-- > drivers/gpu/drm/i915/i915_vma.h | 2 +- > drivers/gpu/drm/i915/i915_vma_types.h | 2 - > drivers/gpu/drm/i915/selftests/i915_gem.c | 5 +- > .../gpu/drm/i915/selftests/i915_gem_evict.c | 4 +- > drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 15 ++-- > .../drm/i915/selftests/intel_memory_region.c | 4 +- > drivers/gpu/drm/i915/selftests/mock_gtt.c | 8 +- > 36 files changed, 451 insertions(+), 238 deletions(-) > > diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c > index c5eacfdba1a5..7c5fddb203ba 100644 > --- a/drivers/gpu/drm/i915/display/intel_dpt.c > +++ b/drivers/gpu/drm/i915/display/intel_dpt.c > @@ -43,24 +43,24 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) > static void dpt_insert_page(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_dpt *dpt = i915_vm_to_dpt(vm); > gen8_pte_t __iomem *base = dpt->iomem; > > gen8_set_pte(base + offset / I915_GTT_PAGE_SIZE, > - vm->pte_encode(addr, level, flags)); > + vm->pte_encode(addr, pat_index, flags)); > } > > static void dpt_insert_entries(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_dpt *dpt = i915_vm_to_dpt(vm); > gen8_pte_t __iomem *base = dpt->iomem; > - const gen8_pte_t pte_encode = vm->pte_encode(0, level, flags); > + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); > struct sgt_iter sgt_iter; > dma_addr_t addr; > int i; > @@ -83,7 +83,7 @@ static void dpt_clear_range(struct i915_address_space *vm, > static void dpt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > u32 pte_flags; > @@ -98,7 +98,7 @@ static void dpt_bind_vma(struct i915_address_space *vm, > if (vma_res->bi.lmem) > pte_flags |= PTE_LM; > > - vm->insert_entries(vm, vma_res, cache_level, pte_flags); > + vm->insert_entries(vm, vma_res, pat_index, pte_flags); > > vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; > > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c > index d2d5a24301b2..05107a6efe45 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c > @@ -27,8 +27,15 @@ static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj) > if (IS_DGFX(i915)) > return false; > > - return !(obj->cache_level == I915_CACHE_NONE || > - obj->cache_level == I915_CACHE_WT); > + /* > + * For objects created by userspace through GEM_CREATE with pat_index > + * set by set_pat extension, i915_gem_object_has_cache_level() will > + * always return true, because the coherency of such object is managed > + * by userspace. Othereise the call here would fall back to checking > + * whether the object is un-cached or write-through. > + */ > + return !(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || > + i915_gem_object_has_cache_level(obj, I915_CACHE_WT)); > } > > bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj) > @@ -267,7 +274,13 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, > { > int ret; > > - if (obj->cache_level == cache_level) > + /* > + * For objects created by userspace through GEM_CREATE with pat_index > + * set by set_pat extension, simply return 0 here without touching > + * the cache setting, because such objects should have an immutable > + * cache setting by desgin and always managed by userspace. > + */ > + if (i915_gem_object_has_cache_level(obj, cache_level)) > return 0; > > ret = i915_gem_object_wait(obj, > @@ -278,10 +291,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, > return ret; > > /* Always invalidate stale cachelines */ > - if (obj->cache_level != cache_level) { > - i915_gem_object_set_cache_coherency(obj, cache_level); > - obj->cache_dirty = true; > - } > + i915_gem_object_set_cache_coherency(obj, cache_level); > + obj->cache_dirty = true; > > /* The cache-level will be applied when each vma is rebound. */ > return i915_gem_object_unbind(obj, > @@ -306,20 +317,22 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data, > goto out; > } > > - switch (obj->cache_level) { > - case I915_CACHE_LLC: > - case I915_CACHE_L3_LLC: > - args->caching = I915_CACHING_CACHED; > - break; > + /* > + * This ioctl should be disabled for the objects with pat_index > + * set by user space. > + */ > + if (obj->pat_set_by_user) { > + err = -EOPNOTSUPP; > + goto out; > + } > > - case I915_CACHE_WT: > + if (i915_gem_object_has_cache_level(obj, I915_CACHE_LLC) || > + i915_gem_object_has_cache_level(obj, I915_CACHE_L3_LLC)) > + args->caching = I915_CACHING_CACHED; > + else if (i915_gem_object_has_cache_level(obj, I915_CACHE_WT)) > args->caching = I915_CACHING_DISPLAY; > - break; > - > - default: > + else > args->caching = I915_CACHING_NONE; > - break; > - } > out: > rcu_read_unlock(); > return err; > @@ -364,6 +377,15 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data, > if (!obj) > return -ENOENT; > > + /* > + * This ioctl should be disabled for the objects with pat_index > + * set by user space. > + */ > + if (obj->pat_set_by_user) { > + ret = -EOPNOTSUPP; > + goto out; > + } > + > /* > * The caching mode of proxy object is handled by its generator, and > * not allowed to be changed by userspace. > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c > index 3aeede6aee4d..5fb459ea4294 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c > @@ -640,9 +640,15 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache, > if (DBG_FORCE_RELOC == FORCE_GTT_RELOC) > return false; > > + /* > + * For objects created by userspace through GEM_CREATE with pat_index > + * set by set_pat extension, i915_gem_object_has_cache_level() always > + * return true, otherwise the call would fall back to checking whether > + * the object is un-cached. > + */ > return (cache->has_llc || > obj->cache_dirty || > - obj->cache_level != I915_CACHE_NONE); > + !i915_gem_object_has_cache_level(obj, I915_CACHE_NONE)); > } > > static int eb_reserve_vma(struct i915_execbuffer *eb, > @@ -1324,7 +1330,10 @@ static void *reloc_iomap(struct i915_vma *batch, > if (drm_mm_node_allocated(&cache->node)) { > ggtt->vm.insert_page(&ggtt->vm, > i915_gem_object_get_dma_address(obj, page), > - offset, I915_CACHE_NONE, 0); > + offset, > + i915_gem_get_pat_index(ggtt->vm.i915, > + I915_CACHE_NONE), > + 0); > } else { > offset += page << PAGE_SHIFT; > } > @@ -1464,7 +1473,7 @@ eb_relocate_entry(struct i915_execbuffer *eb, > reloc_cache_unmap(&eb->reloc_cache); > mutex_lock(&vma->vm->mutex); > err = i915_vma_bind(target->vma, > - target->vma->obj->cache_level, > + target->vma->obj->pat_index, > PIN_GLOBAL, NULL, NULL); > mutex_unlock(&vma->vm->mutex); > reloc_cache_remap(&eb->reloc_cache, ev->vma->obj); > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c > index 3dbacdf0911a..4e7a838ab7bd 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c > @@ -383,7 +383,16 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf) > } > > /* Access to snoopable pages through the GTT is incoherent. */ > - if (obj->cache_level != I915_CACHE_NONE && !HAS_LLC(i915)) { > + /* > + * For objects created by userspace through GEM_CREATE with pat_index > + * set by set_pat extension, coherency is managed by userspace, make > + * sure we don't fail handling the vm fault by calling > + * i915_gem_object_has_cache_level() which always return true for such > + * objects. Otherwise this helper function would fall back to checking > + * whether the object is un-cached. > + */ > + if (!(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || > + HAS_LLC(i915))) { > ret = -EFAULT; > goto err_unpin; > } > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c > index 8c70a0ec7d2f..46a19b099ec8 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c > @@ -54,6 +54,24 @@ unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, > return INTEL_INFO(i915)->cachelevel_to_pat[level]; > } > > +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, > + enum i915_cache_level lvl) > +{ > + /* > + * In case the pat_index is set by user space, this kernel mode > + * driver should leave the coherency to be managed by user space, > + * simply return true here. > + */ > + if (obj->pat_set_by_user) > + return true; > + > + /* > + * Otherwise the pat_index should have been converted from cache_level > + * so that the following comparison is valid. > + */ > + return obj->pat_index == i915_gem_get_pat_index(obj_to_i915(obj), lvl); > +} > + > struct drm_i915_gem_object *i915_gem_object_alloc(void) > { > struct drm_i915_gem_object *obj; > @@ -133,7 +151,7 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, > { > struct drm_i915_private *i915 = to_i915(obj->base.dev); > > - obj->cache_level = cache_level; > + obj->pat_index = i915_gem_get_pat_index(i915, cache_level); > > if (cache_level != I915_CACHE_NONE) > obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | > @@ -148,6 +166,37 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, > !IS_DGFX(i915); > } > > +/** > + * i915_gem_object_set_pat_index - set PAT index to be used in PTE encode > + * @obj: #drm_i915_gem_object > + * @pat_index: PAT index > + * > + * This is a clone of i915_gem_object_set_cache_coherency taking pat index > + * instead of cache_level as its second argument. > + */ > +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, > + unsigned int pat_index) > +{ > + struct drm_i915_private *i915 = to_i915(obj->base.dev); > + > + if (obj->pat_index == pat_index) > + return; > + > + obj->pat_index = pat_index; > + > + if (pat_index != i915_gem_get_pat_index(i915, I915_CACHE_NONE)) > + obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | > + I915_BO_CACHE_COHERENT_FOR_WRITE); > + else if (HAS_LLC(i915)) > + obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ; > + else > + obj->cache_coherent = 0; > + > + obj->cache_dirty = > + !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) && > + !IS_DGFX(i915); > +} > + > bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj) > { > struct drm_i915_private *i915 = to_i915(obj->base.dev); > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h > index 284e1aa396cd..884a17275b3a 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h > +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h > @@ -34,6 +34,8 @@ static inline bool i915_gem_object_size_2big(u64 size) > > unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, > enum i915_cache_level level); > +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, > + enum i915_cache_level lvl); > void i915_gem_init__objects(struct drm_i915_private *i915); > > void i915_objects_module_exit(void); > @@ -764,6 +766,8 @@ bool i915_gem_object_has_unknown_state(struct drm_i915_gem_object *obj); > > void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, > unsigned int cache_level); > +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, > + unsigned int pat_index); > bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj); > void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj); > void i915_gem_object_flush_if_display_locked(struct drm_i915_gem_object *obj); > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h > index bf0bd8e11355..adf4c2b28135 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h > +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h > @@ -364,15 +364,42 @@ struct drm_i915_gem_object { > #define I915_BO_FLAG_STRUCT_PAGE BIT(0) /* Object backed by struct pages */ > #define I915_BO_FLAG_IOMEM BIT(1) /* Object backed by IO memory */ > /** > - * @cache_level: The desired GTT caching level. > + * @pat_index: The desired PAT index. > + * > + * See hardware specification for valid PAT indices for each platform. > + * This field used to contain a value of enum i915_cache_level. It's > + * changed to an unsigned int because PAT indices are being used by > + * both UMD and KMD for caching policy control after GEN12. > + * For backward compatibility, this field will continue to contain > + * value of i915_cache_level for pre-GEN12 platforms so that the PTE > + * encode functions for these legacy platforms can stay the same. > + * In the meantime platform specific tables are created to translate > + * i915_cache_level into pat index, for more details check the macros > + * defined i915/i915_pci.c, e.g. PVC_CACHELEVEL. > + */ Dunno - this still reads a bit wonky to me. It was unsigned int already and also it does not hold i915_cache_level on legacy platforms, right? > + unsigned int pat_index:6; > + /** > + * @pat_set_by_user: Indicate whether pat_index is set by user space > * > - * See enum i915_cache_level for possible values, along with what > - * each does. > + * This field is set to false by default, only set to true if the > + * pat_index is set by user space. By design, user space is capable of > + * managing caching behavior by setting pat_index, in which case this > + * kernel mode driver should never touch the pat_index. > */ > - unsigned int cache_level:3; > + unsigned int pat_set_by_user; Unsigned int:1 to fit into the bitfield sandwitch better? > /** > * @cache_coherent: > * > + * Note: with the change above which replaced cache_level with > + * pat_index, the use of cache_coherent is limited to objects created > + * by kernel. Check for @pat_set_by_user to find out if an object is > + * created by userspace or kernel. A later patch will disable the pat_set_by_user is not about who created the object, no? And it is not in a later patch any longer. > + * ioctl's to change cache settings for objects with pat_index set by > + * userspace. Please don't assume @cache_coherent having the flags set > + * as describe here for these objects. Also see the helper function > + * i915_gem_object_has_cache_level() for possible ways to bypass the > + * use of this field. > + * > * Track whether the pages are coherent with the GPU if reading or > * writing through the CPU caches. The largely depends on the > * @cache_level setting. > @@ -446,6 +473,15 @@ struct drm_i915_gem_object { > /** > * @cache_dirty: > * > + * Note: with the change above which replaced cache_level with > + * pat_index, the use of cache_dirty is limited to objects created > + * by kernel. Check for @pat_set_by_user to find out if an object is > + * created by userspace or kernel. A later patch will disable the > + * ioctl's to change cache settings for objects with pat_index set by > + * userspace. Please don't assume @cache_dirty is set as describe here > + * for these objects. Also see i915_gem_object_has_cache_level() for > + * possible ways to bypass the use of this field. > + * > * Track if we are we dirty with writes through the CPU cache for this > * object. As a result reading directly from main memory might yield > * stale data. > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c > index ee492d823f1b..3b094d36a0b0 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c > @@ -565,7 +565,9 @@ static void dbg_poison(struct i915_ggtt *ggtt, > > ggtt->vm.insert_page(&ggtt->vm, addr, > ggtt->error_capture.start, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(ggtt->vm.i915, > + I915_CACHE_NONE), > + 0); > mb(); > > s = io_mapping_map_wc(&ggtt->iomap, > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c > index 69eb20ed4d47..7078af2f8f79 100644 > --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c > +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c > @@ -214,7 +214,8 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, > > intel_engine_pm_get(to_gt(i915)->migrate.context->engine); > ret = intel_context_migrate_clear(to_gt(i915)->migrate.context, deps, > - dst_st->sgl, dst_level, > + dst_st->sgl, > + i915_gem_get_pat_index(i915, dst_level), > i915_ttm_gtt_binds_lmem(dst_mem), > 0, &rq); > } else { > @@ -228,9 +229,10 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, > intel_engine_pm_get(to_gt(i915)->migrate.context->engine); > ret = intel_context_migrate_copy(to_gt(i915)->migrate.context, > deps, src_rsgt->table.sgl, > - src_level, > + i915_gem_get_pat_index(i915, src_level), > i915_ttm_gtt_binds_lmem(bo->resource), > - dst_st->sgl, dst_level, > + dst_st->sgl, > + i915_gem_get_pat_index(i915, dst_level), > i915_ttm_gtt_binds_lmem(dst_mem), > &rq); > > diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c > index d1057574e966..df6c9a84252c 100644 > --- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c > +++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c > @@ -354,7 +354,7 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single) > > obj->write_domain = I915_GEM_DOMAIN_CPU; > obj->read_domains = I915_GEM_DOMAIN_CPU; > - obj->cache_level = I915_CACHE_NONE; > + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); > > return obj; > } > diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c > index fe6c37fd7859..a93a90b15907 100644 > --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c > +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c > @@ -219,7 +219,7 @@ static int __igt_lmem_pages_migrate(struct intel_gt *gt, > continue; > > err = intel_migrate_clear(>->migrate, &ww, deps, > - obj->mm.pages->sgl, obj->cache_level, > + obj->mm.pages->sgl, obj->pat_index, > i915_gem_object_is_lmem(obj), > 0xdeadbeaf, &rq); > if (rq) { > diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c > index 56279908ed30..a93d8f9f8bc1 100644 > --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c > +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c > @@ -1222,7 +1222,7 @@ static int __igt_mmap_migrate(struct intel_memory_region **placements, > } > > err = intel_context_migrate_clear(to_gt(i915)->migrate.context, NULL, > - obj->mm.pages->sgl, obj->cache_level, > + obj->mm.pages->sgl, obj->pat_index, > i915_gem_object_is_lmem(obj), > expand32(POISON_INUSE), &rq); > i915_gem_object_unpin_pages(obj); > diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c > index 5aaacc53fa4c..c2bdc133c89a 100644 > --- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c > +++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c > @@ -109,7 +109,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm, > > static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); > @@ -117,7 +117,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, > unsigned int first_entry = vma_res->start / I915_GTT_PAGE_SIZE; > unsigned int act_pt = first_entry / GEN6_PTES; > unsigned int act_pte = first_entry % GEN6_PTES; > - const u32 pte_encode = vm->pte_encode(0, cache_level, flags); > + const u32 pte_encode = vm->pte_encode(0, pat_index, flags); > struct sgt_dma iter = sgt_dma(vma_res); > gen6_pte_t *vaddr; > > @@ -227,7 +227,9 @@ static int gen6_ppgtt_init_scratch(struct gen6_ppgtt *ppgtt) > > vm->scratch[0]->encode = > vm->pte_encode(px_dma(vm->scratch[0]), > - I915_CACHE_NONE, PTE_READ_ONLY); > + i915_gem_get_pat_index(vm->i915, > + I915_CACHE_NONE), > + PTE_READ_ONLY); > > vm->scratch[1] = vm->alloc_pt_dma(vm, I915_GTT_PAGE_SIZE_4K); > if (IS_ERR(vm->scratch[1])) { > @@ -278,7 +280,7 @@ static void gen6_ppgtt_cleanup(struct i915_address_space *vm) > static void pd_vma_bind(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 unused) > { > struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); > diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c > index bb6998d67133..f948d33e5ec5 100644 > --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c > +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c > @@ -29,7 +29,7 @@ static u64 gen8_pde_encode(const dma_addr_t addr, > } > > static u64 gen8_pte_encode(dma_addr_t addr, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; > @@ -40,7 +40,12 @@ static u64 gen8_pte_encode(dma_addr_t addr, > if (flags & PTE_LM) > pte |= GEN12_PPGTT_PTE_LM; > > - switch (level) { > + /* > + * For pre-gen12 platforms pat_index is the same as enum > + * i915_cache_level, so the switch-case here is still valid. > + * See translation table defined by LEGACY_CACHELEVEL. > + */ > + switch (pat_index) { > case I915_CACHE_NONE: > pte |= PPAT_UNCACHED; > break; > @@ -55,9 +60,9 @@ static u64 gen8_pte_encode(dma_addr_t addr, > return pte; > } > > -static u64 mtl_pte_encode(dma_addr_t addr, > - enum i915_cache_level level, > - u32 flags) > +static u64 gen12_pte_encode(dma_addr_t addr, > + unsigned int pat_index, > + u32 flags) > { > gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; > > @@ -67,24 +72,17 @@ static u64 mtl_pte_encode(dma_addr_t addr, > if (flags & PTE_LM) > pte |= GEN12_PPGTT_PTE_LM; > > - switch (level) { > - case I915_CACHE_NONE: > - pte |= GEN12_PPGTT_PTE_PAT1; > - break; > - case I915_CACHE_LLC: > - case I915_CACHE_L3_LLC: > - pte |= GEN12_PPGTT_PTE_PAT0 | GEN12_PPGTT_PTE_PAT1; > - break; > - case I915_CACHE_WT: > + if (pat_index & BIT(0)) > pte |= GEN12_PPGTT_PTE_PAT0; > - break; > - default: > - /* This should never happen. Added to deal with the compile > - * error due to the addition of I915_MAX_CACHE_LEVEL. Will > - * be removed by the pat_index patch. > - */ > - break; > - } > + > + if (pat_index & BIT(1)) > + pte |= GEN12_PPGTT_PTE_PAT1; > + > + if (pat_index & BIT(2)) > + pte |= GEN12_PPGTT_PTE_PAT2; > + > + if (pat_index & BIT(3)) > + pte |= MTL_PPGTT_PTE_PAT3; > > return pte; > } > @@ -457,11 +455,11 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt, > struct i915_page_directory *pdp, > struct sgt_dma *iter, > u64 idx, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > struct i915_page_directory *pd; > - const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, cache_level, flags); > + const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, pat_index, flags); > gen8_pte_t *vaddr; > > pd = i915_pd_entry(pdp, gen8_pd_index(idx, 2)); > @@ -504,10 +502,10 @@ static void > xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > struct sgt_dma *iter, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); > + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); > unsigned int rem = sg_dma_len(iter->sg); > u64 start = vma_res->start; > u64 end = start + vma_res->vma_size; > @@ -612,10 +610,10 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, > static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > struct sgt_dma *iter, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); > + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); > unsigned int rem = sg_dma_len(iter->sg); > u64 start = vma_res->start; > > @@ -735,7 +733,7 @@ static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, > > static void gen8_ppgtt_insert(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(vm); > @@ -743,9 +741,9 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, > > if (vma_res->bi.page_sizes.sg > I915_GTT_PAGE_SIZE) { > if (GRAPHICS_VER_FULL(vm->i915) >= IP_VER(12, 50)) > - xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); > + xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); > else > - gen8_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); > + gen8_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); > } else { > u64 idx = vma_res->start >> GEN8_PTE_SHIFT; > > @@ -754,7 +752,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, > gen8_pdp_for_page_index(vm, idx); > > idx = gen8_ppgtt_insert_pte(ppgtt, pdp, &iter, idx, > - cache_level, flags); > + pat_index, flags); > } while (idx); > > vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; > @@ -764,7 +762,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, > static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > u64 idx = offset >> GEN8_PTE_SHIFT; > @@ -778,14 +776,14 @@ static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, > GEM_BUG_ON(pt->is_compact); > > vaddr = px_vaddr(pt); > - vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, level, flags); > + vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, pat_index, flags); > drm_clflush_virt_range(&vaddr[gen8_pd_index(idx, 0)], sizeof(*vaddr)); > } > > static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > u64 idx = offset >> GEN8_PTE_SHIFT; > @@ -808,20 +806,20 @@ static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, > } > > vaddr = px_vaddr(pt); > - vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, level, flags); > + vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, pat_index, flags); > } > > static void xehpsdv_ppgtt_insert_entry(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > if (flags & PTE_LM) > return __xehpsdv_ppgtt_insert_entry_lm(vm, addr, offset, > - level, flags); > + pat_index, flags); > > - return gen8_ppgtt_insert_entry(vm, addr, offset, level, flags); > + return gen8_ppgtt_insert_entry(vm, addr, offset, pat_index, flags); > } > > static int gen8_init_scratch(struct i915_address_space *vm) > @@ -856,7 +854,9 @@ static int gen8_init_scratch(struct i915_address_space *vm) > > vm->scratch[0]->encode = > vm->pte_encode(px_dma(vm->scratch[0]), > - I915_CACHE_NONE, pte_flags); > + i915_gem_get_pat_index(vm->i915, > + I915_CACHE_NONE), > + pte_flags); > > for (i = 1; i <= vm->top; i++) { > struct drm_i915_gem_object *obj; > @@ -998,8 +998,8 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, > */ > ppgtt->vm.alloc_scratch_dma = alloc_pt_dma; > > - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70)) > - ppgtt->vm.pte_encode = mtl_pte_encode; > + if (GRAPHICS_VER(gt->i915) >= 12) > + ppgtt->vm.pte_encode = gen12_pte_encode; > else > ppgtt->vm.pte_encode = gen8_pte_encode; > > diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h > index f541d19264b4..19c635441642 100644 > --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h > +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h > @@ -10,13 +10,12 @@ > > struct i915_address_space; > struct intel_gt; > -enum i915_cache_level; > > struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, > unsigned long lmem_pt_obj_flags); > > u64 gen8_ggtt_pte_encode(dma_addr_t addr, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags); > > #endif > diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c > index c8390d03fce2..2a7942fac798 100644 > --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c > +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c > @@ -221,7 +221,7 @@ static void guc_ggtt_invalidate(struct i915_ggtt *ggtt) > } > > static u64 mtl_ggtt_pte_encode(dma_addr_t addr, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; > @@ -231,30 +231,17 @@ static u64 mtl_ggtt_pte_encode(dma_addr_t addr, > if (flags & PTE_LM) > pte |= GEN12_GGTT_PTE_LM; > > - switch (level) { > - case I915_CACHE_NONE: > - pte |= MTL_GGTT_PTE_PAT1; > - break; > - case I915_CACHE_LLC: > - case I915_CACHE_L3_LLC: > - pte |= MTL_GGTT_PTE_PAT0 | MTL_GGTT_PTE_PAT1; > - break; > - case I915_CACHE_WT: > + if (pat_index & BIT(0)) > pte |= MTL_GGTT_PTE_PAT0; > - break; > - default: > - /* This should never happen. Added to deal with the compile > - * error due to the addition of I915_MAX_CACHE_LEVEL. Will > - * be removed by the pat_index patch. > - */ > - break; > - } > + > + if (pat_index & BIT(1)) > + pte |= MTL_GGTT_PTE_PAT1; > > return pte; > } > > u64 gen8_ggtt_pte_encode(dma_addr_t addr, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; > @@ -273,25 +260,25 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) > static void gen8_ggtt_insert_page(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); > gen8_pte_t __iomem *pte = > (gen8_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; > > - gen8_set_pte(pte, ggtt->vm.pte_encode(addr, level, flags)); > + gen8_set_pte(pte, ggtt->vm.pte_encode(addr, pat_index, flags)); > > ggtt->invalidate(ggtt); > } > > static void gen8_ggtt_insert_entries(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); > - const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, level, flags); > + const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, pat_index, flags); > gen8_pte_t __iomem *gte; > gen8_pte_t __iomem *end; > struct sgt_iter iter; > @@ -348,14 +335,14 @@ static void gen8_ggtt_clear_range(struct i915_address_space *vm, > static void gen6_ggtt_insert_page(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); > gen6_pte_t __iomem *pte = > (gen6_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; > > - iowrite32(vm->pte_encode(addr, level, flags), pte); > + iowrite32(vm->pte_encode(addr, pat_index, flags), pte); > > ggtt->invalidate(ggtt); > } > @@ -368,7 +355,7 @@ static void gen6_ggtt_insert_page(struct i915_address_space *vm, > */ > static void gen6_ggtt_insert_entries(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); > @@ -385,7 +372,7 @@ static void gen6_ggtt_insert_entries(struct i915_address_space *vm, > iowrite32(vm->scratch[0]->encode, gte++); > end += (vma_res->node_size + vma_res->guard) / I915_GTT_PAGE_SIZE; > for_each_sgt_daddr(addr, iter, vma_res->bi.pages) > - iowrite32(vm->pte_encode(addr, level, flags), gte++); > + iowrite32(vm->pte_encode(addr, pat_index, flags), gte++); > GEM_BUG_ON(gte > end); > > /* Fill the allocated but "unused" space beyond the end of the buffer */ > @@ -420,14 +407,15 @@ struct insert_page { > struct i915_address_space *vm; > dma_addr_t addr; > u64 offset; > - enum i915_cache_level level; > + unsigned int pat_index; > }; > > static int bxt_vtd_ggtt_insert_page__cb(void *_arg) > { > struct insert_page *arg = _arg; > > - gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, arg->level, 0); > + gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, > + arg->pat_index, 0); > bxt_vtd_ggtt_wa(arg->vm); > > return 0; > @@ -436,10 +424,10 @@ static int bxt_vtd_ggtt_insert_page__cb(void *_arg) > static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 unused) > { > - struct insert_page arg = { vm, addr, offset, level }; > + struct insert_page arg = { vm, addr, offset, pat_index }; > > stop_machine(bxt_vtd_ggtt_insert_page__cb, &arg, NULL); > } > @@ -447,7 +435,7 @@ static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, > struct insert_entries { > struct i915_address_space *vm; > struct i915_vma_resource *vma_res; > - enum i915_cache_level level; > + unsigned int pat_index; > u32 flags; > }; > > @@ -455,7 +443,8 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) > { > struct insert_entries *arg = _arg; > > - gen8_ggtt_insert_entries(arg->vm, arg->vma_res, arg->level, arg->flags); > + gen8_ggtt_insert_entries(arg->vm, arg->vma_res, > + arg->pat_index, arg->flags); > bxt_vtd_ggtt_wa(arg->vm); > > return 0; > @@ -463,10 +452,10 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) > > static void bxt_vtd_ggtt_insert_entries__BKL(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > - struct insert_entries arg = { vm, vma_res, level, flags }; > + struct insert_entries arg = { vm, vma_res, pat_index, flags }; > > stop_machine(bxt_vtd_ggtt_insert_entries__cb, &arg, NULL); > } > @@ -495,7 +484,7 @@ static void gen6_ggtt_clear_range(struct i915_address_space *vm, > void intel_ggtt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > u32 pte_flags; > @@ -512,7 +501,7 @@ void intel_ggtt_bind_vma(struct i915_address_space *vm, > if (vma_res->bi.lmem) > pte_flags |= PTE_LM; > > - vm->insert_entries(vm, vma_res, cache_level, pte_flags); > + vm->insert_entries(vm, vma_res, pat_index, pte_flags); > vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; > } > > @@ -661,7 +650,7 @@ static int init_ggtt(struct i915_ggtt *ggtt) > static void aliasing_gtt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > u32 pte_flags; > @@ -673,10 +662,10 @@ static void aliasing_gtt_bind_vma(struct i915_address_space *vm, > > if (flags & I915_VMA_LOCAL_BIND) > ppgtt_bind_vma(&i915_vm_to_ggtt(vm)->alias->vm, > - stash, vma_res, cache_level, flags); > + stash, vma_res, pat_index, flags); > > if (flags & I915_VMA_GLOBAL_BIND) > - vm->insert_entries(vm, vma_res, cache_level, pte_flags); > + vm->insert_entries(vm, vma_res, pat_index, pte_flags); > > vma_res->bound_flags |= flags; > } > @@ -933,7 +922,9 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size) > > ggtt->vm.scratch[0]->encode = > ggtt->vm.pte_encode(px_dma(ggtt->vm.scratch[0]), > - I915_CACHE_NONE, pte_flags); > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), > + pte_flags); > > return 0; > } > @@ -1022,6 +1013,11 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt) > return ggtt_probe_common(ggtt, size); > } > > +/* > + * For pre-gen8 platforms pat_index is the same as enum i915_cache_level, > + * so these PTE encode functions are left with using cache_level. > + * See translation table LEGACY_CACHELEVEL. > + */ > static u64 snb_pte_encode(dma_addr_t addr, > enum i915_cache_level level, > u32 flags) > @@ -1302,7 +1298,9 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) > */ > vma->resource->bound_flags = 0; > vma->ops->bind_vma(vm, NULL, vma->resource, > - obj ? obj->cache_level : 0, > + obj ? obj->pat_index : > + i915_gem_get_pat_index(vm->i915, > + I915_CACHE_NONE), > was_bound); > > if (obj) { /* only used during resume => exclusive access */ > diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h > index 9aff343beaa8..4d6296cdbcfd 100644 > --- a/drivers/gpu/drm/i915/gt/intel_gtt.h > +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h > @@ -163,8 +163,6 @@ typedef u64 gen8_pte_t; > #define MTL_3_COH_2W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 3) > #define MTL_2_COH_1W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 2) > > -enum i915_cache_level; > - > struct drm_i915_gem_object; > struct i915_fence_reg; > struct i915_vma; > @@ -232,7 +230,7 @@ struct i915_vma_ops { > void (*bind_vma)(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > /* > * Unmap an object from an address space. This usually consists of > @@ -304,7 +302,7 @@ struct i915_address_space { > (*alloc_scratch_dma)(struct i915_address_space *vm, int sz); > > u64 (*pte_encode)(dma_addr_t addr, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags); /* Create a valid PTE */ > #define PTE_READ_ONLY BIT(0) > #define PTE_LM BIT(1) > @@ -319,20 +317,20 @@ struct i915_address_space { > void (*insert_page)(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void (*insert_entries)(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void (*raw_insert_page)(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void (*raw_insert_entries)(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void (*cleanup)(struct i915_address_space *vm); > > @@ -579,7 +577,7 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt, > void intel_ggtt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void intel_ggtt_unbind_vma(struct i915_address_space *vm, > struct i915_vma_resource *vma_res); > @@ -657,7 +655,7 @@ void gen6_ggtt_invalidate(struct i915_ggtt *ggtt); > void ppgtt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags); > void ppgtt_unbind_vma(struct i915_address_space *vm, > struct i915_vma_resource *vma_res); > diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c > index e0998879a0e1..6023288b0e2d 100644 > --- a/drivers/gpu/drm/i915/gt/intel_migrate.c > +++ b/drivers/gpu/drm/i915/gt/intel_migrate.c > @@ -45,7 +45,9 @@ static void xehpsdv_toggle_pdes(struct i915_address_space *vm, > * Insert a dummy PTE into every PT that will map to LMEM to ensure > * we have a correctly setup PDE structure for later use. > */ > - vm->insert_page(vm, 0, d->offset, I915_CACHE_NONE, PTE_LM); > + vm->insert_page(vm, 0, d->offset, > + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), > + PTE_LM); > GEM_BUG_ON(!pt->is_compact); > d->offset += SZ_2M; > } > @@ -63,7 +65,9 @@ static void xehpsdv_insert_pte(struct i915_address_space *vm, > * alignment is 64K underneath for the pt, and we are careful > * not to access the space in the void. > */ > - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, PTE_LM); > + vm->insert_page(vm, px_dma(pt), d->offset, > + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), > + PTE_LM); > d->offset += SZ_64K; > } > > @@ -73,7 +77,8 @@ static void insert_pte(struct i915_address_space *vm, > { > struct insert_pte_data *d = data; > > - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, > + vm->insert_page(vm, px_dma(pt), d->offset, > + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), > i915_gem_object_is_lmem(pt->base) ? PTE_LM : 0); > d->offset += PAGE_SIZE; > } > @@ -356,13 +361,13 @@ static int max_pte_pkt_size(struct i915_request *rq, int pkt) > > static int emit_pte(struct i915_request *rq, > struct sgt_dma *it, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > u64 offset, > int length) > { > bool has_64K_pages = HAS_64K_PAGES(rq->engine->i915); > - const u64 encode = rq->context->vm->pte_encode(0, cache_level, > + const u64 encode = rq->context->vm->pte_encode(0, pat_index, > is_lmem ? PTE_LM : 0); > struct intel_ring *ring = rq->ring; > int pkt, dword_length; > @@ -673,17 +678,17 @@ int > intel_context_migrate_copy(struct intel_context *ce, > const struct i915_deps *deps, > struct scatterlist *src, > - enum i915_cache_level src_cache_level, > + unsigned int src_pat_index, > bool src_is_lmem, > struct scatterlist *dst, > - enum i915_cache_level dst_cache_level, > + unsigned int dst_pat_index, > bool dst_is_lmem, > struct i915_request **out) > { > struct sgt_dma it_src = sg_sgt(src), it_dst = sg_sgt(dst), it_ccs; > struct drm_i915_private *i915 = ce->engine->i915; > u64 ccs_bytes_to_cpy = 0, bytes_to_cpy; > - enum i915_cache_level ccs_cache_level; > + unsigned int ccs_pat_index; > u32 src_offset, dst_offset; > u8 src_access, dst_access; > struct i915_request *rq; > @@ -707,12 +712,12 @@ intel_context_migrate_copy(struct intel_context *ce, > dst_sz = scatter_list_length(dst); > if (src_is_lmem) { > it_ccs = it_dst; > - ccs_cache_level = dst_cache_level; > + ccs_pat_index = dst_pat_index; > ccs_is_src = false; > } else if (dst_is_lmem) { > bytes_to_cpy = dst_sz; > it_ccs = it_src; > - ccs_cache_level = src_cache_level; > + ccs_pat_index = src_pat_index; > ccs_is_src = true; > } > > @@ -773,7 +778,7 @@ intel_context_migrate_copy(struct intel_context *ce, > src_sz = calculate_chunk_sz(i915, src_is_lmem, > bytes_to_cpy, ccs_bytes_to_cpy); > > - len = emit_pte(rq, &it_src, src_cache_level, src_is_lmem, > + len = emit_pte(rq, &it_src, src_pat_index, src_is_lmem, > src_offset, src_sz); > if (!len) { > err = -EINVAL; > @@ -784,7 +789,7 @@ intel_context_migrate_copy(struct intel_context *ce, > goto out_rq; > } > > - err = emit_pte(rq, &it_dst, dst_cache_level, dst_is_lmem, > + err = emit_pte(rq, &it_dst, dst_pat_index, dst_is_lmem, > dst_offset, len); > if (err < 0) > goto out_rq; > @@ -811,7 +816,7 @@ intel_context_migrate_copy(struct intel_context *ce, > goto out_rq; > > ccs_sz = GET_CCS_BYTES(i915, len); > - err = emit_pte(rq, &it_ccs, ccs_cache_level, false, > + err = emit_pte(rq, &it_ccs, ccs_pat_index, false, > ccs_is_src ? src_offset : dst_offset, > ccs_sz); > if (err < 0) > @@ -979,7 +984,7 @@ int > intel_context_migrate_clear(struct intel_context *ce, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > u32 value, > struct i915_request **out) > @@ -1027,7 +1032,7 @@ intel_context_migrate_clear(struct intel_context *ce, > if (err) > goto out_rq; > > - len = emit_pte(rq, &it, cache_level, is_lmem, offset, CHUNK_SZ); > + len = emit_pte(rq, &it, pat_index, is_lmem, offset, CHUNK_SZ); > if (len <= 0) { > err = len; > goto out_rq; > @@ -1074,10 +1079,10 @@ int intel_migrate_copy(struct intel_migrate *m, > struct i915_gem_ww_ctx *ww, > const struct i915_deps *deps, > struct scatterlist *src, > - enum i915_cache_level src_cache_level, > + unsigned int src_pat_index, > bool src_is_lmem, > struct scatterlist *dst, > - enum i915_cache_level dst_cache_level, > + unsigned int dst_pat_index, > bool dst_is_lmem, > struct i915_request **out) > { > @@ -1098,8 +1103,8 @@ int intel_migrate_copy(struct intel_migrate *m, > goto out; > > err = intel_context_migrate_copy(ce, deps, > - src, src_cache_level, src_is_lmem, > - dst, dst_cache_level, dst_is_lmem, > + src, src_pat_index, src_is_lmem, > + dst, dst_pat_index, dst_is_lmem, > out); > > intel_context_unpin(ce); > @@ -1113,7 +1118,7 @@ intel_migrate_clear(struct intel_migrate *m, > struct i915_gem_ww_ctx *ww, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > u32 value, > struct i915_request **out) > @@ -1134,7 +1139,7 @@ intel_migrate_clear(struct intel_migrate *m, > if (err) > goto out; > > - err = intel_context_migrate_clear(ce, deps, sg, cache_level, > + err = intel_context_migrate_clear(ce, deps, sg, pat_index, > is_lmem, value, out); > > intel_context_unpin(ce); > diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.h b/drivers/gpu/drm/i915/gt/intel_migrate.h > index ccc677ec4aa3..11fc09a00c4b 100644 > --- a/drivers/gpu/drm/i915/gt/intel_migrate.h > +++ b/drivers/gpu/drm/i915/gt/intel_migrate.h > @@ -16,7 +16,6 @@ struct i915_request; > struct i915_gem_ww_ctx; > struct intel_gt; > struct scatterlist; > -enum i915_cache_level; > > int intel_migrate_init(struct intel_migrate *m, struct intel_gt *gt); > > @@ -26,20 +25,20 @@ int intel_migrate_copy(struct intel_migrate *m, > struct i915_gem_ww_ctx *ww, > const struct i915_deps *deps, > struct scatterlist *src, > - enum i915_cache_level src_cache_level, > + unsigned int src_pat_index, > bool src_is_lmem, > struct scatterlist *dst, > - enum i915_cache_level dst_cache_level, > + unsigned int dst_pat_index, > bool dst_is_lmem, > struct i915_request **out); > > int intel_context_migrate_copy(struct intel_context *ce, > const struct i915_deps *deps, > struct scatterlist *src, > - enum i915_cache_level src_cache_level, > + unsigned int src_pat_index, > bool src_is_lmem, > struct scatterlist *dst, > - enum i915_cache_level dst_cache_level, > + unsigned int dst_pat_index, > bool dst_is_lmem, > struct i915_request **out); > > @@ -48,7 +47,7 @@ intel_migrate_clear(struct intel_migrate *m, > struct i915_gem_ww_ctx *ww, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > u32 value, > struct i915_request **out); > @@ -56,7 +55,7 @@ int > intel_context_migrate_clear(struct intel_context *ce, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > u32 value, > struct i915_request **out); > diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/gt/intel_ppgtt.c > index 7ecfa672f738..436756bfbb1a 100644 > --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c > +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c > @@ -181,7 +181,7 @@ struct i915_ppgtt *i915_ppgtt_create(struct intel_gt *gt, > void ppgtt_bind_vma(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > u32 pte_flags; > @@ -199,7 +199,7 @@ void ppgtt_bind_vma(struct i915_address_space *vm, > if (vma_res->bi.lmem) > pte_flags |= PTE_LM; > > - vm->insert_entries(vm, vma_res, cache_level, pte_flags); > + vm->insert_entries(vm, vma_res, pat_index, pte_flags); > wmb(); > } > > diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c > index e677f2da093d..3def5ca72dec 100644 > --- a/drivers/gpu/drm/i915/gt/selftest_migrate.c > +++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c > @@ -137,7 +137,7 @@ static int copy(struct intel_migrate *migrate, > static int intel_context_copy_ccs(struct intel_context *ce, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool write_to_ccs, > struct i915_request **out) > { > @@ -185,7 +185,7 @@ static int intel_context_copy_ccs(struct intel_context *ce, > if (err) > goto out_rq; > > - len = emit_pte(rq, &it, cache_level, true, offset, CHUNK_SZ); > + len = emit_pte(rq, &it, pat_index, true, offset, CHUNK_SZ); > if (len <= 0) { > err = len; > goto out_rq; > @@ -223,7 +223,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, > struct i915_gem_ww_ctx *ww, > const struct i915_deps *deps, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool write_to_ccs, > struct i915_request **out) > { > @@ -243,7 +243,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, > if (err) > goto out; > > - err = intel_context_copy_ccs(ce, deps, sg, cache_level, > + err = intel_context_copy_ccs(ce, deps, sg, pat_index, > write_to_ccs, out); > > intel_context_unpin(ce); > @@ -300,7 +300,7 @@ static int clear(struct intel_migrate *migrate, > /* Write the obj data into ccs surface */ > err = intel_migrate_ccs_copy(migrate, &ww, NULL, > obj->mm.pages->sgl, > - obj->cache_level, > + obj->pat_index, > true, &rq); > if (rq && !err) { > if (i915_request_wait(rq, 0, HZ) < 0) { > @@ -351,7 +351,7 @@ static int clear(struct intel_migrate *migrate, > > err = intel_migrate_ccs_copy(migrate, &ww, NULL, > obj->mm.pages->sgl, > - obj->cache_level, > + obj->pat_index, > false, &rq); > if (rq && !err) { > if (i915_request_wait(rq, 0, HZ) < 0) { > @@ -414,9 +414,9 @@ static int __migrate_copy(struct intel_migrate *migrate, > struct i915_request **out) > { > return intel_migrate_copy(migrate, ww, NULL, > - src->mm.pages->sgl, src->cache_level, > + src->mm.pages->sgl, src->pat_index, > i915_gem_object_is_lmem(src), > - dst->mm.pages->sgl, dst->cache_level, > + dst->mm.pages->sgl, dst->pat_index, > i915_gem_object_is_lmem(dst), > out); > } > @@ -428,9 +428,9 @@ static int __global_copy(struct intel_migrate *migrate, > struct i915_request **out) > { > return intel_context_migrate_copy(migrate->context, NULL, > - src->mm.pages->sgl, src->cache_level, > + src->mm.pages->sgl, src->pat_index, > i915_gem_object_is_lmem(src), > - dst->mm.pages->sgl, dst->cache_level, > + dst->mm.pages->sgl, dst->pat_index, > i915_gem_object_is_lmem(dst), > out); > } > @@ -455,7 +455,7 @@ static int __migrate_clear(struct intel_migrate *migrate, > { > return intel_migrate_clear(migrate, ww, NULL, > obj->mm.pages->sgl, > - obj->cache_level, > + obj->pat_index, > i915_gem_object_is_lmem(obj), > value, out); > } > @@ -468,7 +468,7 @@ static int __global_clear(struct intel_migrate *migrate, > { > return intel_context_migrate_clear(migrate->context, NULL, > obj->mm.pages->sgl, > - obj->cache_level, > + obj->pat_index, > i915_gem_object_is_lmem(obj), > value, out); > } > @@ -648,7 +648,7 @@ static int live_emit_pte_full_ring(void *arg) > */ > pr_info("%s emite_pte ring space=%u\n", __func__, rq->ring->space); > it = sg_sgt(obj->mm.pages->sgl); > - len = emit_pte(rq, &it, obj->cache_level, false, 0, CHUNK_SZ); > + len = emit_pte(rq, &it, obj->pat_index, false, 0, CHUNK_SZ); > if (!len) { > err = -EINVAL; > goto out_rq; > @@ -844,7 +844,7 @@ static int wrap_ktime_compare(const void *A, const void *B) > > static int __perf_clear_blt(struct intel_context *ce, > struct scatterlist *sg, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > bool is_lmem, > size_t sz) > { > @@ -858,7 +858,7 @@ static int __perf_clear_blt(struct intel_context *ce, > > t0 = ktime_get(); > > - err = intel_context_migrate_clear(ce, NULL, sg, cache_level, > + err = intel_context_migrate_clear(ce, NULL, sg, pat_index, > is_lmem, 0, &rq); > if (rq) { > if (i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT) < 0) > @@ -904,7 +904,8 @@ static int perf_clear_blt(void *arg) > > err = __perf_clear_blt(gt->migrate.context, > dst->mm.pages->sgl, > - I915_CACHE_NONE, > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > i915_gem_object_is_lmem(dst), > sizes[i]); > > @@ -919,10 +920,10 @@ static int perf_clear_blt(void *arg) > > static int __perf_copy_blt(struct intel_context *ce, > struct scatterlist *src, > - enum i915_cache_level src_cache_level, > + unsigned int src_pat_index, > bool src_is_lmem, > struct scatterlist *dst, > - enum i915_cache_level dst_cache_level, > + unsigned int dst_pat_index, > bool dst_is_lmem, > size_t sz) > { > @@ -937,9 +938,9 @@ static int __perf_copy_blt(struct intel_context *ce, > t0 = ktime_get(); > > err = intel_context_migrate_copy(ce, NULL, > - src, src_cache_level, > + src, src_pat_index, > src_is_lmem, > - dst, dst_cache_level, > + dst, dst_pat_index, > dst_is_lmem, > &rq); > if (rq) { > @@ -994,10 +995,12 @@ static int perf_copy_blt(void *arg) > > err = __perf_copy_blt(gt->migrate.context, > src->mm.pages->sgl, > - I915_CACHE_NONE, > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > i915_gem_object_is_lmem(src), > dst->mm.pages->sgl, > - I915_CACHE_NONE, > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > i915_gem_object_is_lmem(dst), > sz); > > diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c > index a9e0a91bc0e0..79aa6ac66ad2 100644 > --- a/drivers/gpu/drm/i915/gt/selftest_reset.c > +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c > @@ -86,7 +86,9 @@ __igt_reset_stolen(struct intel_gt *gt, > > ggtt->vm.insert_page(&ggtt->vm, dma, > ggtt->error_capture.start, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > + 0); > mb(); > > s = io_mapping_map_wc(&ggtt->iomap, > @@ -127,7 +129,9 @@ __igt_reset_stolen(struct intel_gt *gt, > > ggtt->vm.insert_page(&ggtt->vm, dma, > ggtt->error_capture.start, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > + 0); > mb(); > > s = io_mapping_map_wc(&ggtt->iomap, > diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c > index 9f536c251179..39c3ec12df1a 100644 > --- a/drivers/gpu/drm/i915/gt/selftest_timeline.c > +++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c > @@ -836,7 +836,7 @@ static int setup_watcher(struct hwsp_watcher *w, struct intel_gt *gt, > return PTR_ERR(obj); > > /* keep the same cache settings as timeline */ > - i915_gem_object_set_cache_coherency(obj, tl->hwsp_ggtt->obj->cache_level); > + i915_gem_object_set_pat_index(obj, tl->hwsp_ggtt->obj->pat_index); > w->map = i915_gem_object_pin_map_unlocked(obj, > page_unmask_bits(tl->hwsp_ggtt->obj->mm.mapping)); > if (IS_ERR(w->map)) { > diff --git a/drivers/gpu/drm/i915/gt/selftest_tlb.c b/drivers/gpu/drm/i915/gt/selftest_tlb.c > index e6cac1f15d6e..4493c8518e91 100644 > --- a/drivers/gpu/drm/i915/gt/selftest_tlb.c > +++ b/drivers/gpu/drm/i915/gt/selftest_tlb.c > @@ -36,6 +36,8 @@ pte_tlbinv(struct intel_context *ce, > u64 length, > struct rnd_state *prng) > { > + const unsigned int pat_index = > + i915_gem_get_pat_index(ce->vm->i915, I915_CACHE_NONE); > struct drm_i915_gem_object *batch; > struct drm_mm_node vb_node; > struct i915_request *rq; > @@ -155,7 +157,7 @@ pte_tlbinv(struct intel_context *ce, > /* Flip the PTE between A and B */ > if (i915_gem_object_is_lmem(vb->obj)) > pte_flags |= PTE_LM; > - ce->vm->insert_entries(ce->vm, &vb_res, 0, pte_flags); > + ce->vm->insert_entries(ce->vm, &vb_res, pat_index, pte_flags); > > /* Flush the PTE update to concurrent HW */ > tlbinv(ce->vm, addr & -length, length); > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c > index 4ec7df9ed5ff..e34c79120090 100644 > --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c > +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c > @@ -957,9 +957,15 @@ static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw) > pte_flags |= PTE_LM; > > if (ggtt->vm.raw_insert_entries) > - ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); > + ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, > + i915_gem_get_pat_index(ggtt->vm.i915, > + I915_CACHE_NONE), > + pte_flags); > else > - ggtt->vm.insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); > + ggtt->vm.insert_entries(&ggtt->vm, dummy, > + i915_gem_get_pat_index(ggtt->vm.i915, > + I915_CACHE_NONE), > + pte_flags); > } > > static void uc_fw_unbind_ggtt(struct intel_uc_fw *uc_fw) > diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c > index 41389a32e998..76ccd4e03e31 100644 > --- a/drivers/gpu/drm/i915/i915_debugfs.c > +++ b/drivers/gpu/drm/i915/i915_debugfs.c > @@ -139,21 +139,54 @@ static const char *stringify_vma_type(const struct i915_vma *vma) > return "ppgtt"; > } > > -static const char *i915_cache_level_str(struct drm_i915_private *i915, int type) > -{ > - switch (type) { > - case I915_CACHE_NONE: return " uncached"; > - case I915_CACHE_LLC: return HAS_LLC(i915) ? " LLC" : " snooped"; > - case I915_CACHE_L3_LLC: return " L3+LLC"; > - case I915_CACHE_WT: return " WT"; > - default: return ""; > +static const char *i915_cache_level_str(struct drm_i915_gem_object *obj) > +{ > + struct drm_i915_private *i915 = obj_to_i915(obj); > + > + if (IS_METEORLAKE(i915)) { > + switch (obj->pat_index) { > + case 0: return " WB"; > + case 1: return " WT"; > + case 2: return " UC"; > + case 3: return " WB (1-Way Coh)"; > + case 4: return " WB (2-Way Coh)"; > + default: return " not defined"; > + } > + } else if (IS_PONTEVECCHIO(i915)) { > + switch (obj->pat_index) { > + case 0: return " UC"; > + case 1: return " WC"; > + case 2: return " WT"; > + case 3: return " WB"; > + case 4: return " WT (CLOS1)"; > + case 5: return " WB (CLOS1)"; > + case 6: return " WT (CLOS2)"; > + case 7: return " WT (CLOS2)"; > + default: return " not defined"; > + } > + } else if (GRAPHICS_VER(i915) >= 12) { > + switch (obj->pat_index) { > + case 0: return " WB"; > + case 1: return " WC"; > + case 2: return " WT"; > + case 3: return " UC"; > + default: return " not defined"; > + } > + } else { > + switch (obj->pat_index) { > + case 0: return " UC"; > + case 1: return HAS_LLC(i915) ? > + " LLC" : " snooped"; > + case 2: return " L3+LLC"; > + case 3: return " WT"; > + default: return " not defined"; > + } > } > } > > void > i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) > { > - struct drm_i915_private *dev_priv = to_i915(obj->base.dev); > struct i915_vma *vma; > int pin_count = 0; > > @@ -165,7 +198,7 @@ i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) > obj->base.size / 1024, > obj->read_domains, > obj->write_domain, > - i915_cache_level_str(dev_priv, obj->cache_level), > + i915_cache_level_str(obj), > obj->mm.dirty ? " dirty" : "", > obj->mm.madv == I915_MADV_DONTNEED ? " purgeable" : ""); > if (obj->base.name) > diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c > index 0a78bdbd36b1..38933c00f705 100644 > --- a/drivers/gpu/drm/i915/i915_gem.c > +++ b/drivers/gpu/drm/i915/i915_gem.c > @@ -420,8 +420,11 @@ i915_gem_gtt_pread(struct drm_i915_gem_object *obj, > page_length = remain < page_length ? remain : page_length; > if (drm_mm_node_allocated(&node)) { > ggtt->vm.insert_page(&ggtt->vm, > - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), > - node.start, I915_CACHE_NONE, 0); > + i915_gem_object_get_dma_address(obj, > + offset >> PAGE_SHIFT), > + node.start, > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), 0); > } else { > page_base += offset & PAGE_MASK; > } > @@ -598,8 +601,11 @@ i915_gem_gtt_pwrite_fast(struct drm_i915_gem_object *obj, > /* flush the write before we modify the GGTT */ > intel_gt_flush_ggtt_writes(ggtt->vm.gt); > ggtt->vm.insert_page(&ggtt->vm, > - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), > - node.start, I915_CACHE_NONE, 0); > + i915_gem_object_get_dma_address(obj, > + offset >> PAGE_SHIFT), > + node.start, > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), 0); > wmb(); /* flush modifications to the GGTT (insert_page) */ > } else { > page_base += offset & PAGE_MASK; > @@ -1142,6 +1148,19 @@ int i915_gem_init(struct drm_i915_private *dev_priv) > unsigned int i; > int ret; > > + /* > + * In the proccess of replacing cache_level with pat_index a tricky > + * dependency is created on the definition of the enum i915_cache_level. > + * in case this enum is changed, PTE encode would be broken. _I_n > + * Add a WARNING here. And remove when we completely quit using this > + * enum > + */ > + WARN_ON_ONCE(I915_CACHE_NONE != 0 || > + I915_CACHE_LLC != 1 || > + I915_CACHE_L3_LLC != 2 || > + I915_CACHE_WT != 3 || > + I915_MAX_CACHE_LEVEL != 4); I thought BUILD_BUG_ON would be better. > + > /* We need to fallback to 4K pages if host doesn't support huge gtt. */ > if (intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv)) > RUNTIME_INFO(dev_priv)->page_sizes = I915_GTT_PAGE_SIZE_4K; > diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c > index 8ffdd7f47e93..566536499954 100644 > --- a/drivers/gpu/drm/i915/i915_gpu_error.c > +++ b/drivers/gpu/drm/i915/i915_gpu_error.c > @@ -1122,10 +1122,14 @@ i915_vma_coredump_create(const struct intel_gt *gt, > mutex_lock(&ggtt->error_mutex); > if (ggtt->vm.raw_insert_page) > ggtt->vm.raw_insert_page(&ggtt->vm, dma, slot, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > + 0); > else > ggtt->vm.insert_page(&ggtt->vm, dma, slot, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(gt->i915, > + I915_CACHE_NONE), > + 0); > mb(); > > s = io_mapping_map_wc(&ggtt->iomap, slot, PAGE_SIZE); > diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c > index 20a44788999e..a814775a363d 100644 > --- a/drivers/gpu/drm/i915/i915_vma.c > +++ b/drivers/gpu/drm/i915/i915_vma.c > @@ -315,7 +315,7 @@ struct i915_vma_work { > struct i915_vma_resource *vma_res; > struct drm_i915_gem_object *obj; > struct i915_sw_dma_fence_cb cb; > - enum i915_cache_level cache_level; > + unsigned int pat_index; > unsigned int flags; > }; > > @@ -334,7 +334,7 @@ static void __vma_bind(struct dma_fence_work *work) > return; > > vma_res->ops->bind_vma(vma_res->vm, &vw->stash, > - vma_res, vw->cache_level, vw->flags); > + vma_res, vw->pat_index, vw->flags); > } > > static void __vma_release(struct dma_fence_work *work) > @@ -426,7 +426,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, > /** > * i915_vma_bind - Sets up PTEs for an VMA in it's corresponding address space. > * @vma: VMA to map > - * @cache_level: mapping cache level > + * @pat_index: PAT index to set in PTE > * @flags: flags like global or local mapping > * @work: preallocated worker for allocating and binding the PTE > * @vma_res: pointer to a preallocated vma resource. The resource is either > @@ -437,7 +437,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, > * Note that DMA addresses are also the only part of the SG table we care about. > */ > int i915_vma_bind(struct i915_vma *vma, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags, > struct i915_vma_work *work, > struct i915_vma_resource *vma_res) > @@ -507,7 +507,7 @@ int i915_vma_bind(struct i915_vma *vma, > struct dma_fence *prev; > > work->vma_res = i915_vma_resource_get(vma->resource); > - work->cache_level = cache_level; > + work->pat_index = pat_index; > work->flags = bind_flags; > > /* > @@ -537,7 +537,7 @@ int i915_vma_bind(struct i915_vma *vma, > > return ret; > } > - vma->ops->bind_vma(vma->vm, NULL, vma->resource, cache_level, > + vma->ops->bind_vma(vma->vm, NULL, vma->resource, pat_index, > bind_flags); > } > > @@ -814,7 +814,7 @@ i915_vma_insert(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, > color = 0; > > if (i915_vm_has_cache_coloring(vma->vm)) > - color = vma->obj->cache_level; > + color = vma->obj->pat_index; > > if (flags & PIN_OFFSET_FIXED) { > u64 offset = flags & PIN_OFFSET_MASK; > @@ -1518,7 +1518,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, > > GEM_BUG_ON(!vma->pages); > err = i915_vma_bind(vma, > - vma->obj->cache_level, > + vma->obj->pat_index, > flags, work, vma_res); > vma_res = NULL; > if (err) > diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h > index 38c8c66ed724..9a9729205d5b 100644 > --- a/drivers/gpu/drm/i915/i915_vma.h > +++ b/drivers/gpu/drm/i915/i915_vma.h > @@ -250,7 +250,7 @@ i915_vma_compare(struct i915_vma *vma, > > struct i915_vma_work *i915_vma_work(void); > int i915_vma_bind(struct i915_vma *vma, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags, > struct i915_vma_work *work, > struct i915_vma_resource *vma_res); > diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h > index 77fda2244d16..64472b7f0e77 100644 > --- a/drivers/gpu/drm/i915/i915_vma_types.h > +++ b/drivers/gpu/drm/i915/i915_vma_types.h > @@ -32,8 +32,6 @@ > > #include "gem/i915_gem_object_types.h" > > -enum i915_cache_level; > - > /** > * DOC: Global GTT views > * > diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c > index d91d0ade8abd..61da4ed9d521 100644 > --- a/drivers/gpu/drm/i915/selftests/i915_gem.c > +++ b/drivers/gpu/drm/i915/selftests/i915_gem.c > @@ -57,7 +57,10 @@ static void trash_stolen(struct drm_i915_private *i915) > u32 __iomem *s; > int x; > > - ggtt->vm.insert_page(&ggtt->vm, dma, slot, I915_CACHE_NONE, 0); > + ggtt->vm.insert_page(&ggtt->vm, dma, slot, > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), > + 0); > > s = io_mapping_map_atomic_wc(&ggtt->iomap, slot); > for (x = 0; x < PAGE_SIZE / sizeof(u32); x++) { > diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c > index f68ef4074088..f8fe3681c3dc 100644 > --- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c > +++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c > @@ -246,7 +246,7 @@ static int igt_evict_for_cache_color(void *arg) > struct drm_mm_node target = { > .start = I915_GTT_PAGE_SIZE * 2, > .size = I915_GTT_PAGE_SIZE, > - .color = I915_CACHE_LLC, > + .color = i915_gem_get_pat_index(gt->i915, I915_CACHE_LLC), > }; > struct drm_i915_gem_object *obj; > struct i915_vma *vma; > @@ -309,7 +309,7 @@ static int igt_evict_for_cache_color(void *arg) > /* Attempt to remove the first *pinned* vma, by removing the (empty) > * neighbour -- this should fail. > */ > - target.color = I915_CACHE_L3_LLC; > + target.color = i915_gem_get_pat_index(gt->i915, I915_CACHE_L3_LLC); > > mutex_lock(&ggtt->vm.mutex); > err = i915_gem_evict_for_node(&ggtt->vm, NULL, &target, 0); > diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c > index 154801f1c468..36940ef10108 100644 > --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c > +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c > @@ -135,7 +135,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size) > > obj->write_domain = I915_GEM_DOMAIN_CPU; > obj->read_domains = I915_GEM_DOMAIN_CPU; > - obj->cache_level = I915_CACHE_NONE; > + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); > > /* Preallocate the "backing storage" */ > if (i915_gem_object_pin_pages_unlocked(obj)) > @@ -359,7 +359,9 @@ static int lowlevel_hole(struct i915_address_space *vm, > > with_intel_runtime_pm(vm->gt->uncore->rpm, wakeref) > vm->insert_entries(vm, mock_vma_res, > - I915_CACHE_NONE, 0); > + i915_gem_get_pat_index(vm->i915, > + I915_CACHE_NONE), > + 0); > } > count = n; > > @@ -1377,7 +1379,10 @@ static int igt_ggtt_page(void *arg) > > ggtt->vm.insert_page(&ggtt->vm, > i915_gem_object_get_dma_address(obj, 0), > - offset, I915_CACHE_NONE, 0); > + offset, > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), > + 0); > } > > order = i915_random_order(count, &prng); > @@ -1510,7 +1515,7 @@ static int reserve_gtt_with_resource(struct i915_vma *vma, u64 offset) > mutex_lock(&vm->mutex); > err = i915_gem_gtt_reserve(vm, NULL, &vma->node, obj->base.size, > offset, > - obj->cache_level, > + obj->pat_index, > 0); > if (!err) { > i915_vma_resource_init_from_vma(vma_res, vma); > @@ -1690,7 +1695,7 @@ static int insert_gtt_with_resource(struct i915_vma *vma) > > mutex_lock(&vm->mutex); > err = i915_gem_gtt_insert(vm, NULL, &vma->node, obj->base.size, 0, > - obj->cache_level, 0, vm->total, 0); > + obj->pat_index, 0, vm->total, 0); > if (!err) { > i915_vma_resource_init_from_vma(vma_res, vma); > vma->resource = vma_res; > diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c > index 3b18e5905c86..d985d9bae2e8 100644 > --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c > +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c > @@ -1070,7 +1070,9 @@ static int igt_lmem_write_cpu(void *arg) > /* Put the pages into a known state -- from the gpu for added fun */ > intel_engine_pm_get(engine); > err = intel_context_migrate_clear(engine->gt->migrate.context, NULL, > - obj->mm.pages->sgl, I915_CACHE_NONE, > + obj->mm.pages->sgl, > + i915_gem_get_pat_index(i915, > + I915_CACHE_NONE), > true, 0xdeadbeaf, &rq); > if (rq) { > dma_resv_add_fence(obj->base.resv, &rq->fence, > diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c > index ece97e4faacb..a516c0aa88fd 100644 > --- a/drivers/gpu/drm/i915/selftests/mock_gtt.c > +++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c > @@ -27,21 +27,21 @@ > static void mock_insert_page(struct i915_address_space *vm, > dma_addr_t addr, > u64 offset, > - enum i915_cache_level level, > + unsigned int pat_index, > u32 flags) > { > } > > static void mock_insert_entries(struct i915_address_space *vm, > struct i915_vma_resource *vma_res, > - enum i915_cache_level level, u32 flags) > + unsigned int pat_index, u32 flags) > { > } > > static void mock_bind_ppgtt(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > GEM_BUG_ON(flags & I915_VMA_GLOBAL_BIND); > @@ -94,7 +94,7 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name) > static void mock_bind_ggtt(struct i915_address_space *vm, > struct i915_vm_pt_stash *stash, > struct i915_vma_resource *vma_res, > - enum i915_cache_level cache_level, > + unsigned int pat_index, > u32 flags) > { > } With a pinky promise to improve this all in the near future I won't grumble to loudly. :) I haven't read all the details, I leave that to other reviewers, and also assuming some final tweaks as indicated above please. Regards, Tvrtko ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level 2023-05-09 12:15 ` Tvrtko Ursulin @ 2023-05-09 17:12 ` Yang, Fei 2023-05-11 12:02 ` Tvrtko Ursulin 0 siblings, 1 reply; 10+ messages in thread From: Yang, Fei @ 2023-05-09 17:12 UTC (permalink / raw) To: Tvrtko Ursulin, intel-gfx@lists.freedesktop.org Cc: Chris Wilson, Roper, Matthew D, dri-devel@lists.freedesktop.org [-- Attachment #1: Type: text/plain, Size: 104173 bytes --] > On 09/05/2023 00:48, fei.yang@intel.com wrote: >> From: Fei Yang <fei.yang@intel.com> >> >> Currently the KMD is using enum i915_cache_level to set caching policy for >> buffer objects. This is flaky because the PAT index which really controls >> the caching behavior in PTE has far more levels than what's defined in the >> enum. In addition, the PAT index is platform dependent, having to translate >> between i915_cache_level and PAT index is not reliable, and makes the code >> more complicated. >> >> From UMD's perspective there is also a necessity to set caching policy for >> performance fine tuning. It's much easier for the UMD to directly use PAT >> index because the behavior of each PAT index is clearly defined in Bspec. >> Having the abstracted i915_cache_level sitting in between would only cause >> more ambiguity. PAT is expected to work much like MOCS already works today, >> and by design userspace is expected to select the index that exactly >> matches the desired behavior described in the hardware specification. >> >> For these reasons this patch replaces i915_cache_level with PAT index. Also >> note, the cache_level is not completely removed yet, because the KMD still >> has the need of creating buffer objects with simple cache settings such as >> cached, uncached, or writethrough. For kernel objects, cache_level is used >> for simplicity and backward compatibility. For Pre-gen12 platforms PAT can >> have 1:1 mapping to i915_cache_level, so these two are interchangeable. see >> the use of LEGACY_CACHELEVEL. >> >> One consequence of this change is that gen8_pte_encode is no longer working >> for gen12 platforms due to the fact that gen12 platforms has different PAT >> definitions. In the meantime the mtl_pte_encode introduced specfically for >> MTL becomes generic for all gen12 platforms. This patch renames the MTL >> PTE encode function into gen12_pte_encode and apply it to all gen12. Even >> though this change looks unrelated, but separating them would temporarily >> break gen12 PTE encoding, thus squash them in one patch. >> >> Special note: this patch changes the way caching behavior is controlled in >> the sense that some objects are left to be managed by userspace. For such >> objects we need to be careful not to change the userspace settings.There >> are kerneldoc and comments added around obj->cache_coherent, cache_dirty, >> and how to bypass the checkings by i915_gem_object_has_cache_level. For >> full understanding, these changes need to be looked at together with the >> two follow-up patches, one disables the {set|get}_caching ioctl's and the >> other adds set_pat extension to the GEM_CREATE uAPI. >> >> Bspec: 63019 >> >> Cc: Chris Wilson <chris.p.wilson@linux.intel.com> >> Signed-off-by: Fei Yang <fei.yang@intel.com> >> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> >> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> >> >> To be squashed >> --- >> drivers/gpu/drm/i915/display/intel_dpt.c | 12 +-- >> drivers/gpu/drm/i915/gem/i915_gem_domain.c | 58 +++++++++---- >> .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 15 +++- >> drivers/gpu/drm/i915/gem/i915_gem_mman.c | 11 ++- >> drivers/gpu/drm/i915/gem/i915_gem_object.c | 51 ++++++++++- >> drivers/gpu/drm/i915/gem/i915_gem_object.h | 4 + >> .../gpu/drm/i915/gem/i915_gem_object_types.h | 44 +++++++++- >> drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 4 +- >> drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 8 +- >> .../gpu/drm/i915/gem/selftests/huge_pages.c | 2 +- >> .../drm/i915/gem/selftests/i915_gem_migrate.c | 2 +- >> .../drm/i915/gem/selftests/i915_gem_mman.c | 2 +- >> drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 10 ++- >> drivers/gpu/drm/i915/gt/gen8_ppgtt.c | 84 +++++++++---------- >> drivers/gpu/drm/i915/gt/gen8_ppgtt.h | 3 +- >> drivers/gpu/drm/i915/gt/intel_ggtt.c | 82 +++++++++--------- >> drivers/gpu/drm/i915/gt/intel_gtt.h | 18 ++-- >> drivers/gpu/drm/i915/gt/intel_migrate.c | 47 ++++++----- >> drivers/gpu/drm/i915/gt/intel_migrate.h | 13 ++- >> drivers/gpu/drm/i915/gt/intel_ppgtt.c | 4 +- >> drivers/gpu/drm/i915/gt/selftest_migrate.c | 47 ++++++----- >> drivers/gpu/drm/i915/gt/selftest_reset.c | 8 +- >> drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- >> drivers/gpu/drm/i915/gt/selftest_tlb.c | 4 +- >> drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 10 ++- >> drivers/gpu/drm/i915/i915_debugfs.c | 53 +++++++++--- >> drivers/gpu/drm/i915/i915_gem.c | 27 +++++- >> drivers/gpu/drm/i915/i915_gpu_error.c | 8 +- >> drivers/gpu/drm/i915/i915_vma.c | 16 ++-- >> drivers/gpu/drm/i915/i915_vma.h | 2 +- >> drivers/gpu/drm/i915/i915_vma_types.h | 2 - >> drivers/gpu/drm/i915/selftests/i915_gem.c | 5 +- >> .../gpu/drm/i915/selftests/i915_gem_evict.c | 4 +- >> drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 15 ++-- >> .../drm/i915/selftests/intel_memory_region.c | 4 +- >> drivers/gpu/drm/i915/selftests/mock_gtt.c | 8 +- >> 36 files changed, 451 insertions(+), 238 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/display/intel_dpt.c b/drivers/gpu/drm/i915/display/intel_dpt.c >> index c5eacfdba1a5..7c5fddb203ba 100644 >> --- a/drivers/gpu/drm/i915/display/intel_dpt.c >> +++ b/drivers/gpu/drm/i915/display/intel_dpt.c >> @@ -43,24 +43,24 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) >> static void dpt_insert_page(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_dpt *dpt = i915_vm_to_dpt(vm); >> gen8_pte_t __iomem *base = dpt->iomem; >> >> gen8_set_pte(base + offset / I915_GTT_PAGE_SIZE, >> - vm->pte_encode(addr, level, flags)); >> + vm->pte_encode(addr, pat_index, flags)); >> } >> >> static void dpt_insert_entries(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_dpt *dpt = i915_vm_to_dpt(vm); >> gen8_pte_t __iomem *base = dpt->iomem; >> - const gen8_pte_t pte_encode = vm->pte_encode(0, level, flags); >> + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); >> struct sgt_iter sgt_iter; >> dma_addr_t addr; >> int i; >> @@ -83,7 +83,7 @@ static void dpt_clear_range(struct i915_address_space *vm, >> static void dpt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> u32 pte_flags; >> @@ -98,7 +98,7 @@ static void dpt_bind_vma(struct i915_address_space *vm, >> if (vma_res->bi.lmem) >> pte_flags |= PTE_LM; >> >> - vm->insert_entries(vm, vma_res, cache_level, pte_flags); >> + vm->insert_entries(vm, vma_res, pat_index, pte_flags); >> >> vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; >> >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c >> index d2d5a24301b2..05107a6efe45 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c >> @@ -27,8 +27,15 @@ static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj) >> if (IS_DGFX(i915)) >> return false; >> >> - return !(obj->cache_level == I915_CACHE_NONE || >> - obj->cache_level == I915_CACHE_WT); >> + /* >> + * For objects created by userspace through GEM_CREATE with pat_index >> + * set by set_pat extension, i915_gem_object_has_cache_level() will >> + * always return true, because the coherency of such object is managed >> + * by userspace. Othereise the call here would fall back to checking >> + * whether the object is un-cached or write-through. >> + */ >> + return !(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || >> + i915_gem_object_has_cache_level(obj, I915_CACHE_WT)); >> } >> >> bool i915_gem_cpu_write_needs_clflush(struct drm_i915_gem_object *obj) >> @@ -267,7 +274,13 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, >> { >> int ret; >> >> - if (obj->cache_level == cache_level) >> + /* >> + * For objects created by userspace through GEM_CREATE with pat_index >> + * set by set_pat extension, simply return 0 here without touching >> + * the cache setting, because such objects should have an immutable >> + * cache setting by desgin and always managed by userspace. >> + */ >> + if (i915_gem_object_has_cache_level(obj, cache_level)) >> return 0; >> >> ret = i915_gem_object_wait(obj, >> @@ -278,10 +291,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj, >> return ret; >> >> /* Always invalidate stale cachelines */ >> - if (obj->cache_level != cache_level) { >> - i915_gem_object_set_cache_coherency(obj, cache_level); >> - obj->cache_dirty = true; >> - } >> + i915_gem_object_set_cache_coherency(obj, cache_level); >> + obj->cache_dirty = true; >> >> /* The cache-level will be applied when each vma is rebound. */ >> return i915_gem_object_unbind(obj, >> @@ -306,20 +317,22 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data, >> goto out; >> } >> >> - switch (obj->cache_level) { >> - case I915_CACHE_LLC: >> - case I915_CACHE_L3_LLC: >> - args->caching = I915_CACHING_CACHED; >> - break; >> + /* >> + * This ioctl should be disabled for the objects with pat_index >> + * set by user space. >> + */ >> + if (obj->pat_set_by_user) { >> + err = -EOPNOTSUPP; >> + goto out; >> + } >> >> - case I915_CACHE_WT: >> + if (i915_gem_object_has_cache_level(obj, I915_CACHE_LLC) || >> + i915_gem_object_has_cache_level(obj, I915_CACHE_L3_LLC)) >> + args->caching = I915_CACHING_CACHED; >> + else if (i915_gem_object_has_cache_level(obj, I915_CACHE_WT)) >> args->caching = I915_CACHING_DISPLAY; >> - break; >> - >> - default: >> + else >> args->caching = I915_CACHING_NONE; >> - break; >> - } >> out: >> rcu_read_unlock(); >> return err; >> @@ -364,6 +377,15 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data, >> if (!obj) >> return -ENOENT; >> >> + /* >> + * This ioctl should be disabled for the objects with pat_index >> + * set by user space. >> + */ >> + if (obj->pat_set_by_user) { >> + ret = -EOPNOTSUPP; >> + goto out; >> + } >> + >> /* >> * The caching mode of proxy object is handled by its generator, and >> * not allowed to be changed by userspace. >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c >> index 3aeede6aee4d..5fb459ea4294 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c >> @@ -640,9 +640,15 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache, >> if (DBG_FORCE_RELOC == FORCE_GTT_RELOC) >> return false; >> >> + /* >> + * For objects created by userspace through GEM_CREATE with pat_index >> + * set by set_pat extension, i915_gem_object_has_cache_level() always >> + * return true, otherwise the call would fall back to checking whether >> + * the object is un-cached. >> + */ >> return (cache->has_llc || >> obj->cache_dirty || >> - obj->cache_level != I915_CACHE_NONE); >> + !i915_gem_object_has_cache_level(obj, I915_CACHE_NONE)); >> } >> >> static int eb_reserve_vma(struct i915_execbuffer *eb, >> @@ -1324,7 +1330,10 @@ static void *reloc_iomap(struct i915_vma *batch, >> if (drm_mm_node_allocated(&cache->node)) { >> ggtt->vm.insert_page(&ggtt->vm, >> i915_gem_object_get_dma_address(obj, page), >> - offset, I915_CACHE_NONE, 0); >> + offset, >> + i915_gem_get_pat_index(ggtt->vm.i915, >> + I915_CACHE_NONE), >> + 0); >> } else { >> offset += page << PAGE_SHIFT; >> } >> @@ -1464,7 +1473,7 @@ eb_relocate_entry(struct i915_execbuffer *eb, >> reloc_cache_unmap(&eb->reloc_cache); >> mutex_lock(&vma->vm->mutex); >> err = i915_vma_bind(target->vma, >> - target->vma->obj->cache_level, >> + target->vma->obj->pat_index, >> PIN_GLOBAL, NULL, NULL); >> mutex_unlock(&vma->vm->mutex); >> reloc_cache_remap(&eb->reloc_cache, ev->vma->obj); >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c >> index 3dbacdf0911a..4e7a838ab7bd 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_mman.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_mman.c >> @@ -383,7 +383,16 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf) >> } >> >> /* Access to snoopable pages through the GTT is incoherent. */ >> - if (obj->cache_level != I915_CACHE_NONE && !HAS_LLC(i915)) { >> + /* >> + * For objects created by userspace through GEM_CREATE with pat_index >> + * set by set_pat extension, coherency is managed by userspace, make >> + * sure we don't fail handling the vm fault by calling >> + * i915_gem_object_has_cache_level() which always return true for such >> + * objects. Otherwise this helper function would fall back to checking >> + * whether the object is un-cached. >> + */ >> + if (!(i915_gem_object_has_cache_level(obj, I915_CACHE_NONE) || >> + HAS_LLC(i915))) { >> ret = -EFAULT; >> goto err_unpin; >> } >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c >> index 8c70a0ec7d2f..46a19b099ec8 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c >> @@ -54,6 +54,24 @@ unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, >> return INTEL_INFO(i915)->cachelevel_to_pat[level]; >> } >> >> +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, >> + enum i915_cache_level lvl) >> +{ >> + /* >> + * In case the pat_index is set by user space, this kernel mode >> + * driver should leave the coherency to be managed by user space, >> + * simply return true here. >> + */ >> + if (obj->pat_set_by_user) >> + return true; >> + >> + /* >> + * Otherwise the pat_index should have been converted from cache_level >> + * so that the following comparison is valid. >> + */ >> + return obj->pat_index == i915_gem_get_pat_index(obj_to_i915(obj), lvl); >> +} >> + >> struct drm_i915_gem_object *i915_gem_object_alloc(void) >> { >> struct drm_i915_gem_object *obj; >> @@ -133,7 +151,7 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, >> { >> struct drm_i915_private *i915 = to_i915(obj->base.dev); >> >> - obj->cache_level = cache_level; >> + obj->pat_index = i915_gem_get_pat_index(i915, cache_level); >> >> if (cache_level != I915_CACHE_NONE) >> obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | >> @@ -148,6 +166,37 @@ void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, >> !IS_DGFX(i915); >> } >> >> +/** >> + * i915_gem_object_set_pat_index - set PAT index to be used in PTE encode >> + * @obj: #drm_i915_gem_object >> + * @pat_index: PAT index >> + * >> + * This is a clone of i915_gem_object_set_cache_coherency taking pat index >> + * instead of cache_level as its second argument. >> + */ >> +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, >> + unsigned int pat_index) >> +{ >> + struct drm_i915_private *i915 = to_i915(obj->base.dev); >> + >> + if (obj->pat_index == pat_index) >> + return; >> + >> + obj->pat_index = pat_index; >> + >> + if (pat_index != i915_gem_get_pat_index(i915, I915_CACHE_NONE)) >> + obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ | >> + I915_BO_CACHE_COHERENT_FOR_WRITE); >> + else if (HAS_LLC(i915)) >> + obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ; >> + else >> + obj->cache_coherent = 0; >> + >> + obj->cache_dirty = >> + !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE) && >> + !IS_DGFX(i915); >> +} >> + >> bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj) >> { >> struct drm_i915_private *i915 = to_i915(obj->base.dev); >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h >> index 284e1aa396cd..884a17275b3a 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h >> @@ -34,6 +34,8 @@ static inline bool i915_gem_object_size_2big(u64 size) >> >> unsigned int i915_gem_get_pat_index(struct drm_i915_private *i915, >> enum i915_cache_level level); >> +bool i915_gem_object_has_cache_level(const struct drm_i915_gem_object *obj, >> + enum i915_cache_level lvl); >> void i915_gem_init__objects(struct drm_i915_private *i915); >> >> void i915_objects_module_exit(void); >> @@ -764,6 +766,8 @@ bool i915_gem_object_has_unknown_state(struct drm_i915_gem_object *obj); >> >> void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj, >> unsigned int cache_level); >> +void i915_gem_object_set_pat_index(struct drm_i915_gem_object *obj, >> + unsigned int pat_index); >> bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj); >> void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj); >> void i915_gem_object_flush_if_display_locked(struct drm_i915_gem_object *obj); >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h >> index bf0bd8e11355..adf4c2b28135 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h >> @@ -364,15 +364,42 @@ struct drm_i915_gem_object { >> #define I915_BO_FLAG_STRUCT_PAGE BIT(0) /* Object backed by struct pages */ >> #define I915_BO_FLAG_IOMEM BIT(1) /* Object backed by IO memory */ >> /** >> - * @cache_level: The desired GTT caching level. >> + * @pat_index: The desired PAT index. >> + * >> + * See hardware specification for valid PAT indices for each platform. >> + * This field used to contain a value of enum i915_cache_level. It's >> + * changed to an unsigned int because PAT indices are being used by >> + * both UMD and KMD for caching policy control after GEN12. >> + * For backward compatibility, this field will continue to contain >> + * value of i915_cache_level for pre-GEN12 platforms so that the PTE >> + * encode functions for these legacy platforms can stay the same. >> + * In the meantime platform specific tables are created to translate >> + * i915_cache_level into pat index, for more details check the macros >> + * defined i915/i915_pci.c, e.g. PVC_CACHELEVEL. >> + */ > > Dunno - this still reads a bit wonky to me. It was unsigned int already > and also it does not hold i915_cache_level on legacy platforms, right? v8 sent with some clarification. >> + unsigned int pat_index:6; >> + /** >> + * @pat_set_by_user: Indicate whether pat_index is set by user space >> * >> - * See enum i915_cache_level for possible values, along with what >> - * each does. >> + * This field is set to false by default, only set to true if the >> + * pat_index is set by user space. By design, user space is capable of >> + * managing caching behavior by setting pat_index, in which case this >> + * kernel mode driver should never touch the pat_index. >> */ >> - unsigned int cache_level:3; >> + unsigned int pat_set_by_user; > > Unsigned int:1 to fit into the bitfield sandwitch better? okay >> /** >> * @cache_coherent: >> * >> + * Note: with the change above which replaced cache_level with >> + * pat_index, the use of cache_coherent is limited to objects created >> + * by kernel. Check for @pat_set_by_user to find out if an object is >> + * created by userspace or kernel. A later patch will disable the > >pat_set_by_user is not about who created the object, no? And it is not >in a later patch any longer. > >> + * ioctl's to change cache settings for objects with pat_index set by >> + * userspace. Please don't assume @cache_coherent having the flags set >> + * as describe here for these objects. Also see the helper function >> + * i915_gem_object_has_cache_level() for possible ways to bypass the >> + * use of this field. >> + * >> * Track whether the pages are coherent with the GPU if reading or >> * writing through the CPU caches. The largely depends on the >> * @cache_level setting. >> @@ -446,6 +473,15 @@ struct drm_i915_gem_object { >> /** >> * @cache_dirty: >> * >> + * Note: with the change above which replaced cache_level with >> + * pat_index, the use of cache_dirty is limited to objects created >> + * by kernel. Check for @pat_set_by_user to find out if an object is >> + * created by userspace or kernel. A later patch will disable the >> + * ioctl's to change cache settings for objects with pat_index set by >> + * userspace. Please don't assume @cache_dirty is set as describe here >> + * for these objects. Also see i915_gem_object_has_cache_level() for >> + * possible ways to bypass the use of this field. >> + * >> * Track if we are we dirty with writes through the CPU cache for this >> * object. As a result reading directly from main memory might yield >> * stale data. >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c >> index ee492d823f1b..3b094d36a0b0 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c >> @@ -565,7 +565,9 @@ static void dbg_poison(struct i915_ggtt *ggtt, >> >> ggtt->vm.insert_page(&ggtt->vm, addr, >> ggtt->error_capture.start, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(ggtt->vm.i915, >> + I915_CACHE_NONE), >> + 0); >> mb(); >> >> s = io_mapping_map_wc(&ggtt->iomap, >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c >> index 69eb20ed4d47..7078af2f8f79 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c >> @@ -214,7 +214,8 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, >> >> intel_engine_pm_get(to_gt(i915)->migrate.context->engine); >> ret = intel_context_migrate_clear(to_gt(i915)->migrate.context, deps, >> - dst_st->sgl, dst_level, >> + dst_st->sgl, >> + i915_gem_get_pat_index(i915, dst_level), >> i915_ttm_gtt_binds_lmem(dst_mem), >> 0, &rq); >> } else { >> @@ -228,9 +229,10 @@ static struct dma_fence *i915_ttm_accel_move(struct ttm_buffer_object *bo, >> intel_engine_pm_get(to_gt(i915)->migrate.context->engine); >> ret = intel_context_migrate_copy(to_gt(i915)->migrate.context, >> deps, src_rsgt->table.sgl, >> - src_level, >> + i915_gem_get_pat_index(i915, src_level), >> i915_ttm_gtt_binds_lmem(bo->resource), >> - dst_st->sgl, dst_level, >> + dst_st->sgl, >> + i915_gem_get_pat_index(i915, dst_level), >> i915_ttm_gtt_binds_lmem(dst_mem), >> &rq); >> >> diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c >> index d1057574e966..df6c9a84252c 100644 >> --- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c >> +++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c >> @@ -354,7 +354,7 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single) >> >> obj->write_domain = I915_GEM_DOMAIN_CPU; >> obj->read_domains = I915_GEM_DOMAIN_CPU; >> - obj->cache_level = I915_CACHE_NONE; >> + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); >> >> return obj; >> } >> diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c >> index fe6c37fd7859..a93a90b15907 100644 >> --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c >> +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c >> @@ -219,7 +219,7 @@ static int __igt_lmem_pages_migrate(struct intel_gt *gt, >> continue; >> >> err = intel_migrate_clear(>->migrate, &ww, deps, >> - obj->mm.pages->sgl, obj->cache_level, >> + obj->mm.pages->sgl, obj->pat_index, >> i915_gem_object_is_lmem(obj), >> 0xdeadbeaf, &rq); >> if (rq) { >> diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c >> index 56279908ed30..a93d8f9f8bc1 100644 >> --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c >> +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c >> @@ -1222,7 +1222,7 @@ static int __igt_mmap_migrate(struct intel_memory_region **placements, >> } >> >> err = intel_context_migrate_clear(to_gt(i915)->migrate.context, NULL, >> - obj->mm.pages->sgl, obj->cache_level, >> + obj->mm.pages->sgl, obj->pat_index, >> i915_gem_object_is_lmem(obj), >> expand32(POISON_INUSE), &rq); >> i915_gem_object_unpin_pages(obj); >> diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c >> index 5aaacc53fa4c..c2bdc133c89a 100644 >> --- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c >> +++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c >> @@ -109,7 +109,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm, >> >> static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm); >> @@ -117,7 +117,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm, >> unsigned int first_entry = vma_res->start / I915_GTT_PAGE_SIZE; >> unsigned int act_pt = first_entry / GEN6_PTES; >> unsigned int act_pte = first_entry % GEN6_PTES; >> - const u32 pte_encode = vm->pte_encode(0, cache_level, flags); >> + const u32 pte_encode = vm->pte_encode(0, pat_index, flags); >> struct sgt_dma iter = sgt_dma(vma_res); >> gen6_pte_t *vaddr; >> >> @@ -227,7 +227,9 @@ static int gen6_ppgtt_init_scratch(struct gen6_ppgtt *ppgtt) >> >> vm->scratch[0]->encode = >> vm->pte_encode(px_dma(vm->scratch[0]), >> - I915_CACHE_NONE, PTE_READ_ONLY); >> + i915_gem_get_pat_index(vm->i915, >> + I915_CACHE_NONE), >> + PTE_READ_ONLY); >> >> vm->scratch[1] = vm->alloc_pt_dma(vm, I915_GTT_PAGE_SIZE_4K); >> if (IS_ERR(vm->scratch[1])) { >> @@ -278,7 +280,7 @@ static void gen6_ppgtt_cleanup(struct i915_address_space *vm) >> static void pd_vma_bind(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 unused) >> { >> struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); >> diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c >> index bb6998d67133..f948d33e5ec5 100644 >> --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.c >> +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.c >> @@ -29,7 +29,7 @@ static u64 gen8_pde_encode(const dma_addr_t addr, >> } >> >> static u64 gen8_pte_encode(dma_addr_t addr, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; >> @@ -40,7 +40,12 @@ static u64 gen8_pte_encode(dma_addr_t addr, >> if (flags & PTE_LM) >> pte |= GEN12_PPGTT_PTE_LM; >> >> - switch (level) { >> + /* >> + * For pre-gen12 platforms pat_index is the same as enum >> + * i915_cache_level, so the switch-case here is still valid. >> + * See translation table defined by LEGACY_CACHELEVEL. >> + */ >> + switch (pat_index) { >> case I915_CACHE_NONE: >> pte |= PPAT_UNCACHED; >> break; >> @@ -55,9 +60,9 @@ static u64 gen8_pte_encode(dma_addr_t addr, >> return pte; >> } >> >> -static u64 mtl_pte_encode(dma_addr_t addr, >> - enum i915_cache_level level, >> - u32 flags) >> +static u64 gen12_pte_encode(dma_addr_t addr, >> + unsigned int pat_index, >> + u32 flags) >> { >> gen8_pte_t pte = addr | GEN8_PAGE_PRESENT | GEN8_PAGE_RW; >> >> @@ -67,24 +72,17 @@ static u64 mtl_pte_encode(dma_addr_t addr, >> if (flags & PTE_LM) >> pte |= GEN12_PPGTT_PTE_LM; >> >> - switch (level) { >> - case I915_CACHE_NONE: >> - pte |= GEN12_PPGTT_PTE_PAT1; >> - break; >> - case I915_CACHE_LLC: >> - case I915_CACHE_L3_LLC: >> - pte |= GEN12_PPGTT_PTE_PAT0 | GEN12_PPGTT_PTE_PAT1; >> - break; >> - case I915_CACHE_WT: >> + if (pat_index & BIT(0)) >> pte |= GEN12_PPGTT_PTE_PAT0; >> - break; >> - default: >> - /* This should never happen. Added to deal with the compile >> - * error due to the addition of I915_MAX_CACHE_LEVEL. Will >> - * be removed by the pat_index patch. >> - */ >> - break; >> - } >> + >> + if (pat_index & BIT(1)) >> + pte |= GEN12_PPGTT_PTE_PAT1; >> + >> + if (pat_index & BIT(2)) >> + pte |= GEN12_PPGTT_PTE_PAT2; >> + >> + if (pat_index & BIT(3)) >> + pte |= MTL_PPGTT_PTE_PAT3; >> >> return pte; >> } >> @@ -457,11 +455,11 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt, >> struct i915_page_directory *pdp, >> struct sgt_dma *iter, >> u64 idx, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_page_directory *pd; >> - const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, cache_level, flags); >> + const gen8_pte_t pte_encode = ppgtt->vm.pte_encode(0, pat_index, flags); >> gen8_pte_t *vaddr; >> >> pd = i915_pd_entry(pdp, gen8_pd_index(idx, 2)); >> @@ -504,10 +502,10 @@ static void >> xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> struct sgt_dma *iter, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); >> + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); >> unsigned int rem = sg_dma_len(iter->sg); >> u64 start = vma_res->start; >> u64 end = start + vma_res->vma_size; >> @@ -612,10 +610,10 @@ xehpsdv_ppgtt_insert_huge(struct i915_address_space *vm, >> static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> struct sgt_dma *iter, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> - const gen8_pte_t pte_encode = vm->pte_encode(0, cache_level, flags); >> + const gen8_pte_t pte_encode = vm->pte_encode(0, pat_index, flags); >> unsigned int rem = sg_dma_len(iter->sg); >> u64 start = vma_res->start; >> >> @@ -735,7 +733,7 @@ static void gen8_ppgtt_insert_huge(struct i915_address_space *vm, >> >> static void gen8_ppgtt_insert(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(vm); >> @@ -743,9 +741,9 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, >> >> if (vma_res->bi.page_sizes.sg > I915_GTT_PAGE_SIZE) { >> if (GRAPHICS_VER_FULL(vm->i915) >= IP_VER(12, 50)) >> - xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); >> + xehpsdv_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); >> else >> - gen8_ppgtt_insert_huge(vm, vma_res, &iter, cache_level, flags); >> + gen8_ppgtt_insert_huge(vm, vma_res, &iter, pat_index, flags); >> } else { >> u64 idx = vma_res->start >> GEN8_PTE_SHIFT; >> >> @@ -754,7 +752,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, >> gen8_pdp_for_page_index(vm, idx); >> >> idx = gen8_ppgtt_insert_pte(ppgtt, pdp, &iter, idx, >> - cache_level, flags); >> + pat_index, flags); >> } while (idx); >> >> vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; >> @@ -764,7 +762,7 @@ static void gen8_ppgtt_insert(struct i915_address_space *vm, >> static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> u64 idx = offset >> GEN8_PTE_SHIFT; >> @@ -778,14 +776,14 @@ static void gen8_ppgtt_insert_entry(struct i915_address_space *vm, >> GEM_BUG_ON(pt->is_compact); >> >> vaddr = px_vaddr(pt); >> - vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, level, flags); >> + vaddr[gen8_pd_index(idx, 0)] = vm->pte_encode(addr, pat_index, flags); >> drm_clflush_virt_range(&vaddr[gen8_pd_index(idx, 0)], sizeof(*vaddr)); >> } >> >> static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> u64 idx = offset >> GEN8_PTE_SHIFT; >> @@ -808,20 +806,20 @@ static void __xehpsdv_ppgtt_insert_entry_lm(struct i915_address_space *vm, >> } >> >> vaddr = px_vaddr(pt); >> - vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, level, flags); >> + vaddr[gen8_pd_index(idx, 0) / 16] = vm->pte_encode(addr, pat_index, flags); >> } >> >> static void xehpsdv_ppgtt_insert_entry(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> if (flags & PTE_LM) >> return __xehpsdv_ppgtt_insert_entry_lm(vm, addr, offset, >> - level, flags); >> + pat_index, flags); >> >> - return gen8_ppgtt_insert_entry(vm, addr, offset, level, flags); >> + return gen8_ppgtt_insert_entry(vm, addr, offset, pat_index, flags); >> } >> >> static int gen8_init_scratch(struct i915_address_space *vm) >> @@ -856,7 +854,9 @@ static int gen8_init_scratch(struct i915_address_space *vm) >> >> vm->scratch[0]->encode = >> vm->pte_encode(px_dma(vm->scratch[0]), >> - I915_CACHE_NONE, pte_flags); >> + i915_gem_get_pat_index(vm->i915, >> + I915_CACHE_NONE), >> + pte_flags); >> >> for (i = 1; i <= vm->top; i++) { >> struct drm_i915_gem_object *obj; >> @@ -998,8 +998,8 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, >> */ >> ppgtt->vm.alloc_scratch_dma = alloc_pt_dma; >> >> - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 70)) >> - ppgtt->vm.pte_encode = mtl_pte_encode; >> + if (GRAPHICS_VER(gt->i915) >= 12) >> + ppgtt->vm.pte_encode = gen12_pte_encode; >> else >> ppgtt->vm.pte_encode = gen8_pte_encode; >> >> diff --git a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h >> index f541d19264b4..19c635441642 100644 >> --- a/drivers/gpu/drm/i915/gt/gen8_ppgtt.h >> +++ b/drivers/gpu/drm/i915/gt/gen8_ppgtt.h >> @@ -10,13 +10,12 @@ >> >> struct i915_address_space; >> struct intel_gt; >> -enum i915_cache_level; >> >> struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt, >> unsigned long lmem_pt_obj_flags); >> >> u64 gen8_ggtt_pte_encode(dma_addr_t addr, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags); >> >> #endif >> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> index c8390d03fce2..2a7942fac798 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c >> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c >> @@ -221,7 +221,7 @@ static void guc_ggtt_invalidate(struct i915_ggtt *ggtt) >> } >> >> static u64 mtl_ggtt_pte_encode(dma_addr_t addr, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; >> @@ -231,30 +231,17 @@ static u64 mtl_ggtt_pte_encode(dma_addr_t addr, >> if (flags & PTE_LM) >> pte |= GEN12_GGTT_PTE_LM; >> >> - switch (level) { >> - case I915_CACHE_NONE: >> - pte |= MTL_GGTT_PTE_PAT1; >> - break; >> - case I915_CACHE_LLC: >> - case I915_CACHE_L3_LLC: >> - pte |= MTL_GGTT_PTE_PAT0 | MTL_GGTT_PTE_PAT1; >> - break; >> - case I915_CACHE_WT: >> + if (pat_index & BIT(0)) >> pte |= MTL_GGTT_PTE_PAT0; >> - break; >> - default: >> - /* This should never happen. Added to deal with the compile >> - * error due to the addition of I915_MAX_CACHE_LEVEL. Will >> - * be removed by the pat_index patch. >> - */ >> - break; >> - } >> + >> + if (pat_index & BIT(1)) >> + pte |= MTL_GGTT_PTE_PAT1; >> >> return pte; >> } >> >> u64 gen8_ggtt_pte_encode(dma_addr_t addr, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> gen8_pte_t pte = addr | GEN8_PAGE_PRESENT; >> @@ -273,25 +260,25 @@ static void gen8_set_pte(void __iomem *addr, gen8_pte_t pte) >> static void gen8_ggtt_insert_page(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); >> gen8_pte_t __iomem *pte = >> (gen8_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; >> >> - gen8_set_pte(pte, ggtt->vm.pte_encode(addr, level, flags)); >> + gen8_set_pte(pte, ggtt->vm.pte_encode(addr, pat_index, flags)); >> >> ggtt->invalidate(ggtt); >> } >> >> static void gen8_ggtt_insert_entries(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); >> - const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, level, flags); >> + const gen8_pte_t pte_encode = ggtt->vm.pte_encode(0, pat_index, flags); >> gen8_pte_t __iomem *gte; >> gen8_pte_t __iomem *end; >> struct sgt_iter iter; >> @@ -348,14 +335,14 @@ static void gen8_ggtt_clear_range(struct i915_address_space *vm, >> static void gen6_ggtt_insert_page(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); >> gen6_pte_t __iomem *pte = >> (gen6_pte_t __iomem *)ggtt->gsm + offset / I915_GTT_PAGE_SIZE; >> >> - iowrite32(vm->pte_encode(addr, level, flags), pte); >> + iowrite32(vm->pte_encode(addr, pat_index, flags), pte); >> >> ggtt->invalidate(ggtt); >> } >> @@ -368,7 +355,7 @@ static void gen6_ggtt_insert_page(struct i915_address_space *vm, >> */ >> static void gen6_ggtt_insert_entries(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> struct i915_ggtt *ggtt = i915_vm_to_ggtt(vm); >> @@ -385,7 +372,7 @@ static void gen6_ggtt_insert_entries(struct i915_address_space *vm, >> iowrite32(vm->scratch[0]->encode, gte++); >> end += (vma_res->node_size + vma_res->guard) / I915_GTT_PAGE_SIZE; >> for_each_sgt_daddr(addr, iter, vma_res->bi.pages) >> - iowrite32(vm->pte_encode(addr, level, flags), gte++); >> + iowrite32(vm->pte_encode(addr, pat_index, flags), gte++); >> GEM_BUG_ON(gte > end); >> >> /* Fill the allocated but "unused" space beyond the end of the buffer */ >> @@ -420,14 +407,15 @@ struct insert_page { >> struct i915_address_space *vm; >> dma_addr_t addr; >> u64 offset; >> - enum i915_cache_level level; >> + unsigned int pat_index; >> }; >> >> static int bxt_vtd_ggtt_insert_page__cb(void *_arg) >> { >> struct insert_page *arg = _arg; >> >> - gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, arg->level, 0); >> + gen8_ggtt_insert_page(arg->vm, arg->addr, arg->offset, >> + arg->pat_index, 0); >> bxt_vtd_ggtt_wa(arg->vm); >> >> return 0; >> @@ -436,10 +424,10 @@ static int bxt_vtd_ggtt_insert_page__cb(void *_arg) >> static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 unused) >> { >> - struct insert_page arg = { vm, addr, offset, level }; >> + struct insert_page arg = { vm, addr, offset, pat_index }; >> >> stop_machine(bxt_vtd_ggtt_insert_page__cb, &arg, NULL); >> } >> @@ -447,7 +435,7 @@ static void bxt_vtd_ggtt_insert_page__BKL(struct i915_address_space *vm, >> struct insert_entries { >> struct i915_address_space *vm; >> struct i915_vma_resource *vma_res; >> - enum i915_cache_level level; >> + unsigned int pat_index; >> u32 flags; >> }; >> >> @@ -455,7 +443,8 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) >> { >> struct insert_entries *arg = _arg; >> >> - gen8_ggtt_insert_entries(arg->vm, arg->vma_res, arg->level, arg->flags); >> + gen8_ggtt_insert_entries(arg->vm, arg->vma_res, >> + arg->pat_index, arg->flags); >> bxt_vtd_ggtt_wa(arg->vm); >> >> return 0; >> @@ -463,10 +452,10 @@ static int bxt_vtd_ggtt_insert_entries__cb(void *_arg) >> >> static void bxt_vtd_ggtt_insert_entries__BKL(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> - struct insert_entries arg = { vm, vma_res, level, flags }; >> + struct insert_entries arg = { vm, vma_res, pat_index, flags }; >> >> stop_machine(bxt_vtd_ggtt_insert_entries__cb, &arg, NULL); >> } >> @@ -495,7 +484,7 @@ static void gen6_ggtt_clear_range(struct i915_address_space *vm, >> void intel_ggtt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> u32 pte_flags; >> @@ -512,7 +501,7 @@ void intel_ggtt_bind_vma(struct i915_address_space *vm, >> if (vma_res->bi.lmem) >> pte_flags |= PTE_LM; >> >> - vm->insert_entries(vm, vma_res, cache_level, pte_flags); >> + vm->insert_entries(vm, vma_res, pat_index, pte_flags); >> vma_res->page_sizes_gtt = I915_GTT_PAGE_SIZE; >> } >> >> @@ -661,7 +650,7 @@ static int init_ggtt(struct i915_ggtt *ggtt) >> static void aliasing_gtt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> u32 pte_flags; >> @@ -673,10 +662,10 @@ static void aliasing_gtt_bind_vma(struct i915_address_space *vm, >> >> if (flags & I915_VMA_LOCAL_BIND) >> ppgtt_bind_vma(&i915_vm_to_ggtt(vm)->alias->vm, >> - stash, vma_res, cache_level, flags); >> + stash, vma_res, pat_index, flags); >> >> if (flags & I915_VMA_GLOBAL_BIND) >> - vm->insert_entries(vm, vma_res, cache_level, pte_flags); >> + vm->insert_entries(vm, vma_res, pat_index, pte_flags); >> >> vma_res->bound_flags |= flags; >> } >> @@ -933,7 +922,9 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size) >> >> ggtt->vm.scratch[0]->encode = >> ggtt->vm.pte_encode(px_dma(ggtt->vm.scratch[0]), >> - I915_CACHE_NONE, pte_flags); >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), >> + pte_flags); >> >> return 0; >> } >> @@ -1022,6 +1013,11 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt) >> return ggtt_probe_common(ggtt, size); >> } >> >> +/* >> + * For pre-gen8 platforms pat_index is the same as enum i915_cache_level, >> + * so these PTE encode functions are left with using cache_level. >> + * See translation table LEGACY_CACHELEVEL. >> + */ >> static u64 snb_pte_encode(dma_addr_t addr, >> enum i915_cache_level level, >> u32 flags) >> @@ -1302,7 +1298,9 @@ bool i915_ggtt_resume_vm(struct i915_address_space *vm) >> */ >> vma->resource->bound_flags = 0; >> vma->ops->bind_vma(vm, NULL, vma->resource, >> - obj ? obj->cache_level : 0, >> + obj ? obj->pat_index : >> + i915_gem_get_pat_index(vm->i915, >> + I915_CACHE_NONE), >> was_bound); >> >> if (obj) { /* only used during resume => exclusive access */ >> diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h >> index 9aff343beaa8..4d6296cdbcfd 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_gtt.h >> +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h >> @@ -163,8 +163,6 @@ typedef u64 gen8_pte_t; >> #define MTL_3_COH_2W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 3) >> #define MTL_2_COH_1W REG_FIELD_PREP(MTL_PAT_INDEX_COH_MODE_MASK, 2) >> >> -enum i915_cache_level; >> - >> struct drm_i915_gem_object; >> struct i915_fence_reg; >> struct i915_vma; >> @@ -232,7 +230,7 @@ struct i915_vma_ops { >> void (*bind_vma)(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> /* >> * Unmap an object from an address space. This usually consists of >> @@ -304,7 +302,7 @@ struct i915_address_space { >> (*alloc_scratch_dma)(struct i915_address_space *vm, int sz); >> >> u64 (*pte_encode)(dma_addr_t addr, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags); /* Create a valid PTE */ >> #define PTE_READ_ONLY BIT(0) >> #define PTE_LM BIT(1) >> @@ -319,20 +317,20 @@ struct i915_address_space { >> void (*insert_page)(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void (*insert_entries)(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void (*raw_insert_page)(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void (*raw_insert_entries)(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void (*cleanup)(struct i915_address_space *vm); >> >> @@ -579,7 +577,7 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt, >> void intel_ggtt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void intel_ggtt_unbind_vma(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res); >> @@ -657,7 +655,7 @@ void gen6_ggtt_invalidate(struct i915_ggtt *ggtt); >> void ppgtt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags); >> void ppgtt_unbind_vma(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res); >> diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.c b/drivers/gpu/drm/i915/gt/intel_migrate.c >> index e0998879a0e1..6023288b0e2d 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_migrate.c >> +++ b/drivers/gpu/drm/i915/gt/intel_migrate.c >> @@ -45,7 +45,9 @@ static void xehpsdv_toggle_pdes(struct i915_address_space *vm, >> * Insert a dummy PTE into every PT that will map to LMEM to ensure >> * we have a correctly setup PDE structure for later use. >> */ >> - vm->insert_page(vm, 0, d->offset, I915_CACHE_NONE, PTE_LM); >> + vm->insert_page(vm, 0, d->offset, >> + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), >> + PTE_LM); >> GEM_BUG_ON(!pt->is_compact); >> d->offset += SZ_2M; >> } >> @@ -63,7 +65,9 @@ static void xehpsdv_insert_pte(struct i915_address_space *vm, >> * alignment is 64K underneath for the pt, and we are careful >> * not to access the space in the void. >> */ >> - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, PTE_LM); >> + vm->insert_page(vm, px_dma(pt), d->offset, >> + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), >> + PTE_LM); >> d->offset += SZ_64K; >> } >> >> @@ -73,7 +77,8 @@ static void insert_pte(struct i915_address_space *vm, >> { >> struct insert_pte_data *d = data; >> >> - vm->insert_page(vm, px_dma(pt), d->offset, I915_CACHE_NONE, >> + vm->insert_page(vm, px_dma(pt), d->offset, >> + i915_gem_get_pat_index(vm->i915, I915_CACHE_NONE), >> i915_gem_object_is_lmem(pt->base) ? PTE_LM : 0); >> d->offset += PAGE_SIZE; >> } >> @@ -356,13 +361,13 @@ static int max_pte_pkt_size(struct i915_request *rq, int pkt) >> >> static int emit_pte(struct i915_request *rq, >> struct sgt_dma *it, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> u64 offset, >> int length) >> { >> bool has_64K_pages = HAS_64K_PAGES(rq->engine->i915); >> - const u64 encode = rq->context->vm->pte_encode(0, cache_level, >> + const u64 encode = rq->context->vm->pte_encode(0, pat_index, >> is_lmem ? PTE_LM : 0); >> struct intel_ring *ring = rq->ring; >> int pkt, dword_length; >> @@ -673,17 +678,17 @@ int >> intel_context_migrate_copy(struct intel_context *ce, >> const struct i915_deps *deps, >> struct scatterlist *src, >> - enum i915_cache_level src_cache_level, >> + unsigned int src_pat_index, >> bool src_is_lmem, >> struct scatterlist *dst, >> - enum i915_cache_level dst_cache_level, >> + unsigned int dst_pat_index, >> bool dst_is_lmem, >> struct i915_request **out) >> { >> struct sgt_dma it_src = sg_sgt(src), it_dst = sg_sgt(dst), it_ccs; >> struct drm_i915_private *i915 = ce->engine->i915; >> u64 ccs_bytes_to_cpy = 0, bytes_to_cpy; >> - enum i915_cache_level ccs_cache_level; >> + unsigned int ccs_pat_index; >> u32 src_offset, dst_offset; >> u8 src_access, dst_access; >> struct i915_request *rq; >> @@ -707,12 +712,12 @@ intel_context_migrate_copy(struct intel_context *ce, >> dst_sz = scatter_list_length(dst); >> if (src_is_lmem) { >> it_ccs = it_dst; >> - ccs_cache_level = dst_cache_level; >> + ccs_pat_index = dst_pat_index; >> ccs_is_src = false; >> } else if (dst_is_lmem) { >> bytes_to_cpy = dst_sz; >> it_ccs = it_src; >> - ccs_cache_level = src_cache_level; >> + ccs_pat_index = src_pat_index; >> ccs_is_src = true; >> } >> >> @@ -773,7 +778,7 @@ intel_context_migrate_copy(struct intel_context *ce, >> src_sz = calculate_chunk_sz(i915, src_is_lmem, >> bytes_to_cpy, ccs_bytes_to_cpy); >> >> - len = emit_pte(rq, &it_src, src_cache_level, src_is_lmem, >> + len = emit_pte(rq, &it_src, src_pat_index, src_is_lmem, >> src_offset, src_sz); >> if (!len) { >> err = -EINVAL; >> @@ -784,7 +789,7 @@ intel_context_migrate_copy(struct intel_context *ce, >> goto out_rq; >> } >> >> - err = emit_pte(rq, &it_dst, dst_cache_level, dst_is_lmem, >> + err = emit_pte(rq, &it_dst, dst_pat_index, dst_is_lmem, >> dst_offset, len); >> if (err < 0) >> goto out_rq; >> @@ -811,7 +816,7 @@ intel_context_migrate_copy(struct intel_context *ce, >> goto out_rq; >> >> ccs_sz = GET_CCS_BYTES(i915, len); >> - err = emit_pte(rq, &it_ccs, ccs_cache_level, false, >> + err = emit_pte(rq, &it_ccs, ccs_pat_index, false, >> ccs_is_src ? src_offset : dst_offset, >> ccs_sz); >> if (err < 0) >> @@ -979,7 +984,7 @@ int >> intel_context_migrate_clear(struct intel_context *ce, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> u32 value, >> struct i915_request **out) >> @@ -1027,7 +1032,7 @@ intel_context_migrate_clear(struct intel_context *ce, >> if (err) >> goto out_rq; >> >> - len = emit_pte(rq, &it, cache_level, is_lmem, offset, CHUNK_SZ); >> + len = emit_pte(rq, &it, pat_index, is_lmem, offset, CHUNK_SZ); >> if (len <= 0) { >> err = len; >> goto out_rq; >> @@ -1074,10 +1079,10 @@ int intel_migrate_copy(struct intel_migrate *m, >> struct i915_gem_ww_ctx *ww, >> const struct i915_deps *deps, >> struct scatterlist *src, >> - enum i915_cache_level src_cache_level, >> + unsigned int src_pat_index, >> bool src_is_lmem, >> struct scatterlist *dst, >> - enum i915_cache_level dst_cache_level, >> + unsigned int dst_pat_index, >> bool dst_is_lmem, >> struct i915_request **out) >> { >> @@ -1098,8 +1103,8 @@ int intel_migrate_copy(struct intel_migrate *m, >> goto out; >> >> err = intel_context_migrate_copy(ce, deps, >> - src, src_cache_level, src_is_lmem, >> - dst, dst_cache_level, dst_is_lmem, >> + src, src_pat_index, src_is_lmem, >> + dst, dst_pat_index, dst_is_lmem, >> out); >> >> intel_context_unpin(ce); >> @@ -1113,7 +1118,7 @@ intel_migrate_clear(struct intel_migrate *m, >> struct i915_gem_ww_ctx *ww, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> u32 value, >> struct i915_request **out) >> @@ -1134,7 +1139,7 @@ intel_migrate_clear(struct intel_migrate *m, >> if (err) >> goto out; >> >> - err = intel_context_migrate_clear(ce, deps, sg, cache_level, >> + err = intel_context_migrate_clear(ce, deps, sg, pat_index, >> is_lmem, value, out); >> >> intel_context_unpin(ce); >> diff --git a/drivers/gpu/drm/i915/gt/intel_migrate.h b/drivers/gpu/drm/i915/gt/intel_migrate.h >> index ccc677ec4aa3..11fc09a00c4b 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_migrate.h >> +++ b/drivers/gpu/drm/i915/gt/intel_migrate.h >> @@ -16,7 +16,6 @@ struct i915_request; >> struct i915_gem_ww_ctx; >> struct intel_gt; >> struct scatterlist; >> -enum i915_cache_level; >> >> int intel_migrate_init(struct intel_migrate *m, struct intel_gt *gt); >> >> @@ -26,20 +25,20 @@ int intel_migrate_copy(struct intel_migrate *m, >> struct i915_gem_ww_ctx *ww, >> const struct i915_deps *deps, >> struct scatterlist *src, >> - enum i915_cache_level src_cache_level, >> + unsigned int src_pat_index, >> bool src_is_lmem, >> struct scatterlist *dst, >> - enum i915_cache_level dst_cache_level, >> + unsigned int dst_pat_index, >> bool dst_is_lmem, >> struct i915_request **out); >> >> int intel_context_migrate_copy(struct intel_context *ce, >> const struct i915_deps *deps, >> struct scatterlist *src, >> - enum i915_cache_level src_cache_level, >> + unsigned int src_pat_index, >> bool src_is_lmem, >> struct scatterlist *dst, >> - enum i915_cache_level dst_cache_level, >> + unsigned int dst_pat_index, >> bool dst_is_lmem, >> struct i915_request **out); >> >> @@ -48,7 +47,7 @@ intel_migrate_clear(struct intel_migrate *m, >> struct i915_gem_ww_ctx *ww, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> u32 value, >> struct i915_request **out); >> @@ -56,7 +55,7 @@ int >> intel_context_migrate_clear(struct intel_context *ce, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> u32 value, >> struct i915_request **out); >> diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/gt/intel_ppgtt.c >> index 7ecfa672f738..436756bfbb1a 100644 >> --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c >> +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c >> @@ -181,7 +181,7 @@ struct i915_ppgtt *i915_ppgtt_create(struct intel_gt *gt, >> void ppgtt_bind_vma(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> u32 pte_flags; >> @@ -199,7 +199,7 @@ void ppgtt_bind_vma(struct i915_address_space *vm, >> if (vma_res->bi.lmem) >> pte_flags |= PTE_LM; >> >> - vm->insert_entries(vm, vma_res, cache_level, pte_flags); >> + vm->insert_entries(vm, vma_res, pat_index, pte_flags); >> wmb(); >> } >> >> diff --git a/drivers/gpu/drm/i915/gt/selftest_migrate.c b/drivers/gpu/drm/i915/gt/selftest_migrate.c >> index e677f2da093d..3def5ca72dec 100644 >> --- a/drivers/gpu/drm/i915/gt/selftest_migrate.c >> +++ b/drivers/gpu/drm/i915/gt/selftest_migrate.c >> @@ -137,7 +137,7 @@ static int copy(struct intel_migrate *migrate, >> static int intel_context_copy_ccs(struct intel_context *ce, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool write_to_ccs, >> struct i915_request **out) >> { >> @@ -185,7 +185,7 @@ static int intel_context_copy_ccs(struct intel_context *ce, >> if (err) >> goto out_rq; >> >> - len = emit_pte(rq, &it, cache_level, true, offset, CHUNK_SZ); >> + len = emit_pte(rq, &it, pat_index, true, offset, CHUNK_SZ); >> if (len <= 0) { >> err = len; >> goto out_rq; >> @@ -223,7 +223,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, >> struct i915_gem_ww_ctx *ww, >> const struct i915_deps *deps, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool write_to_ccs, >> struct i915_request **out) >> { >> @@ -243,7 +243,7 @@ intel_migrate_ccs_copy(struct intel_migrate *m, >> if (err) >> goto out; >> >> - err = intel_context_copy_ccs(ce, deps, sg, cache_level, >> + err = intel_context_copy_ccs(ce, deps, sg, pat_index, >> write_to_ccs, out); >> >> intel_context_unpin(ce); >> @@ -300,7 +300,7 @@ static int clear(struct intel_migrate *migrate, >> /* Write the obj data into ccs surface */ >> err = intel_migrate_ccs_copy(migrate, &ww, NULL, >> obj->mm.pages->sgl, >> - obj->cache_level, >> + obj->pat_index, >> true, &rq); >> if (rq && !err) { >> if (i915_request_wait(rq, 0, HZ) < 0) { >> @@ -351,7 +351,7 @@ static int clear(struct intel_migrate *migrate, >> >> err = intel_migrate_ccs_copy(migrate, &ww, NULL, >> obj->mm.pages->sgl, >> - obj->cache_level, >> + obj->pat_index, >> false, &rq); >> if (rq && !err) { >> if (i915_request_wait(rq, 0, HZ) < 0) { >> @@ -414,9 +414,9 @@ static int __migrate_copy(struct intel_migrate *migrate, >> struct i915_request **out) >> { >> return intel_migrate_copy(migrate, ww, NULL, >> - src->mm.pages->sgl, src->cache_level, >> + src->mm.pages->sgl, src->pat_index, >> i915_gem_object_is_lmem(src), >> - dst->mm.pages->sgl, dst->cache_level, >> + dst->mm.pages->sgl, dst->pat_index, >> i915_gem_object_is_lmem(dst), >> out); >> } >> @@ -428,9 +428,9 @@ static int __global_copy(struct intel_migrate *migrate, >> struct i915_request **out) >> { >> return intel_context_migrate_copy(migrate->context, NULL, >> - src->mm.pages->sgl, src->cache_level, >> + src->mm.pages->sgl, src->pat_index, >> i915_gem_object_is_lmem(src), >> - dst->mm.pages->sgl, dst->cache_level, >> + dst->mm.pages->sgl, dst->pat_index, >> i915_gem_object_is_lmem(dst), >> out); >> } >> @@ -455,7 +455,7 @@ static int __migrate_clear(struct intel_migrate *migrate, >> { >> return intel_migrate_clear(migrate, ww, NULL, >> obj->mm.pages->sgl, >> - obj->cache_level, >> + obj->pat_index, >> i915_gem_object_is_lmem(obj), >> value, out); >> } >> @@ -468,7 +468,7 @@ static int __global_clear(struct intel_migrate *migrate, >> { >> return intel_context_migrate_clear(migrate->context, NULL, >> obj->mm.pages->sgl, >> - obj->cache_level, >> + obj->pat_index, >> i915_gem_object_is_lmem(obj), >> value, out); >> } >> @@ -648,7 +648,7 @@ static int live_emit_pte_full_ring(void *arg) >> */ >> pr_info("%s emite_pte ring space=%u\n", __func__, rq->ring->space); >> it = sg_sgt(obj->mm.pages->sgl); >> - len = emit_pte(rq, &it, obj->cache_level, false, 0, CHUNK_SZ); >> + len = emit_pte(rq, &it, obj->pat_index, false, 0, CHUNK_SZ); >> if (!len) { >> err = -EINVAL; >> goto out_rq; >> @@ -844,7 +844,7 @@ static int wrap_ktime_compare(const void *A, const void *B) >> >> static int __perf_clear_blt(struct intel_context *ce, >> struct scatterlist *sg, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> bool is_lmem, >> size_t sz) >> { >> @@ -858,7 +858,7 @@ static int __perf_clear_blt(struct intel_context *ce, >> >> t0 = ktime_get(); >> >> - err = intel_context_migrate_clear(ce, NULL, sg, cache_level, >> + err = intel_context_migrate_clear(ce, NULL, sg, pat_index, >> is_lmem, 0, &rq); >> if (rq) { >> if (i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT) < 0) >> @@ -904,7 +904,8 @@ static int perf_clear_blt(void *arg) >> >> err = __perf_clear_blt(gt->migrate.context, >> dst->mm.pages->sgl, >> - I915_CACHE_NONE, >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> i915_gem_object_is_lmem(dst), >> sizes[i]); >> >> @@ -919,10 +920,10 @@ static int perf_clear_blt(void *arg) >> >> static int __perf_copy_blt(struct intel_context *ce, >> struct scatterlist *src, >> - enum i915_cache_level src_cache_level, >> + unsigned int src_pat_index, >> bool src_is_lmem, >> struct scatterlist *dst, >> - enum i915_cache_level dst_cache_level, >> + unsigned int dst_pat_index, >> bool dst_is_lmem, >> size_t sz) >> { >> @@ -937,9 +938,9 @@ static int __perf_copy_blt(struct intel_context *ce, >> t0 = ktime_get(); >> >> err = intel_context_migrate_copy(ce, NULL, >> - src, src_cache_level, >> + src, src_pat_index, >> src_is_lmem, >> - dst, dst_cache_level, >> + dst, dst_pat_index, >> dst_is_lmem, >> &rq); >> if (rq) { >> @@ -994,10 +995,12 @@ static int perf_copy_blt(void *arg) >> >> err = __perf_copy_blt(gt->migrate.context, >> src->mm.pages->sgl, >> - I915_CACHE_NONE, >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> i915_gem_object_is_lmem(src), >> dst->mm.pages->sgl, >> - I915_CACHE_NONE, >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> i915_gem_object_is_lmem(dst), >> sz); >> >> diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c >> index a9e0a91bc0e0..79aa6ac66ad2 100644 >> --- a/drivers/gpu/drm/i915/gt/selftest_reset.c >> +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c >> @@ -86,7 +86,9 @@ __igt_reset_stolen(struct intel_gt *gt, >> >> ggtt->vm.insert_page(&ggtt->vm, dma, >> ggtt->error_capture.start, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> + 0); >> mb(); >> >> s = io_mapping_map_wc(&ggtt->iomap, >> @@ -127,7 +129,9 @@ __igt_reset_stolen(struct intel_gt *gt, >> >> ggtt->vm.insert_page(&ggtt->vm, dma, >> ggtt->error_capture.start, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> + 0); >> mb(); >> >> s = io_mapping_map_wc(&ggtt->iomap, >> diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c >> index 9f536c251179..39c3ec12df1a 100644 >> --- a/drivers/gpu/drm/i915/gt/selftest_timeline.c >> +++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c >> @@ -836,7 +836,7 @@ static int setup_watcher(struct hwsp_watcher *w, struct intel_gt *gt, >> return PTR_ERR(obj); >> >> /* keep the same cache settings as timeline */ >> - i915_gem_object_set_cache_coherency(obj, tl->hwsp_ggtt->obj->cache_level); >> + i915_gem_object_set_pat_index(obj, tl->hwsp_ggtt->obj->pat_index); >> w->map = i915_gem_object_pin_map_unlocked(obj, >> page_unmask_bits(tl->hwsp_ggtt->obj->mm.mapping)); >> if (IS_ERR(w->map)) { >> diff --git a/drivers/gpu/drm/i915/gt/selftest_tlb.c b/drivers/gpu/drm/i915/gt/selftest_tlb.c >> index e6cac1f15d6e..4493c8518e91 100644 >> --- a/drivers/gpu/drm/i915/gt/selftest_tlb.c >> +++ b/drivers/gpu/drm/i915/gt/selftest_tlb.c >> @@ -36,6 +36,8 @@ pte_tlbinv(struct intel_context *ce, >> u64 length, >> struct rnd_state *prng) >> { >> + const unsigned int pat_index = >> + i915_gem_get_pat_index(ce->vm->i915, I915_CACHE_NONE); >> struct drm_i915_gem_object *batch; >> struct drm_mm_node vb_node; >> struct i915_request *rq; >> @@ -155,7 +157,7 @@ pte_tlbinv(struct intel_context *ce, >> /* Flip the PTE between A and B */ >> if (i915_gem_object_is_lmem(vb->obj)) >> pte_flags |= PTE_LM; >> - ce->vm->insert_entries(ce->vm, &vb_res, 0, pte_flags); >> + ce->vm->insert_entries(ce->vm, &vb_res, pat_index, pte_flags); >> >> /* Flush the PTE update to concurrent HW */ >> tlbinv(ce->vm, addr & -length, length); >> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c >> index 4ec7df9ed5ff..e34c79120090 100644 >> --- a/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c >> +++ b/drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c >> @@ -957,9 +957,15 @@ static void uc_fw_bind_ggtt(struct intel_uc_fw *uc_fw) >> pte_flags |= PTE_LM; >> >> if (ggtt->vm.raw_insert_entries) >> - ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); >> + ggtt->vm.raw_insert_entries(&ggtt->vm, dummy, >> + i915_gem_get_pat_index(ggtt->vm.i915, >> + I915_CACHE_NONE), >> + pte_flags); >> else >> - ggtt->vm.insert_entries(&ggtt->vm, dummy, I915_CACHE_NONE, pte_flags); >> + ggtt->vm.insert_entries(&ggtt->vm, dummy, >> + i915_gem_get_pat_index(ggtt->vm.i915, >> + I915_CACHE_NONE), >> + pte_flags); >> } >> >> static void uc_fw_unbind_ggtt(struct intel_uc_fw *uc_fw) >> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c >> index 41389a32e998..76ccd4e03e31 100644 >> --- a/drivers/gpu/drm/i915/i915_debugfs.c >> +++ b/drivers/gpu/drm/i915/i915_debugfs.c >> @@ -139,21 +139,54 @@ static const char *stringify_vma_type(const struct i915_vma *vma) >> return "ppgtt"; >> } >> >> -static const char *i915_cache_level_str(struct drm_i915_private *i915, int type) >> -{ >> - switch (type) { >> - case I915_CACHE_NONE: return " uncached"; >> - case I915_CACHE_LLC: return HAS_LLC(i915) ? " LLC" : " snooped"; >> - case I915_CACHE_L3_LLC: return " L3+LLC"; >> - case I915_CACHE_WT: return " WT"; >> - default: return ""; >> +static const char *i915_cache_level_str(struct drm_i915_gem_object *obj) >> +{ >> + struct drm_i915_private *i915 = obj_to_i915(obj); >> + >> + if (IS_METEORLAKE(i915)) { >> + switch (obj->pat_index) { >> + case 0: return " WB"; >> + case 1: return " WT"; >> + case 2: return " UC"; >> + case 3: return " WB (1-Way Coh)"; >> + case 4: return " WB (2-Way Coh)"; >> + default: return " not defined"; >> + } >> + } else if (IS_PONTEVECCHIO(i915)) { >> + switch (obj->pat_index) { >> + case 0: return " UC"; >> + case 1: return " WC"; >> + case 2: return " WT"; >> + case 3: return " WB"; >> + case 4: return " WT (CLOS1)"; >> + case 5: return " WB (CLOS1)"; >> + case 6: return " WT (CLOS2)"; >> + case 7: return " WT (CLOS2)"; >> + default: return " not defined"; >> + } >> + } else if (GRAPHICS_VER(i915) >= 12) { >> + switch (obj->pat_index) { >> + case 0: return " WB"; >> + case 1: return " WC"; >> + case 2: return " WT"; >> + case 3: return " UC"; >> + default: return " not defined"; >> + } >> + } else { >> + switch (obj->pat_index) { >> + case 0: return " UC"; >> + case 1: return HAS_LLC(i915) ? >> + " LLC" : " snooped"; >> + case 2: return " L3+LLC"; >> + case 3: return " WT"; >> + default: return " not defined"; >> + } >> } >> } >> >> void >> i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) >> { >> - struct drm_i915_private *dev_priv = to_i915(obj->base.dev); >> struct i915_vma *vma; >> int pin_count = 0; >> >> @@ -165,7 +198,7 @@ i915_debugfs_describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj) >> obj->base.size / 1024, >> obj->read_domains, >> obj->write_domain, >> - i915_cache_level_str(dev_priv, obj->cache_level), >> + i915_cache_level_str(obj), >> obj->mm.dirty ? " dirty" : "", >> obj->mm.madv == I915_MADV_DONTNEED ? " purgeable" : ""); >> if (obj->base.name) >> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c >> index 0a78bdbd36b1..38933c00f705 100644 >> --- a/drivers/gpu/drm/i915/i915_gem.c >> +++ b/drivers/gpu/drm/i915/i915_gem.c >> @@ -420,8 +420,11 @@ i915_gem_gtt_pread(struct drm_i915_gem_object *obj, >> page_length = remain < page_length ? remain : page_length; >> if (drm_mm_node_allocated(&node)) { >> ggtt->vm.insert_page(&ggtt->vm, >> - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), >> - node.start, I915_CACHE_NONE, 0); >> + i915_gem_object_get_dma_address(obj, >> + offset >> PAGE_SHIFT), >> + node.start, >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), 0); >> } else { >> page_base += offset & PAGE_MASK; >> } >> @@ -598,8 +601,11 @@ i915_gem_gtt_pwrite_fast(struct drm_i915_gem_object *obj, >> /* flush the write before we modify the GGTT */ >> intel_gt_flush_ggtt_writes(ggtt->vm.gt); >> ggtt->vm.insert_page(&ggtt->vm, >> - i915_gem_object_get_dma_address(obj, offset >> PAGE_SHIFT), >> - node.start, I915_CACHE_NONE, 0); >> + i915_gem_object_get_dma_address(obj, >> + offset >> PAGE_SHIFT), >> + node.start, >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), 0); >> wmb(); /* flush modifications to the GGTT (insert_page) */ >> } else { >> page_base += offset & PAGE_MASK; >> @@ -1142,6 +1148,19 @@ int i915_gem_init(struct drm_i915_private *dev_priv) >> unsigned int i; >> int ret; >> >> + /* >> + * In the proccess of replacing cache_level with pat_index a tricky >> + * dependency is created on the definition of the enum i915_cache_level. >> + * in case this enum is changed, PTE encode would be broken. > >_I_n Sorry, what does this mean? >> + * Add a WARNING here. And remove when we completely quit using this >> + * enum >> + */ >> + WARN_ON_ONCE(I915_CACHE_NONE != 0 || >> + I915_CACHE_LLC != 1 || >> + I915_CACHE_L3_LLC != 2 || >> + I915_CACHE_WT != 3 || >> + I915_MAX_CACHE_LEVEL != 4); > > I thought BUILD_BUG_ON would be better. Of course, don't know what I was thinking, updated in v8. >> + >> /* We need to fallback to 4K pages if host doesn't support huge gtt. */ >> if (intel_vgpu_active(dev_priv) && !intel_vgpu_has_huge_gtt(dev_priv)) >> RUNTIME_INFO(dev_priv)->page_sizes = I915_GTT_PAGE_SIZE_4K; >> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c >> index 8ffdd7f47e93..566536499954 100644 >> --- a/drivers/gpu/drm/i915/i915_gpu_error.c >> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c >> @@ -1122,10 +1122,14 @@ i915_vma_coredump_create(const struct intel_gt *gt, >> mutex_lock(&ggtt->error_mutex); >> if (ggtt->vm.raw_insert_page) >> ggtt->vm.raw_insert_page(&ggtt->vm, dma, slot, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> + 0); >> else >> ggtt->vm.insert_page(&ggtt->vm, dma, slot, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(gt->i915, >> + I915_CACHE_NONE), >> + 0); >> mb(); >> >> s = io_mapping_map_wc(&ggtt->iomap, slot, PAGE_SIZE); >> diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c >> index 20a44788999e..a814775a363d 100644 >> --- a/drivers/gpu/drm/i915/i915_vma.c >> +++ b/drivers/gpu/drm/i915/i915_vma.c >> @@ -315,7 +315,7 @@ struct i915_vma_work { >> struct i915_vma_resource *vma_res; >> struct drm_i915_gem_object *obj; >> struct i915_sw_dma_fence_cb cb; >> - enum i915_cache_level cache_level; >> + unsigned int pat_index; >> unsigned int flags; >> }; >> >> @@ -334,7 +334,7 @@ static void __vma_bind(struct dma_fence_work *work) >> return; >> >> vma_res->ops->bind_vma(vma_res->vm, &vw->stash, >> - vma_res, vw->cache_level, vw->flags); >> + vma_res, vw->pat_index, vw->flags); >> } >> >> static void __vma_release(struct dma_fence_work *work) >> @@ -426,7 +426,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, >> /** >> * i915_vma_bind - Sets up PTEs for an VMA in it's corresponding address space. >> * @vma: VMA to map >> - * @cache_level: mapping cache level >> + * @pat_index: PAT index to set in PTE >> * @flags: flags like global or local mapping >> * @work: preallocated worker for allocating and binding the PTE >> * @vma_res: pointer to a preallocated vma resource. The resource is either >> @@ -437,7 +437,7 @@ i915_vma_resource_init_from_vma(struct i915_vma_resource *vma_res, >> * Note that DMA addresses are also the only part of the SG table we care about. >> */ >> int i915_vma_bind(struct i915_vma *vma, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags, >> struct i915_vma_work *work, >> struct i915_vma_resource *vma_res) >> @@ -507,7 +507,7 @@ int i915_vma_bind(struct i915_vma *vma, >> struct dma_fence *prev; >> >> work->vma_res = i915_vma_resource_get(vma->resource); >> - work->cache_level = cache_level; >> + work->pat_index = pat_index; >> work->flags = bind_flags; >> >> /* >> @@ -537,7 +537,7 @@ int i915_vma_bind(struct i915_vma *vma, >> >> return ret; >> } >> - vma->ops->bind_vma(vma->vm, NULL, vma->resource, cache_level, >> + vma->ops->bind_vma(vma->vm, NULL, vma->resource, pat_index, >> bind_flags); >> } >> >> @@ -814,7 +814,7 @@ i915_vma_insert(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, >> color = 0; >> >> if (i915_vm_has_cache_coloring(vma->vm)) >> - color = vma->obj->cache_level; >> + color = vma->obj->pat_index; >> >> if (flags & PIN_OFFSET_FIXED) { >> u64 offset = flags & PIN_OFFSET_MASK; >> @@ -1518,7 +1518,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww, >> >> GEM_BUG_ON(!vma->pages); >> err = i915_vma_bind(vma, >> - vma->obj->cache_level, >> + vma->obj->pat_index, >> flags, work, vma_res); >> vma_res = NULL; >> if (err) >> diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h >> index 38c8c66ed724..9a9729205d5b 100644 >> --- a/drivers/gpu/drm/i915/i915_vma.h >> +++ b/drivers/gpu/drm/i915/i915_vma.h >> @@ -250,7 +250,7 @@ i915_vma_compare(struct i915_vma *vma, >> >> struct i915_vma_work *i915_vma_work(void); >> int i915_vma_bind(struct i915_vma *vma, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags, >> struct i915_vma_work *work, >> struct i915_vma_resource *vma_res); >> diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h >> index 77fda2244d16..64472b7f0e77 100644 >> --- a/drivers/gpu/drm/i915/i915_vma_types.h >> +++ b/drivers/gpu/drm/i915/i915_vma_types.h >> @@ -32,8 +32,6 @@ >> >> #include "gem/i915_gem_object_types.h" >> >> -enum i915_cache_level; >> - >> /** >> * DOC: Global GTT views >> * >> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c >> index d91d0ade8abd..61da4ed9d521 100644 >> --- a/drivers/gpu/drm/i915/selftests/i915_gem.c >> +++ b/drivers/gpu/drm/i915/selftests/i915_gem.c >> @@ -57,7 +57,10 @@ static void trash_stolen(struct drm_i915_private *i915) >> u32 __iomem *s; >> int x; >> >> - ggtt->vm.insert_page(&ggtt->vm, dma, slot, I915_CACHE_NONE, 0); >> + ggtt->vm.insert_page(&ggtt->vm, dma, slot, >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), >> + 0); >> >> s = io_mapping_map_atomic_wc(&ggtt->iomap, slot); >> for (x = 0; x < PAGE_SIZE / sizeof(u32); x++) { >> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c >> index f68ef4074088..f8fe3681c3dc 100644 >> --- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c >> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c >> @@ -246,7 +246,7 @@ static int igt_evict_for_cache_color(void *arg) >> struct drm_mm_node target = { >> .start = I915_GTT_PAGE_SIZE * 2, >> .size = I915_GTT_PAGE_SIZE, >> - .color = I915_CACHE_LLC, >> + .color = i915_gem_get_pat_index(gt->i915, I915_CACHE_LLC), >> }; >> struct drm_i915_gem_object *obj; >> struct i915_vma *vma; >> @@ -309,7 +309,7 @@ static int igt_evict_for_cache_color(void *arg) >> /* Attempt to remove the first *pinned* vma, by removing the (empty) >> * neighbour -- this should fail. >> */ >> - target.color = I915_CACHE_L3_LLC; >> + target.color = i915_gem_get_pat_index(gt->i915, I915_CACHE_L3_LLC); >> >> mutex_lock(&ggtt->vm.mutex); >> err = i915_gem_evict_for_node(&ggtt->vm, NULL, &target, 0); >> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c >> index 154801f1c468..36940ef10108 100644 >> --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c >> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c >> @@ -135,7 +135,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size) >> >> obj->write_domain = I915_GEM_DOMAIN_CPU; >> obj->read_domains = I915_GEM_DOMAIN_CPU; >> - obj->cache_level = I915_CACHE_NONE; >> + obj->pat_index = i915_gem_get_pat_index(i915, I915_CACHE_NONE); >> >> /* Preallocate the "backing storage" */ >> if (i915_gem_object_pin_pages_unlocked(obj)) >> @@ -359,7 +359,9 @@ static int lowlevel_hole(struct i915_address_space *vm, >> >> with_intel_runtime_pm(vm->gt->uncore->rpm, wakeref) >> vm->insert_entries(vm, mock_vma_res, >> - I915_CACHE_NONE, 0); >> + i915_gem_get_pat_index(vm->i915, >> + I915_CACHE_NONE), >> + 0); >> } >> count = n; >> >> @@ -1377,7 +1379,10 @@ static int igt_ggtt_page(void *arg) >> >> ggtt->vm.insert_page(&ggtt->vm, >> i915_gem_object_get_dma_address(obj, 0), >> - offset, I915_CACHE_NONE, 0); >> + offset, >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), >> + 0); >> } >> >> order = i915_random_order(count, &prng); >> @@ -1510,7 +1515,7 @@ static int reserve_gtt_with_resource(struct i915_vma *vma, u64 offset) >> mutex_lock(&vm->mutex); >> err = i915_gem_gtt_reserve(vm, NULL, &vma->node, obj->base.size, >> offset, >> - obj->cache_level, >> + obj->pat_index, >> 0); >> if (!err) { >> i915_vma_resource_init_from_vma(vma_res, vma); >> @@ -1690,7 +1695,7 @@ static int insert_gtt_with_resource(struct i915_vma *vma) >> >> mutex_lock(&vm->mutex); >> err = i915_gem_gtt_insert(vm, NULL, &vma->node, obj->base.size, 0, >> - obj->cache_level, 0, vm->total, 0); >> + obj->pat_index, 0, vm->total, 0); >> if (!err) { >> i915_vma_resource_init_from_vma(vma_res, vma); >> vma->resource = vma_res; >> diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c >> index 3b18e5905c86..d985d9bae2e8 100644 >> --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c >> +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c >> @@ -1070,7 +1070,9 @@ static int igt_lmem_write_cpu(void *arg) >> /* Put the pages into a known state -- from the gpu for added fun */ >> intel_engine_pm_get(engine); >> err = intel_context_migrate_clear(engine->gt->migrate.context, NULL, >> - obj->mm.pages->sgl, I915_CACHE_NONE, >> + obj->mm.pages->sgl, >> + i915_gem_get_pat_index(i915, >> + I915_CACHE_NONE), >> true, 0xdeadbeaf, &rq); >> if (rq) { >> dma_resv_add_fence(obj->base.resv, &rq->fence, >> diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c >> index ece97e4faacb..a516c0aa88fd 100644 >> --- a/drivers/gpu/drm/i915/selftests/mock_gtt.c >> +++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c >> @@ -27,21 +27,21 @@ >> static void mock_insert_page(struct i915_address_space *vm, >> dma_addr_t addr, >> u64 offset, >> - enum i915_cache_level level, >> + unsigned int pat_index, >> u32 flags) >> { >> } >> >> static void mock_insert_entries(struct i915_address_space *vm, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level level, u32 flags) >> + unsigned int pat_index, u32 flags) >> { >> } >> >> static void mock_bind_ppgtt(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> GEM_BUG_ON(flags & I915_VMA_GLOBAL_BIND); >> @@ -94,7 +94,7 @@ struct i915_ppgtt *mock_ppgtt(struct drm_i915_private *i915, const char *name) >> static void mock_bind_ggtt(struct i915_address_space *vm, >> struct i915_vm_pt_stash *stash, >> struct i915_vma_resource *vma_res, >> - enum i915_cache_level cache_level, >> + unsigned int pat_index, >> u32 flags) >> { >> } > > With a pinky promise to improve this all in the near future I won't > grumble to loudly. :) I haven't read all the details, I leave that to > other reviewers, and also assuming some final tweaks as indicated above > please. Thanks for all the suggestions, really appreciated. May I add your Acked-by? -Fei > Regards, > > Tvrtko [-- Attachment #2: Type: text/html, Size: 254881 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level 2023-05-09 17:12 ` Yang, Fei @ 2023-05-11 12:02 ` Tvrtko Ursulin 0 siblings, 0 replies; 10+ messages in thread From: Tvrtko Ursulin @ 2023-05-11 12:02 UTC (permalink / raw) To: Yang, Fei, intel-gfx@lists.freedesktop.org Cc: Chris Wilson, Roper, Matthew D, dri-devel@lists.freedesktop.org On 09/05/2023 18:12, Yang, Fei wrote: > > On 09/05/2023 00:48, fei.yang@intel.com wrote: > >> From: Fei Yang <fei.yang@intel.com> > >> > >> Currently the KMD is using enum i915_cache_level to set caching > policy for > >> buffer objects. This is flaky because the PAT index which really > controls > >> the caching behavior in PTE has far more levels than what's defined > in the > >> enum. In addition, the PAT index is platform dependent, having to > translate > >> between i915_cache_level and PAT index is not reliable, and makes > the code > >> more complicated. > >> > >> From UMD's perspective there is also a necessity to set caching > policy for > >> performance fine tuning. It's much easier for the UMD to directly > use PAT > >> index because the behavior of each PAT index is clearly defined in > Bspec. > >> Having the abstracted i915_cache_level sitting in between would only > cause > >> more ambiguity. PAT is expected to work much like MOCS already works > today, > >> and by design userspace is expected to select the index that exactly > >> matches the desired behavior described in the hardware specification. > >> > >> For these reasons this patch replaces i915_cache_level with PAT > index. Also > >> note, the cache_level is not completely removed yet, because the KMD > still > >> has the need of creating buffer objects with simple cache settings > such as > >> cached, uncached, or writethrough. For kernel objects, cache_level > is used > >> for simplicity and backward compatibility. For Pre-gen12 platforms > PAT can > >> have 1:1 mapping to i915_cache_level, so these two are > interchangeable. see > >> the use of LEGACY_CACHELEVEL. > >> > >> One consequence of this change is that gen8_pte_encode is no longer > working > >> for gen12 platforms due to the fact that gen12 platforms has > different PAT > >> definitions. In the meantime the mtl_pte_encode introduced > specfically for > >> MTL becomes generic for all gen12 platforms. This patch renames the MTL > >> PTE encode function into gen12_pte_encode and apply it to all gen12. > Even > >> though this change looks unrelated, but separating them would > temporarily > >> break gen12 PTE encoding, thus squash them in one patch. > >> > >> Special note: this patch changes the way caching behavior is > controlled in > >> the sense that some objects are left to be managed by userspace. For > such > >> objects we need to be careful not to change the userspace settings.There > >> are kerneldoc and comments added around obj->cache_coherent, > cache_dirty, > >> and how to bypass the checkings by i915_gem_object_has_cache_level. For > >> full understanding, these changes need to be looked at together with the > >> two follow-up patches, one disables the {set|get}_caching ioctl's > and the > >> other adds set_pat extension to the GEM_CREATE uAPI. > >> > >> Bspec: 63019 > >> > >> Cc: Chris Wilson <chris.p.wilson@linux.intel.com> > >> Signed-off-by: Fei Yang <fei.yang@intel.com> > >> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> > >> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> [snip] > >> + node.start, > >> + i915_gem_get_pat_index(i915, > >> + > I915_CACHE_NONE), 0); > >> wmb(); /* flush modifications to the GGTT > (insert_page) */ > >> } else { > >> page_base += offset & PAGE_MASK; > >> @@ -1142,6 +1148,19 @@ int i915_gem_init(struct drm_i915_private > *dev_priv) > >> unsigned int i; > >> int ret; > >> > >> + /* > >> + * In the proccess of replacing cache_level with pat_index a > tricky > >> + * dependency is created on the definition of the enum > i915_cache_level. > >> + * in case this enum is changed, PTE encode would be broken. > > > >_I_n > > Sorry, what does this mean? Start of sentence, capital 'i'. [snip] > > With a pinky promise to improve this all in the near future I won't > > grumble to loudly. :) I haven't read all the details, I leave that to > > other reviewers, and also assuming some final tweaks as indicated above > > please. > > Thanks for all the suggestions, really appreciated. > May I add your Acked-by? I can't make myself do it since I really don't like the design that much. That's why I said I will not grumble too loudly. Jira for follow up clean since we both agreed something more elegant is possible would be appreciated though. Regards, Tvrtko ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Intel-gfx] ✗ Fi.CI.SPARSE: warning for drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 1/2] drm/i915: preparation for using PAT index fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level fei.yang @ 2023-05-09 1:18 ` Patchwork 2023-05-09 1:32 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork 2023-05-09 5:47 ` [Intel-gfx] ✓ Fi.CI.IGT: " Patchwork 4 siblings, 0 replies; 10+ messages in thread From: Patchwork @ 2023-05-09 1:18 UTC (permalink / raw) To: Yang, Fei; +Cc: intel-gfx == Series Details == Series: drm/i915: use pat_index instead of cache_level URL : https://patchwork.freedesktop.org/series/117480/ State : warning == Summary == Error: dim sparse failed Sparse version: v0.6.2 Fast mode used, each commit won't be checked separately. ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Intel-gfx] ✓ Fi.CI.BAT: success for drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang ` (2 preceding siblings ...) 2023-05-09 1:18 ` [Intel-gfx] ✗ Fi.CI.SPARSE: warning for " Patchwork @ 2023-05-09 1:32 ` Patchwork 2023-05-09 5:47 ` [Intel-gfx] ✓ Fi.CI.IGT: " Patchwork 4 siblings, 0 replies; 10+ messages in thread From: Patchwork @ 2023-05-09 1:32 UTC (permalink / raw) To: Yang, Fei; +Cc: intel-gfx [-- Attachment #1: Type: text/plain, Size: 3400 bytes --] == Series Details == Series: drm/i915: use pat_index instead of cache_level URL : https://patchwork.freedesktop.org/series/117480/ State : success == Summary == CI Bug Log - changes from CI_DRM_13124 -> Patchwork_117480v1 ==================================================== Summary ------- **SUCCESS** No regressions found. External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/index.html Participating hosts (40 -> 39) ------------------------------ Missing (1): fi-snb-2520m Known issues ------------ Here are the changes found in Patchwork_117480v1 that come from known issues: ### IGT changes ### #### Issues hit #### * igt@i915_selftest@live@gt_lrc: - bat-dg2-11: [PASS][1] -> [INCOMPLETE][2] ([i915#7609] / [i915#7913] / [i915#7953]) [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/bat-dg2-11/igt@i915_selftest@live@gt_lrc.html [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/bat-dg2-11/igt@i915_selftest@live@gt_lrc.html * igt@i915_selftest@live@slpc: - bat-rpls-2: [PASS][3] -> [DMESG-WARN][4] ([i915#6367]) [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/bat-rpls-2/igt@i915_selftest@live@slpc.html [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/bat-rpls-2/igt@i915_selftest@live@slpc.html * igt@kms_pipe_crc_basic@nonblocking-crc@pipe-d-dp-1: - bat-dg2-8: [PASS][5] -> [FAIL][6] ([i915#7932]) [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/bat-dg2-8/igt@kms_pipe_crc_basic@nonblocking-crc@pipe-d-dp-1.html [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/bat-dg2-8/igt@kms_pipe_crc_basic@nonblocking-crc@pipe-d-dp-1.html #### Possible fixes #### * igt@i915_selftest@live@gt_heartbeat: - fi-glk-j4005: [DMESG-FAIL][7] ([i915#5334]) -> [PASS][8] [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/fi-glk-j4005/igt@i915_selftest@live@gt_heartbeat.html [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/fi-glk-j4005/igt@i915_selftest@live@gt_heartbeat.html {name}: This element is suppressed. This means it is ignored when computing the status of the difference (SUCCESS, WARNING, or FAILURE). [i915#4817]: https://gitlab.freedesktop.org/drm/intel/issues/4817 [i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334 [i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367 [i915#7609]: https://gitlab.freedesktop.org/drm/intel/issues/7609 [i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913 [i915#7932]: https://gitlab.freedesktop.org/drm/intel/issues/7932 [i915#7953]: https://gitlab.freedesktop.org/drm/intel/issues/7953 Build changes ------------- * Linux: CI_DRM_13124 -> Patchwork_117480v1 CI-20190529: 20190529 CI_DRM_13124: 28153baed517db8d009844ee992f850a88c9eb33 @ git://anongit.freedesktop.org/gfx-ci/linux IGT_7283: ce51f53938690f581b315fa045d41155a5c6ecd3 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git Patchwork_117480v1: 28153baed517db8d009844ee992f850a88c9eb33 @ git://anongit.freedesktop.org/gfx-ci/linux ### Linux commits e947e617e7c3 drm/i915: use pat_index instead of cache_level 9ac93e110e4e drm/i915: preparation for using PAT index == Logs == For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/index.html [-- Attachment #2: Type: text/html, Size: 4030 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Intel-gfx] ✓ Fi.CI.IGT: success for drm/i915: use pat_index instead of cache_level 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang ` (3 preceding siblings ...) 2023-05-09 1:32 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork @ 2023-05-09 5:47 ` Patchwork 4 siblings, 0 replies; 10+ messages in thread From: Patchwork @ 2023-05-09 5:47 UTC (permalink / raw) To: Yang, Fei; +Cc: intel-gfx [-- Attachment #1: Type: text/plain, Size: 9359 bytes --] == Series Details == Series: drm/i915: use pat_index instead of cache_level URL : https://patchwork.freedesktop.org/series/117480/ State : success == Summary == CI Bug Log - changes from CI_DRM_13124_full -> Patchwork_117480v1_full ==================================================== Summary ------- **SUCCESS** No regressions found. Participating hosts (7 -> 7) ------------------------------ No changes in participating hosts Known issues ------------ Here are the changes found in Patchwork_117480v1_full that come from known issues: ### IGT changes ### #### Issues hit #### * igt@gem_exec_fair@basic-none-solo@rcs0: - shard-apl: [PASS][1] -> [FAIL][2] ([i915#2842]) [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-apl7/igt@gem_exec_fair@basic-none-solo@rcs0.html [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-apl4/igt@gem_exec_fair@basic-none-solo@rcs0.html * igt@gem_exec_fair@basic-pace-share@rcs0: - shard-glk: [PASS][3] -> [FAIL][4] ([i915#2842]) [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-glk2/igt@gem_exec_fair@basic-pace-share@rcs0.html [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk8/igt@gem_exec_fair@basic-pace-share@rcs0.html * igt@gem_exec_fair@basic-throttle@rcs0: - shard-glk: NOTRUN -> [FAIL][5] ([i915#2842]) [5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk7/igt@gem_exec_fair@basic-throttle@rcs0.html * igt@gem_lmem_swapping@verify-random-ccs: - shard-glk: NOTRUN -> [SKIP][6] ([fdo#109271] / [i915#4613]) +1 similar issue [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk7/igt@gem_lmem_swapping@verify-random-ccs.html * igt@gem_userptr_blits@vma-merge: - shard-glk: NOTRUN -> [FAIL][7] ([i915#3318]) [7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk7/igt@gem_userptr_blits@vma-merge.html * igt@gen9_exec_parse@allowed-single: - shard-apl: [PASS][8] -> [ABORT][9] ([i915#5566]) [8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-apl6/igt@gen9_exec_parse@allowed-single.html [9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-apl7/igt@gen9_exec_parse@allowed-single.html * igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs: - shard-glk: NOTRUN -> [SKIP][10] ([fdo#109271] / [i915#3886]) +6 similar issues [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk3/igt@kms_ccs@pipe-c-bad-rotation-90-y_tiled_gen12_mc_ccs.html * igt@kms_color@ctm-green-to-red@pipe-a-hdmi-a-1: - shard-snb: NOTRUN -> [SKIP][11] ([fdo#109271]) +40 similar issues [11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-snb1/igt@kms_color@ctm-green-to-red@pipe-a-hdmi-a-1.html * igt@kms_cursor_crc@cursor-random-max-size: - shard-glk: NOTRUN -> [SKIP][12] ([fdo#109271]) +94 similar issues [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk7/igt@kms_cursor_crc@cursor-random-max-size.html * igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2: - shard-glk: [PASS][13] -> [FAIL][14] ([i915#79]) [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-glk2/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk8/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html * igt@kms_psr2_sf@overlay-plane-update-continuous-sf: - shard-glk: NOTRUN -> [SKIP][15] ([fdo#109271] / [i915#658]) +1 similar issue [15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk7/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html #### Possible fixes #### * igt@drm_fdinfo@most-busy-idle-check-all@rcs0: - {shard-rkl}: [FAIL][16] ([i915#7742]) -> [PASS][17] [16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-rkl-6/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html [17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-rkl-1/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html * igt@gem_ctx_exec@basic-nohangcheck: - {shard-tglu}: [FAIL][18] ([i915#6268]) -> [PASS][19] [18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-tglu-5/igt@gem_ctx_exec@basic-nohangcheck.html [19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-tglu-8/igt@gem_ctx_exec@basic-nohangcheck.html * igt@gem_exec_fair@basic-none@vecs0: - {shard-rkl}: [FAIL][20] ([i915#2842]) -> [PASS][21] [20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-rkl-6/igt@gem_exec_fair@basic-none@vecs0.html [21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-rkl-1/igt@gem_exec_fair@basic-none@vecs0.html * igt@gem_ppgtt@blt-vs-render-ctxn: - shard-snb: [FAIL][22] ([i915#8295]) -> [PASS][23] [22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-snb6/igt@gem_ppgtt@blt-vs-render-ctxn.html [23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-snb4/igt@gem_ppgtt@blt-vs-render-ctxn.html * igt@i915_module_load@reload-with-fault-injection: - {shard-dg1}: [DMESG-WARN][24] ([i915#8420]) -> [PASS][25] [24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-dg1-17/igt@i915_module_load@reload-with-fault-injection.html [25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-dg1-14/igt@i915_module_load@reload-with-fault-injection.html * igt@i915_pm_rpm@modeset-lpsp-stress-no-wait: - {shard-rkl}: [SKIP][26] ([i915#1397]) -> [PASS][27] [26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-rkl-2/igt@i915_pm_rpm@modeset-lpsp-stress-no-wait.html [27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-rkl-7/igt@i915_pm_rpm@modeset-lpsp-stress-no-wait.html * igt@kms_flip@plain-flip-ts-check-interruptible@b-hdmi-a2: - shard-glk: [FAIL][28] ([i915#2122]) -> [PASS][29] [28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13124/shard-glk1/igt@kms_flip@plain-flip-ts-check-interruptible@b-hdmi-a2.html [29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/shard-glk8/igt@kms_flip@plain-flip-ts-check-interruptible@b-hdmi-a2.html {name}: This element is suppressed. This means it is ignored when computing the status of the difference (SUCCESS, WARNING, or FAILURE). [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375 [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271 [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189 [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072 [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397 [i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122 [i915#2681]: https://gitlab.freedesktop.org/drm/intel/issues/2681 [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842 [i915#315]: https://gitlab.freedesktop.org/drm/intel/issues/315 [i915#3318]: https://gitlab.freedesktop.org/drm/intel/issues/3318 [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555 [i915#3591]: https://gitlab.freedesktop.org/drm/intel/issues/3591 [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886 [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955 [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070 [i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078 [i915#4258]: https://gitlab.freedesktop.org/drm/intel/issues/4258 [i915#4579]: https://gitlab.freedesktop.org/drm/intel/issues/4579 [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613 [i915#4816]: https://gitlab.freedesktop.org/drm/intel/issues/4816 [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176 [i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566 [i915#6268]: https://gitlab.freedesktop.org/drm/intel/issues/6268 [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658 [i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742 [i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79 [i915#7953]: https://gitlab.freedesktop.org/drm/intel/issues/7953 [i915#7975]: https://gitlab.freedesktop.org/drm/intel/issues/7975 [i915#8213]: https://gitlab.freedesktop.org/drm/intel/issues/8213 [i915#8295]: https://gitlab.freedesktop.org/drm/intel/issues/8295 [i915#8420]: https://gitlab.freedesktop.org/drm/intel/issues/8420 Build changes ------------- * Linux: CI_DRM_13124 -> Patchwork_117480v1 CI-20190529: 20190529 CI_DRM_13124: 28153baed517db8d009844ee992f850a88c9eb33 @ git://anongit.freedesktop.org/gfx-ci/linux IGT_7283: ce51f53938690f581b315fa045d41155a5c6ecd3 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git Patchwork_117480v1: 28153baed517db8d009844ee992f850a88c9eb33 @ git://anongit.freedesktop.org/gfx-ci/linux piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit == Logs == For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_117480v1/index.html [-- Attachment #2: Type: text/html, Size: 9725 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-05-11 12:03 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-05-08 23:48 [Intel-gfx] [PATCH v7 0/2] drm/i915: use pat_index instead of cache_level fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 1/2] drm/i915: preparation for using PAT index fei.yang 2023-05-08 23:48 ` [Intel-gfx] [PATCH v7 2/2] drm/i915: use pat_index instead of cache_level fei.yang 2023-05-09 11:08 ` Andi Shyti 2023-05-09 12:15 ` Tvrtko Ursulin 2023-05-09 17:12 ` Yang, Fei 2023-05-11 12:02 ` Tvrtko Ursulin 2023-05-09 1:18 ` [Intel-gfx] ✗ Fi.CI.SPARSE: warning for " Patchwork 2023-05-09 1:32 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork 2023-05-09 5:47 ` [Intel-gfx] ✓ Fi.CI.IGT: " Patchwork
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox