Re: [Intel-gfx] [PATCH] drm/i915: Document and future-proof preemption control policy

[Wayne Boyer <wayne.boyer@intel.com>]

On 9/7/22 2:24 PM, Matt Roper wrote:
> Intel hardware allows some preemption settings to be controlled either
> by the kernel-mode driver exclusively, or placed under control of the
> user-mode drivers; on Linux we always select the userspace control
> option.  The various registers involved in this are not documented very
> clearly; let's add some clarifying comments to help explain how this all
> works and provide some history on why our Linux drivers take the
> approach they do (which I believe differs from the path taken by certain
> other operating systems' drivers).
> 
> While we're at it, let's also remove the graphics version 12 upper bound
> on this programming.  As described, we don't have any plans to move away
> from UMD control of preemption settings on future platforms, and there's
> currently no reason to believe that the hardware will fundamentally
> change how these registers and settings work after version 12.
> 
> Bspec: 45921, 45858, 45863
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
> Suggested-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Signed-off-by: Matt Roper <matthew.d.roper@intel.com>

Reviewed-by: Wayne Boyer <wayne.boyer@intel.com>

> ---
>   drivers/gpu/drm/i915/gt/intel_workarounds.c | 58 +++++++++++++++++++--
>   1 file changed, 55 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
> index 6d2003d598e6..3e5a41378e81 100644
> --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
> +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
> @@ -2389,12 +2389,64 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
>   			     FF_DOP_CLOCK_GATE_DISABLE);
>   	}
>   
> -	if (IS_GRAPHICS_VER(i915, 9, 12)) {
> -		/* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl,tgl */
> +	/*
> +	 * Intel platforms that support fine-grained preemption (i.e., gen9 and
> +	 * beyond) allow the kernel-mode driver to choose between two different
> +	 * options for controlling preemption granularity and behavior.
> +	 *
> +	 * Option 1 (hardware default):
> +	 *   Preemption settings are controlled in a global manner via
> +	 *   kernel-only register CS_DEBUG_MODE1 (0x20EC).  Any granularity
> +	 *   and settings chosen by the kernel-mode driver will apply to all
> +	 *   userspace clients.
> +	 *
> +	 * Option 2:
> +	 *   Preemption settings are controlled on a per-context basis via
> +	 *   register CS_CHICKEN1 (0x2580).  CS_CHICKEN1 is saved/restored on
> +	 *   context switch and is writable by userspace (e.g., via
> +	 *   MI_LOAD_REGISTER_IMMEDIATE instructions placed in a batch buffer)
> +	 *   which allows different userspace drivers/clients to select
> +	 *   different settings, or to change those settings on the fly in
> +	 *   response to runtime needs.  This option was known by name
> +	 *   "FtrPerCtxtPreemptionGranularityControl" at one time, although
> +	 *   that name is somewhat misleading as other non-granularity
> +	 *   preemption settings are also impacted by this decision.
> +	 *
> +	 * On Linux, our policy has always been to let userspace drivers
> +	 * control preemption granularity/settings (Option 2).  This was
> +	 * originally mandatory on gen9 to prevent ABI breakage (old gen9
> +	 * userspace developed before object-level preemption was enabled would
> +	 * not behave well if i915 were to go with Option 1 and enable that
> +	 * preemption in a global manner).  On gen9 each context would have
> +	 * object-level preemption disabled by default (see
> +	 * WaDisable3DMidCmdPreemption in gen9_ctx_workarounds_init), but
> +	 * userspace drivers could opt-in to object-level preemption as they
> +	 * saw fit.  For post-gen9 platforms, we continue to utilize Option 2;
> +	 * even though it is no longer necessary for ABI compatibility when
> +	 * enabling a new platform, it does ensure that userspace will be able
> +	 * to implement any workarounds that show up requiring temporary
> +	 * adjustments to preemption behavior at runtime.
> +	 *
> +	 * Notes/Workarounds:
> +	 *  - Wa_14015141709:  On DG2 and early steppings of MTL,
> +	 *      CS_CHICKEN1[0] does not disable object-level preemption as
> +	 *      it is supposed to (nor does CS_DEBUG_MODE1[0] if we had been
> +	 *      using Option 1).  Effectively this means userspace is unable
> +	 *      to disable object-level preemption on these platforms/steppings
> +	 *      despite the setting here.
> +	 *
> +	 *  - Wa_16013994831:  May require that userspace program
> +	 *      CS_CHICKEN1[10] when certain runtime conditions are true.
> +	 *      Userspace requires Option 2 to be in effect for their update of
> +	 *      CS_CHICKEN1[10] to be effective.
> +	 *
> +	 * Other workarounds may appear in the future that will also require
> +	 * Option 2 behavior to allow proper userspace implementation.
> +	 */
> +	if (GRAPHICS_VER(i915) >= 9)
>   		wa_masked_en(wal,
>   			     GEN7_FF_SLICE_CS_CHICKEN1,
>   			     GEN9_FFSC_PERCTX_PREEMPT_CTRL);
> -	}
>   
>   	if (IS_SKYLAKE(i915) ||
>   	    IS_KABYLAKE(i915) ||

-- 
--
Wayne Boyer
Graphics Software Engineer
VTT-OSGC Platform Enablement