[PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[Chris Wilson]

Ville noticed that the last mocs entry is used unconditionally by the HW
when it performs cache evictions, and noted that while the value is not
meant to be writable by the driver, we should program it to a reasonable
value nevertheless.

As it turns out, we can change the value of mocs:63 and the value we
were programming into it would cause hard hangs in conjunction with
atomic operations.

Suggested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2707
Fixes: 3bbaba0ceaa2 ("drm/i915: Added Programming of the MOCS")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: <stable@vger.kernel.org> # v4.3+
---
 drivers/gpu/drm/i915/gt/intel_mocs.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
index 254873e1646e..6ae512847f64 100644
--- a/drivers/gpu/drm/i915/gt/intel_mocs.c
+++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
@@ -131,7 +131,10 @@ static const struct drm_i915_mocs_entry skl_mocs_table[] = {
 	GEN9_MOCS_ENTRIES,
 	MOCS_ENTRY(I915_MOCS_CACHED,
 		   LE_3_WB | LE_TC_2_LLC_ELLC | LE_LRUM(3),
-		   L3_3_WB)
+		   L3_3_WB),
+	MOCS_ENTRY(63,
+		   LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
+		   L3_1_UC)
 };
 
 /* NOTE: the LE_TGT_CACHE is not used on Broxton */
-- 
2.20.1

Re: [PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[Ville Syrjälä]

On Thu, Nov 26, 2020 at 10:55:39AM +0000, Chris Wilson wrote:
> Ville noticed that the last mocs entry is used unconditionally by the HW
> when it performs cache evictions, and noted that while the value is not
> meant to be writable by the driver, we should program it to a reasonable
> value nevertheless.
> 
> As it turns out, we can change the value of mocs:63 and the value we
> were programming into it would cause hard hangs in conjunction with
> atomic operations.
> 
> Suggested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2707
> Fixes: 3bbaba0ceaa2 ("drm/i915: Added Programming of the MOCS")
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
> Cc: Jason Ekstrand <jason@jlekstrand.net>
> Cc: <stable@vger.kernel.org> # v4.3+
> ---
>  drivers/gpu/drm/i915/gt/intel_mocs.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
> index 254873e1646e..6ae512847f64 100644
> --- a/drivers/gpu/drm/i915/gt/intel_mocs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
> @@ -131,7 +131,10 @@ static const struct drm_i915_mocs_entry skl_mocs_table[] = {
>  	GEN9_MOCS_ENTRIES,
>  	MOCS_ENTRY(I915_MOCS_CACHED,
>  		   LE_3_WB | LE_TC_2_LLC_ELLC | LE_LRUM(3),
> -		   L3_3_WB)
> +		   L3_3_WB),
> +	MOCS_ENTRY(63,

Wonder if we should give these magic MOCS entries actual names?

Anyways, matches my reading of the spec
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>

> +		   LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
> +		   L3_1_UC)
>  };
>  
>  /* NOTE: the LE_TGT_CACHE is not used on Broxton */
> -- 
> 2.20.1

-- 
Ville Syrjälä
Intel

[PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[Chris Wilson]

Ville noticed that the last mocs entry is used unconditionally by the HW
when it performs cache evictions, and noted that while the value is not
meant to be writable by the driver, we should program it to a reasonable
value nevertheless.

As it turns out, we can change the value of mocs:63 and the value we
were programming into it would cause hard hangs in conjunction with
atomic operations.

v2: Add details from bspec about how it is used by HW

Suggested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2707
Fixes: 3bbaba0ceaa2 ("drm/i915: Added Programming of the MOCS")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: <stable@vger.kernel.org> # v4.3+
---
 drivers/gpu/drm/i915/gt/intel_mocs.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
index 254873e1646e..26cedde80476 100644
--- a/drivers/gpu/drm/i915/gt/intel_mocs.c
+++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
@@ -131,7 +131,19 @@ static const struct drm_i915_mocs_entry skl_mocs_table[] = {
 	GEN9_MOCS_ENTRIES,
 	MOCS_ENTRY(I915_MOCS_CACHED,
 		   LE_3_WB | LE_TC_2_LLC_ELLC | LE_LRUM(3),
-		   L3_3_WB)
+		   L3_3_WB),
+
+	/*
+	 * mocs:63
+	 * - used by the L3 for all its evictions.
+	 *   Thus it is expected to allow LLC cacheability to enable coherent
+	 *   flows to be maintained.
+	 * - used to force L3 uncachable cycles.
+	 *   Thus it is expected to make the surce L3 uncacheable.
+	 */
+	MOCS_ENTRY(63,
+		   LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
+		   L3_1_UC)
 };
 
 /* NOTE: the LE_TGT_CACHE is not used on Broxton */
-- 
2.20.1

Re: [PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[Ville Syrjälä]

On Thu, Nov 26, 2020 at 02:08:41PM +0000, Chris Wilson wrote:
> Ville noticed that the last mocs entry is used unconditionally by the HW
> when it performs cache evictions, and noted that while the value is not
> meant to be writable by the driver, we should program it to a reasonable
> value nevertheless.
> 
> As it turns out, we can change the value of mocs:63 and the value we
> were programming into it would cause hard hangs in conjunction with
> atomic operations.
> 
> v2: Add details from bspec about how it is used by HW
> 
> Suggested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2707
> Fixes: 3bbaba0ceaa2 ("drm/i915: Added Programming of the MOCS")
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
> Cc: Jason Ekstrand <jason@jlekstrand.net>
> Cc: <stable@vger.kernel.org> # v4.3+
> ---
>  drivers/gpu/drm/i915/gt/intel_mocs.c | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
> index 254873e1646e..26cedde80476 100644
> --- a/drivers/gpu/drm/i915/gt/intel_mocs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
> @@ -131,7 +131,19 @@ static const struct drm_i915_mocs_entry skl_mocs_table[] = {
>  	GEN9_MOCS_ENTRIES,
>  	MOCS_ENTRY(I915_MOCS_CACHED,
>  		   LE_3_WB | LE_TC_2_LLC_ELLC | LE_LRUM(3),
> -		   L3_3_WB)
> +		   L3_3_WB),
> +
> +	/*
> +	 * mocs:63
> +	 * - used by the L3 for all its evictions.
> +	 *   Thus it is expected to allow LLC cacheability to enable coherent
> +	 *   flows to be maintained.
> +	 * - used to force L3 uncachable cycles.
> +	 *   Thus it is expected to make the surce L3 uncacheable.

"surce"?

> +	 */
> +	MOCS_ENTRY(63,
> +		   LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
> +		   L3_1_UC)
>  };
>  
>  /* NOTE: the LE_TGT_CACHE is not used on Broxton */
> -- 
> 2.20.1

-- 
Ville Syrjälä
Intel

Re: [PATCH] drm/i915/gt: Program mocs:63 for cache eviction on gen9

[Jason Ekstrand]

I just pulled drm-tip with both this and "drm/i915/gt: Declare gen9
has 64 mocs entries!" and the hang persists.  You really had my hopes
up there....  I think we need to merge the L3$ disable patch.

--Jason


On Thu, Nov 26, 2020 at 9:50 AM Ville Syrjälä
<ville.syrjala@linux.intel.com> wrote:
>
> On Thu, Nov 26, 2020 at 02:08:41PM +0000, Chris Wilson wrote:
> > Ville noticed that the last mocs entry is used unconditionally by the HW
> > when it performs cache evictions, and noted that while the value is not
> > meant to be writable by the driver, we should program it to a reasonable
> > value nevertheless.
> >
> > As it turns out, we can change the value of mocs:63 and the value we
> > were programming into it would cause hard hangs in conjunction with
> > atomic operations.
> >
> > v2: Add details from bspec about how it is used by HW
> >
> > Suggested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> > Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2707
> > Fixes: 3bbaba0ceaa2 ("drm/i915: Added Programming of the MOCS")
> > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
> > Cc: Jason Ekstrand <jason@jlekstrand.net>
> > Cc: <stable@vger.kernel.org> # v4.3+
> > ---
> >  drivers/gpu/drm/i915/gt/intel_mocs.c | 14 +++++++++++++-
> >  1 file changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_mocs.c b/drivers/gpu/drm/i915/gt/intel_mocs.c
> > index 254873e1646e..26cedde80476 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_mocs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_mocs.c
> > @@ -131,7 +131,19 @@ static const struct drm_i915_mocs_entry skl_mocs_table[] = {
> >       GEN9_MOCS_ENTRIES,
> >       MOCS_ENTRY(I915_MOCS_CACHED,
> >                  LE_3_WB | LE_TC_2_LLC_ELLC | LE_LRUM(3),
> > -                L3_3_WB)
> > +                L3_3_WB),
> > +
> > +     /*
> > +      * mocs:63
> > +      * - used by the L3 for all its evictions.
> > +      *   Thus it is expected to allow LLC cacheability to enable coherent
> > +      *   flows to be maintained.
> > +      * - used to force L3 uncachable cycles.
> > +      *   Thus it is expected to make the surce L3 uncacheable.
>
> "surce"?
>
> > +      */
> > +     MOCS_ENTRY(63,
> > +                LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
> > +                L3_1_UC)
> >  };
> >
> >  /* NOTE: the LE_TGT_CACHE is not used on Broxton */
> > --
> > 2.20.1
>
> --
> Ville Syrjälä
> Intel