Re: [PATCH v12 11/13] drm/xe: Force flush system memory AuxCCS framebuffers before scan out

["Ville Syrjälä" <ville.syrjala@linux.intel.com>]

On Thu, Oct 02, 2025 at 06:04:28PM +0100, Tvrtko Ursulin wrote:
> 
> On 02/10/2025 17:23, Ville Syrjälä wrote:
> > On Thu, Oct 02, 2025 at 03:01:08PM +0100, Tvrtko Ursulin wrote:
> >>
> >> Hi,
> >>
> >> On 26/09/2025 20:35, Ville Syrjälä wrote:
> >>> On Fri, Sep 26, 2025 at 10:41:56AM +0300, Ville Syrjälä wrote:
> >>>> I reverse engineered this a bit and there's definitely a
> >>>> MOCS issue at play.
> >>>>
> >>>> First I noticed that if filled the entire MOCS table with
> >>>> UC the problem went away. I then filled the entire table
> >>>> with WB and essentially bisected what I need to make UC
> >>>> to fix it. And I had to repeat that same process starting
> >>>> from the other end of table.
> >>>>
> >>>> Looks like there is some undocumented magic in the hardware.
> >>>>
> >>>> MOCS 61 really is special:
> >>>> - MOCS 61 UC, others WB, select MOCS 61 -> no corruption
> >>>>
> >>>> MOCS 0 and 63 are special in other ways:
> >>>> - MOCS X UC, others WB, select MOCS X -> corruption
> >>>> - MOCS X+0 UC, others WB, select MOCS X -> corruption
> >>>> - MOCS X+63 UC, others WB, select MOCS X -> corruption
> >>>> - MOCS X+0+63 UC, others WB, select MOCS X -> no corruption
> >>>>     where X != 61
> >>>
> >>> OK, the MOCS 63 issue was caused by me having L3=WB still in
> >>> MOCS X. If I change MOCS X to L3=UC, MOCS 63 no longer makes
> >>> a difference. I suppose that means MOCS 63 is still used for
> >>> L3 evictions, even though bspec no longer mentions that fact
> >>> explicitly.
> >>>
> >>> So MOCS 0 is the thing that really matters for CCS. And for
> >>> MOCS 0 only the LLC WB vs. UC selection matters. L3 WB vs. UC
> >>> doesn't seem to make any difference.
> >>>
> >>> It's interesting that MOCS 60 is documented as a "CCS special case",
> >>> but in reality it's MOCS 0 that matters for CCS. I wonder if some
> >>> wires got crossed in the hw design and the wrong MOCS entry ended
> >>> up being used for CCS and no one noticed...
> >>
> >> Oh wow, that is an amazing discovery!
> >>
> >> I verified it on my end too. Setting MOCS 0 to uncached and cache dirt
> >> is gone. No need to the explicit cache flush patch on first pin.
> >>
> >> Luckily ADL is unsupported so we could change it to UC. I will send a
> >> series for CI to see what it will say.
> 
> So the MOCS 0 UC experiment did not seem to be 100% glitch free. It 
> *looks* it helps, maybe even a lot, but not fully - three tests still 
> failed due CRC mismatches.
> 
> > I think the real fix is to change igt to use MOCS 61 for tgl/adl.
> > That is what Mesa uses as well.
> I somehow glossed over the fact you initially wrote 61 worked fine for 
> you and focused only on your X+0+63 combinations. :(
> 
> 61 works fine for me locally too. Very curious hw behaviour.
> 
> It would be nice to do a CI run with IGT changed to 61 but AFAIK the xe 
> patchwork/CI does not support the Test-with tag.
> 
> > Looks like Mesa uses a different MOCS for DG1 and DG2. Those
> > do seem to like up with what's in bspec, so probably someone
> > needs to just copy the whole MOCS thing from Mesa into igt.
> 
> I can have a look.
> 
> > Looks like Mesa doesn't even use a UC MOCS for anything except
> > on MTL, so possibly we can just change the TGL MOCS 0 to be the
> > same WB as on ADL, and maybe that gives some performance benefit
> > in some cases.
> 
> On xe, i915 or both?

Both.

Does xe not program the table already according to bspec? I doubt
we should really care about the "ancient Mesa + xe + TGL" case,
so the special TGL MOCS table shouldn't be needed on xe IMO.

> 
> >>>> I didn't actually test all values of X there, but I did spot
> >>>> check a handful of them.
> >>>>
> >>>> Also, ADL is affected, but TGL doesn't seem to be. Though I
> >>>> still need to check the situation on TGL a bit more thoroughly.
> >>>
> >>> TGL actually works exactly the same as ADL. The only reason why
> >>> TGL worked correctly out of the box was that we use a different
> >>> MOCS table for TGL/RKL (IIRC because we started out with the
> >>> wrong table and early Mesa versions depended on that), and in
> >>> that table MOCS 0 is just 0x0, whereas on ADL MOCS 0 is WB.
> >>
> >> Kind of sounds familiar but the only commit I found was 3f027d61663f
> >> ("drm/i915/gt: Add separate MOCS table for Gen12 devices other than
> >> TGL/RKL") but it is about MOCS 1. What am I missing? Are the hw defaults
> >> maybe different and not the code?
> > 
> > The defaults are somehow populated differently dependign on
> > unused_entries_index which is also being set in a very confusing
> > way (first set it to 1(PTE) on everything and the overwritten
> > with some other value for some of the platforms). The code could
> > certainly use a good cleanup pass.
> > 
> > Anyways, the default index ends up being different on TGL and ADL
> > and thus MOCS 0 ends up different as well.
> 
> Yep. I missed it and forgot about cfbe5291a189 ("drm/i915/gt: Initialize 
> unused MOCS entries with device specific values").
> 
> > Since MOCS 0 seems to be special, we should probablya populate
> > it explicitly. And I suppose we should first figure out if
> > other platforms are also affected.
> 
> Yeah. If we could only get the full understanding on the details of 
> "specialness".

I filed a bspec issue for it now. I guess we'll see if anyone
cares anymore...

And I do still want to reverse engineer this on other platforms
as well.

-- 
Ville Syrjälä
Intel