Write GFX_FLSH_CNT after updating GGTT entries

Write GFX_FLSH_CNT after updating GGTT entries

[Zhi Wang]

Hi Gurus:
     I'm curious about the register GFX_FLSH_CNT(0x101008) in 
i915_gem_gtt.c. Does these register exist in recently generations? After 
digging into b-spec, it looks only BXT and CHV has this register. Does 
the desktop platform also have this register which needs to be written 
after updating GGTT MMIOs?

BTW: Looks windows driver haven't used this MMIO... So whose behavior is 
the right behavior?

Thanks,
Zhi.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Ville Syrjälä]

On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
> Hi Gurus:
>      I'm curious about the register GFX_FLSH_CNT(0x101008) in 
> i915_gem_gtt.c. Does these register exist in recently generations? After 
> digging into b-spec, it looks only BXT and CHV has this register. Does 
> the desktop platform also have this register which needs to be written 
> after updating GGTT MMIOs?
> 
> BTW: Looks windows driver haven't used this MMIO... So whose behavior is 
> the right behavior?

As I understand it that register flushes the CPU GTT TLBs, and we need
to do it because of the WC mapping we have for the GTT PTEs. If we used
UC mapping we wouldn't need it since there's supposedly an automagic
TLB flush that happens on PTE writes.

BSpec is bad at finding some registers via bxml. Using dtsearch and
looking for both 0x<offset> and <offset>h is the method I use to track
such things down.

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Zhi Wang]

Hi Ville:

Thanks for the answer! :) Learned a lot.

I think the following scenario should be typical for a general PCI 
devices(perhaps a dedicated video card). How do other PCI devices handle 
this kinds of WC MMIO writes without GFX_FLSH_CNT? Only support UC mapping?

Thanks,
Zhi.

On 11/19/15 18:35, Ville Syrjälä wrote:
> On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
>> Hi Gurus:
>>       I'm curious about the register GFX_FLSH_CNT(0x101008) in
>> i915_gem_gtt.c. Does these register exist in recently generations? After
>> digging into b-spec, it looks only BXT and CHV has this register. Does
>> the desktop platform also have this register which needs to be written
>> after updating GGTT MMIOs?
>>
>> BTW: Looks windows driver haven't used this MMIO... So whose behavior is
>> the right behavior?
>
> As I understand it that register flushes the CPU GTT TLBs, and we need
> to do it because of the WC mapping we have for the GTT PTEs. If we used
> UC mapping we wouldn't need it since there's supposedly an automagic
> TLB flush that happens on PTE writes.
>
> BSpec is bad at finding some registers via bxml. Using dtsearch and
> looking for both 0x<offset> and <offset>h is the method I use to track
> such things down.
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Ville Syrjälä]

On Thu, Nov 19, 2015 at 09:04:11PM +0800, Zhi Wang wrote:
> Hi Ville:
> 
> Thanks for the answer! :) Learned a lot.
> 
> I think the following scenario should be typical for a general PCI 
> devices(perhaps a dedicated video card). How do other PCI devices handle 
> this kinds of WC MMIO writes without GFX_FLSH_CNT? Only support UC mapping?

The standard rule is that you can't use WC, except for prefetchable BARs.

> 
> Thanks,
> Zhi.
> 
> On 11/19/15 18:35, Ville Syrjälä wrote:
> > On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
> >> Hi Gurus:
> >>       I'm curious about the register GFX_FLSH_CNT(0x101008) in
> >> i915_gem_gtt.c. Does these register exist in recently generations? After
> >> digging into b-spec, it looks only BXT and CHV has this register. Does
> >> the desktop platform also have this register which needs to be written
> >> after updating GGTT MMIOs?
> >>
> >> BTW: Looks windows driver haven't used this MMIO... So whose behavior is
> >> the right behavior?
> >
> > As I understand it that register flushes the CPU GTT TLBs, and we need
> > to do it because of the WC mapping we have for the GTT PTEs. If we used
> > UC mapping we wouldn't need it since there's supposedly an automagic
> > TLB flush that happens on PTE writes.
> >
> > BSpec is bad at finding some registers via bxml. Using dtsearch and
> > looking for both 0x<offset> and <offset>h is the method I use to track
> > such things down.
> >

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Zhi Wang]

Thanks!

OK. Then I check my box:

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd 
Gen Core processor Graphics Controller (rev 09) (prog-if 00 [VGA 
controller])
	Subsystem: Hewlett-Packard Company Device 3396
	Flags: bus master, fast devsel, latency 0, IRQ 27
	Memory at f7400000 (64-bit, non-prefetchable) [size=4M]

So this should be our own magic to play WC on a non-prefechable BAR 
especially for GGTT MMIOs?

Thanks,
Zhi.

On 11/19/15 21:26, Ville Syrjälä wrote:
> On Thu, Nov 19, 2015 at 09:04:11PM +0800, Zhi Wang wrote:
>> Hi Ville:
>>
>> Thanks for the answer! :) Learned a lot.
>>
>> I think the following scenario should be typical for a general PCI
>> devices(perhaps a dedicated video card). How do other PCI devices handle
>> this kinds of WC MMIO writes without GFX_FLSH_CNT? Only support UC mapping?
>
> The standard rule is that you can't use WC, except for prefetchable BARs.
>
>>
>> Thanks,
>> Zhi.
>>
>> On 11/19/15 18:35, Ville Syrjälä wrote:
>>> On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
>>>> Hi Gurus:
>>>>        I'm curious about the register GFX_FLSH_CNT(0x101008) in
>>>> i915_gem_gtt.c. Does these register exist in recently generations? After
>>>> digging into b-spec, it looks only BXT and CHV has this register. Does
>>>> the desktop platform also have this register which needs to be written
>>>> after updating GGTT MMIOs?
>>>>
>>>> BTW: Looks windows driver haven't used this MMIO... So whose behavior is
>>>> the right behavior?
>>>
>>> As I understand it that register flushes the CPU GTT TLBs, and we need
>>> to do it because of the WC mapping we have for the GTT PTEs. If we used
>>> UC mapping we wouldn't need it since there's supposedly an automagic
>>> TLB flush that happens on PTE writes.
>>>
>>> BSpec is bad at finding some registers via bxml. Using dtsearch and
>>> looking for both 0x<offset> and <offset>h is the method I use to track
>>> such things down.
>>>
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Tian, Kevin]

> From: Ville Syrjälä [mailto:ville.syrjala@linux.intel.com]
> Sent: Thursday, November 19, 2015 6:35 PM
> 
> On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
> > Hi Gurus:
> >      I'm curious about the register GFX_FLSH_CNT(0x101008) in
> > i915_gem_gtt.c. Does these register exist in recently generations? After
> > digging into b-spec, it looks only BXT and CHV has this register. Does
> > the desktop platform also have this register which needs to be written
> > after updating GGTT MMIOs?
> >
> > BTW: Looks windows driver haven't used this MMIO... So whose behavior is
> > the right behavior?
> 
> As I understand it that register flushes the CPU GTT TLBs, and we need
> to do it because of the WC mapping we have for the GTT PTEs. If we used
> UC mapping we wouldn't need it since there's supposedly an automagic
> TLB flush that happens on PTE writes.
> 
> BSpec is bad at finding some registers via bxml. Using dtsearch and
> looking for both 0x<offset> and <offset>h is the method I use to track
> such things down.
> 

Curious how much gain is observed by using WC vs. using UC on GTT
entries?

Thanks
Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Chris Wilson]

On Fri, Nov 20, 2015 at 09:23:12AM +0000, Tian, Kevin wrote:
> > From: Ville Syrjälä [mailto:ville.syrjala@linux.intel.com]
> > Sent: Thursday, November 19, 2015 6:35 PM
> > 
> > On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
> > > Hi Gurus:
> > >      I'm curious about the register GFX_FLSH_CNT(0x101008) in
> > > i915_gem_gtt.c. Does these register exist in recently generations? After
> > > digging into b-spec, it looks only BXT and CHV has this register. Does
> > > the desktop platform also have this register which needs to be written
> > > after updating GGTT MMIOs?
> > >
> > > BTW: Looks windows driver haven't used this MMIO... So whose behavior is
> > > the right behavior?
> > 
> > As I understand it that register flushes the CPU GTT TLBs, and we need
> > to do it because of the WC mapping we have for the GTT PTEs. If we used
> > UC mapping we wouldn't need it since there's supposedly an automagic
> > TLB flush that happens on PTE writes.
> > 
> > BSpec is bad at finding some registers via bxml. Using dtsearch and
> > looking for both 0x<offset> and <offset>h is the method I use to track
> > such things down.
> > 
> 
> Curious how much gain is observed by using WC vs. using UC on GTT
> entries?

Think back yonder when everything goes through the GGTT, and where
writing the PTEs was slower than allocating a bunch of pages and
applications would insist on submitting new objects every batch.

It was very easy to have workloads where UC GGTT updates were the
ratelimiting step. A WC update is ~8x faster, and sufficient to move the
bottleneck elsewhere.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: Write GFX_FLSH_CNT after updating GGTT entries

[Tian, Kevin]

> From: Chris Wilson [mailto:chris@chris-wilson.co.uk]
> Sent: Friday, November 20, 2015 5:40 PM
> 
> On Fri, Nov 20, 2015 at 09:23:12AM +0000, Tian, Kevin wrote:
> > > From: Ville Syrjälä [mailto:ville.syrjala@linux.intel.com]
> > > Sent: Thursday, November 19, 2015 6:35 PM
> > >
> > > On Thu, Nov 19, 2015 at 06:20:23PM +0800, Zhi Wang wrote:
> > > > Hi Gurus:
> > > >      I'm curious about the register GFX_FLSH_CNT(0x101008) in
> > > > i915_gem_gtt.c. Does these register exist in recently generations? After
> > > > digging into b-spec, it looks only BXT and CHV has this register. Does
> > > > the desktop platform also have this register which needs to be written
> > > > after updating GGTT MMIOs?
> > > >
> > > > BTW: Looks windows driver haven't used this MMIO... So whose behavior is
> > > > the right behavior?
> > >
> > > As I understand it that register flushes the CPU GTT TLBs, and we need
> > > to do it because of the WC mapping we have for the GTT PTEs. If we used
> > > UC mapping we wouldn't need it since there's supposedly an automagic
> > > TLB flush that happens on PTE writes.
> > >
> > > BSpec is bad at finding some registers via bxml. Using dtsearch and
> > > looking for both 0x<offset> and <offset>h is the method I use to track
> > > such things down.
> > >
> >
> > Curious how much gain is observed by using WC vs. using UC on GTT
> > entries?
> 
> Think back yonder when everything goes through the GGTT, and where
> writing the PTEs was slower than allocating a bunch of pages and
> applications would insist on submitting new objects every batch.
> 
> It was very easy to have workloads where UC GGTT updates were the
> ratelimiting step. A WC update is ~8x faster, and sufficient to move the
> bottleneck elsewhere.
> -Chris

8x is clearly worthy of it. Today w/ PPGTT this optimization is
somehow less important then. :-)

Thanks
Kevin
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx