About dealing with CSB.context element switch in execlist mode.

About dealing with CSB.context element switch in execlist mode.

[Wang, Zhi A]

[-- Attachment #1.1: Type: text/plain, Size: 799 bytes --]

Hi Gurus:
I'm wondering what's the right approach to deal with the context switch reason: element_switch? According to b-spec, one ELSP submission may include two elements, when one element is finished, HW will move to process next element, the previous context will be scheduled out with a "element_switch" context switch reason.

I saw that i915 would try to start a new ELSP write which may contain two new elements when it found a "element_switch" CSB in the context switch handler. I'm a bit confused here, as HW may be still running a context at this time, I'm not sure if two new elements can be submitted at this time. So I think maybe my understanding about this context switch reason might be wrong.

Anyone can educate me how to deal with the "element_switch" CSB?

Thanks,
Zhi.

[-- Attachment #1.2: Type: text/html, Size: 22326 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Michel Thierry]

On 11/24/2015 1:33 PM, Wang, Zhi A wrote:
> Hi Gurus:
>
> I’m wondering what’s the right approach to deal with the context switch
> reason: element_switch? According to b-spec, one ELSP submission may
> include two elements, when one element is finished, HW will move to
> process next element, the previous context will be scheduled out with a
> “element_switch” context switch reason.

Correct, in fact you get 2 flags, element_switch & context_complete.

>
> I saw that i915 would try to start a new ELSP write which may contain
> two new elements when it found a “element_switch” CSB in the context
> switch handler. I’m a bit confused here, as HW may be still running a
> context at this time, I’m not sure if two new elements can be submitted
> at this time. So I think maybe my understanding about this context
> switch reason might be wrong.

The driver is trying to speed up the execution when there are +3 
contexts queued. The first _new_ element you mention is in fact the 
running context.

>
> Anyone can educate me how to deal with the “element_switch” CSB?
>

Let's say you have contexts A, B, C and D already in queue, so A & B are 
sent to the ELSP.

After element_switch (A completed) B starts automatically, at this point 
the driver will send a new execlist with B and C, effectively 
lite-restoring B (note it cannot be a different context since that will 
cause a preemption).

And when B completes, the driver sends C & D. Lite-restoring the 2nd ctx 
of each execlist allows us to send these 4 contexts with only 3 ELSP writes.

> Thanks,
>
> Zhi.
>
-Michel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Wang, Zhi A]

Wow, that's nice! Thanks Michel for the clear explanation! That's just the answer I'm looking for! :)

Thanks,
Zhi.

> -----Original Message-----
> From: Thierry, Michel
> Sent: Wednesday, November 25, 2015 8:48 PM
> To: Wang, Zhi A; intel-gfx@lists.freedesktop.org
> Cc: Han, Xu; Li, Weinan Z; He, Min
> Subject: Re: [Intel-gfx] About dealing with CSB.context element switch in
> execlist mode.
> 
> On 11/24/2015 1:33 PM, Wang, Zhi A wrote:
> > Hi Gurus:
> >
> > I'm wondering what's the right approach to deal with the context
> > switch
> > reason: element_switch? According to b-spec, one ELSP submission may
> > include two elements, when one element is finished, HW will move to
> > process next element, the previous context will be scheduled out with
> > a "element_switch" context switch reason.
> 
> Correct, in fact you get 2 flags, element_switch & context_complete.
> 
> >
> > I saw that i915 would try to start a new ELSP write which may contain
> > two new elements when it found a "element_switch" CSB in the context
> > switch handler. I'm a bit confused here, as HW may be still running a
> > context at this time, I'm not sure if two new elements can be
> > submitted at this time. So I think maybe my understanding about this
> > context switch reason might be wrong.
> 
> The driver is trying to speed up the execution when there are +3 contexts
> queued. The first _new_ element you mention is in fact the running context.
> 
> >
> > Anyone can educate me how to deal with the "element_switch" CSB?
> >
> 
> Let's say you have contexts A, B, C and D already in queue, so A & B are
> sent to the ELSP.
> 
> After element_switch (A completed) B starts automatically, at this point
> the driver will send a new execlist with B and C, effectively
> lite-restoring B (note it cannot be a different context since that will
> cause a preemption).
> 
> And when B completes, the driver sends C & D. Lite-restoring the 2nd ctx
> of each execlist allows us to send these 4 contexts with only 3 ELSP writes.
> 
> > Thanks,
> >
> > Zhi.
> >
> -Michel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Wang, Zhi A]

Another question about EXECLIST is: Can a preemption happen between element switch?

I know this is beyond the scope of i915 a little. I'm just curious if it's possible.

Let's say we have context A B C

At first, we submit context A B in one ELSP write.
Then, we submit context C in another ELSP write at some time.

If context A or B is running and gets preempted, then there will be CSB.preempted in CSB buffer. This is the normal behavior.

I'm wondering that if there is any possibility that a preemption can happen between the two elements.

Then the CSB should look like:

[CSB 0 idle-to-active] 
[CSB 1 CTX A element_switch/context_complete] 
[CSB 2 CTX C active-to-idle/context_complete]

Is it possible?

Thanks,
Zhi.

> -----Original Message-----
> From: Thierry, Michel
> Sent: Wednesday, November 25, 2015 8:48 PM
> To: Wang, Zhi A; intel-gfx@lists.freedesktop.org
> Cc: Han, Xu; Li, Weinan Z; He, Min
> Subject: Re: [Intel-gfx] About dealing with CSB.context element switch in
> execlist mode.
> 
> On 11/24/2015 1:33 PM, Wang, Zhi A wrote:
> > Hi Gurus:
> >
> > I'm wondering what's the right approach to deal with the context
> > switch
> > reason: element_switch? According to b-spec, one ELSP submission may
> > include two elements, when one element is finished, HW will move to
> > process next element, the previous context will be scheduled out with
> > a "element_switch" context switch reason.
> 
> Correct, in fact you get 2 flags, element_switch & context_complete.
> 
> >
> > I saw that i915 would try to start a new ELSP write which may contain
> > two new elements when it found a "element_switch" CSB in the context
> > switch handler. I'm a bit confused here, as HW may be still running a
> > context at this time, I'm not sure if two new elements can be
> > submitted at this time. So I think maybe my understanding about this
> > context switch reason might be wrong.
> 
> The driver is trying to speed up the execution when there are +3 contexts
> queued. The first _new_ element you mention is in fact the running context.
> 
> >
> > Anyone can educate me how to deal with the "element_switch" CSB?
> >
> 
> Let's say you have contexts A, B, C and D already in queue, so A & B are
> sent to the ELSP.
> 
> After element_switch (A completed) B starts automatically, at this point
> the driver will send a new execlist with B and C, effectively
> lite-restoring B (note it cannot be a different context since that will
> cause a preemption).
> 
> And when B completes, the driver sends C & D. Lite-restoring the 2nd ctx
> of each execlist allows us to send these 4 contexts with only 3 ELSP writes.
> 
> > Thanks,
> >
> > Zhi.
> >
> -Michel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Michel Thierry]

On 11/25/2015 1:00 PM, Wang, Zhi A wrote:
> Another question about EXECLIST is: Can a preemption happen between element switch?
>
> I know this is beyond the scope of i915 a little. I'm just curious if it's possible.
>
> Let's say we have context A B C
>
> At first, we submit context A B in one ELSP write.
> Then, we submit context C in another ELSP write at some time.
>
> If context A or B is running and gets preempted, then there will be CSB.preempted in CSB buffer. This is the normal behavior.
>
> I'm wondering that if there is any possibility that a preemption can happen between the two elements.
>
> Then the CSB should look like:
>
> [CSB 0 idle-to-active]
> [CSB 1 CTX A element_switch/context_complete]
> [CSB 2 CTX C active-to-idle/context_complete]
>
> Is it possible?

I would expect to always have a preempted event in the CSB, even in the 
remote case that A already completed and B hasn't started; there was an 
active execlist and it has been replaced by a new one:

  [CSB 0 idle-to-active]
  [CSB 1 CTX A element_switch/context_complete]
  [CSB 2 preempted ]
  [CSB 3 CTX C active-to-idle/context_complete]
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Wang, Zhi A]

OK. I see. Thanks Michel! :) Have a nice day. :)

Thanks,
Zhi.

> -----Original Message-----
> From: Thierry, Michel
> Sent: Wednesday, November 25, 2015 9:15 PM
> To: Wang, Zhi A; intel-gfx@lists.freedesktop.org
> Cc: Han, Xu; Li, Weinan Z; He, Min; Lv, Zhiyuan; Tian, Kevin
> Subject: Re: [Intel-gfx] About dealing with CSB.context element switch in
> execlist mode.
> 
> On 11/25/2015 1:00 PM, Wang, Zhi A wrote:
> > Another question about EXECLIST is: Can a preemption happen between
> element switch?
> >
> > I know this is beyond the scope of i915 a little. I'm just curious if it's possible.
> >
> > Let's say we have context A B C
> >
> > At first, we submit context A B in one ELSP write.
> > Then, we submit context C in another ELSP write at some time.
> >
> > If context A or B is running and gets preempted, then there will be
> CSB.preempted in CSB buffer. This is the normal behavior.
> >
> > I'm wondering that if there is any possibility that a preemption can happen
> between the two elements.
> >
> > Then the CSB should look like:
> >
> > [CSB 0 idle-to-active]
> > [CSB 1 CTX A element_switch/context_complete] [CSB 2 CTX C
> > active-to-idle/context_complete]
> >
> > Is it possible?
> 
> I would expect to always have a preempted event in the CSB, even in the
> remote case that A already completed and B hasn't started; there was an
> active execlist and it has been replaced by a new one:
> 
>   [CSB 0 idle-to-active]
>   [CSB 1 CTX A element_switch/context_complete]
>   [CSB 2 preempted ]
>   [CSB 3 CTX C active-to-idle/context_complete]
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: About dealing with CSB.context element switch in execlist mode.

[Dave Gordon]

On 25/11/15 13:17, Wang, Zhi A wrote:
> OK. I see. Thanks Michel! :) Have a nice day. :)
>
> Thanks,
> Zhi.
>
>> -----Original Message-----
>> From: Thierry, Michel
>> Sent: Wednesday, November 25, 2015 9:15 PM
>> To: Wang, Zhi A; intel-gfx@lists.freedesktop.org
>> Cc: Han, Xu; Li, Weinan Z; He, Min; Lv, Zhiyuan; Tian, Kevin
>> Subject: Re: [Intel-gfx] About dealing with CSB.context element switch in
>> execlist mode.
>>
>> On 11/25/2015 1:00 PM, Wang, Zhi A wrote:
>>> Another question about EXECLIST is: Can a preemption happen between
>> element switch?
>>>
>>> I know this is beyond the scope of i915 a little. I'm just curious if it's possible.
>>>
>>> Let's say we have context A B C
>>>
>>> At first, we submit context A B in one ELSP write.
>>> Then, we submit context C in another ELSP write at some time.
>>>
>>> If context A or B is running and gets preempted, then there will be
>> CSB.preempted in CSB buffer. This is the normal behavior.
>>>
>>> I'm wondering that if there is any possibility that a preemption can happen
>> between the two elements.
>>>
>>> Then the CSB should look like:
>>>
>>> [CSB 0 idle-to-active]
>>> [CSB 1 CTX A element_switch/context_complete] [CSB 2 CTX C
>>> active-to-idle/context_complete]
>>>
>>> Is it possible?
>>
>> I would expect to always have a preempted event in the CSB, even in the
>> remote case that A already completed and B hasn't started; there was an
>> active execlist and it has been replaced by a new one:
>>
>>    [CSB 0 idle-to-active]
>>    [CSB 1 CTX A element_switch/context_complete]
>>    [CSB 2 preempted ]
>>    [CSB 3 CTX C active-to-idle/context_complete]
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx

A few examples of CSB sequences you may see:

CSB[1]: 0x000.00000 / 0x00000001  I->A
CSB[2]: 0x201.018df / 0x00000018                 A->I DONE
CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x201.018df / 0x00000018                 A->I DONE
CSB[5]: 0x000.00000 / 0x00000001  I->A
CSB[0]: 0x201.018df / 0x00000018                 A->I DONE

This is as simple as it gets. A single context is repeatedly submitted, 
and each time it completes before being resubmitted.

CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x601.018e6 / 0x00008002       PRMT                LITE
CSB[5]: 0x601.018e6 / 0x00000018                 A->I DONE
CSB[0]: 0x000.00000 / 0x00000001  I->A
CSB[1]: 0x601.018e6 / 0x00008002       PRMT                LITE
CSB[2]: 0x601.018e6 / 0x00000018                 A->I DONE

Here again a single context is in use, but this time it was submitted, 
then updated and resubmitted while still executing (lite restore), then 
completed. This whole sequence repeats twice.

CSB[4]: 0x001.02935 / 0x00000018                 A->I DONE
CSB[5]: 0x000.00000 / 0x00000001  I->A
CSB[0]: 0x001.02935 / 0x00000018                 A->I DONE
CSB[1]: 0x000.00000 / 0x00000001  I->A
CSB[2]: 0x001.02935 / 0x00000002       PRMT
CSB[3]: 0x001.02950 / 0x00000018                 A->I DONE

This is a genuine preemption. We see context A (02935) being submitted 
and completed twice, but during the third execution is gets preempted. 
The last line shows the completion of context B (02950) that preempted 
context A.

CSB[3]: 0x000.00000 / 0x00000001  I->A
CSB[4]: 0x001.018ed / 0x00000002       PRMT
CSB[5]: 0x001.01908 / 0x00000018                 A->I DONE
CSB[0]: 0x000.00000 / 0x00000001  I->A
CSB[1]: 0x001.01908 / 0x00000018                 A->I DONE
CSB[2]: 0x000.00000 / 0x00000001  I->A

Here, context A (018ed) was started, then preempted by a higher-priority 
context B (01908), which got to run to completion twice before another 
context (presumably A) was (re)started. We infer that A (or whatever 
other context was most recently submitted) was still running at the 
instant when this trace was captured.

The PRMT (without LITE) event always indicates the identity of the 
context that was preempted. In the case of preemption after a dual 
submission, it may be either the first or the second workload that gets 
preempted, but it must be one or the other (there's no "idle" window 
between finishing the first and starting the second in which preemption 
could occur). In addition, preemption is actually disabled while context 
restore is in progress, so if the ELSP is written after the GPU has 
committed to loading a context, the preemption won't happen until the 
load is completed. [This is subject to change, but this how it's set up 
on Skylake today].

HTH,
.Dave.

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx