Request for feedback - Sprite flip notification support

Request for feedback - Sprite flip notification support

[Vijay Purushothaman]

Hello,

In our current driver implementation we support flip notifications only 
for primary plane. So, in a full screen video playback scenario where 
only one sprite plane is active, the user space is forced to rely on 
primary plane flip notification even though there is no real need for 
this plane to be active. Ideally we should be able to support flip 
notifications for any given plane. Switching off the primary plane (when 
not used) will help in better memory self refresh & decent power savings..

We do have a hack in android product trees which supports flip 
notifications for one sprite plane. unfortunately this hack in its 
current form cannot be considered for up streaming...

My current thinking is to have an array of unpin_work items to match the 
number of planes. Is anyone working on this or thought about this 
scenario in detail? Any pointers / restrictions that needs to considered 
for a generic implementation of this feature?

*Thanks,
Vijay
*

Re: Request for feedback - Sprite flip notification support

[Ville Syrjälä]

On Wed, Feb 05, 2014 at 08:35:11PM +0530, Vijay Purushothaman wrote:
> Hello,
> 
> In our current driver implementation we support flip notifications only 
> for primary plane. So, in a full screen video playback scenario where 
> only one sprite plane is active, the user space is forced to rely on 
> primary plane flip notification even though there is no real need for 
> this plane to be active. Ideally we should be able to support flip 
> notifications for any given plane. Switching off the primary plane (when 
> not used) will help in better memory self refresh & decent power savings..
> 
> We do have a hack in android product trees which supports flip 
> notifications for one sprite plane. unfortunately this hack in its 
> current form cannot be considered for up streaming...
> 
> My current thinking is to have an array of unpin_work items to match the 
> number of planes. Is anyone working on this or thought about this 
> scenario in detail? Any pointers / restrictions that needs to considered 
> for a generic implementation of this feature?

The plan is to implement the nuclear page flip which will take care of
all planes in the same way.

-- 
Ville Syrjälä
Intel OTC

Re: Request for feedback - Sprite flip notification support

[Vijay Purushothaman]

On 2/5/2014 8:43 PM, Ville Syrjälä wrote:
> On Wed, Feb 05, 2014 at 08:35:11PM +0530, Vijay Purushothaman wrote:
>> Hello,
>>
>> In our current driver implementation we support flip notifications only
>> for primary plane. So, in a full screen video playback scenario where
>> only one sprite plane is active, the user space is forced to rely on
>> primary plane flip notification even though there is no real need for
>> this plane to be active. Ideally we should be able to support flip
>> notifications for any given plane. Switching off the primary plane (when
>> not used) will help in better memory self refresh & decent power savings..
>>
>> We do have a hack in android product trees which supports flip
>> notifications for one sprite plane. unfortunately this hack in its
>> current form cannot be considered for up streaming...
>>
>> My current thinking is to have an array of unpin_work items to match the
>> number of planes. Is anyone working on this or thought about this
>> scenario in detail? Any pointers / restrictions that needs to considered
>> for a generic implementation of this feature?
>
> The plan is to implement the nuclear page flip which will take care of
> all planes in the same way.
>
Thanks Ville. If the nuclear page flip is part of your bigger atomic 
mode set framework, is there a way you can split this into smaller sets 
for merge? Multiple product trees will benefit from the nuclear page flip.

Is there anything that i can help with? Like testing your patches with 
android user space?

Thanks,
Vijay

Re: Request for feedback - Sprite flip notification support

[Ville Syrjälä]

On Wed, Feb 05, 2014 at 09:25:36PM +0530, Vijay Purushothaman wrote:
> On 2/5/2014 8:43 PM, Ville Syrjälä wrote:
> > On Wed, Feb 05, 2014 at 08:35:11PM +0530, Vijay Purushothaman wrote:
> >> Hello,
> >>
> >> In our current driver implementation we support flip notifications only
> >> for primary plane. So, in a full screen video playback scenario where
> >> only one sprite plane is active, the user space is forced to rely on
> >> primary plane flip notification even though there is no real need for
> >> this plane to be active. Ideally we should be able to support flip
> >> notifications for any given plane. Switching off the primary plane (when
> >> not used) will help in better memory self refresh & decent power savings..
> >>
> >> We do have a hack in android product trees which supports flip
> >> notifications for one sprite plane. unfortunately this hack in its
> >> current form cannot be considered for up streaming...
> >>
> >> My current thinking is to have an array of unpin_work items to match the
> >> number of planes. Is anyone working on this or thought about this
> >> scenario in detail? Any pointers / restrictions that needs to considered
> >> for a generic implementation of this feature?
> >
> > The plan is to implement the nuclear page flip which will take care of
> > all planes in the same way.
> >
> Thanks Ville. If the nuclear page flip is part of your bigger atomic 
> mode set framework, is there a way you can split this into smaller sets 
> for merge? Multiple product trees will benefit from the nuclear page flip.

I've split things up already somewhat. Some has landed some has not.

Currently I have my minimal "atomic update of sprite+primary during
setplane" series I need to get in. It shouldn't need major work anymore,
just some minor tweaks. But I realized I need to limit this to just
pch platforms for now. Making it work reliably on gmch platforms
require some extra interrupt related work. The main thing here is that
it adds the mechanism to do the update atomically for the entire pipe.

After that I need to post the last bits of my watermark update saga.
This too will initially be limited to pch platforms only. Obviously
watermarks need to updated correctly to avoid underruns when planes
get shuffled around.

> Is there anything that i can help with? Like testing your patches with 
> android user space?

There's nothing to test at this point unless you want to test my old
branch from year ago or something.

What needs to be done:
- review the latest atomic framework patches from Rob Clark
- expose primary/cursor planes as drm_planes
  * this could in theory be skipped, but it'll lead to cruft in the API
    we need to maintain until the end of time. Also I think restructing
    stuff internally to this direction will be a good idea anyway to make
    the nuclear flip code neater. This more or less involves collecting
    the plane state to some plane_config type of structure, and being
    able to pre-compute that
- try to collect the necessary missing bits from my last
  atomic branch to implement the nuclear flip
  * the flip helper thing to update an arbitrary collection of planes
    atomically, maybe could be simplified a bit
  * mechanism to queue nuclear flips and make them wait for the
    GPU to finish writing to all the relevant buffers before issuing
    the actual flips/updates
- finally hook up the atomic ioctl to do the nuclear flip
  * pre-pin all buffers, pre-compute plane configs, pre-compute
    watermarks, check everything, wait for the GPU, and finally
    do the update

For the atomic modeset side some of that's the same really. There too we
also need to pre-compute the plane configs and pre-pin buffers. Most of
the rest we already pre-compute via the pipe config. One major thing
left out of the pipe config pre-compute currently is PLLs. We compute
that stuff way too late still. We also need to massage the modeset code
some more to make it capable of modesetting multiple pipes at once.

-- 
Ville Syrjälä
Intel OTC

Re: Request for feedback - Sprite flip notification support

[Vijay Purushothaman]

On 2/5/2014 10:18 PM, Ville Syrjälä wrote:
> On Wed, Feb 05, 2014 at 09:25:36PM +0530, Vijay Purushothaman wrote:
>> On 2/5/2014 8:43 PM, Ville Syrjälä wrote:
>>> On Wed, Feb 05, 2014 at 08:35:11PM +0530, Vijay Purushothaman wrote:
>>>> Hello,
>>>>
>>>> In our current driver implementation we support flip notifications only
>>>> for primary plane. So, in a full screen video playback scenario where
>>>> only one sprite plane is active, the user space is forced to rely on
>>>> primary plane flip notification even though there is no real need for
>>>> this plane to be active. Ideally we should be able to support flip
>>>> notifications for any given plane. Switching off the primary plane (when
>>>> not used) will help in better memory self refresh & decent power savings..
>>>>
>>>> We do have a hack in android product trees which supports flip
>>>> notifications for one sprite plane. unfortunately this hack in its
>>>> current form cannot be considered for up streaming...
>>>>
>>>> My current thinking is to have an array of unpin_work items to match the
>>>> number of planes. Is anyone working on this or thought about this
>>>> scenario in detail? Any pointers / restrictions that needs to considered
>>>> for a generic implementation of this feature?
>>>
>>> The plan is to implement the nuclear page flip which will take care of
>>> all planes in the same way.
>>>
>> Thanks Ville. If the nuclear page flip is part of your bigger atomic
>> mode set framework, is there a way you can split this into smaller sets
>> for merge? Multiple product trees will benefit from the nuclear page flip.
>
> I've split things up already somewhat. Some has landed some has not.
>
> Currently I have my minimal "atomic update of sprite+primary during
> setplane" series I need to get in. It shouldn't need major work anymore,
> just some minor tweaks. But I realized I need to limit this to just
> pch platforms for now. Making it work reliably on gmch platforms
> require some extra interrupt related work. The main thing here is that
> it adds the mechanism to do the update atomically for the entire pipe.
>
> After that I need to post the last bits of my watermark update saga.
> This too will initially be limited to pch platforms only. Obviously
> watermarks need to updated correctly to avoid underruns when planes
> get shuffled around.
>
>> Is there anything that i can help with? Like testing your patches with
>> android user space?
>
> There's nothing to test at this point unless you want to test my old
> branch from year ago or something.
>
> What needs to be done:
> - review the latest atomic framework patches from Rob Clark
> - expose primary/cursor planes as drm_planes
>    * this could in theory be skipped, but it'll lead to cruft in the API
>      we need to maintain until the end of time. Also I think restructing
>      stuff internally to this direction will be a good idea anyway to make
>      the nuclear flip code neater. This more or less involves collecting
>      the plane state to some plane_config type of structure, and being
>      able to pre-compute that
> - try to collect the necessary missing bits from my last
>    atomic branch to implement the nuclear flip
>    * the flip helper thing to update an arbitrary collection of planes
>      atomically, maybe could be simplified a bit
>    * mechanism to queue nuclear flips and make them wait for the
>      GPU to finish writing to all the relevant buffers before issuing
>      the actual flips/updates
> - finally hook up the atomic ioctl to do the nuclear flip
>    * pre-pin all buffers, pre-compute plane configs, pre-compute
>      watermarks, check everything, wait for the GPU, and finally
>      do the update
>
> For the atomic modeset side some of that's the same really. There too we
> also need to pre-compute the plane configs and pre-pin buffers. Most of
> the rest we already pre-compute via the pipe config. One major thing
> left out of the pipe config pre-compute currently is PLLs. We compute
> that stuff way too late still. We also need to massage the modeset code
> some more to make it capable of modesetting multiple pipes at once.
>

Thanks for the detailed answer. This should solve most of the display 
issues in the product trees.. considering the amount of work involved 
this looks more like a long term solution. Would it be okay if we submit 
a short term solution for sprite flip notifications? This would help us 
to force a standard approach across multiple android product kernels. We 
can revert this fix once the atomic mode set / nuclear page flip is ready.

Thanks,
Vijay

Re: Request for feedback - Sprite flip notification support

[Vijay Purushothaman]

On 2/6/2014 12:28 PM, Vijay Purushothaman wrote:
> On 2/5/2014 10:18 PM, Ville Syrjälä wrote:
>> On Wed, Feb 05, 2014 at 09:25:36PM +0530, Vijay Purushothaman wrote:
>>> On 2/5/2014 8:43 PM, Ville Syrjälä wrote:
>>>> On Wed, Feb 05, 2014 at 08:35:11PM +0530, Vijay Purushothaman wrote:
>>>>> Hello,
>>>>>
>>>>> In our current driver implementation we support flip notifications
>>>>> only
>>>>> for primary plane. So, in a full screen video playback scenario where
>>>>> only one sprite plane is active, the user space is forced to rely on
>>>>> primary plane flip notification even though there is no real need for
>>>>> this plane to be active. Ideally we should be able to support flip
>>>>> notifications for any given plane. Switching off the primary plane
>>>>> (when
>>>>> not used) will help in better memory self refresh & decent power
>>>>> savings..
>>>>>
>>>>> We do have a hack in android product trees which supports flip
>>>>> notifications for one sprite plane. unfortunately this hack in its
>>>>> current form cannot be considered for up streaming...
>>>>>
>>>>> My current thinking is to have an array of unpin_work items to
>>>>> match the
>>>>> number of planes. Is anyone working on this or thought about this
>>>>> scenario in detail? Any pointers / restrictions that needs to
>>>>> considered
>>>>> for a generic implementation of this feature?
>>>>
>>>> The plan is to implement the nuclear page flip which will take care of
>>>> all planes in the same way.
>>>>
>>> Thanks Ville. If the nuclear page flip is part of your bigger atomic
>>> mode set framework, is there a way you can split this into smaller sets
>>> for merge? Multiple product trees will benefit from the nuclear page
>>> flip.
>>
>> I've split things up already somewhat. Some has landed some has not.
>>
>> Currently I have my minimal "atomic update of sprite+primary during
>> setplane" series I need to get in. It shouldn't need major work anymore,
>> just some minor tweaks. But I realized I need to limit this to just
>> pch platforms for now. Making it work reliably on gmch platforms
>> require some extra interrupt related work. The main thing here is that
>> it adds the mechanism to do the update atomically for the entire pipe.
>>
>> After that I need to post the last bits of my watermark update saga.
>> This too will initially be limited to pch platforms only. Obviously
>> watermarks need to updated correctly to avoid underruns when planes
>> get shuffled around.
>>
>>> Is there anything that i can help with? Like testing your patches with
>>> android user space?
>>
>> There's nothing to test at this point unless you want to test my old
>> branch from year ago or something.
>>
>> What needs to be done:
>> - review the latest atomic framework patches from Rob Clark
>> - expose primary/cursor planes as drm_planes
>>    * this could in theory be skipped, but it'll lead to cruft in the API
>>      we need to maintain until the end of time. Also I think restructing
>>      stuff internally to this direction will be a good idea anyway to
>> make
>>      the nuclear flip code neater. This more or less involves collecting
>>      the plane state to some plane_config type of structure, and being
>>      able to pre-compute that
>> - try to collect the necessary missing bits from my last
>>    atomic branch to implement the nuclear flip
>>    * the flip helper thing to update an arbitrary collection of planes
>>      atomically, maybe could be simplified a bit
>>    * mechanism to queue nuclear flips and make them wait for the
>>      GPU to finish writing to all the relevant buffers before issuing
>>      the actual flips/updates
>> - finally hook up the atomic ioctl to do the nuclear flip
>>    * pre-pin all buffers, pre-compute plane configs, pre-compute
>>      watermarks, check everything, wait for the GPU, and finally
>>      do the update
>>
>> For the atomic modeset side some of that's the same really. There too we
>> also need to pre-compute the plane configs and pre-pin buffers. Most of
>> the rest we already pre-compute via the pipe config. One major thing
>> left out of the pipe config pre-compute currently is PLLs. We compute
>> that stuff way too late still. We also need to massage the modeset code
>> some more to make it capable of modesetting multiple pipes at once.
>>
>
> Thanks for the detailed answer. This should solve most of the display
> issues in the product trees.. considering the amount of work involved
> this looks more like a long term solution. Would it be okay if we submit
> a short term solution for sprite flip notifications? This would help us
> to force a standard approach across multiple android product kernels. We
> can revert this fix once the atomic mode set / nuclear page flip is ready.
>
Ville / Daniel,

Ping.. Could you please suggest whether this is okay? If you think this 
is not worth or if nuclear page flip is on the horizon i will focus on 
display self refresh patches..

Thanks,
Vijay

Re: Request for feedback - Sprite flip notification support

[Daniel Vetter]

On Fri, Feb 7, 2014 at 7:36 AM, Vijay Purushothaman
<vijay.a.purushothaman@intel.com> wrote:
> Ping.. Could you please suggest whether this is okay? If you think this is
> not worth or if nuclear page flip is on the horizon i will focus on display
> self refresh patches..

Since we need to keep userspace ABIs working essentially forever I
prefer if we don't have to add temporary fixes. If it fits into
Ville's overall plans though we could merge all the backend patches
though, as preparation for the new atomic interfaces.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch