[RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Marek Szyprowski]

Dear All,

I would like to add support for configuring alpha modes: straight and 
pre-multiplied for blending to Exynos DRM driver. For those who see 
those terms for the first time:
1. straight alpha mode means means that all A and RGB values are from 
full 0.255 range,
2. pre-multiplied alpha mode means that A is from 0..255 range and RGB 
values are scaled to 0..ALPHA range (where ALPHA is the A value for the 
given pixel).
The pre-multiplied mode is quite common in texture processing.

I did a little research and found that there is no common approach for 
defining straight or pre-multiplied alpha modes for ARGB framebuffers.

 From reading the sources and the register names I found that following 
drivers use pre-multiplied modes for ARGB framebuffers:
radeon (at least for ARGB cursors),
amdgpu (cursor),
intel (all ARGB planes),
rock chip.

On the other hand, there are drm drivers which support ARGB modes and 
use straight alpha modes:
omap,
msm.

For the rest of drivers, which might deal with per-pixel alpha, I was 
not able to determine from the code if the pixel RGB values are 
interpreted as per-multiplied or straight:
atmel_hlcdc,
sti,
mdp5,
shmobile,
rcar,
vc4,
imx.

Exynos DRM driver now mixes pre-multiplied and straight modes, depending 
on the CRTC sub-driver.

While checking the code I found a following comment in 
drm/i915/intel_display.c in skl_plane_ctl_format() function:
/*
  * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
  * to be already pre-multiplied. We need to add a knob (or a different
  * DRM_FORMAT) for user-space to configure that.
  */

The question is how to cleanup this ambiguities and let generic 
userspace to properly use ARGB/ARGB modes with proper blending mode. I 
came up with the following 3 solutions:

1. Define new fourcc values for pre-multiplied (or/and straight) alpha 
modes,
2. Introduce a new framebuffer flag for pre-multiplied or straight alpha 
(or both),
3. Introduce a new plane property for controlling alpha blending mode.

The first solution has serious compatibility problem, because we can 
either map existing fourcc to pre-multiplied (to follow radeon/amdgpu, 
intel and rockchip behavior) or straight mode. Both ways it will break 
some existing applications. To avoid compatibility problem we would need 
to add 2 more sets of fourcc and leave existing ARGB modes as 'driver 
dependent'.

The second solution is probably a bit less intrusive, but it suffers for 
the similar compatibility problem. To make this feature compatible with 
existing code, probably 2 flags will be needed: 
DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED and 
DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no 
flag, the driver can use its default mode.

Third solution (additional plane property) has been already proposed: 
https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor 
comments. Separate property lets at least drivers to notify userspace 
about driver's default alpha mode and lets generic application to 
properly set the requested mode. The only problem is that the alpha mode 
is not directly configured on the framebuffer object, where in my 
opinion it should be stored.

Please let me know which approach You like best and which should I take 
for introducing generic way of configuring alpha mode.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Daniel Vetter]

On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
> Dear All,
> 
> I would like to add support for configuring alpha modes: straight and
> pre-multiplied for blending to Exynos DRM driver. For those who see those
> terms for the first time:
> 1. straight alpha mode means means that all A and RGB values are from full
> 0.255 range,
> 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
> values are scaled to 0..ALPHA range (where ALPHA is the A value for the
> given pixel).
> The pre-multiplied mode is quite common in texture processing.
> 
> I did a little research and found that there is no common approach for
> defining straight or pre-multiplied alpha modes for ARGB framebuffers.
> 
> From reading the sources and the register names I found that following
> drivers use pre-multiplied modes for ARGB framebuffers:
> radeon (at least for ARGB cursors),
> amdgpu (cursor),
> intel (all ARGB planes),
> rock chip.
> 
> On the other hand, there are drm drivers which support ARGB modes and use
> straight alpha modes:
> omap,
> msm.
> 
> For the rest of drivers, which might deal with per-pixel alpha, I was not
> able to determine from the code if the pixel RGB values are interpreted as
> per-multiplied or straight:
> atmel_hlcdc,
> sti,
> mdp5,
> shmobile,
> rcar,
> vc4,
> imx.

Imo in case of doubt/mixed definitions the semantics of the big desktop
drivers should win. True generic userspace is most likely developed on
those, everything else should just adjust. And in most cases we can get
away with that on arm drivers since they tend to ship userspace and kernel
in lockstep. At least where it really matters.

> Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
> the CRTC sub-driver.
> 
> While checking the code I found a following comment in
> drm/i915/intel_display.c in skl_plane_ctl_format() function:
> /*
>  * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
>  * to be already pre-multiplied. We need to add a knob (or a different
>  * DRM_FORMAT) for user-space to configure that.
>  */
> 
> The question is how to cleanup this ambiguities and let generic userspace to
> properly use ARGB/ARGB modes with proper blending mode. I came up with the
> following 3 solutions:
> 
> 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
> modes,
> 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
> both),
> 3. Introduce a new plane property for controlling alpha blending mode.
> 
> The first solution has serious compatibility problem, because we can either
> map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
> rockchip behavior) or straight mode. Both ways it will break some existing
> applications. To avoid compatibility problem we would need to add 2 more
> sets of fourcc and leave existing ARGB modes as 'driver dependent'.
> 
> The second solution is probably a bit less intrusive, but it suffers for the
> similar compatibility problem. To make this feature compatible with existing
> code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
> and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
> flag, the driver can use its default mode.
> 
> Third solution (additional plane property) has been already proposed:
> https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
> comments. Separate property lets at least drivers to notify userspace about
> driver's default alpha mode and lets generic application to properly set the
> requested mode. The only problem is that the alpha mode is not directly
> configured on the framebuffer object, where in my opinion it should be
> stored.
> 
> Please let me know which approach You like best and which should I take for
> introducing generic way of configuring alpha mode.

One idea that was tossed around a few times is to go full generic and
implement something like glBlendFunc for planes, except that we'd also
pre-multiplied/straight versions where it makes sense. For drivers that
can't support pre-multiplied alpha they could simply leave out these
values from the blend-func property (like we already allow with e.g.
rotation, when a driver can't do 90/270°).

Wrt backwards compat a property would work well too: If it's not there
then userspace should assume old behaviour (which should be pre-multiplied
really, but might be somewhat broken on some drivers/kernel).

Damien had some RFC about all this way back, but can't find it right now.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Ville Syrjälä]

On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
> On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
> > Dear All,
> > 
> > I would like to add support for configuring alpha modes: straight and
> > pre-multiplied for blending to Exynos DRM driver. For those who see those
> > terms for the first time:
> > 1. straight alpha mode means means that all A and RGB values are from full
> > 0.255 range,
> > 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
> > values are scaled to 0..ALPHA range (where ALPHA is the A value for the
> > given pixel).
> > The pre-multiplied mode is quite common in texture processing.
> > 
> > I did a little research and found that there is no common approach for
> > defining straight or pre-multiplied alpha modes for ARGB framebuffers.
> > 
> > From reading the sources and the register names I found that following
> > drivers use pre-multiplied modes for ARGB framebuffers:
> > radeon (at least for ARGB cursors),
> > amdgpu (cursor),
> > intel (all ARGB planes),
> > rock chip.
> > 
> > On the other hand, there are drm drivers which support ARGB modes and use
> > straight alpha modes:
> > omap,
> > msm.
> > 
> > For the rest of drivers, which might deal with per-pixel alpha, I was not
> > able to determine from the code if the pixel RGB values are interpreted as
> > per-multiplied or straight:
> > atmel_hlcdc,
> > sti,
> > mdp5,
> > shmobile,
> > rcar,
> > vc4,
> > imx.
> 
> Imo in case of doubt/mixed definitions the semantics of the big desktop
> drivers should win. True generic userspace is most likely developed on
> those, everything else should just adjust. And in most cases we can get
> away with that on arm drivers since they tend to ship userspace and kernel
> in lockstep. At least where it really matters.
> 
> > Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
> > the CRTC sub-driver.
> > 
> > While checking the code I found a following comment in
> > drm/i915/intel_display.c in skl_plane_ctl_format() function:
> > /*
> >  * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
> >  * to be already pre-multiplied. We need to add a knob (or a different
> >  * DRM_FORMAT) for user-space to configure that.
> >  */
> > 
> > The question is how to cleanup this ambiguities and let generic userspace to
> > properly use ARGB/ARGB modes with proper blending mode. I came up with the
> > following 3 solutions:
> > 
> > 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
> > modes,
> > 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
> > both),
> > 3. Introduce a new plane property for controlling alpha blending mode.
> > 
> > The first solution has serious compatibility problem, because we can either
> > map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
> > rockchip behavior) or straight mode. Both ways it will break some existing
> > applications. To avoid compatibility problem we would need to add 2 more
> > sets of fourcc and leave existing ARGB modes as 'driver dependent'.
> > 
> > The second solution is probably a bit less intrusive, but it suffers for the
> > similar compatibility problem. To make this feature compatible with existing
> > code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
> > and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
> > flag, the driver can use its default mode.
> > 
> > Third solution (additional plane property) has been already proposed:
> > https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
> > comments. Separate property lets at least drivers to notify userspace about
> > driver's default alpha mode and lets generic application to properly set the
> > requested mode. The only problem is that the alpha mode is not directly
> > configured on the framebuffer object, where in my opinion it should be
> > stored.
> > 
> > Please let me know which approach You like best and which should I take for
> > introducing generic way of configuring alpha mode.
> 
> One idea that was tossed around a few times is to go full generic and
> implement something like glBlendFunc for planes, except that we'd also
> pre-multiplied/straight versions where it makes sense. For drivers that
> can't support pre-multiplied alpha they could simply leave out these
> values from the blend-func property (like we already allow with e.g.
> rotation, when a driver can't do 90/270°).
> 
> Wrt backwards compat a property would work well too: If it's not there
> then userspace should assume old behaviour (which should be pre-multiplied
> really, but might be somewhat broken on some drivers/kernel).
> 
> Damien had some RFC about all this way back, but can't find it right now.

I remembe he said he wanted to work on it, but I don't remember seeing
anything come out of that. I guess he got swallowed by patchwork, or I
just missed the patch.

This mail might include some of my more recent thoughts on this matter
http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Marek Szyprowski]

Hello,

On 2016-01-11 16:22, Ville Syrjälä wrote:
> On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
>> On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
>>> Dear All,
>>>
>>> I would like to add support for configuring alpha modes: straight and
>>> pre-multiplied for blending to Exynos DRM driver. For those who see those
>>> terms for the first time:
>>> 1. straight alpha mode means means that all A and RGB values are from full
>>> 0.255 range,
>>> 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
>>> values are scaled to 0..ALPHA range (where ALPHA is the A value for the
>>> given pixel).
>>> The pre-multiplied mode is quite common in texture processing.
>>>
>>> I did a little research and found that there is no common approach for
>>> defining straight or pre-multiplied alpha modes for ARGB framebuffers.
>>>
>>> >From reading the sources and the register names I found that following
>>> drivers use pre-multiplied modes for ARGB framebuffers:
>>> radeon (at least for ARGB cursors),
>>> amdgpu (cursor),
>>> intel (all ARGB planes),
>>> rock chip.
>>>
>>> On the other hand, there are drm drivers which support ARGB modes and use
>>> straight alpha modes:
>>> omap,
>>> msm.
>>>
>>> For the rest of drivers, which might deal with per-pixel alpha, I was not
>>> able to determine from the code if the pixel RGB values are interpreted as
>>> per-multiplied or straight:
>>> atmel_hlcdc,
>>> sti,
>>> mdp5,
>>> shmobile,
>>> rcar,
>>> vc4,
>>> imx.
>> Imo in case of doubt/mixed definitions the semantics of the big desktop
>> drivers should win. True generic userspace is most likely developed on
>> those, everything else should just adjust. And in most cases we can get
>> away with that on arm drivers since they tend to ship userspace and kernel
>> in lockstep. At least where it really matters.
>>
>>> Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
>>> the CRTC sub-driver.
>>>
>>> While checking the code I found a following comment in
>>> drm/i915/intel_display.c in skl_plane_ctl_format() function:
>>> /*
>>>   * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
>>>   * to be already pre-multiplied. We need to add a knob (or a different
>>>   * DRM_FORMAT) for user-space to configure that.
>>>   */
>>>
>>> The question is how to cleanup this ambiguities and let generic userspace to
>>> properly use ARGB/ARGB modes with proper blending mode. I came up with the
>>> following 3 solutions:
>>>
>>> 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
>>> modes,
>>> 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
>>> both),
>>> 3. Introduce a new plane property for controlling alpha blending mode.
>>>
>>> The first solution has serious compatibility problem, because we can either
>>> map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
>>> rockchip behavior) or straight mode. Both ways it will break some existing
>>> applications. To avoid compatibility problem we would need to add 2 more
>>> sets of fourcc and leave existing ARGB modes as 'driver dependent'.
>>>
>>> The second solution is probably a bit less intrusive, but it suffers for the
>>> similar compatibility problem. To make this feature compatible with existing
>>> code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
>>> and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
>>> flag, the driver can use its default mode.
>>>
>>> Third solution (additional plane property) has been already proposed:
>>> https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
>>> comments. Separate property lets at least drivers to notify userspace about
>>> driver's default alpha mode and lets generic application to properly set the
>>> requested mode. The only problem is that the alpha mode is not directly
>>> configured on the framebuffer object, where in my opinion it should be
>>> stored.
>>>
>>> Please let me know which approach You like best and which should I take for
>>> introducing generic way of configuring alpha mode.
>> One idea that was tossed around a few times is to go full generic and
>> implement something like glBlendFunc for planes, except that we'd also
>> pre-multiplied/straight versions where it makes sense. For drivers that
>> can't support pre-multiplied alpha they could simply leave out these
>> values from the blend-func property (like we already allow with e.g.
>> rotation, when a driver can't do 90/270°).
>>
>> Wrt backwards compat a property would work well too: If it's not there
>> then userspace should assume old behaviour (which should be pre-multiplied
>> really, but might be somewhat broken on some drivers/kernel).
>>
>> Damien had some RFC about all this way back, but can't find it right now.
> I remembe he said he wanted to work on it, but I don't remember seeing
> anything come out of that. I guess he got swallowed by patchwork, or I
> just missed the patch.
>
> This mail might include some of my more recent thoughts on this matter
> http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html

I also found this patch: http://patchwork.freedesktop.org/patch/23000/

However I still don't get a complete picture how it would work. If I got it
right, the real blending function is defined by a pair of the values 
from the
proposed enums, where one applies to the background (the blending result of
deeper planes) and the second applies to the processed plane. Some
combinations of the proposed enums doesn't make much sense (and especially
changing the constant zero and one means simply changing the order of the
planes), so I wonder if it might be easier to simply create an enum with
all sane values for blending and document it really well? This way drivers
will simply initialize only the supported blending modes.

I assume that this way also the separate enums can be defined for straight
and pre-multiplied alpha blending modes.

The other problem mentioned somewhere is where to attach the blending
property. Some parameters (like straight/pre-multiplied or enabling global
alpha) can be set on per-plane basis, but I can imagine that some parameters
are global for the given crtc.

The another item related to blending is color keying. It was not mentioned
in the above proposal at all, but some hardware can do that. I've noticed
that some hardware also is able to do color keying only for exact RGB(A)
values and some have separate registers to set RGB tolerance. This also
need to be standardized somehow.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Daniel Vetter]

On Wed, Jan 13, 2016 at 03:13:35PM +0100, Marek Szyprowski wrote:
> Hello,
> 
> On 2016-01-11 16:22, Ville Syrjälä wrote:
> >On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
> >>On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
> >>>Dear All,
> >>>
> >>>I would like to add support for configuring alpha modes: straight and
> >>>pre-multiplied for blending to Exynos DRM driver. For those who see those
> >>>terms for the first time:
> >>>1. straight alpha mode means means that all A and RGB values are from full
> >>>0.255 range,
> >>>2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
> >>>values are scaled to 0..ALPHA range (where ALPHA is the A value for the
> >>>given pixel).
> >>>The pre-multiplied mode is quite common in texture processing.
> >>>
> >>>I did a little research and found that there is no common approach for
> >>>defining straight or pre-multiplied alpha modes for ARGB framebuffers.
> >>>
> >>>>From reading the sources and the register names I found that following
> >>>drivers use pre-multiplied modes for ARGB framebuffers:
> >>>radeon (at least for ARGB cursors),
> >>>amdgpu (cursor),
> >>>intel (all ARGB planes),
> >>>rock chip.
> >>>
> >>>On the other hand, there are drm drivers which support ARGB modes and use
> >>>straight alpha modes:
> >>>omap,
> >>>msm.
> >>>
> >>>For the rest of drivers, which might deal with per-pixel alpha, I was not
> >>>able to determine from the code if the pixel RGB values are interpreted as
> >>>per-multiplied or straight:
> >>>atmel_hlcdc,
> >>>sti,
> >>>mdp5,
> >>>shmobile,
> >>>rcar,
> >>>vc4,
> >>>imx.
> >>Imo in case of doubt/mixed definitions the semantics of the big desktop
> >>drivers should win. True generic userspace is most likely developed on
> >>those, everything else should just adjust. And in most cases we can get
> >>away with that on arm drivers since they tend to ship userspace and kernel
> >>in lockstep. At least where it really matters.
> >>
> >>>Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
> >>>the CRTC sub-driver.
> >>>
> >>>While checking the code I found a following comment in
> >>>drm/i915/intel_display.c in skl_plane_ctl_format() function:
> >>>/*
> >>>  * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
> >>>  * to be already pre-multiplied. We need to add a knob (or a different
> >>>  * DRM_FORMAT) for user-space to configure that.
> >>>  */
> >>>
> >>>The question is how to cleanup this ambiguities and let generic userspace to
> >>>properly use ARGB/ARGB modes with proper blending mode. I came up with the
> >>>following 3 solutions:
> >>>
> >>>1. Define new fourcc values for pre-multiplied (or/and straight) alpha
> >>>modes,
> >>>2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
> >>>both),
> >>>3. Introduce a new plane property for controlling alpha blending mode.
> >>>
> >>>The first solution has serious compatibility problem, because we can either
> >>>map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
> >>>rockchip behavior) or straight mode. Both ways it will break some existing
> >>>applications. To avoid compatibility problem we would need to add 2 more
> >>>sets of fourcc and leave existing ARGB modes as 'driver dependent'.
> >>>
> >>>The second solution is probably a bit less intrusive, but it suffers for the
> >>>similar compatibility problem. To make this feature compatible with existing
> >>>code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
> >>>and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
> >>>flag, the driver can use its default mode.
> >>>
> >>>Third solution (additional plane property) has been already proposed:
> >>>https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
> >>>comments. Separate property lets at least drivers to notify userspace about
> >>>driver's default alpha mode and lets generic application to properly set the
> >>>requested mode. The only problem is that the alpha mode is not directly
> >>>configured on the framebuffer object, where in my opinion it should be
> >>>stored.
> >>>
> >>>Please let me know which approach You like best and which should I take for
> >>>introducing generic way of configuring alpha mode.
> >>One idea that was tossed around a few times is to go full generic and
> >>implement something like glBlendFunc for planes, except that we'd also
> >>pre-multiplied/straight versions where it makes sense. For drivers that
> >>can't support pre-multiplied alpha they could simply leave out these
> >>values from the blend-func property (like we already allow with e.g.
> >>rotation, when a driver can't do 90/270°).
> >>
> >>Wrt backwards compat a property would work well too: If it's not there
> >>then userspace should assume old behaviour (which should be pre-multiplied
> >>really, but might be somewhat broken on some drivers/kernel).
> >>
> >>Damien had some RFC about all this way back, but can't find it right now.
> >I remembe he said he wanted to work on it, but I don't remember seeing
> >anything come out of that. I guess he got swallowed by patchwork, or I
> >just missed the patch.
> >
> >This mail might include some of my more recent thoughts on this matter
> >http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html
> 
> I also found this patch: http://patchwork.freedesktop.org/patch/23000/
> 
> However I still don't get a complete picture how it would work. If I got it
> right, the real blending function is defined by a pair of the values from
> the
> proposed enums, where one applies to the background (the blending result of
> deeper planes) and the second applies to the processed plane. Some
> combinations of the proposed enums doesn't make much sense (and especially
> changing the constant zero and one means simply changing the order of the
> planes), so I wonder if it might be easier to simply create an enum with
> all sane values for blending and document it really well? This way drivers
> will simply initialize only the supported blending modes.
> 
> I assume that this way also the separate enums can be defined for straight
> and pre-multiplied alpha blending modes.

Yeah, aliasing is always a big concern for stuff like this, hence why we
have normalized zpos, functions to reduce rotations and stuff like that.

If it's possible to define a set of enums that work, and dont accidentally
interact with zpos in fun ways then I guess we could go ahead with that.
But would need some serious thought.

I think in the end it's probably best if we look at blending/zpos as one
overall feature, where the kerneldoc/spec for the feature explains how
things should work and how it all interacts.

> The other problem mentioned somewhere is where to attach the blending
> property. Some parameters (like straight/pre-multiplied or enabling global
> alpha) can be set on per-plane basis, but I can imagine that some parameters
> are global for the given crtc.

Yeah, we can do both per-plane and global props on the crtc. E.g. we
probably need a backgroun color (for chips that support anything else than
black), and that needs to be on the crtc and would always be the lowest
plane.

Lots of interactions here.

Thinking about this: Should we maybe extract a new drm_prop_blending, so
that we can pull together zpos, background color, blending modes and all
the helpers to register props into one file? Plus then add one overview
section which explains in detail the model we expect this to work with?

> The another item related to blending is color keying. It was not mentioned
> in the above proposal at all, but some hardware can do that. I've noticed
> that some hardware also is able to do color keying only for exact RGB(A)
> values and some have separate registers to set RGB tolerance. This also
> need to be standardized somehow.

tbh I have no idea how color keying should interact with blending. At
least with intel hw it's either alpha or keying, and color keying has
become fairly unpopular.

I think it'd be best if we just consider alpha blending for now, and then
figure out in a 2nd step how to integrate color keying (if anyone really
cares enough about this). Otherwise this will become too big and we'll
never get it merged.

One more: Where/how to you plan to create the open source demonstration
compositor? Just testcases arent' enough, it should be something real like
wayland, hwc, ozone or xf86-video-modesetting.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Ville Syrjälä]

On Wed, Jan 13, 2016 at 03:13:35PM +0100, Marek Szyprowski wrote:
> Hello,
> 
> On 2016-01-11 16:22, Ville Syrjälä wrote:
> > On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
> >> On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
> >>> Dear All,
> >>>
> >>> I would like to add support for configuring alpha modes: straight and
> >>> pre-multiplied for blending to Exynos DRM driver. For those who see those
> >>> terms for the first time:
> >>> 1. straight alpha mode means means that all A and RGB values are from full
> >>> 0.255 range,
> >>> 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
> >>> values are scaled to 0..ALPHA range (where ALPHA is the A value for the
> >>> given pixel).
> >>> The pre-multiplied mode is quite common in texture processing.
> >>>
> >>> I did a little research and found that there is no common approach for
> >>> defining straight or pre-multiplied alpha modes for ARGB framebuffers.
> >>>
> >>> >From reading the sources and the register names I found that following
> >>> drivers use pre-multiplied modes for ARGB framebuffers:
> >>> radeon (at least for ARGB cursors),
> >>> amdgpu (cursor),
> >>> intel (all ARGB planes),
> >>> rock chip.
> >>>
> >>> On the other hand, there are drm drivers which support ARGB modes and use
> >>> straight alpha modes:
> >>> omap,
> >>> msm.
> >>>
> >>> For the rest of drivers, which might deal with per-pixel alpha, I was not
> >>> able to determine from the code if the pixel RGB values are interpreted as
> >>> per-multiplied or straight:
> >>> atmel_hlcdc,
> >>> sti,
> >>> mdp5,
> >>> shmobile,
> >>> rcar,
> >>> vc4,
> >>> imx.
> >> Imo in case of doubt/mixed definitions the semantics of the big desktop
> >> drivers should win. True generic userspace is most likely developed on
> >> those, everything else should just adjust. And in most cases we can get
> >> away with that on arm drivers since they tend to ship userspace and kernel
> >> in lockstep. At least where it really matters.
> >>
> >>> Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
> >>> the CRTC sub-driver.
> >>>
> >>> While checking the code I found a following comment in
> >>> drm/i915/intel_display.c in skl_plane_ctl_format() function:
> >>> /*
> >>>   * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
> >>>   * to be already pre-multiplied. We need to add a knob (or a different
> >>>   * DRM_FORMAT) for user-space to configure that.
> >>>   */
> >>>
> >>> The question is how to cleanup this ambiguities and let generic userspace to
> >>> properly use ARGB/ARGB modes with proper blending mode. I came up with the
> >>> following 3 solutions:
> >>>
> >>> 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
> >>> modes,
> >>> 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
> >>> both),
> >>> 3. Introduce a new plane property for controlling alpha blending mode.
> >>>
> >>> The first solution has serious compatibility problem, because we can either
> >>> map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
> >>> rockchip behavior) or straight mode. Both ways it will break some existing
> >>> applications. To avoid compatibility problem we would need to add 2 more
> >>> sets of fourcc and leave existing ARGB modes as 'driver dependent'.
> >>>
> >>> The second solution is probably a bit less intrusive, but it suffers for the
> >>> similar compatibility problem. To make this feature compatible with existing
> >>> code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
> >>> and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
> >>> flag, the driver can use its default mode.
> >>>
> >>> Third solution (additional plane property) has been already proposed:
> >>> https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
> >>> comments. Separate property lets at least drivers to notify userspace about
> >>> driver's default alpha mode and lets generic application to properly set the
> >>> requested mode. The only problem is that the alpha mode is not directly
> >>> configured on the framebuffer object, where in my opinion it should be
> >>> stored.
> >>>
> >>> Please let me know which approach You like best and which should I take for
> >>> introducing generic way of configuring alpha mode.
> >> One idea that was tossed around a few times is to go full generic and
> >> implement something like glBlendFunc for planes, except that we'd also
> >> pre-multiplied/straight versions where it makes sense. For drivers that
> >> can't support pre-multiplied alpha they could simply leave out these
> >> values from the blend-func property (like we already allow with e.g.
> >> rotation, when a driver can't do 90/270°).
> >>
> >> Wrt backwards compat a property would work well too: If it's not there
> >> then userspace should assume old behaviour (which should be pre-multiplied
> >> really, but might be somewhat broken on some drivers/kernel).
> >>
> >> Damien had some RFC about all this way back, but can't find it right now.
> > I remembe he said he wanted to work on it, but I don't remember seeing
> > anything come out of that. I guess he got swallowed by patchwork, or I
> > just missed the patch.
> >
> > This mail might include some of my more recent thoughts on this matter
> > http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html
> 
> I also found this patch: http://patchwork.freedesktop.org/patch/23000/
> 
> However I still don't get a complete picture how it would work. If I got it
> right, the real blending function is defined by a pair of the values 
> from the
> proposed enums, where one applies to the background (the blending result of
> deeper planes) and the second applies to the processed plane. Some
> combinations of the proposed enums doesn't make much sense (and especially
> changing the constant zero and one means simply changing the order of the
> planes), so I wonder if it might be easier to simply create an enum with
> all sane values for blending and document it really well? This way drivers
> will simply initialize only the supported blending modes.

That was my idea more or less too. We'd still model things based on the
GL stuff, but we only advertize specific src+dst blend factor
combinations. Ie. a single enum property where each value contains both
the src and dst blend factor.

> 
> I assume that this way also the separate enums can be defined for straight
> and pre-multiplied alpha blending modes.
> 
> The other problem mentioned somewhere is where to attach the blending
> property. Some parameters (like straight/pre-multiplied or enabling global
> alpha) can be set on per-plane basis, but I can imagine that some parameters
> are global for the given crtc.

I'm quite convinced these days that we need to go for a per-plane property.
This of course means that when dealing with multiple planes the user may
still end up requesting a set of blend equations that can't be supported,
but I don't see any real way to avoid that.

> 
> The another item related to blending is color keying. It was not mentioned
> in the above proposal at all, but some hardware can do that. I've noticed
> that some hardware also is able to do color keying only for exact RGB(A)
> values and some have separate registers to set RGB tolerance. This also
> need to be standardized somehow.

Yeah, I think typically you have value+mask, min+max+mask, or just
min+max. It's also hardware dependent whether those are applied against
the raw pixel value, or perhaps some range expanded value, or even color
space converted value. Another hardware dependent fact is exactly which
planes take part in the color keying. I suppose source keying should be
easy since it should apply to a single plane, but destination keying at
least can only be effective between a specific subset of planes (which
can change depending on which planes are actually enabled). Also the
plane z order may be affected by destination keying.

So there are quite a few hardware depenent variables in the mix, and so
I'm not sure there's any reasonable way to abstract it all.

Maybe it's best if we just define a set of standardish properties and
each driver can pick whichever ones it can do, and we just ignore all
the other hardware dependent details. So we could have something like:
dst-color-key-min, dst-color-key-max, dst-color-key-value,
dst-color-key-mask
and the same for source keying. And we can define that the values be
encoded as something like 16:16:16:16 ARGB/AYCbCr (well I guess we'd
leave out the alpha from color keying values), the driver being free
to ignore as many low bits as it wants.

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Marek Szyprowski]

Hello,

On 2016-01-13 15:58, Ville Syrjälä wrote:
> On Wed, Jan 13, 2016 at 03:13:35PM +0100, Marek Szyprowski wrote:
>> On 2016-01-11 16:22, Ville Syrjälä wrote:
>>> On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
>>>> On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
>>>>> Dear All,
>>>>>
>>>>> I would like to add support for configuring alpha modes: straight and
>>>>> pre-multiplied for blending to Exynos DRM driver. For those who see those
>>>>> terms for the first time:
>>>>> 1. straight alpha mode means means that all A and RGB values are from full
>>>>> 0.255 range,
>>>>> 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
>>>>> values are scaled to 0..ALPHA range (where ALPHA is the A value for the
>>>>> given pixel).
>>>>> The pre-multiplied mode is quite common in texture processing.
>>>>>
>>>>> I did a little research and found that there is no common approach for
>>>>> defining straight or pre-multiplied alpha modes for ARGB framebuffers.
>>>>>
>>>>> >From reading the sources and the register names I found that following
>>>>> drivers use pre-multiplied modes for ARGB framebuffers:
>>>>> radeon (at least for ARGB cursors),
>>>>> amdgpu (cursor),
>>>>> intel (all ARGB planes),
>>>>> rock chip.
>>>>>
>>>>> On the other hand, there are drm drivers which support ARGB modes and use
>>>>> straight alpha modes:
>>>>> omap,
>>>>> msm.
>>>>>
>>>>> For the rest of drivers, which might deal with per-pixel alpha, I was not
>>>>> able to determine from the code if the pixel RGB values are interpreted as
>>>>> per-multiplied or straight:
>>>>> atmel_hlcdc,
>>>>> sti,
>>>>> mdp5,
>>>>> shmobile,
>>>>> rcar,
>>>>> vc4,
>>>>> imx.
>>>> Imo in case of doubt/mixed definitions the semantics of the big desktop
>>>> drivers should win. True generic userspace is most likely developed on
>>>> those, everything else should just adjust. And in most cases we can get
>>>> away with that on arm drivers since they tend to ship userspace and kernel
>>>> in lockstep. At least where it really matters.
>>>>
>>>>> Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
>>>>> the CRTC sub-driver.
>>>>>
>>>>> While checking the code I found a following comment in
>>>>> drm/i915/intel_display.c in skl_plane_ctl_format() function:
>>>>> /*
>>>>>    * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
>>>>>    * to be already pre-multiplied. We need to add a knob (or a different
>>>>>    * DRM_FORMAT) for user-space to configure that.
>>>>>    */
>>>>>
>>>>> The question is how to cleanup this ambiguities and let generic userspace to
>>>>> properly use ARGB/ARGB modes with proper blending mode. I came up with the
>>>>> following 3 solutions:
>>>>>
>>>>> 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
>>>>> modes,
>>>>> 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
>>>>> both),
>>>>> 3. Introduce a new plane property for controlling alpha blending mode.
>>>>>
>>>>> The first solution has serious compatibility problem, because we can either
>>>>> map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
>>>>> rockchip behavior) or straight mode. Both ways it will break some existing
>>>>> applications. To avoid compatibility problem we would need to add 2 more
>>>>> sets of fourcc and leave existing ARGB modes as 'driver dependent'.
>>>>>
>>>>> The second solution is probably a bit less intrusive, but it suffers for the
>>>>> similar compatibility problem. To make this feature compatible with existing
>>>>> code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
>>>>> and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
>>>>> flag, the driver can use its default mode.
>>>>>
>>>>> Third solution (additional plane property) has been already proposed:
>>>>> https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
>>>>> comments. Separate property lets at least drivers to notify userspace about
>>>>> driver's default alpha mode and lets generic application to properly set the
>>>>> requested mode. The only problem is that the alpha mode is not directly
>>>>> configured on the framebuffer object, where in my opinion it should be
>>>>> stored.
>>>>>
>>>>> Please let me know which approach You like best and which should I take for
>>>>> introducing generic way of configuring alpha mode.
>>>> One idea that was tossed around a few times is to go full generic and
>>>> implement something like glBlendFunc for planes, except that we'd also
>>>> pre-multiplied/straight versions where it makes sense. For drivers that
>>>> can't support pre-multiplied alpha they could simply leave out these
>>>> values from the blend-func property (like we already allow with e.g.
>>>> rotation, when a driver can't do 90/270°).
>>>>
>>>> Wrt backwards compat a property would work well too: If it's not there
>>>> then userspace should assume old behaviour (which should be pre-multiplied
>>>> really, but might be somewhat broken on some drivers/kernel).
>>>>
>>>> Damien had some RFC about all this way back, but can't find it right now.
>>> I remembe he said he wanted to work on it, but I don't remember seeing
>>> anything come out of that. I guess he got swallowed by patchwork, or I
>>> just missed the patch.
>>>
>>> This mail might include some of my more recent thoughts on this matter
>>> http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html
>> I also found this patch: http://patchwork.freedesktop.org/patch/23000/
>>
>> However I still don't get a complete picture how it would work. If I got it
>> right, the real blending function is defined by a pair of the values
>> from the
>> proposed enums, where one applies to the background (the blending result of
>> deeper planes) and the second applies to the processed plane. Some
>> combinations of the proposed enums doesn't make much sense (and especially
>> changing the constant zero and one means simply changing the order of the
>> planes), so I wonder if it might be easier to simply create an enum with
>> all sane values for blending and document it really well? This way drivers
>> will simply initialize only the supported blending modes.
> That was my idea more or less too. We'd still model things based on the
> GL stuff, but we only advertize specific src+dst blend factor
> combinations. Ie. a single enum property where each value contains both
> the src and dst blend factor.

Okay, I've did some analysis of what Exynos hardware can do and it 
turned out that there only a few modes which really makes sense. 
Assuming that the order of planes is first determined by zpos 
properties, we already have well defined what is background (result of 
blending of all deeper planes) and what is blended plane (the terms 
source and destination used in previous discussion about blending are 
not very intuitive for me).

Now lets consider the basic blending equation:
ResultRGB = A * backgroundRGB + B * planeRGB

where:
planeRGB is RGB pixel values of blended plane
backgroundRGB is RGB pixel values of plane's background (i.e. result of 
previous blending or RGB pixel values of deeper plane)

where A and B can be one of:
ZERO, ONE, CONST_ALPHA (constant plane property), (1-CONST_ALPHA), 
PIXEL_ALPHA (alpha value from given pixel), (1-PIXEL_ALPHA), 
PIXEL_CONST_ALPHA (CONST_ALPHA * PIXEL_ALPHA), (1-PIXEL_CONST_ALPHA)

This gives following useful modes:
1. NO_BLENDING: A=ZERO, B=ONE - no blending, global and per-pixel alpha 
ignored (regardless selected fourcc mode)
2. CONST_ALPHA_BLEND: A=(1-CONST_ALPHA), B=CONST_ALPHA - blending with 
global plane alpha, per-pixel alpha ignored (regardless selected fourcc 
mode)
3. PIXEL_ALPHA_BLEND: A=(1-PIXEL_ALPHA), B=PIXEL_ALPHA - blending with 
per-pixel alpha, assuming plane with alpha pre-multiplied RGB pixel 
values (global alpha ignored, needs fourcc mode with alpha channel 
otherwise same as 1)
4. PIXEL_CONST_ALPHA_BLEND: A=(1-PIXEL_CONST_ALPHA), B=PIXEL_CONST_ALPHA 
- blending with both global and per-pixel alpha multiplied, assuming 
plane with alpha pre-multiplied RGB pixel values (needs fourcc mode with 
alpha channel otherwise same as 2)

If we want to solve pre-multiplied/straight alpha modes issue by 
introducing additional blending mode, we should consider following:
5. PIXEL_ALPHA_BLEND_STRAIGHT: A=(1-PIXEL_ALPHA), B=PIXEL_ALPHA - 
blending with per-pixel alpha, assuming plane with straight RGB pixel 
values (to be multiplied by pixel alpha value),
6. PIXEL_CONST_ALPHA_BLEND_STRAIGHT: A=(1-CONST_PIXEL_ALPHA), 
B=CONST_PIXEL_ALPHA - blending with both global and per-pixel alpha, 
assuming plane with straight RGB pixel values (to be multiplied by pixel 
alpha value)

Are there any other useful modes implemented by your hardware? The above 
defined modes are quite easy to get embedded into single enum and have 
well defined behavior.

>> I assume that this way also the separate enums can be defined for straight
>> and pre-multiplied alpha blending modes.
>>
>> The other problem mentioned somewhere is where to attach the blending
>> property. Some parameters (like straight/pre-multiplied or enabling global
>> alpha) can be set on per-plane basis, but I can imagine that some parameters
>> are global for the given crtc.
> I'm quite convinced these days that we need to go for a per-plane property.
> This of course means that when dealing with multiple planes the user may
> still end up requesting a set of blend equations that can't be supported,
> but I don't see any real way to avoid that.

Right, especially if straight/pre-multiplied alpha will be selected by 
blending mode, it must be per-plane.

>> The another item related to blending is color keying. It was not mentioned
>> in the above proposal at all, but some hardware can do that. I've noticed
>> that some hardware also is able to do color keying only for exact RGB(A)
>> values and some have separate registers to set RGB tolerance. This also
>> need to be standardized somehow.
> Yeah, I think typically you have value+mask, min+max+mask, or just
> min+max. It's also hardware dependent whether those are applied against
> the raw pixel value, or perhaps some range expanded value, or even color
> space converted value. Another hardware dependent fact is exactly which
> planes take part in the color keying. I suppose source keying should be
> easy since it should apply to a single plane, but destination keying at
> least can only be effective between a specific subset of planes (which
> can change depending on which planes are actually enabled). Also the
> plane z order may be affected by destination keying.
>
> So there are quite a few hardware depenent variables in the mix, and so
> I'm not sure there's any reasonable way to abstract it all.
>
> Maybe it's best if we just define a set of standardish properties and
> each driver can pick whichever ones it can do, and we just ignore all
> the other hardware dependent details. So we could have something like:
> dst-color-key-min, dst-color-key-max, dst-color-key-value,
> dst-color-key-mask
> and the same for source keying. And we can define that the values be
> encoded as something like 16:16:16:16 ARGB/AYCbCr (well I guess we'd
> leave out the alpha from color keying values), the driver being free
> to ignore as many low bits as it wants.

Huh, I wasn't aware of those source/destination keying modes.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Ville Syrjälä]

On Mon, Jan 18, 2016 at 11:14:41AM +0100, Marek Szyprowski wrote:
> Hello,
> 
> On 2016-01-13 15:58, Ville Syrjälä wrote:
> > On Wed, Jan 13, 2016 at 03:13:35PM +0100, Marek Szyprowski wrote:
> >> On 2016-01-11 16:22, Ville Syrjälä wrote:
> >>> On Mon, Jan 11, 2016 at 04:07:50PM +0100, Daniel Vetter wrote:
> >>>> On Mon, Jan 11, 2016 at 03:18:44PM +0100, Marek Szyprowski wrote:
> >>>>> Dear All,
> >>>>>
> >>>>> I would like to add support for configuring alpha modes: straight and
> >>>>> pre-multiplied for blending to Exynos DRM driver. For those who see those
> >>>>> terms for the first time:
> >>>>> 1. straight alpha mode means means that all A and RGB values are from full
> >>>>> 0.255 range,
> >>>>> 2. pre-multiplied alpha mode means that A is from 0..255 range and RGB
> >>>>> values are scaled to 0..ALPHA range (where ALPHA is the A value for the
> >>>>> given pixel).
> >>>>> The pre-multiplied mode is quite common in texture processing.
> >>>>>
> >>>>> I did a little research and found that there is no common approach for
> >>>>> defining straight or pre-multiplied alpha modes for ARGB framebuffers.
> >>>>>
> >>>>> >From reading the sources and the register names I found that following
> >>>>> drivers use pre-multiplied modes for ARGB framebuffers:
> >>>>> radeon (at least for ARGB cursors),
> >>>>> amdgpu (cursor),
> >>>>> intel (all ARGB planes),
> >>>>> rock chip.
> >>>>>
> >>>>> On the other hand, there are drm drivers which support ARGB modes and use
> >>>>> straight alpha modes:
> >>>>> omap,
> >>>>> msm.
> >>>>>
> >>>>> For the rest of drivers, which might deal with per-pixel alpha, I was not
> >>>>> able to determine from the code if the pixel RGB values are interpreted as
> >>>>> per-multiplied or straight:
> >>>>> atmel_hlcdc,
> >>>>> sti,
> >>>>> mdp5,
> >>>>> shmobile,
> >>>>> rcar,
> >>>>> vc4,
> >>>>> imx.
> >>>> Imo in case of doubt/mixed definitions the semantics of the big desktop
> >>>> drivers should win. True generic userspace is most likely developed on
> >>>> those, everything else should just adjust. And in most cases we can get
> >>>> away with that on arm drivers since they tend to ship userspace and kernel
> >>>> in lockstep. At least where it really matters.
> >>>>
> >>>>> Exynos DRM driver now mixes pre-multiplied and straight modes, depending on
> >>>>> the CRTC sub-driver.
> >>>>>
> >>>>> While checking the code I found a following comment in
> >>>>> drm/i915/intel_display.c in skl_plane_ctl_format() function:
> >>>>> /*
> >>>>>    * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
> >>>>>    * to be already pre-multiplied. We need to add a knob (or a different
> >>>>>    * DRM_FORMAT) for user-space to configure that.
> >>>>>    */
> >>>>>
> >>>>> The question is how to cleanup this ambiguities and let generic userspace to
> >>>>> properly use ARGB/ARGB modes with proper blending mode. I came up with the
> >>>>> following 3 solutions:
> >>>>>
> >>>>> 1. Define new fourcc values for pre-multiplied (or/and straight) alpha
> >>>>> modes,
> >>>>> 2. Introduce a new framebuffer flag for pre-multiplied or straight alpha (or
> >>>>> both),
> >>>>> 3. Introduce a new plane property for controlling alpha blending mode.
> >>>>>
> >>>>> The first solution has serious compatibility problem, because we can either
> >>>>> map existing fourcc to pre-multiplied (to follow radeon/amdgpu, intel and
> >>>>> rockchip behavior) or straight mode. Both ways it will break some existing
> >>>>> applications. To avoid compatibility problem we would need to add 2 more
> >>>>> sets of fourcc and leave existing ARGB modes as 'driver dependent'.
> >>>>>
> >>>>> The second solution is probably a bit less intrusive, but it suffers for the
> >>>>> similar compatibility problem. To make this feature compatible with existing
> >>>>> code, probably 2 flags will be needed: DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED
> >>>>> and DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no
> >>>>> flag, the driver can use its default mode.
> >>>>>
> >>>>> Third solution (additional plane property) has been already proposed:
> >>>>> https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor
> >>>>> comments. Separate property lets at least drivers to notify userspace about
> >>>>> driver's default alpha mode and lets generic application to properly set the
> >>>>> requested mode. The only problem is that the alpha mode is not directly
> >>>>> configured on the framebuffer object, where in my opinion it should be
> >>>>> stored.
> >>>>>
> >>>>> Please let me know which approach You like best and which should I take for
> >>>>> introducing generic way of configuring alpha mode.
> >>>> One idea that was tossed around a few times is to go full generic and
> >>>> implement something like glBlendFunc for planes, except that we'd also
> >>>> pre-multiplied/straight versions where it makes sense. For drivers that
> >>>> can't support pre-multiplied alpha they could simply leave out these
> >>>> values from the blend-func property (like we already allow with e.g.
> >>>> rotation, when a driver can't do 90/270°).
> >>>>
> >>>> Wrt backwards compat a property would work well too: If it's not there
> >>>> then userspace should assume old behaviour (which should be pre-multiplied
> >>>> really, but might be somewhat broken on some drivers/kernel).
> >>>>
> >>>> Damien had some RFC about all this way back, but can't find it right now.
> >>> I remembe he said he wanted to work on it, but I don't remember seeing
> >>> anything come out of that. I guess he got swallowed by patchwork, or I
> >>> just missed the patch.
> >>>
> >>> This mail might include some of my more recent thoughts on this matter
> >>> http://lists.freedesktop.org/archives/intel-gfx/2014-April/042956.html
> >> I also found this patch: http://patchwork.freedesktop.org/patch/23000/
> >>
> >> However I still don't get a complete picture how it would work. If I got it
> >> right, the real blending function is defined by a pair of the values
> >> from the
> >> proposed enums, where one applies to the background (the blending result of
> >> deeper planes) and the second applies to the processed plane. Some
> >> combinations of the proposed enums doesn't make much sense (and especially
> >> changing the constant zero and one means simply changing the order of the
> >> planes), so I wonder if it might be easier to simply create an enum with
> >> all sane values for blending and document it really well? This way drivers
> >> will simply initialize only the supported blending modes.
> > That was my idea more or less too. We'd still model things based on the
> > GL stuff, but we only advertize specific src+dst blend factor
> > combinations. Ie. a single enum property where each value contains both
> > the src and dst blend factor.
> 
> Okay, I've did some analysis of what Exynos hardware can do and it 
> turned out that there only a few modes which really makes sense. 
> Assuming that the order of planes is first determined by zpos 
> properties, we already have well defined what is background (result of 
> blending of all deeper planes) and what is blended plane (the terms 
> source and destination used in previous discussion about blending are 
> not very intuitive for me).
> 
> Now lets consider the basic blending equation:
> ResultRGB = A * backgroundRGB + B * planeRGB
> 
> where:
> planeRGB is RGB pixel values of blended plane
> backgroundRGB is RGB pixel values of plane's background (i.e. result of 
> previous blending or RGB pixel values of deeper plane)
> 
> where A and B can be one of:
> ZERO, ONE, CONST_ALPHA (constant plane property), (1-CONST_ALPHA), 
> PIXEL_ALPHA (alpha value from given pixel), (1-PIXEL_ALPHA), 
> PIXEL_CONST_ALPHA (CONST_ALPHA * PIXEL_ALPHA), (1-PIXEL_CONST_ALPHA)
> 
> This gives following useful modes:
> 1. NO_BLENDING: A=ZERO, B=ONE - no blending, global and per-pixel alpha 
> ignored (regardless selected fourcc mode)
> 2. CONST_ALPHA_BLEND: A=(1-CONST_ALPHA), B=CONST_ALPHA - blending with 
> global plane alpha, per-pixel alpha ignored (regardless selected fourcc 
> mode)
> 3. PIXEL_ALPHA_BLEND: A=(1-PIXEL_ALPHA), B=PIXEL_ALPHA - blending with 
> per-pixel alpha, assuming plane with alpha pre-multiplied RGB pixel 
> values (global alpha ignored, needs fourcc mode with alpha channel 
> otherwise same as 1)
> 4. PIXEL_CONST_ALPHA_BLEND: A=(1-PIXEL_CONST_ALPHA), B=PIXEL_CONST_ALPHA 
> - blending with both global and per-pixel alpha multiplied, assuming 
> plane with alpha pre-multiplied RGB pixel values (needs fourcc mode with 
> alpha channel otherwise same as 2)
> 
> If we want to solve pre-multiplied/straight alpha modes issue by 
> introducing additional blending mode, we should consider following:
> 5. PIXEL_ALPHA_BLEND_STRAIGHT: A=(1-PIXEL_ALPHA), B=PIXEL_ALPHA - 
> blending with per-pixel alpha, assuming plane with straight RGB pixel 
> values (to be multiplied by pixel alpha value),
> 6. PIXEL_CONST_ALPHA_BLEND_STRAIGHT: A=(1-CONST_PIXEL_ALPHA), 
> B=CONST_PIXEL_ALPHA - blending with both global and per-pixel alpha, 
> assuming plane with straight RGB pixel values (to be multiplied by pixel 
> alpha value)
> 
> Are there any other useful modes implemented by your hardware? The above 
> defined modes are quite easy to get embedded into single enum and have 
> well defined behavior.

IIRC some of our hardware had a bunch more options available. To avoid
putting too many limitations into the API (which I'm sure we'd end up
regretting at some point if history is any indication) I'd just call
the enum values as:
S_ONE_D_ZERO
S_ONE_D_ONE_MINUS_SRC_ALPHA
S_SRC_ALPHA_D_ONE_MINUS_SRC_ALPHA
etc.
or something like that. I suppose the actual strings shouldn't matter
too much if people would just use the values and we'd define the values
to have real meaning. Eg. ((dst_factor << whatever) | src_factor)

As far as pre-multiplication goes, I think a separate bool property would
be good enough. And I've always found it easier to think in terms of
which operations the hardware will perform, so I'd call it something
like "src_premultiply", and when it's true the hardware will do an extra
premultiplication step before blending, and when it's false no extra step
is performed.

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel