Re: [Intel-gfx] [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline

[Pekka Paalanen <ppaalanen@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 7966 bytes --]

On Thu, 7 Sep 2023 12:31:47 +0000
"Shankar, Uma" <uma.shankar@intel.com> wrote:

> > -----Original Message-----
> > From: Pekka Paalanen <ppaalanen@gmail.com>
> > Sent: Tuesday, September 5, 2023 5:03 PM
> > To: Shankar, Uma <uma.shankar@intel.com>
> > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org; wayland-
> > devel@lists.freedesktop.org; Harry Wentland <harry.wentland@amd.com>;
> > Sebastian Wick <sebastian.wick@redhat.com>; ville.syrjala@linux.intel.com;
> > Jonas Adahl <jadahl@redhat.com>
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color Pipeline
> > 
> > On Mon, 4 Sep 2023 13:44:49 +0000
> > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> >   
> > > > -----Original Message-----
> > > > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf
> > > > Of Pekka Paalanen
> > > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > > To: Shankar, Uma <uma.shankar@intel.com>
> > > > Cc: intel-gfx@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > > <chaitanya.kumar.borah@intel.com>; dri-devel@lists.freedesktop.org;
> > > > wayland- devel@lists.freedesktop.org
> > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > > Plane Color Pipeline
> > > >
> > > > On Wed, 30 Aug 2023 08:59:36 +0000
> > > > "Shankar, Uma" <uma.shankar@intel.com> wrote:
> > > >  
> > > > > > -----Original Message-----
> > > > > > From: Harry Wentland <harry.wentland@amd.com>
> > > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > > To: Shankar, Uma <uma.shankar@intel.com>;
> > > > > > intel-gfx@lists.freedesktop.org; dri-
> > > > > > devel@lists.freedesktop.org
> > > > > > Cc: Borah, Chaitanya Kumar <chaitanya.kumar.borah@intel.com>;
> > > > > > wayland- devel@lists.freedesktop.org
> > > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > > proposed Plane Color Pipeline
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 2023-08-29 12:03, Uma Shankar wrote:  
> > > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > > >
> > > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > > <chaitanya.kumar.borah@intel.com>
> > > > > > > Signed-off-by: Uma Shankar <uma.shankar@intel.com>
> > > > > > > ---
> > > > > > >   .../gpu/rfc/plane_color_pipeline.rst          | 394 ++++++++++++++++++
> > > > > > >   1 file changed, 394 insertions(+)
> > > > > > >   create mode 100644
> > > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > >
> > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..60ce515b6ea7
> > > > > > > --- /dev/null
> > > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst  
> > > >
> > > > ...
> > > >
> > > > Hi Uma!  
> > >
> > > Thanks Pekka for the feedback and useful inputs.  
> > 
> > Hi Uma,
> > 
> > sorry to say, but the overall feeling I get from this proposal is that it is just the
> > current color related KMS properties wrapped in a pipeline blob. That is not the
> > re-design I believe we are looking for, and the feeling is based on several details
> > that are just copied from the current property design. Also the "private" stuff has
> > to go.  
> 
> Hi Pekka,
> Ok, got the concerns in general.  We will try to evaluate in deeper detail the
> property based design and come back if there are some issues or inputs.
>  
> At Intel we don't need private as of now, but we thought of having an option to
> enable any custom hardware or vendor. But we can drop the same for now if
> community doesn't feel the need for it.
> 
> > All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap
> > distribution, and parametrized curves are good development, but right now we
> > are looking at things one step higher level: the overall color pipeline design and
> > how to represent any operation. Most of this series is considering details below
> > the current attention level, hence I'm paying attention only to the first few
> > patches.  
> 
> We will need to precisely describe the hardware in userspace. Number of luts, precision,
> segments etc.., we can't just pass EOTF's as enum from userspace and let driver put
> hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour.
> This will be needed as there are multiple colorspaces possible and even LUTS can be
> used to perform tone mapping. So, we need userspace to be able to program luts directly.

Hi Uma,

yes, we do need to expose freely programmable LUTs when hardware has
them. That's why I say it is good development.

However, this is not an either-or situation.

We must also be able to expose fixed-function curve blocks when
hardware has them. Please, do not confuse this with a descriptive
design. This is not about saying "this FB is using PQ encoding, convert
it to NNN for me".

This is about defining an operation, that is mathematically defined as
"the PQ EOTF with normalized domain and range", for example. This is
prescriptive, because the exact mathematical formula of the operation
is defined, and it does not depend on any properties of the block's
input or output. It contains no opinions on how to convert something to
another, like a conversion from a color system and viewing environment
to another does (e.g. sRGB<->BT.2100/PQ). Every driver exposing this
operation must implement it exactly the same, with a small allowed error
tolerance.

There are no limitations on how it can be used. Userspace can choose to
apply that formula on anything it likes, and use the result in any way
it likes, even if it is an utterly non-standard pipeline not making any
sense *to us*.

You could argue that an operation to "convert PQ to HLG" is also
prescriptive for example, and yes, it is if implementations have to
adhere to a single specific formula for it. But if implementations are
allowed choose any formula they believe is the best to implement that
operation, then it is descriptive: "I have PQ content, I want HLG out,
do whatever, I don't care".

If you really have a fixed-function hardware block that literally uses
one specific formula to convert PQ to HLG, you can and probably should
expose that as a colorop. It is prescriptive, because the formula is
fully known to userspace, and userspace will choose to use it for its
formula, and not because it converts PQ to HLG.

However, if you had to combine multiple hardware blocks to achieve the
PQ-to-HLG formula, then that does not make sense to expose as a colorop
(other than for backward compatibility if your previous hardware
generation had it as a single fixed-function block). In practise, it
would be too rigid to be useful in more than few specific situations.
It would be much more flexible to expose the actual hardware blocks you
have, and let userspace use them any way it likes. This is about
finding the right balance in abstraction for UAPI.

> This is something we must expose to userspace. We will check if this can be fitted in
> property based approach.

I'm sure it can be exposed. The trivial option is to define a colorop
with the specific operation type that defines a property that will hold
the LUT configuration blob you have designed.

You do not need to force literally everything into a million
properties, you can still use blobs where they make sense, like needing
a variable or great number of elements of some type.

As such, the overall color pipeline UAPI design has little effect
on how you would design your LUT UAPI structures.


Thanks,
pq

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]