[PATCH v3 1/4] media: v4l: dev-decoder: Add source change V4L2_EVENT_SRC_CH_COLORSPACE

[PATCH v3 1/4] media: v4l: dev-decoder: Add source change V4L2_EVENT_SRC_CH_COLORSPACE

[ming.qian]

From: Ming Qian <ming.qian@oss.nxp.com>

Add a new source change V4L2_EVENT_SRC_CH_COLORSPACE that
indicates colorspace change in the stream.
The change V4L2_EVENT_SRC_CH_RESOLUTION will always affect
the allocation, but V4L2_EVENT_SRC_CH_COLORSPACE won't.

Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
---
v3
- Improve the description according to comments

 .../userspace-api/media/v4l/vidioc-dqevent.rst       | 12 ++++++++++++
 .../userspace-api/media/videodev2.h.rst.exceptions   |  1 +
 include/uapi/linux/videodev2.h                       |  1 +
 3 files changed, 14 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst b/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
index 8db103760930..f317af57a92c 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
@@ -369,6 +369,18 @@ call.
 	loss of signal and so restarting streaming I/O is required in order for
 	the hardware to synchronize to the video signal.
 
+    * - ``V4L2_EVENT_SRC_CH_COLORSPACE``
+      - 0x0002
+      - This event gets triggered when a colorspace change is detected at
+	an input. This can come from a video decoder or a video receiver.
+	Applications will query the new colorspace information
+	(if any, the signal may also have been lost). If the signal is lost,
+	then that is a CH_RESOLUTION change, not CH_COLORSPACE.
+
+	For stateful decoders follow the guidelines in :ref:`decoder`.
+	If CH_COLORSPACE is set, but not CH_RESOLUTION, then only the
+	colorspace changed and there is no need to reallocate buffers.
+
 Return Value
 ============
 
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index 35d3456cc812..ac47c6d9448b 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -526,6 +526,7 @@ replace define V4L2_EVENT_CTRL_CH_RANGE ctrl-changes-flags
 replace define V4L2_EVENT_CTRL_CH_DIMENSIONS ctrl-changes-flags
 
 replace define V4L2_EVENT_SRC_CH_RESOLUTION src-changes-flags
+replace define V4L2_EVENT_SRC_CH_COLORSPACE src-changes-flags
 
 replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_motion_det`
 
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index c8cb2796130f..242242c8e57b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2559,6 +2559,7 @@ struct v4l2_event_frame_sync {
 };
 
 #define V4L2_EVENT_SRC_CH_RESOLUTION		(1 << 0)
+#define V4L2_EVENT_SRC_CH_COLORSPACE		(1 << 1)
 
 struct v4l2_event_src_change {
 	__u32 changes;
-- 
2.43.0-rc1

[PATCH v3 2/4] media: docs: dev-decoder: Trigger dynamic source change for colorspace

[ming.qian]

From: Ming Qian <ming.qian@oss.nxp.com>

If colorspace changes, the client needs to renegotiate the pipeline,
otherwise the decoded frame may not be displayed correctly.

When a colorspace change in the stream, the decoder sends a
V4L2_EVENT_SOURCE_CHANGE event with changes set to
V4L2_EVENT_SRC_CH_COLORSPACE. After client receive this source change
event, then client can switch to the correct stream setting. And each
frame can be displayed properly.

So add colorspace as a trigger parameter for dynamic resolution change.

Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
---
v2
- Add V4L2_EVENT_SRC_CH_COLORSPACE for colorspace source change event

 .../userspace-api/media/v4l/dev-decoder.rst     | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst b/Documentation/userspace-api/media/v4l/dev-decoder.rst
index ef8e8cf31f90..51d6da3eea4a 100644
--- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
+++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
@@ -784,8 +784,8 @@ before the sequence started. Last of the buffers will have the
 must check if there is any pending event and:
 
 * if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
-  ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution
-  Change` sequence needs to be followed,
+  ``V4L2_EVENT_SRC_CH_RESOLUTION`` or ``V4L2_EVENT_SRC_CH_COLORSPACE`` is pending,
+  the `Dynamic Resolution Change` sequence needs to be followed,
 
 * if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence needs
   to be followed.
@@ -932,13 +932,17 @@ reflected by corresponding queries):
 
 * the minimum number of buffers needed for decoding,
 
-* bit-depth of the bitstream has been changed.
+* bit-depth of the bitstream has been changed,
+
+* colorspace of the bitstream has been changed.
 
 Whenever that happens, the decoder must proceed as follows:
 
 1.  After encountering a resolution change in the stream, the decoder sends a
     ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
-    ``V4L2_EVENT_SRC_CH_RESOLUTION``.
+    ``V4L2_EVENT_SRC_CH_RESOLUTION``, or a colorspace change in the stream, the
+    decoder sends a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
+    ``V4L2_EVENT_SRC_CH_COLORSPACE``.
 
     .. important::
 
@@ -946,6 +950,11 @@ Whenever that happens, the decoder must proceed as follows:
        values applying to the stream after the resolution change, including
        queue formats, selection rectangles and controls.
 
+.. note::
+        A ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
+        ``V4L2_EVENT_SRC_CH_RESOLUTION`` will affect the allocation, but
+        ``V4L2_EVENT_SRC_CH_COLORSPACE`` won't.
+
 2.  The decoder will then process and decode all remaining buffers from before
     the resolution change point.
 
-- 
2.43.0-rc1

Re: [PATCH v3 2/4] media: docs: dev-decoder: Trigger dynamic source change for colorspace

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 4021 bytes --]

Hi Ming,

Le vendredi 18 avril 2025 à 16:54 +0800, ming.qian@oss.nxp.com a écrit :
> From: Ming Qian <ming.qian@oss.nxp.com>
> 
> If colorspace changes, the client needs to renegotiate the pipeline,
> otherwise the decoded frame may not be displayed correctly.
> 
> When a colorspace change in the stream, the decoder sends a
> V4L2_EVENT_SOURCE_CHANGE event with changes set to
> V4L2_EVENT_SRC_CH_COLORSPACE. After client receive this source change
> event, then client can switch to the correct stream setting. And each
> frame can be displayed properly.

sorry for the long delay. While reading this, in any case userspace have to read
the new format. Why can't userspace compare the old and new v4l2_format and
decide to avoid re-allocation that way ?

There is also backward compatbility issues for driver that was sending
V4L2_EVENT_SRC_CH_RESOLUTION for colorspace change before. Despite the costly
re-allocation, userspace only watching for V4L2_EVENT_SRC_CH_RESOLUTION would
endup not updating the colorspace anymore.

Combining both would be an option, but then V4L2_EVENT_SRC_CH_RESOLUTION means
any v4l2_format changes, which is awkward. What do you think of leaving to
userspace the task of comparing the old and new v4l2_format ?

Nicolas

> 
> So add colorspace as a trigger parameter for dynamic resolution change.
> 
> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
> ---
> v2
> - Add V4L2_EVENT_SRC_CH_COLORSPACE for colorspace source change event
> 
>  .../userspace-api/media/v4l/dev-decoder.rst     | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> index ef8e8cf31f90..51d6da3eea4a 100644
> --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> @@ -784,8 +784,8 @@ before the sequence started. Last of the buffers will have
> the
>  must check if there is any pending event and:
>  
>  * if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> -  ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution
> -  Change` sequence needs to be followed,
> +  ``V4L2_EVENT_SRC_CH_RESOLUTION`` or ``V4L2_EVENT_SRC_CH_COLORSPACE`` is
> pending,
> +  the `Dynamic Resolution Change` sequence needs to be followed,
>  
>  * if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence
> needs
>    to be followed.
> @@ -932,13 +932,17 @@ reflected by corresponding queries):
>  
>  * the minimum number of buffers needed for decoding,
>  
> -* bit-depth of the bitstream has been changed.
> +* bit-depth of the bitstream has been changed,
> +
> +* colorspace of the bitstream has been changed.
>  
>  Whenever that happens, the decoder must proceed as follows:
>  
>  1.  After encountering a resolution change in the stream, the decoder sends a
>      ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> -    ``V4L2_EVENT_SRC_CH_RESOLUTION``.
> +    ``V4L2_EVENT_SRC_CH_RESOLUTION``, or a colorspace change in the stream,
> the
> +    decoder sends a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set
> to
> +    ``V4L2_EVENT_SRC_CH_COLORSPACE``.
>  
>      .. important::
>  
> @@ -946,6 +950,11 @@ Whenever that happens, the decoder must proceed as
> follows:
>         values applying to the stream after the resolution change, including
>         queue formats, selection rectangles and controls.
>  
> +.. note::
> +        A ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> +        ``V4L2_EVENT_SRC_CH_RESOLUTION`` will affect the allocation, but
> +        ``V4L2_EVENT_SRC_CH_COLORSPACE`` won't.
> +
>  2.  The decoder will then process and decode all remaining buffers from
> before
>      the resolution change point.
>  

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v3 2/4] media: docs: dev-decoder: Trigger dynamic source change for colorspace

[Ming Qian(OSS)]

Hi Nicolas,

On 8/1/2025 11:23 PM, Nicolas Dufresne wrote:
> Hi Ming,
> 
> Le vendredi 18 avril 2025 à 16:54 +0800, ming.qian@oss.nxp.com a écrit :
>> From: Ming Qian <ming.qian@oss.nxp.com>
>>
>> If colorspace changes, the client needs to renegotiate the pipeline,
>> otherwise the decoded frame may not be displayed correctly.
>>
>> When a colorspace change in the stream, the decoder sends a
>> V4L2_EVENT_SOURCE_CHANGE event with changes set to
>> V4L2_EVENT_SRC_CH_COLORSPACE. After client receive this source change
>> event, then client can switch to the correct stream setting. And each
>> frame can be displayed properly.
> 
> sorry for the long delay. While reading this, in any case userspace have to read
> the new format. Why can't userspace compare the old and new v4l2_format and
> decide to avoid re-allocation that way ?
> 
> There is also backward compatbility issues for driver that was sending
> V4L2_EVENT_SRC_CH_RESOLUTION for colorspace change before. Despite the costly
> re-allocation, userspace only watching for V4L2_EVENT_SRC_CH_RESOLUTION would
> endup not updating the colorspace anymore.
> 
> Combining both would be an option, but then V4L2_EVENT_SRC_CH_RESOLUTION means
> any v4l2_format changes, which is awkward. What do you think of leaving to
> userspace the task of comparing the old and new v4l2_format ?
> 
> Nicolas
> 

Sorry for the delay response (I don't understand why I received this
email several months late.)

I agree that comparing the old and new v4l2_format is feasible. And
there are currently four conditions that can trigger source change.
- coded resolution (OUTPUT width and height),
- visible resolution (selection rectangles),
- the minimum number of buffers needed for decoding,
- bit-depth of the bitstream has been changed.

Therefore, comparing only v4l2_format is insufficient. This is much more
complicated than checking a flag. I'm not sure if existing applications
have already done this.

I also think it's OK for driver to send V4L2_EVENT_SRC_CH_RESOLUTION for
colorspace change, This will only incur additional allocation overhead,
without causing any functional issues.

Therefore, from a driver perspective:
- V4L2_EVENT_SRC_CH_RESOLUTION for format change      OK
- V4L2_EVENT_SRC_CH_RESOLUTION for colorspace chagne  OK, but 
re-allocation cost
- V4L2_EVENT_SOURCE_CHANGE for colorspace change      OK

from a client perspective:
- V4L2_EVENT_SRC_CH_RESOLUTION without compareing format    OK, 
re-allocation
- V4L2_EVENT_SRC_CH_RESOLUTION with comparing format        OK, 
additional support required
- V4L2_EVENT_SOURCE_CHANGE                                  OK, 
additional support required


I believe that adding a V4L2_EVENT_SOURCE_CHANGE flag will help simplify
the process and will not cause too many backward compatibility issues.

Regards,
Ming

>>
>> So add colorspace as a trigger parameter for dynamic resolution change.
>>
>> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
>> ---
>> v2
>> - Add V4L2_EVENT_SRC_CH_COLORSPACE for colorspace source change event
>>
>>   .../userspace-api/media/v4l/dev-decoder.rst     | 17 +++++++++++++----
>>   1 file changed, 13 insertions(+), 4 deletions(-)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>> b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>> index ef8e8cf31f90..51d6da3eea4a 100644
>> --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>> +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>> @@ -784,8 +784,8 @@ before the sequence started. Last of the buffers will have
>> the
>>   must check if there is any pending event and:
>>   
>>   * if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>> -  ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution
>> -  Change` sequence needs to be followed,
>> +  ``V4L2_EVENT_SRC_CH_RESOLUTION`` or ``V4L2_EVENT_SRC_CH_COLORSPACE`` is
>> pending,
>> +  the `Dynamic Resolution Change` sequence needs to be followed,
>>   
>>   * if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence
>> needs
>>     to be followed.
>> @@ -932,13 +932,17 @@ reflected by corresponding queries):
>>   
>>   * the minimum number of buffers needed for decoding,
>>   
>> -* bit-depth of the bitstream has been changed.
>> +* bit-depth of the bitstream has been changed,
>> +
>> +* colorspace of the bitstream has been changed.
>>   
>>   Whenever that happens, the decoder must proceed as follows:
>>   
>>   1.  After encountering a resolution change in the stream, the decoder sends a
>>       ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>> -    ``V4L2_EVENT_SRC_CH_RESOLUTION``.
>> +    ``V4L2_EVENT_SRC_CH_RESOLUTION``, or a colorspace change in the stream,
>> the
>> +    decoder sends a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set
>> to
>> +    ``V4L2_EVENT_SRC_CH_COLORSPACE``.
>>   
>>       .. important::
>>   
>> @@ -946,6 +950,11 @@ Whenever that happens, the decoder must proceed as
>> follows:
>>          values applying to the stream after the resolution change, including
>>          queue formats, selection rectangles and controls.
>>   
>> +.. note::
>> +        A ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>> +        ``V4L2_EVENT_SRC_CH_RESOLUTION`` will affect the allocation, but
>> +        ``V4L2_EVENT_SRC_CH_COLORSPACE`` won't.
>> +
>>   2.  The decoder will then process and decode all remaining buffers from
>> before
>>       the resolution change point.
>>   

Re: [PATCH v3 2/4] media: docs: dev-decoder: Trigger dynamic source change for colorspace

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 7553 bytes --]

Hi,

I'm reviewing my backlog and this one has been stalled.

Le lundi 03 novembre 2025 à 09:45 +0800, Ming Qian(OSS) a écrit :
> Hi Nicolas,
> 
> On 8/1/2025 11:23 PM, Nicolas Dufresne wrote:
> > Hi Ming,
> > 
> > Le vendredi 18 avril 2025 à 16:54 +0800, ming.qian@oss.nxp.com a écrit :
> > > From: Ming Qian <ming.qian@oss.nxp.com>
> > > 
> > > If colorspace changes, the client needs to renegotiate the pipeline,
> > > otherwise the decoded frame may not be displayed correctly.
> > > 
> > > When a colorspace change in the stream, the decoder sends a
> > > V4L2_EVENT_SOURCE_CHANGE event with changes set to
> > > V4L2_EVENT_SRC_CH_COLORSPACE. After client receive this source change
> > > event, then client can switch to the correct stream setting. And each
> > > frame can be displayed properly.
> > 
> > sorry for the long delay. While reading this, in any case userspace have to read
> > the new format. Why can't userspace compare the old and new v4l2_format and
> > decide to avoid re-allocation that way ?
> > 
> > There is also backward compatbility issues for driver that was sending
> > V4L2_EVENT_SRC_CH_RESOLUTION for colorspace change before. Despite the costly
> > re-allocation, userspace only watching for V4L2_EVENT_SRC_CH_RESOLUTION would
> > endup not updating the colorspace anymore.
> > 
> > Combining both would be an option, but then V4L2_EVENT_SRC_CH_RESOLUTION means
> > any v4l2_format changes, which is awkward. What do you think of leaving to
> > userspace the task of comparing the old and new v4l2_format ?
> > 
> > Nicolas
> > 
> 
> Sorry for the delay response (I don't understand why I received this
> email several months late.)
> 
> I agree that comparing the old and new v4l2_format is feasible. And
> there are currently four conditions that can trigger source change.
> - coded resolution (OUTPUT width and height),
> - visible resolution (selection rectangles),
> - the minimum number of buffers needed for decoding,
> - bit-depth of the bitstream has been changed.
> 
> Therefore, comparing only v4l2_format is insufficient. This is much more
> complicated than checking a flag. I'm not sure if existing applications
> have already done this.
> 
> I also think it's OK for driver to send V4L2_EVENT_SRC_CH_RESOLUTION for
> colorspace change, This will only incur additional allocation overhead,
> without causing any functional issues.
> 
> Therefore, from a driver perspective:
> - V4L2_EVENT_SRC_CH_RESOLUTION for format change      OK
> - V4L2_EVENT_SRC_CH_RESOLUTION for colorspace chagne  OK, but 
> re-allocation cost
> - V4L2_EVENT_SOURCE_CHANGE for colorspace change      OK
> 
> from a client perspective:
> - V4L2_EVENT_SRC_CH_RESOLUTION without compareing format    OK, 
> re-allocation
> - V4L2_EVENT_SRC_CH_RESOLUTION with comparing format        OK, 
> additional support required
> - V4L2_EVENT_SOURCE_CHANGE                                  OK, 
> additional support required
> 
> 
> I believe that adding a V4L2_EVENT_SOURCE_CHANGE flag will help simplify
> the process and will not cause too many backward compatibility issues.

So stepping back a little, the point is the V4L2_EVENT_SRC_CH_RESOLUTION is not
clear enough. For backward compatibility reason we have to keep it though. So
that gives two choices:

1. We don't add new API, and its up to clients to check what actually changes.
Documentation to help this would be nice.

This is easy driver side, you just emit more SOURCE_CHANGE event, and get a
performance hit on client that have not been updated and just naively re-
allocate.

2. We keep V4L2_EVENT_SRC_CH_RESOLUTION, but introduce SRC_CH_REALLOC,
SRC_CH_COLORSPACE, ...

We still endup seding more events. If the flags are exactly 
V4L2_EVENT_SRC_CH_RESOLUTION, we have a legacy driver and can't assume anything.
Just realloc or check in the format details. Then if one or more flags of the
other are present, use this in order to minimize the work.

In all cases, a drain operation must happen, otherwise you would not know at
which boundary this change takes place. I will mark this as "Change Requested"
so it is removed from my queue. If you want to continue this work, I think this
is best plan I can think of with consideration for backward compatibility. If
you go for 1, no new API is needed, but perhaps some better doc would help.

Nicolas


> 
> Regards,
> Ming
> 
> > > 
> > > So add colorspace as a trigger parameter for dynamic resolution change.
> > > 
> > > Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
> > > ---
> > > v2
> > > - Add V4L2_EVENT_SRC_CH_COLORSPACE for colorspace source change event
> > > 
> > >   .../userspace-api/media/v4l/dev-decoder.rst     | 17 +++++++++++++----
> > >   1 file changed, 13 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > index ef8e8cf31f90..51d6da3eea4a 100644
> > > --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
> > > @@ -784,8 +784,8 @@ before the sequence started. Last of the buffers will have
> > > the
> > >   must check if there is any pending event and:
> > >   
> > >   * if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> > > -  ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution
> > > -  Change` sequence needs to be followed,
> > > +  ``V4L2_EVENT_SRC_CH_RESOLUTION`` or ``V4L2_EVENT_SRC_CH_COLORSPACE`` is
> > > pending,
> > > +  the `Dynamic Resolution Change` sequence needs to be followed,
> > >   
> > >   * if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence
> > > needs
> > >     to be followed.
> > > @@ -932,13 +932,17 @@ reflected by corresponding queries):
> > >   
> > >   * the minimum number of buffers needed for decoding,
> > >   
> > > -* bit-depth of the bitstream has been changed.
> > > +* bit-depth of the bitstream has been changed,
> > > +
> > > +* colorspace of the bitstream has been changed.
> > >   
> > >   Whenever that happens, the decoder must proceed as follows:
> > >   
> > >   1.  After encountering a resolution change in the stream, the decoder sends a
> > >       ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> > > -    ``V4L2_EVENT_SRC_CH_RESOLUTION``.
> > > +    ``V4L2_EVENT_SRC_CH_RESOLUTION``, or a colorspace change in the stream,
> > > the
> > > +    decoder sends a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set
> > > to
> > > +    ``V4L2_EVENT_SRC_CH_COLORSPACE``.
> > >   
> > >       .. important::
> > >   
> > > @@ -946,6 +950,11 @@ Whenever that happens, the decoder must proceed as
> > > follows:
> > >          values applying to the stream after the resolution change, including
> > >          queue formats, selection rectangles and controls.
> > >   
> > > +.. note::
> > > +        A ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
> > > +        ``V4L2_EVENT_SRC_CH_RESOLUTION`` will affect the allocation, but
> > > +        ``V4L2_EVENT_SRC_CH_COLORSPACE`` won't.
> > > +
> > >   2.  The decoder will then process and decode all remaining buffers from
> > > before
> > >       the resolution change point.
> > >   

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v3 2/4] media: docs: dev-decoder: Trigger dynamic source change for colorspace

[Ming Qian(OSS)]

Hi Nicolas，

On 11/7/2025 10:50 PM, Nicolas Dufresne wrote:
> Hi,
> 
> I'm reviewing my backlog and this one has been stalled.
> 
> Le lundi 03 novembre 2025 à 09:45 +0800, Ming Qian(OSS) a écrit :
>> Hi Nicolas,
>>
>> On 8/1/2025 11:23 PM, Nicolas Dufresne wrote:
>>> Hi Ming,
>>>
>>> Le vendredi 18 avril 2025 à 16:54 +0800, ming.qian@oss.nxp.com a écrit :
>>>> From: Ming Qian <ming.qian@oss.nxp.com>
>>>>
>>>> If colorspace changes, the client needs to renegotiate the pipeline,
>>>> otherwise the decoded frame may not be displayed correctly.
>>>>
>>>> When a colorspace change in the stream, the decoder sends a
>>>> V4L2_EVENT_SOURCE_CHANGE event with changes set to
>>>> V4L2_EVENT_SRC_CH_COLORSPACE. After client receive this source change
>>>> event, then client can switch to the correct stream setting. And each
>>>> frame can be displayed properly.
>>>
>>> sorry for the long delay. While reading this, in any case userspace have to read
>>> the new format. Why can't userspace compare the old and new v4l2_format and
>>> decide to avoid re-allocation that way ?
>>>
>>> There is also backward compatbility issues for driver that was sending
>>> V4L2_EVENT_SRC_CH_RESOLUTION for colorspace change before. Despite the costly
>>> re-allocation, userspace only watching for V4L2_EVENT_SRC_CH_RESOLUTION would
>>> endup not updating the colorspace anymore.
>>>
>>> Combining both would be an option, but then V4L2_EVENT_SRC_CH_RESOLUTION means
>>> any v4l2_format changes, which is awkward. What do you think of leaving to
>>> userspace the task of comparing the old and new v4l2_format ?
>>>
>>> Nicolas
>>>
>>
>> Sorry for the delay response (I don't understand why I received this
>> email several months late.)
>>
>> I agree that comparing the old and new v4l2_format is feasible. And
>> there are currently four conditions that can trigger source change.
>> - coded resolution (OUTPUT width and height),
>> - visible resolution (selection rectangles),
>> - the minimum number of buffers needed for decoding,
>> - bit-depth of the bitstream has been changed.
>>
>> Therefore, comparing only v4l2_format is insufficient. This is much more
>> complicated than checking a flag. I'm not sure if existing applications
>> have already done this.
>>
>> I also think it's OK for driver to send V4L2_EVENT_SRC_CH_RESOLUTION for
>> colorspace change, This will only incur additional allocation overhead,
>> without causing any functional issues.
>>
>> Therefore, from a driver perspective:
>> - V4L2_EVENT_SRC_CH_RESOLUTION for format change      OK
>> - V4L2_EVENT_SRC_CH_RESOLUTION for colorspace chagne  OK, but
>> re-allocation cost
>> - V4L2_EVENT_SOURCE_CHANGE for colorspace change      OK
>>
>> from a client perspective:
>> - V4L2_EVENT_SRC_CH_RESOLUTION without compareing format    OK,
>> re-allocation
>> - V4L2_EVENT_SRC_CH_RESOLUTION with comparing format        OK,
>> additional support required
>> - V4L2_EVENT_SOURCE_CHANGE                                  OK,
>> additional support required
>>
>>
>> I believe that adding a V4L2_EVENT_SOURCE_CHANGE flag will help simplify
>> the process and will not cause too many backward compatibility issues.
> 
> So stepping back a little, the point is the V4L2_EVENT_SRC_CH_RESOLUTION is not
> clear enough. For backward compatibility reason we have to keep it though. So
> that gives two choices:
> 
> 1. We don't add new API, and its up to clients to check what actually changes.
> Documentation to help this would be nice.
> 
> This is easy driver side, you just emit more SOURCE_CHANGE event, and get a
> performance hit on client that have not been updated and just naively re-
> allocate.
> 
> 2. We keep V4L2_EVENT_SRC_CH_RESOLUTION, but introduce SRC_CH_REALLOC,
> SRC_CH_COLORSPACE, ...
> 
> We still endup seding more events. If the flags are exactly
> V4L2_EVENT_SRC_CH_RESOLUTION, we have a legacy driver and can't assume anything.
> Just realloc or check in the format details. Then if one or more flags of the
> other are present, use this in order to minimize the work.
> 
> In all cases, a drain operation must happen, otherwise you would not know at
> which boundary this change takes place. I will mark this as "Change Requested"
> so it is removed from my queue. If you want to continue this work, I think this
> is best plan I can think of with consideration for backward compatibility. If
> you go for 1, no new API is needed, but perhaps some better doc would help.
> 
> Nicolas
> 
> 

Thank you for your explanation. I agree that method 1 is more reasonable
and easier.
I'll prepare the V4 driver patch based on the method 1.

Regards,
Ming

>>
>> Regards,
>> Ming
>>
>>>>
>>>> So add colorspace as a trigger parameter for dynamic resolution change.
>>>>
>>>> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
>>>> ---
>>>> v2
>>>> - Add V4L2_EVENT_SRC_CH_COLORSPACE for colorspace source change event
>>>>
>>>>    .../userspace-api/media/v4l/dev-decoder.rst     | 17 +++++++++++++----
>>>>    1 file changed, 13 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>> b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>> index ef8e8cf31f90..51d6da3eea4a 100644
>>>> --- a/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/dev-decoder.rst
>>>> @@ -784,8 +784,8 @@ before the sequence started. Last of the buffers will have
>>>> the
>>>>    must check if there is any pending event and:
>>>>    
>>>>    * if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>>>> -  ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution
>>>> -  Change` sequence needs to be followed,
>>>> +  ``V4L2_EVENT_SRC_CH_RESOLUTION`` or ``V4L2_EVENT_SRC_CH_COLORSPACE`` is
>>>> pending,
>>>> +  the `Dynamic Resolution Change` sequence needs to be followed,
>>>>    
>>>>    * if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence
>>>> needs
>>>>      to be followed.
>>>> @@ -932,13 +932,17 @@ reflected by corresponding queries):
>>>>    
>>>>    * the minimum number of buffers needed for decoding,
>>>>    
>>>> -* bit-depth of the bitstream has been changed.
>>>> +* bit-depth of the bitstream has been changed,
>>>> +
>>>> +* colorspace of the bitstream has been changed.
>>>>    
>>>>    Whenever that happens, the decoder must proceed as follows:
>>>>    
>>>>    1.  After encountering a resolution change in the stream, the decoder sends a
>>>>        ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>>>> -    ``V4L2_EVENT_SRC_CH_RESOLUTION``.
>>>> +    ``V4L2_EVENT_SRC_CH_RESOLUTION``, or a colorspace change in the stream,
>>>> the
>>>> +    decoder sends a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set
>>>> to
>>>> +    ``V4L2_EVENT_SRC_CH_COLORSPACE``.
>>>>    
>>>>        .. important::
>>>>    
>>>> @@ -946,6 +950,11 @@ Whenever that happens, the decoder must proceed as
>>>> follows:
>>>>           values applying to the stream after the resolution change, including
>>>>           queue formats, selection rectangles and controls.
>>>>    
>>>> +.. note::
>>>> +        A ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to
>>>> +        ``V4L2_EVENT_SRC_CH_RESOLUTION`` will affect the allocation, but
>>>> +        ``V4L2_EVENT_SRC_CH_COLORSPACE`` won't.
>>>> +
>>>>    2.  The decoder will then process and decode all remaining buffers from
>>>> before
>>>>        the resolution change point.
>>>>    

[PATCH v3 3/4] media: amphion: Clear last_buffer_dequeued flag for DEC_CMD_START

[ming.qian]

From: Ming Qian <ming.qian@oss.nxp.com>

The V4L2_DEC_CMD_START command may be used to handle the dynamic source
change, which will triggers an implicit decoder drain.
The last_buffer_dequeued flag is set in the implicit decoder drain,
so driver need to clear it to continue the following decoding flow.

Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
---
 drivers/media/platform/amphion/vdec.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
index d812b86f001e..ee0a7572ed15 100644
--- a/drivers/media/platform/amphion/vdec.c
+++ b/drivers/media/platform/amphion/vdec.c
@@ -728,6 +728,7 @@ static int vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd
 	switch (cmd->cmd) {
 	case V4L2_DEC_CMD_START:
 		vdec_cmd_start(inst);
+		vb2_clear_last_buffer_dequeued(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx));
 		break;
 	case V4L2_DEC_CMD_STOP:
 		vdec_cmd_stop(inst);
-- 
2.43.0-rc1

[PATCH v3 4/4] media: amphion: Trigger source change if colorspace chagned

[ming.qian]

From: Ming Qian <ming.qian@oss.nxp.com>

After encountering a colorspace change in the stream, the decoder
sends a V4L2_EVENT_SOURCE_CHANGE event with changes set to
V4L2_EVENT_SRC_CH_COLORSPACE.

Then the client can handle the colorspace change without any buffer
reallocation

Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
---
 drivers/media/platform/amphion/vdec.c     | 89 +++++++++++++++--------
 drivers/media/platform/amphion/vpu.h      |  1 +
 drivers/media/platform/amphion/vpu_v4l2.c |  7 +-
 3 files changed, 63 insertions(+), 34 deletions(-)

diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
index ee0a7572ed15..f4979d537b97 100644
--- a/drivers/media/platform/amphion/vdec.c
+++ b/drivers/media/platform/amphion/vdec.c
@@ -369,12 +369,16 @@ static void vdec_handle_resolution_change(struct vpu_inst *inst)
 	if (!vdec->source_change)
 		return;
 
+	if (inst->changes) {
+		vpu_notify_source_change(inst);
+		inst->changes = 0;
+	}
+
 	q = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
 	if (!list_empty(&q->done_list))
 		return;
 
 	vdec->source_change--;
-	vpu_notify_source_change(inst);
 	vpu_set_last_buffer_dequeued(inst, false);
 }
 
@@ -954,10 +958,11 @@ static void vdec_stop_done(struct vpu_inst *inst)
 	vpu_inst_unlock(inst);
 }
 
-static bool vdec_check_source_change(struct vpu_inst *inst)
+static bool vdec_check_source_change(struct vpu_inst *inst, struct vpu_dec_codec_info *hdr)
 {
 	struct vdec_t *vdec = inst->priv;
 	const struct vpu_format *sibling;
+	u32 changes = 0;
 
 	if (!inst->fh.m2m_ctx)
 		return false;
@@ -966,27 +971,41 @@ static bool vdec_check_source_change(struct vpu_inst *inst)
 		return false;
 
 	sibling = vpu_helper_find_sibling(inst, inst->cap_format.type, inst->cap_format.pixfmt);
-	if (sibling && vdec->codec_info.pixfmt == sibling->pixfmt)
-		vdec->codec_info.pixfmt = inst->cap_format.pixfmt;
+	if (sibling && hdr->pixfmt == sibling->pixfmt)
+		hdr->pixfmt = inst->cap_format.pixfmt;
 
 	if (!vb2_is_streaming(v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx)))
-		return true;
-	if (inst->cap_format.pixfmt != vdec->codec_info.pixfmt)
-		return true;
-	if (inst->cap_format.width != vdec->codec_info.decoded_width)
-		return true;
-	if (inst->cap_format.height != vdec->codec_info.decoded_height)
-		return true;
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->cap_format.pixfmt != hdr->pixfmt)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->cap_format.width != hdr->decoded_width)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->cap_format.height != hdr->decoded_height)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
 	if (vpu_get_num_buffers(inst, inst->cap_format.type) < inst->min_buffer_cap)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->crop.left != hdr->offset_x)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->crop.top != hdr->offset_y)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->crop.width != hdr->width)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (inst->crop.height != hdr->height)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+	if (!hdr->progressive)
+		changes |= V4L2_EVENT_SRC_CH_RESOLUTION;
+
+	if (vdec->seq_hdr_found &&
+	    (hdr->color_primaries != vdec->codec_info.color_primaries ||
+	     hdr->transfer_chars != vdec->codec_info.transfer_chars ||
+	     hdr->matrix_coeffs != vdec->codec_info.matrix_coeffs ||
+	     hdr->full_range != vdec->codec_info.full_range))
+		changes |= V4L2_EVENT_SRC_CH_COLORSPACE;
+
+	if (changes) {
+		inst->changes |= changes;
 		return true;
-	if (inst->crop.left != vdec->codec_info.offset_x)
-		return true;
-	if (inst->crop.top != vdec->codec_info.offset_y)
-		return true;
-	if (inst->crop.width != vdec->codec_info.width)
-		return true;
-	if (inst->crop.height != vdec->codec_info.height)
-		return true;
+	}
 
 	return false;
 }
@@ -1337,20 +1356,25 @@ static void vdec_event_seq_hdr(struct vpu_inst *inst, struct vpu_dec_codec_info
 	struct vdec_t *vdec = inst->priv;
 
 	vpu_inst_lock(inst);
-	memcpy(&vdec->codec_info, hdr, sizeof(vdec->codec_info));
 
-	vpu_trace(inst->dev, "[%d] %d x %d, crop : (%d, %d) %d x %d, %d, %d\n",
+	vpu_trace(inst->dev,
+		  "[%d] %d x %d, crop : (%d, %d) %d x %d, %d, %d, colorspace: %d, %d, %d, %d\n",
 		  inst->id,
-		  vdec->codec_info.decoded_width,
-		  vdec->codec_info.decoded_height,
-		  vdec->codec_info.offset_x,
-		  vdec->codec_info.offset_y,
-		  vdec->codec_info.width,
-		  vdec->codec_info.height,
+		  hdr->decoded_width,
+		  hdr->decoded_height,
+		  hdr->offset_x,
+		  hdr->offset_y,
+		  hdr->width,
+		  hdr->height,
 		  hdr->num_ref_frms,
-		  hdr->num_dpb_frms);
+		  hdr->num_dpb_frms,
+		  hdr->color_primaries,
+		  hdr->transfer_chars,
+		  hdr->matrix_coeffs,
+		  hdr->full_range);
 	inst->min_buffer_cap = hdr->num_ref_frms + hdr->num_dpb_frms;
-	vdec->is_source_changed = vdec_check_source_change(inst);
+	vdec->is_source_changed = vdec_check_source_change(inst, hdr);
+	memcpy(&vdec->codec_info, hdr, sizeof(vdec->codec_info));
 	vdec_init_fmt(inst);
 	vdec_init_crop(inst);
 	vdec_init_mbi(inst);
@@ -1379,7 +1403,12 @@ static void vdec_event_resolution_change(struct vpu_inst *inst)
 {
 	struct vdec_t *vdec = inst->priv;
 
-	vpu_trace(inst->dev, "[%d]\n", inst->id);
+	vpu_trace(inst->dev, "[%d] input : %d, decoded : %d, display : %d, sequence : %d\n",
+		  inst->id,
+		  vdec->params.frame_count,
+		  vdec->decoded_frame_count,
+		  vdec->display_frame_count,
+		  vdec->sequence);
 	vpu_inst_lock(inst);
 	vdec->seq_tag = vdec->codec_info.tag;
 	vdec_clear_fs(&vdec->mbi);
diff --git a/drivers/media/platform/amphion/vpu.h b/drivers/media/platform/amphion/vpu.h
index 76bfd6b26170..d8100da160d1 100644
--- a/drivers/media/platform/amphion/vpu.h
+++ b/drivers/media/platform/amphion/vpu.h
@@ -272,6 +272,7 @@ struct vpu_inst {
 	u8 xfer_func;
 	u32 sequence;
 	u32 extra_size;
+	u32 changes;
 
 	u32 flows[16];
 	u32 flow_idx;
diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/platform/amphion/vpu_v4l2.c
index 74668fa362e2..37ef706c29dd 100644
--- a/drivers/media/platform/amphion/vpu_v4l2.c
+++ b/drivers/media/platform/amphion/vpu_v4l2.c
@@ -96,13 +96,12 @@ int vpu_notify_eos(struct vpu_inst *inst)
 
 int vpu_notify_source_change(struct vpu_inst *inst)
 {
-	static const struct v4l2_event ev = {
-		.id = 0,
+	const struct v4l2_event ev = {
 		.type = V4L2_EVENT_SOURCE_CHANGE,
-		.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION
+		.u.src_change.changes = inst->changes
 	};
 
-	vpu_trace(inst->dev, "[%d]\n", inst->id);
+	vpu_trace(inst->dev, "[%d] source change 0x%x\n", inst->id, inst->changes);
 	v4l2_event_queue_fh(&inst->fh, &ev);
 	return 0;
 }
-- 
2.43.0-rc1

Re: [PATCH v3 1/4] media: v4l: dev-decoder: Add source change V4L2_EVENT_SRC_CH_COLORSPACE

[Frank Li]

On Fri, Apr 18, 2025 at 04:54:16PM +0800, ming.qian@oss.nxp.com wrote:
> From: Ming Qian <ming.qian@oss.nxp.com>
>
> Add a new source change V4L2_EVENT_SRC_CH_COLORSPACE that
                   change event V4l2...

> indicates colorspace change in the stream.

wrap at 75 char and add empty line between paragraph.

> The change V4L2_EVENT_SRC_CH_RESOLUTION will always affect
  The event ...

Frank
> the allocation, but V4L2_EVENT_SRC_CH_COLORSPACE won't.
>
> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com>
> ---
> v3
> - Improve the description according to comments
>
>  .../userspace-api/media/v4l/vidioc-dqevent.rst       | 12 ++++++++++++
>  .../userspace-api/media/videodev2.h.rst.exceptions   |  1 +
>  include/uapi/linux/videodev2.h                       |  1 +
>  3 files changed, 14 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst b/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
> index 8db103760930..f317af57a92c 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-dqevent.rst
> @@ -369,6 +369,18 @@ call.
>  	loss of signal and so restarting streaming I/O is required in order for
>  	the hardware to synchronize to the video signal.
>
> +    * - ``V4L2_EVENT_SRC_CH_COLORSPACE``
> +      - 0x0002
> +      - This event gets triggered when a colorspace change is detected at
> +	an input. This can come from a video decoder or a video receiver.
> +	Applications will query the new colorspace information
> +	(if any, the signal may also have been lost). If the signal is lost,
> +	then that is a CH_RESOLUTION change, not CH_COLORSPACE.
> +
> +	For stateful decoders follow the guidelines in :ref:`decoder`.
> +	If CH_COLORSPACE is set, but not CH_RESOLUTION, then only the
> +	colorspace changed and there is no need to reallocate buffers.
> +
>  Return Value
>  ============
>
> diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
> index 35d3456cc812..ac47c6d9448b 100644
> --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
> +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
> @@ -526,6 +526,7 @@ replace define V4L2_EVENT_CTRL_CH_RANGE ctrl-changes-flags
>  replace define V4L2_EVENT_CTRL_CH_DIMENSIONS ctrl-changes-flags
>
>  replace define V4L2_EVENT_SRC_CH_RESOLUTION src-changes-flags
> +replace define V4L2_EVENT_SRC_CH_COLORSPACE src-changes-flags
>
>  replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_motion_det`
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c8cb2796130f..242242c8e57b 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2559,6 +2559,7 @@ struct v4l2_event_frame_sync {
>  };
>
>  #define V4L2_EVENT_SRC_CH_RESOLUTION		(1 << 0)
> +#define V4L2_EVENT_SRC_CH_COLORSPACE		(1 << 1)
>
>  struct v4l2_event_src_change {
>  	__u32 changes;
> --
> 2.43.0-rc1
>