Re: Suggestion regarding x8 gang mode device tree changes

Re: Suggestion regarding x8 gang mode device tree changes

[Hans Verkuil]

On 29/10/2020 02:48, Sowjanya Komatineni wrote:
> Hi Sakari,
> 
> Missed to add you to below patch series for HDMI2CSI bridge support
> 
> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
> 
> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
> 
> Would like to get your suggestion on x8 gang/combined ports capture implementation.
> 
> 
> Tegra VI/CSI don't have native x8. HDMI2CSI bridges uses dual ports with split image on to each port.
> So on Tegra side at SW level we can use dual ports as gang where both ports are simultaneously programmed for combined image capture.
> 
> Currently v2 patches use bus-width with V4L2 parallel bus type for this x8 gang mode implementation.
> So, driver parses endpoint and if bus type is parallel (for x8), it uses v4l2_ep.bus.parallel.bus_width other wise (x4 and below) it uses
> v4l2_ep.bus.mipi_csi2.num_data_lanes for getting lanes from DT.
> 
> x8 is not from native HW and current v2 version does not explicitly add 2nd port in device tree and driver uses consecutive ports as
> combined capture for same video device node.
> 
> From offline discussion with Hans, looks like its better to explicitly specify both ports used as gang/combined capture in device tree on
> Tegra side and bridge side also will expose both TX ports.
> 
> Current CSI driver implementation uses max 2 media pads port@0 as Sink and port@1 as Source. So can add 2nd endpoint in sink node and source
> nodes.
> 
> Below is tc358840 and csi device node change to explicitly add second source.
> 
> With this, VI/CSI driver can be updated to use second port only for identifying total combined ports for capture.
> 
> My understanding is there is no need for creating media links for 2nd port as both ports together are used as combined ports by the Tegra
> driver during streaming for the same csi subdev and video device nodes.

In my opinion both links should be shown in the media controller topology.
Basically these are just independent CSI ports (two independent TX ports on
the tc358840 and two independent RX ports on the Tegra SoC side), that are
combined afterwards into a single video stream.

While this is an HDMI receiver, it is conceivable to have two sensors instead
that combine to a single 3D side-by-side image. In that case each CSI port on
the Tegra goes to a separate sensor. You want to model this.

Regards,

	Hans

> 
> Please provide your suggestions/comments on this so I can take care of exposing both combined ports in v3.
> 
> tc358840@1f {
>     ...
>     ...
>     ports {
>         #address-cells = <1>;
>         #size-cells = <0>;
> 
>         port@0 {
>             reg = <0>;
>             tc358840_out0: endpoint {
>                 link-frequencies = /bits/ 64 <297000000>;
>                 clock-lanes = <0>;
>                 data-lanes = <1 2 3 4>;
>                 remote-endpoint = <&tc358840_csi_in0>;
>             };
>         };
> 
>         port@1 {
>             reg = <1>;
>             tc358840_out1: endpoint {
>                 link-frequencies = /bits/ 64 <297000000>;
>                 clock-lanes = <0>;
>                 data-lanes = <1 2 3 4>;
>                 remote-endpoint = <&tc358840_csi_in1>;
>             };
>         };
>     };
> };
> 
> csi_chan: channel@2 {
>     reg = <2>     /* CSI Port No */
>     ..
>     ..
>     ports {
>         /* port@0 always for Sink pads */
>         port@0 {
>             reg = <0>;  /* Media Pad-0 (Sink) */
>             tc358840_csi_in0: endpoint@0 {
>                 reg = <0>;
>                 data-lanes = <1 2 3 4>;
>                 remote-endpoint = <&tc358840_tx0>;
>             };
> 
>             tc358840_csi_in1: endpoint@1 {
>                 reg = <1>;
>                 data-lanes = <1 2 3 4>;
>                 remote-endpoint = <&tc358840_tx1>;
>             };
>         }
> 
>         /* port@1 always Source pads */
>         port@1 {
>             reg = <1>;  /* Media Pad-1 (Source) */
>             tc358840_csi_in0: endpoint@0 {
>                 reg = <0>;
>                 remote-endpoint = <&tc358840_vi_in0>;
>             };
> 
>             tc358840_csi_in0: endpoint@1 {
>                 reg = <1>;
>                 remote-endpoint = <&tc358840_vi_in1>;
>             };
>         }
>     }
> }
> 
> 
> Thanks
> 
> Sowjanya
> 

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/29/20 2:33 AM, Hans Verkuil wrote:
> On 29/10/2020 02:48, Sowjanya Komatineni wrote:
>> Hi Sakari,
>>
>> Missed to add you to below patch series for HDMI2CSI bridge support
>>
>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>
>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>
>> Would like to get your suggestion on x8 gang/combined ports capture implementation.
>>
>>
>> Tegra VI/CSI don't have native x8. HDMI2CSI bridges uses dual ports with split image on to each port.
>> So on Tegra side at SW level we can use dual ports as gang where both ports are simultaneously programmed for combined image capture.
>>
>> Currently v2 patches use bus-width with V4L2 parallel bus type for this x8 gang mode implementation.
>> So, driver parses endpoint and if bus type is parallel (for x8), it uses v4l2_ep.bus.parallel.bus_width other wise (x4 and below) it uses
>> v4l2_ep.bus.mipi_csi2.num_data_lanes for getting lanes from DT.
>>
>> x8 is not from native HW and current v2 version does not explicitly add 2nd port in device tree and driver uses consecutive ports as
>> combined capture for same video device node.
>>
>>  From offline discussion with Hans, looks like its better to explicitly specify both ports used as gang/combined capture in device tree on
>> Tegra side and bridge side also will expose both TX ports.
>>
>> Current CSI driver implementation uses max 2 media pads port@0 as Sink and port@1 as Source. So can add 2nd endpoint in sink node and source
>> nodes.
>>
>> Below is tc358840 and csi device node change to explicitly add second source.
>>
>> With this, VI/CSI driver can be updated to use second port only for identifying total combined ports for capture.
>>
>> My understanding is there is no need for creating media links for 2nd port as both ports together are used as combined ports by the Tegra
>> driver during streaming for the same csi subdev and video device nodes.
> In my opinion both links should be shown in the media controller topology.
> Basically these are just independent CSI ports (two independent TX ports on
> the tc358840 and two independent RX ports on the Tegra SoC side), that are
> combined afterwards into a single video stream.
>
> While this is an HDMI receiver, it is conceivable to have two sensors instead
> that combine to a single 3D side-by-side image. In that case each CSI port on
> the Tegra goes to a separate sensor. You want to model this.
>
> Regards,
>
> 	Hans

With 2 ports (used as combined ports on Tegra side), as they all are for 
the same subdev and video device, exposing 2nd port and creating media 
links will create 2 source pads and 2 sink pads.

Subdev will be same for both 2 source pads. I am trying to understand 
the usage of creating media links for 2nd port which is used as combined 
port with side-by-side capture thru VI for same video device node.

>> Please provide your suggestions/comments on this so I can take care of exposing both combined ports in v3.
>>
>> tc358840@1f {
>>      ...
>>      ...
>>      ports {
>>          #address-cells = <1>;
>>          #size-cells = <0>;
>>
>>          port@0 {
>>              reg = <0>;
>>              tc358840_out0: endpoint {
>>                  link-frequencies = /bits/ 64 <297000000>;
>>                  clock-lanes = <0>;
>>                  data-lanes = <1 2 3 4>;
>>                  remote-endpoint = <&tc358840_csi_in0>;
>>              };
>>          };
>>
>>          port@1 {
>>              reg = <1>;
>>              tc358840_out1: endpoint {
>>                  link-frequencies = /bits/ 64 <297000000>;
>>                  clock-lanes = <0>;
>>                  data-lanes = <1 2 3 4>;
>>                  remote-endpoint = <&tc358840_csi_in1>;
>>              };
>>          };
>>      };
>> };
>>
>> csi_chan: channel@2 {
>>      reg = <2>     /* CSI Port No */
>>      ..
>>      ..
>>      ports {
>>          /* port@0 always for Sink pads */
>>          port@0 {
>>              reg = <0>;  /* Media Pad-0 (Sink) */
>>              tc358840_csi_in0: endpoint@0 {
>>                  reg = <0>;
>>                  data-lanes = <1 2 3 4>;
>>                  remote-endpoint = <&tc358840_tx0>;
>>              };
>>
>>              tc358840_csi_in1: endpoint@1 {
>>                  reg = <1>;
>>                  data-lanes = <1 2 3 4>;
>>                  remote-endpoint = <&tc358840_tx1>;
>>              };
>>          }
>>
>>          /* port@1 always Source pads */
>>          port@1 {
>>              reg = <1>;  /* Media Pad-1 (Source) */
>>              tc358840_csi_in0: endpoint@0 {
>>                  reg = <0>;
>>                  remote-endpoint = <&tc358840_vi_in0>;
>>              };
>>
>>              tc358840_csi_in0: endpoint@1 {
>>                  reg = <1>;
>>                  remote-endpoint = <&tc358840_vi_in1>;
>>              };
>>          }
>>      }
>> }
>>
>>
>> Thanks
>>
>> Sowjanya
>>

Re: Suggestion regarding x8 gang mode device tree changes

[Sakari Ailus]

Hi Sowjanya,

On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
> Hi Sakari,
> 
> Missed to add you to below patch series for HDMI2CSI bridge support
> 
> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
> 
> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
> 
> Would like to get your suggestion on x8 gang/combined ports capture
> implementation.

The majority of CSI-2 receiver devices support partitioning the lanes among
different PHYs in various ways. They do support usually up to four lanes,
but adding four more lanes is not a reason for making the API different.

So instead, you should implement this as a single port that simply has 8
lanes.

-- 
Kind regards,

Sakari Ailus

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/29/20 7:50 AM, Sakari Ailus wrote:
> Hi Sowjanya,
>
> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>> Hi Sakari,
>>
>> Missed to add you to below patch series for HDMI2CSI bridge support
>>
>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>
>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>
>> Would like to get your suggestion on x8 gang/combined ports capture
>> implementation.
> The majority of CSI-2 receiver devices support partitioning the lanes among
> different PHYs in various ways. They do support usually up to four lanes,
> but adding four more lanes is not a reason for making the API different.
>
> So instead, you should implement this as a single port that simply has 8
> lanes.
>
Thanks Sakari for your reply.

current v2 series treats as 8 lanes. You mean to not expose 2nd port in 
device tree as VI/CSI side takes care of 2nd port as combined to treat 
as 8 lane?

Thanks

Sowjanya

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>
> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>> Hi Sowjanya,
>>
>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>> Hi Sakari,
>>>
>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>
>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/ 
>>>
>>>
>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>
>>> Would like to get your suggestion on x8 gang/combined ports capture
>>> implementation.
>> The majority of CSI-2 receiver devices support partitioning the lanes 
>> among
>> different PHYs in various ways. They do support usually up to four 
>> lanes,
>> but adding four more lanes is not a reason for making the API different.
>>
>> So instead, you should implement this as a single port that simply has 8
>> lanes.
>>
> Thanks Sakari for your reply.
>
> current v2 series treats as 8 lanes. You mean to not expose 2nd port 
> in device tree as VI/CSI side takes care of 2nd port as combined to 
> treat as 8 lane?

AS csi2 bus type supports max 4 data lanes with endpoint parse API.

Currently with x8 as single port, I am using bus-width with bus type as 
parallel in device tree and when using x4 using data-lanes with csi2 bus 
type and driver gets lanes based on either of this from DT.

Instead should we update endpoint parse API for max up to 8 lanes for 
data-lanes?

>
> Thanks
>
> Sowjanya
>

Re: Suggestion regarding x8 gang mode device tree changes

[Sakari Ailus]

On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
> 
> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
> > 
> > On 10/29/20 7:50 AM, Sakari Ailus wrote:
> > > Hi Sowjanya,
> > > 
> > > On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
> > > > Hi Sakari,
> > > > 
> > > > Missed to add you to below patch series for HDMI2CSI bridge support
> > > > 
> > > > https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
> > > > 
> > > > 
> > > > Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
> > > > 
> > > > Would like to get your suggestion on x8 gang/combined ports capture
> > > > implementation.
> > > The majority of CSI-2 receiver devices support partitioning the
> > > lanes among
> > > different PHYs in various ways. They do support usually up to four
> > > lanes,
> > > but adding four more lanes is not a reason for making the API different.
> > > 
> > > So instead, you should implement this as a single port that simply has 8
> > > lanes.
> > > 
> > Thanks Sakari for your reply.
> > 
> > current v2 series treats as 8 lanes. You mean to not expose 2nd port in
> > device tree as VI/CSI side takes care of 2nd port as combined to treat
> > as 8 lane?

Correct.

Although you can have the second port connected if fewer lanes are assigned
to the first one.

How does it work for this device, are the lanes statically allocated to
ports, apart from the special 8 lane mode?

> 
> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
> 
> Currently with x8 as single port, I am using bus-width with bus type as
> parallel in device tree and when using x4 using data-lanes with csi2 bus
> type and driver gets lanes based on either of this from DT.
> 
> Instead should we update endpoint parse API for max up to 8 lanes for
> data-lanes?

Yes, please. Could you send a patch?

The standard AFAIK supports up to four lanes but as we know, hardware
sometimes has more than that.

-- 
Sakari Ailus

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/29/20 9:52 AM, Sakari Ailus wrote:
> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>> Hi Sowjanya,
>>>>
>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>>>> Hi Sakari,
>>>>>
>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>>>
>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>>>>
>>>>>
>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>
>>>>> Would like to get your suggestion on x8 gang/combined ports capture
>>>>> implementation.
>>>> The majority of CSI-2 receiver devices support partitioning the
>>>> lanes among
>>>> different PHYs in various ways. They do support usually up to four
>>>> lanes,
>>>> but adding four more lanes is not a reason for making the API different.
>>>>
>>>> So instead, you should implement this as a single port that simply has 8
>>>> lanes.
>>>>
>>> Thanks Sakari for your reply.
>>>
>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
>>> as 8 lane?
> Correct.
>
> Although you can have the second port connected if fewer lanes are assigned
> to the first one.
>
> How does it work for this device, are the lanes statically allocated to
> ports, apart from the special 8 lane mode?

Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
are programmed for simultaneous streaming during the same video/subdev 
stream ops.

Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
to another x4 port.

HDMI Bridge TX0 -> CSI RX0 (x4 port)

HDMI Bridge TX1 -> CSI RX1 (x4 port)

HDMI bridge side single image is split into 2 x4 ports and on RX side 
image from both ports are captured simultaneously with buffer offsets 
adjusted side-by-side to get combined image for same video buf of video 
device.

Both these 2 x4 ports together are used for streaming by Tegra VI and 
buffer offsets are adjusted side by side for these ports and for video 
device node stream, its single buffer which contains combined image from 
capture.

>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>
>> Currently with x8 as single port, I am using bus-width with bus type as
>> parallel in device tree and when using x4 using data-lanes with csi2 bus
>> type and driver gets lanes based on either of this from DT.
>>
>> Instead should we update endpoint parse API for max up to 8 lanes for
>> data-lanes?
> Yes, please. Could you send a patch?
>
> The standard AFAIK supports up to four lanes but as we know, hardware
> sometimes has more than that.

Sure once Hans also agrees with this to have it as single x8 port (just 
like I have now in v2), will send v3 to update endpoint parse to allow 
upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
to retrieve lanes using csi2 bus type.

Hans, Please confirm if you agree with this.

Re: Suggestion regarding x8 gang mode device tree changes

[Hans Verkuil]

On 29/10/2020 18:07, Sowjanya Komatineni wrote:
> 
> On 10/29/20 9:52 AM, Sakari Ailus wrote:
>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>>> Hi Sowjanya,
>>>>>
>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>>>>> Hi Sakari,
>>>>>>
>>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>>>>
>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>>>>>
>>>>>>
>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>>
>>>>>> Would like to get your suggestion on x8 gang/combined ports capture
>>>>>> implementation.
>>>>> The majority of CSI-2 receiver devices support partitioning the
>>>>> lanes among
>>>>> different PHYs in various ways. They do support usually up to four
>>>>> lanes,
>>>>> but adding four more lanes is not a reason for making the API different.
>>>>>
>>>>> So instead, you should implement this as a single port that simply has 8
>>>>> lanes.
>>>>>
>>>> Thanks Sakari for your reply.
>>>>
>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
>>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
>>>> as 8 lane?
>> Correct.
>>
>> Although you can have the second port connected if fewer lanes are assigned
>> to the first one.
>>
>> How does it work for this device, are the lanes statically allocated to
>> ports, apart from the special 8 lane mode?
> 
> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
> are programmed for simultaneous streaming during the same video/subdev 
> stream ops.
> 
> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
> to another x4 port.
> 
> HDMI Bridge TX0 -> CSI RX0 (x4 port)
> 
> HDMI Bridge TX1 -> CSI RX1 (x4 port)
> 
> HDMI bridge side single image is split into 2 x4 ports and on RX side 
> image from both ports are captured simultaneously with buffer offsets 
> adjusted side-by-side to get combined image for same video buf of video 
> device.
> 
> Both these 2 x4 ports together are used for streaming by Tegra VI and 
> buffer offsets are adjusted side by side for these ports and for video 
> device node stream, its single buffer which contains combined image from 
> capture.
> 
>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>>
>>> Currently with x8 as single port, I am using bus-width with bus type as
>>> parallel in device tree and when using x4 using data-lanes with csi2 bus
>>> type and driver gets lanes based on either of this from DT.
>>>
>>> Instead should we update endpoint parse API for max up to 8 lanes for
>>> data-lanes?
>> Yes, please. Could you send a patch?
>>
>> The standard AFAIK supports up to four lanes but as we know, hardware
>> sometimes has more than that.
> 
> Sure once Hans also agrees with this to have it as single x8 port (just 
> like I have now in v2), will send v3 to update endpoint parse to allow 
> upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
> to retrieve lanes using csi2 bus type.
> 
> Hans, Please confirm if you agree with this.
> 

I'm not sure if I agree with this. Shouldn't a device tree reflect the
hardware? And how would you represent the use case where the ganging
mode stitches together two synced sensors (left and right) into a single
3D side-by-side image? Then you would have:

 Left Sensor TX -> CSI RX0 (x4 port)
Right Sensor TX -> CSI RX1 (x4 port)

And for the tc358840 something similar might be true: in the case of the
Tegra you have this nice ganging mode available, but for other SoCs each
half would have to go to a separate CSI port and captured via a separate
video DMA channel, and software or a GPU is needed to combine the two
halves.

In both these examples it is my understanding that you have to model this
in the DT as separate x4 ports.

Regards,

	Hans

Re: Suggestion regarding x8 gang mode device tree changes

[Sakari Ailus]

Hi Hans,

On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
> > 
> > On 10/29/20 9:52 AM, Sakari Ailus wrote:
> >> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
> >>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
> >>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
> >>>>> Hi Sowjanya,
> >>>>>
> >>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
> >>>>>> Hi Sakari,
> >>>>>>
> >>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
> >>>>>>
> >>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
> >>>>>>
> >>>>>>
> >>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
> >>>>>>
> >>>>>> Would like to get your suggestion on x8 gang/combined ports capture
> >>>>>> implementation.
> >>>>> The majority of CSI-2 receiver devices support partitioning the
> >>>>> lanes among
> >>>>> different PHYs in various ways. They do support usually up to four
> >>>>> lanes,
> >>>>> but adding four more lanes is not a reason for making the API different.
> >>>>>
> >>>>> So instead, you should implement this as a single port that simply has 8
> >>>>> lanes.
> >>>>>
> >>>> Thanks Sakari for your reply.
> >>>>
> >>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
> >>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
> >>>> as 8 lane?
> >> Correct.
> >>
> >> Although you can have the second port connected if fewer lanes are assigned
> >> to the first one.
> >>
> >> How does it work for this device, are the lanes statically allocated to
> >> ports, apart from the special 8 lane mode?
> > 
> > Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
> > are programmed for simultaneous streaming during the same video/subdev 
> > stream ops.
> > 
> > Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
> > to another x4 port.
> > 
> > HDMI Bridge TX0 -> CSI RX0 (x4 port)
> > 
> > HDMI Bridge TX1 -> CSI RX1 (x4 port)
> > 
> > HDMI bridge side single image is split into 2 x4 ports and on RX side 
> > image from both ports are captured simultaneously with buffer offsets 
> > adjusted side-by-side to get combined image for same video buf of video 
> > device.
> > 
> > Both these 2 x4 ports together are used for streaming by Tegra VI and 
> > buffer offsets are adjusted side by side for these ports and for video 
> > device node stream, its single buffer which contains combined image from 
> > capture.
> > 
> >>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
> >>>
> >>> Currently with x8 as single port, I am using bus-width with bus type as
> >>> parallel in device tree and when using x4 using data-lanes with csi2 bus
> >>> type and driver gets lanes based on either of this from DT.
> >>>
> >>> Instead should we update endpoint parse API for max up to 8 lanes for
> >>> data-lanes?
> >> Yes, please. Could you send a patch?
> >>
> >> The standard AFAIK supports up to four lanes but as we know, hardware
> >> sometimes has more than that.
> > 
> > Sure once Hans also agrees with this to have it as single x8 port (just 
> > like I have now in v2), will send v3 to update endpoint parse to allow 
> > upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
> > to retrieve lanes using csi2 bus type.
> > 
> > Hans, Please confirm if you agree with this.
> > 
> 
> I'm not sure if I agree with this. Shouldn't a device tree reflect the
> hardware? And how would you represent the use case where the ganging
> mode stitches together two synced sensors (left and right) into a single
> 3D side-by-side image? Then you would have:
> 
>  Left Sensor TX -> CSI RX0 (x4 port)
> Right Sensor TX -> CSI RX1 (x4 port)
> 
> And for the tc358840 something similar might be true: in the case of the
> Tegra you have this nice ganging mode available, but for other SoCs each
> half would have to go to a separate CSI port and captured via a separate
> video DMA channel, and software or a GPU is needed to combine the two
> halves.
> 
> In both these examples it is my understanding that you have to model this
> in the DT as separate x4 ports.

Do note that a "port" as such is a logical concept. On modern hardware, a
port consists of two or more lanes --- one clock, plus at least one data
lane. Perhaps an example could be useful. For instance, if you have ten
lanes on a device, this could be split into following configurations, based
on the board design:

configuration \ data lanes	port 0	port 1	port 2	port 3

1:				4	4
2:				4	2	1
3:				2	2	2
4:				2	2	1	1

So if you add one more, say:

5:				8

So what we're discussing is just how the lanes are distributed across the
ports.

There are usually hardware specific limitations how the lanes can be
distributed. The interface we have in DT (data-lanes + clock-lanes
properties) allows describing the hardware in general case, so what the
interface allows may not be possible in hardware, but what hardware
implements is supported by the interface.

-- 
Kind regards,

Sakari Ailus

Re: Suggestion regarding x8 gang mode device tree changes

[Hans Verkuil]

On 30/10/2020 10:56, Sakari Ailus wrote:
> Hi Hans,
> 
> On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
>> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
>>>
>>> On 10/29/20 9:52 AM, Sakari Ailus wrote:
>>>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>>>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>>>>> Hi Sowjanya,
>>>>>>>
>>>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>>>>>>> Hi Sakari,
>>>>>>>>
>>>>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>>>>>>
>>>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>>>>>>>
>>>>>>>>
>>>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>>>>
>>>>>>>> Would like to get your suggestion on x8 gang/combined ports capture
>>>>>>>> implementation.
>>>>>>> The majority of CSI-2 receiver devices support partitioning the
>>>>>>> lanes among
>>>>>>> different PHYs in various ways. They do support usually up to four
>>>>>>> lanes,
>>>>>>> but adding four more lanes is not a reason for making the API different.
>>>>>>>
>>>>>>> So instead, you should implement this as a single port that simply has 8
>>>>>>> lanes.
>>>>>>>
>>>>>> Thanks Sakari for your reply.
>>>>>>
>>>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
>>>>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
>>>>>> as 8 lane?
>>>> Correct.
>>>>
>>>> Although you can have the second port connected if fewer lanes are assigned
>>>> to the first one.
>>>>
>>>> How does it work for this device, are the lanes statically allocated to
>>>> ports, apart from the special 8 lane mode?
>>>
>>> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
>>> are programmed for simultaneous streaming during the same video/subdev 
>>> stream ops.
>>>
>>> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
>>> to another x4 port.
>>>
>>> HDMI Bridge TX0 -> CSI RX0 (x4 port)
>>>
>>> HDMI Bridge TX1 -> CSI RX1 (x4 port)
>>>
>>> HDMI bridge side single image is split into 2 x4 ports and on RX side 
>>> image from both ports are captured simultaneously with buffer offsets 
>>> adjusted side-by-side to get combined image for same video buf of video 
>>> device.
>>>
>>> Both these 2 x4 ports together are used for streaming by Tegra VI and 
>>> buffer offsets are adjusted side by side for these ports and for video 
>>> device node stream, its single buffer which contains combined image from 
>>> capture.
>>>
>>>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>>>>
>>>>> Currently with x8 as single port, I am using bus-width with bus type as
>>>>> parallel in device tree and when using x4 using data-lanes with csi2 bus
>>>>> type and driver gets lanes based on either of this from DT.
>>>>>
>>>>> Instead should we update endpoint parse API for max up to 8 lanes for
>>>>> data-lanes?
>>>> Yes, please. Could you send a patch?
>>>>
>>>> The standard AFAIK supports up to four lanes but as we know, hardware
>>>> sometimes has more than that.
>>>
>>> Sure once Hans also agrees with this to have it as single x8 port (just 
>>> like I have now in v2), will send v3 to update endpoint parse to allow 
>>> upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
>>> to retrieve lanes using csi2 bus type.
>>>
>>> Hans, Please confirm if you agree with this.
>>>
>>
>> I'm not sure if I agree with this. Shouldn't a device tree reflect the
>> hardware? And how would you represent the use case where the ganging
>> mode stitches together two synced sensors (left and right) into a single
>> 3D side-by-side image? Then you would have:
>>
>>  Left Sensor TX -> CSI RX0 (x4 port)
>> Right Sensor TX -> CSI RX1 (x4 port)
>>
>> And for the tc358840 something similar might be true: in the case of the
>> Tegra you have this nice ganging mode available, but for other SoCs each
>> half would have to go to a separate CSI port and captured via a separate
>> video DMA channel, and software or a GPU is needed to combine the two
>> halves.
>>
>> In both these examples it is my understanding that you have to model this
>> in the DT as separate x4 ports.
> 
> Do note that a "port" as such is a logical concept. On modern hardware, a
> port consists of two or more lanes --- one clock, plus at least one data
> lane. Perhaps an example could be useful. For instance, if you have ten
> lanes on a device, this could be split into following configurations, based
> on the board design:
> 
> configuration \ data lanes	port 0	port 1	port 2	port 3
> 
> 1:				4	4
> 2:				4	2	1
> 3:				2	2	2
> 4:				2	2	1	1
> 
> So if you add one more, say:
> 
> 5:				8
> 
> So what we're discussing is just how the lanes are distributed across the
> ports.
> 
> There are usually hardware specific limitations how the lanes can be
> distributed. The interface we have in DT (data-lanes + clock-lanes
> properties) allows describing the hardware in general case, so what the
> interface allows may not be possible in hardware, but what hardware
> implements is supported by the interface.
> 

So for this particular instance using a single logical 8-lane port would
make sense, but in the two other scenarios (left/right sensor or supporting
tc358840 in a SoC that doesn't support ganging) I described you would still
have to model it as two 4-lane ports.

Is that correct?

Regards,

	Hans

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/30/20 3:06 AM, Hans Verkuil wrote:
> On 30/10/2020 10:56, Sakari Ailus wrote:
>> Hi Hans,
>>
>> On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
>>> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
>>>> On 10/29/20 9:52 AM, Sakari Ailus wrote:
>>>>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>>>>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>>>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>>>>>> Hi Sowjanya,
>>>>>>>>
>>>>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>>>>>>>> Hi Sakari,
>>>>>>>>>
>>>>>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>>>>>>>
>>>>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>>>>>
>>>>>>>>> Would like to get your suggestion on x8 gang/combined ports capture
>>>>>>>>> implementation.
>>>>>>>> The majority of CSI-2 receiver devices support partitioning the
>>>>>>>> lanes among
>>>>>>>> different PHYs in various ways. They do support usually up to four
>>>>>>>> lanes,
>>>>>>>> but adding four more lanes is not a reason for making the API different.
>>>>>>>>
>>>>>>>> So instead, you should implement this as a single port that simply has 8
>>>>>>>> lanes.
>>>>>>>>
>>>>>>> Thanks Sakari for your reply.
>>>>>>>
>>>>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
>>>>>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
>>>>>>> as 8 lane?
>>>>> Correct.
>>>>>
>>>>> Although you can have the second port connected if fewer lanes are assigned
>>>>> to the first one.
>>>>>
>>>>> How does it work for this device, are the lanes statically allocated to
>>>>> ports, apart from the special 8 lane mode?
>>>> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together
>>>> are programmed for simultaneous streaming during the same video/subdev
>>>> stream ops.
>>>>
>>>> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes
>>>> to another x4 port.
>>>>
>>>> HDMI Bridge TX0 -> CSI RX0 (x4 port)
>>>>
>>>> HDMI Bridge TX1 -> CSI RX1 (x4 port)
>>>>
>>>> HDMI bridge side single image is split into 2 x4 ports and on RX side
>>>> image from both ports are captured simultaneously with buffer offsets
>>>> adjusted side-by-side to get combined image for same video buf of video
>>>> device.
>>>>
>>>> Both these 2 x4 ports together are used for streaming by Tegra VI and
>>>> buffer offsets are adjusted side by side for these ports and for video
>>>> device node stream, its single buffer which contains combined image from
>>>> capture.
>>>>
>>>>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>>>>>
>>>>>> Currently with x8 as single port, I am using bus-width with bus type as
>>>>>> parallel in device tree and when using x4 using data-lanes with csi2 bus
>>>>>> type and driver gets lanes based on either of this from DT.
>>>>>>
>>>>>> Instead should we update endpoint parse API for max up to 8 lanes for
>>>>>> data-lanes?
>>>>> Yes, please. Could you send a patch?
>>>>>
>>>>> The standard AFAIK supports up to four lanes but as we know, hardware
>>>>> sometimes has more than that.
>>>> Sure once Hans also agrees with this to have it as single x8 port (just
>>>> like I have now in v2), will send v3 to update endpoint parse to allow
>>>> upto max 8 data-lanes and will also update Tegra CSI driver accordingly
>>>> to retrieve lanes using csi2 bus type.
>>>>
>>>> Hans, Please confirm if you agree with this.
>>>>
>>> I'm not sure if I agree with this. Shouldn't a device tree reflect the
>>> hardware? And how would you represent the use case where the ganging
>>> mode stitches together two synced sensors (left and right) into a single
>>> 3D side-by-side image? Then you would have:
>>>
>>>   Left Sensor TX -> CSI RX0 (x4 port)
>>> Right Sensor TX -> CSI RX1 (x4 port)
>>>
>>> And for the tc358840 something similar might be true: in the case of the
>>> Tegra you have this nice ganging mode available, but for other SoCs each
>>> half would have to go to a separate CSI port and captured via a separate
>>> video DMA channel, and software or a GPU is needed to combine the two
>>> halves.
>>>
>>> In both these examples it is my understanding that you have to model this
>>> in the DT as separate x4 ports.
>> Do note that a "port" as such is a logical concept. On modern hardware, a
>> port consists of two or more lanes --- one clock, plus at least one data
>> lane. Perhaps an example could be useful. For instance, if you have ten
>> lanes on a device, this could be split into following configurations, based
>> on the board design:
>>
>> configuration \ data lanes	port 0	port 1	port 2	port 3
>>
>> 1:				4	4
>> 2:				4	2	1
>> 3:				2	2	2
>> 4:				2	2	1	1
>>
>> So if you add one more, say:
>>
>> 5:				8
>>
>> So what we're discussing is just how the lanes are distributed across the
>> ports.
>>
>> There are usually hardware specific limitations how the lanes can be
>> distributed. The interface we have in DT (data-lanes + clock-lanes
>> properties) allows describing the hardware in general case, so what the
>> interface allows may not be possible in hardware, but what hardware
>> implements is supported by the interface.
>>
> So for this particular instance using a single logical 8-lane port would
> make sense, but in the two other scenarios (left/right sensor or supporting
> tc358840 in a SoC that doesn't support ganging) I described you would still
> have to model it as two 4-lane ports.
>
> Is that correct?
>
> Regards,
>
> 	Hans

Hi Hans,

As its a single image split here, even it comes from 2 TX ports it goes 
through same video channel where we adjust video buffer offset to align 
side-by-side for combined image.

So if any SoC does not support multiple independent ports (by HW or 
logically), then like Sakari mentioned we expose both x4 ports but in 
that case they both should be exposed to different video channel (DMA 
Buffer allocations).

With this SW should manage to combine captures from these 2 independent 
ports and I am not sure if this is feasible or if we ever had this case 
implemented by any SoC SW so far as SW overhead will impact as well but 
this is different issue.

But isn't most SoC, CSI RX ports are similar instances allowing parallel 
captures as any Receiver supporting multiple ports allows multiple 
streaming right?

Re: Suggestion regarding x8 gang mode device tree changes

[Sowjanya Komatineni]

On 10/30/20 8:26 AM, Sowjanya Komatineni wrote:
>
> On 10/30/20 3:06 AM, Hans Verkuil wrote:
>> On 30/10/2020 10:56, Sakari Ailus wrote:
>>> Hi Hans,
>>>
>>> On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
>>>> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
>>>>> On 10/29/20 9:52 AM, Sakari Ailus wrote:
>>>>>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>>>>>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>>>>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>>>>>>> Hi Sowjanya,
>>>>>>>>>
>>>>>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni 
>>>>>>>>> wrote:
>>>>>>>>>> Hi Sakari,
>>>>>>>>>>
>>>>>>>>>> Missed to add you to below patch series for HDMI2CSI bridge 
>>>>>>>>>> support
>>>>>>>>>>
>>>>>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/ 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>>>>>>
>>>>>>>>>> Would like to get your suggestion on x8 gang/combined ports 
>>>>>>>>>> capture
>>>>>>>>>> implementation.
>>>>>>>>> The majority of CSI-2 receiver devices support partitioning the
>>>>>>>>> lanes among
>>>>>>>>> different PHYs in various ways. They do support usually up to 
>>>>>>>>> four
>>>>>>>>> lanes,
>>>>>>>>> but adding four more lanes is not a reason for making the API 
>>>>>>>>> different.
>>>>>>>>>
>>>>>>>>> So instead, you should implement this as a single port that 
>>>>>>>>> simply has 8
>>>>>>>>> lanes.
>>>>>>>>>
>>>>>>>> Thanks Sakari for your reply.
>>>>>>>>
>>>>>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd 
>>>>>>>> port in
>>>>>>>> device tree as VI/CSI side takes care of 2nd port as combined 
>>>>>>>> to treat
>>>>>>>> as 8 lane?
>>>>>> Correct.
>>>>>>
>>>>>> Although you can have the second port connected if fewer lanes 
>>>>>> are assigned
>>>>>> to the first one.
>>>>>>
>>>>>> How does it work for this device, are the lanes statically 
>>>>>> allocated to
>>>>>> ports, apart from the special 8 lane mode?
>>>>> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together
>>>>> are programmed for simultaneous streaming during the same 
>>>>> video/subdev
>>>>> stream ops.
>>>>>
>>>>> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes 
>>>>> goes
>>>>> to another x4 port.
>>>>>
>>>>> HDMI Bridge TX0 -> CSI RX0 (x4 port)
>>>>>
>>>>> HDMI Bridge TX1 -> CSI RX1 (x4 port)
>>>>>
>>>>> HDMI bridge side single image is split into 2 x4 ports and on RX side
>>>>> image from both ports are captured simultaneously with buffer offsets
>>>>> adjusted side-by-side to get combined image for same video buf of 
>>>>> video
>>>>> device.
>>>>>
>>>>> Both these 2 x4 ports together are used for streaming by Tegra VI and
>>>>> buffer offsets are adjusted side by side for these ports and for 
>>>>> video
>>>>> device node stream, its single buffer which contains combined 
>>>>> image from
>>>>> capture.
>>>>>
>>>>>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>>>>>>
>>>>>>> Currently with x8 as single port, I am using bus-width with bus 
>>>>>>> type as
>>>>>>> parallel in device tree and when using x4 using data-lanes with 
>>>>>>> csi2 bus
>>>>>>> type and driver gets lanes based on either of this from DT.
>>>>>>>
>>>>>>> Instead should we update endpoint parse API for max up to 8 
>>>>>>> lanes for
>>>>>>> data-lanes?
>>>>>> Yes, please. Could you send a patch?
>>>>>>
>>>>>> The standard AFAIK supports up to four lanes but as we know, 
>>>>>> hardware
>>>>>> sometimes has more than that.
>>>>> Sure once Hans also agrees with this to have it as single x8 port 
>>>>> (just
>>>>> like I have now in v2), will send v3 to update endpoint parse to 
>>>>> allow
>>>>> upto max 8 data-lanes and will also update Tegra CSI driver 
>>>>> accordingly
>>>>> to retrieve lanes using csi2 bus type.
>>>>>
>>>>> Hans, Please confirm if you agree with this.
>>>>>
>>>> I'm not sure if I agree with this. Shouldn't a device tree reflect the
>>>> hardware? And how would you represent the use case where the ganging
>>>> mode stitches together two synced sensors (left and right) into a 
>>>> single
>>>> 3D side-by-side image? Then you would have:
>>>>
>>>>   Left Sensor TX -> CSI RX0 (x4 port)
>>>> Right Sensor TX -> CSI RX1 (x4 port)
>>>>
>>>> And for the tc358840 something similar might be true: in the case 
>>>> of the
>>>> Tegra you have this nice ganging mode available, but for other SoCs 
>>>> each
>>>> half would have to go to a separate CSI port and captured via a 
>>>> separate
>>>> video DMA channel, and software or a GPU is needed to combine the two
>>>> halves.
>>>>
>>>> In both these examples it is my understanding that you have to 
>>>> model this
>>>> in the DT as separate x4 ports.
>>> Do note that a "port" as such is a logical concept. On modern 
>>> hardware, a
>>> port consists of two or more lanes --- one clock, plus at least one 
>>> data
>>> lane. Perhaps an example could be useful. For instance, if you have ten
>>> lanes on a device, this could be split into following 
>>> configurations, based
>>> on the board design:
>>>
>>> configuration \ data lanes    port 0    port 1    port 2 port 3
>>>
>>> 1:                4    4
>>> 2:                4    2    1
>>> 3:                2    2    2
>>> 4:                2    2    1    1
>>>
>>> So if you add one more, say:
>>>
>>> 5:                8
>>>
>>> So what we're discussing is just how the lanes are distributed 
>>> across the
>>> ports.
>>>
>>> There are usually hardware specific limitations how the lanes can be
>>> distributed. The interface we have in DT (data-lanes + clock-lanes
>>> properties) allows describing the hardware in general case, so what the
>>> interface allows may not be possible in hardware, but what hardware
>>> implements is supported by the interface.
>>>
>> So for this particular instance using a single logical 8-lane port would
>> make sense, but in the two other scenarios (left/right sensor or 
>> supporting
>> tc358840 in a SoC that doesn't support ganging) I described you would 
>> still
>> have to model it as two 4-lane ports.
>>
>> Is that correct?
>>
>> Regards,
>>
>>     Hans
>
> Hi Hans,
>
> As its a single image split here, even it comes from 2 TX ports it 
> goes through same video channel where we adjust video buffer offset to 
> align side-by-side for combined image.
>
> So if any SoC does not support multiple independent ports (by HW or 
> logically), then like Sakari mentioned we expose both x4 ports but in 
> that case they both should be exposed to different video channel (DMA 
> Buffer allocations).
To be clear, I meant SoC that does not support multiple ports like gang 
or HW native x8 port in the above point.
>
> With this SW should manage to combine captures from these 2 
> independent ports and I am not sure if this is feasible or if we ever 
> had this case implemented by any SoC SW so far as SW overhead will 
> impact as well but this is different issue.
>
> But isn't most SoC, CSI RX ports are similar instances allowing 
> parallel captures as any Receiver supporting multiple ports allows 
> multiple streaming right?
>
Cases we are discussing here are

1. SoC supporting Native x8 or gang ports  on CSI RX: as its single 
image they go thru same video channel

2. SoC does not support gang ports on CSI RX: In this case, different 
video channel should be used

So probably, to cope this with both cases, such sensor/bridge drivers 
should allow creating on CSI TX side both ports as separate (case-2) and 
also as single port (case-1).

Re: Suggestion regarding x8 gang mode device tree changes

[Sakari Ailus]

Hi Hans,

On Fri, Oct 30, 2020 at 11:06:18AM +0100, Hans Verkuil wrote:
> On 30/10/2020 10:56, Sakari Ailus wrote:
> > Hi Hans,
> > 
> > On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
> >> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
> >>>
> >>> On 10/29/20 9:52 AM, Sakari Ailus wrote:
> >>>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
> >>>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
> >>>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
> >>>>>>> Hi Sowjanya,
> >>>>>>>
> >>>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
> >>>>>>>> Hi Sakari,
> >>>>>>>>
> >>>>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
> >>>>>>>>
> >>>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
> >>>>>>>>
> >>>>>>>> Would like to get your suggestion on x8 gang/combined ports capture
> >>>>>>>> implementation.
> >>>>>>> The majority of CSI-2 receiver devices support partitioning the
> >>>>>>> lanes among
> >>>>>>> different PHYs in various ways. They do support usually up to four
> >>>>>>> lanes,
> >>>>>>> but adding four more lanes is not a reason for making the API different.
> >>>>>>>
> >>>>>>> So instead, you should implement this as a single port that simply has 8
> >>>>>>> lanes.
> >>>>>>>
> >>>>>> Thanks Sakari for your reply.
> >>>>>>
> >>>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
> >>>>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
> >>>>>> as 8 lane?
> >>>> Correct.
> >>>>
> >>>> Although you can have the second port connected if fewer lanes are assigned
> >>>> to the first one.
> >>>>
> >>>> How does it work for this device, are the lanes statically allocated to
> >>>> ports, apart from the special 8 lane mode?
> >>>
> >>> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
> >>> are programmed for simultaneous streaming during the same video/subdev 
> >>> stream ops.
> >>>
> >>> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
> >>> to another x4 port.
> >>>
> >>> HDMI Bridge TX0 -> CSI RX0 (x4 port)
> >>>
> >>> HDMI Bridge TX1 -> CSI RX1 (x4 port)
> >>>
> >>> HDMI bridge side single image is split into 2 x4 ports and on RX side 
> >>> image from both ports are captured simultaneously with buffer offsets 
> >>> adjusted side-by-side to get combined image for same video buf of video 
> >>> device.
> >>>
> >>> Both these 2 x4 ports together are used for streaming by Tegra VI and 
> >>> buffer offsets are adjusted side by side for these ports and for video 
> >>> device node stream, its single buffer which contains combined image from 
> >>> capture.
> >>>
> >>>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
> >>>>>
> >>>>> Currently with x8 as single port, I am using bus-width with bus type as
> >>>>> parallel in device tree and when using x4 using data-lanes with csi2 bus
> >>>>> type and driver gets lanes based on either of this from DT.
> >>>>>
> >>>>> Instead should we update endpoint parse API for max up to 8 lanes for
> >>>>> data-lanes?
> >>>> Yes, please. Could you send a patch?
> >>>>
> >>>> The standard AFAIK supports up to four lanes but as we know, hardware
> >>>> sometimes has more than that.
> >>>
> >>> Sure once Hans also agrees with this to have it as single x8 port (just 
> >>> like I have now in v2), will send v3 to update endpoint parse to allow 
> >>> upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
> >>> to retrieve lanes using csi2 bus type.
> >>>
> >>> Hans, Please confirm if you agree with this.
> >>>
> >>
> >> I'm not sure if I agree with this. Shouldn't a device tree reflect the
> >> hardware? And how would you represent the use case where the ganging
> >> mode stitches together two synced sensors (left and right) into a single
> >> 3D side-by-side image? Then you would have:
> >>
> >>  Left Sensor TX -> CSI RX0 (x4 port)
> >> Right Sensor TX -> CSI RX1 (x4 port)
> >>
> >> And for the tc358840 something similar might be true: in the case of the
> >> Tegra you have this nice ganging mode available, but for other SoCs each
> >> half would have to go to a separate CSI port and captured via a separate
> >> video DMA channel, and software or a GPU is needed to combine the two
> >> halves.
> >>
> >> In both these examples it is my understanding that you have to model this
> >> in the DT as separate x4 ports.
> > 
> > Do note that a "port" as such is a logical concept. On modern hardware, a
> > port consists of two or more lanes --- one clock, plus at least one data
> > lane. Perhaps an example could be useful. For instance, if you have ten
> > lanes on a device, this could be split into following configurations, based
> > on the board design:
> > 
> > configuration \ data lanes	port 0	port 1	port 2	port 3
> > 
> > 1:				4	4
> > 2:				4	2	1
> > 3:				2	2	2
> > 4:				2	2	1	1
> > 
> > So if you add one more, say:
> > 
> > 5:				8
> > 
> > So what we're discussing is just how the lanes are distributed across the
> > ports.
> > 
> > There are usually hardware specific limitations how the lanes can be
> > distributed. The interface we have in DT (data-lanes + clock-lanes
> > properties) allows describing the hardware in general case, so what the
> > interface allows may not be possible in hardware, but what hardware
> > implements is supported by the interface.
> > 
> 
> So for this particular instance using a single logical 8-lane port would
> make sense, but in the two other scenarios (left/right sensor or supporting
> tc358840 in a SoC that doesn't support ganging) I described you would still
> have to model it as two 4-lane ports.

If you have two sensors, yes, then it'll be two separate ports; one sensor
connected to each of them. The streams are usually separate, but other
kinds of implementations exist. Still, they generally have no effect on
CSI-2 bus configuration.

-- 
Kind regards,

Sakari Ailus

Re: Suggestion regarding x8 gang mode device tree changes

[Hans Verkuil]

On 30/10/2020 23:31, Sakari Ailus wrote:
> Hi Hans,
> 
> On Fri, Oct 30, 2020 at 11:06:18AM +0100, Hans Verkuil wrote:
>> On 30/10/2020 10:56, Sakari Ailus wrote:
>>> Hi Hans,
>>>
>>> On Fri, Oct 30, 2020 at 10:31:06AM +0100, Hans Verkuil wrote:
>>>> On 29/10/2020 18:07, Sowjanya Komatineni wrote:
>>>>>
>>>>> On 10/29/20 9:52 AM, Sakari Ailus wrote:
>>>>>> On Thu, Oct 29, 2020 at 09:49:57AM -0700, Sowjanya Komatineni wrote:
>>>>>>> On 10/29/20 8:36 AM, Sowjanya Komatineni wrote:
>>>>>>>> On 10/29/20 7:50 AM, Sakari Ailus wrote:
>>>>>>>>> Hi Sowjanya,
>>>>>>>>>
>>>>>>>>> On Wed, Oct 28, 2020 at 06:48:59PM -0700, Sowjanya Komatineni wrote:
>>>>>>>>>> Hi Sakari,
>>>>>>>>>>
>>>>>>>>>> Missed to add you to below patch series for HDMI2CSI bridge support
>>>>>>>>>>
>>>>>>>>>> https://patchwork.kernel.org/project/linux-media/cover/1603768763-25590-1-git-send-email-skomatineni@nvidia.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Patch-10 of this series is for x8 capture from HDMI2CSI bridge.
>>>>>>>>>>
>>>>>>>>>> Would like to get your suggestion on x8 gang/combined ports capture
>>>>>>>>>> implementation.
>>>>>>>>> The majority of CSI-2 receiver devices support partitioning the
>>>>>>>>> lanes among
>>>>>>>>> different PHYs in various ways. They do support usually up to four
>>>>>>>>> lanes,
>>>>>>>>> but adding four more lanes is not a reason for making the API different.
>>>>>>>>>
>>>>>>>>> So instead, you should implement this as a single port that simply has 8
>>>>>>>>> lanes.
>>>>>>>>>
>>>>>>>> Thanks Sakari for your reply.
>>>>>>>>
>>>>>>>> current v2 series treats as 8 lanes. You mean to not expose 2nd port in
>>>>>>>> device tree as VI/CSI side takes care of 2nd port as combined to treat
>>>>>>>> as 8 lane?
>>>>>> Correct.
>>>>>>
>>>>>> Although you can have the second port connected if fewer lanes are assigned
>>>>>> to the first one.
>>>>>>
>>>>>> How does it work for this device, are the lanes statically allocated to
>>>>>> ports, apart from the special 8 lane mode?
>>>>>
>>>>> Tegra CSI each port supports max 4 lanes. For x8, 2 x4 ports together 
>>>>> are programmed for simultaneous streaming during the same video/subdev 
>>>>> stream ops.
>>>>>
>>>>> Physically, CSI RX side 4 lanes goes to x4 port and other 4 lanes goes 
>>>>> to another x4 port.
>>>>>
>>>>> HDMI Bridge TX0 -> CSI RX0 (x4 port)
>>>>>
>>>>> HDMI Bridge TX1 -> CSI RX1 (x4 port)
>>>>>
>>>>> HDMI bridge side single image is split into 2 x4 ports and on RX side 
>>>>> image from both ports are captured simultaneously with buffer offsets 
>>>>> adjusted side-by-side to get combined image for same video buf of video 
>>>>> device.
>>>>>
>>>>> Both these 2 x4 ports together are used for streaming by Tegra VI and 
>>>>> buffer offsets are adjusted side by side for these ports and for video 
>>>>> device node stream, its single buffer which contains combined image from 
>>>>> capture.
>>>>>
>>>>>>> AS csi2 bus type supports max 4 data lanes with endpoint parse API.
>>>>>>>
>>>>>>> Currently with x8 as single port, I am using bus-width with bus type as
>>>>>>> parallel in device tree and when using x4 using data-lanes with csi2 bus
>>>>>>> type and driver gets lanes based on either of this from DT.
>>>>>>>
>>>>>>> Instead should we update endpoint parse API for max up to 8 lanes for
>>>>>>> data-lanes?
>>>>>> Yes, please. Could you send a patch?
>>>>>>
>>>>>> The standard AFAIK supports up to four lanes but as we know, hardware
>>>>>> sometimes has more than that.
>>>>>
>>>>> Sure once Hans also agrees with this to have it as single x8 port (just 
>>>>> like I have now in v2), will send v3 to update endpoint parse to allow 
>>>>> upto max 8 data-lanes and will also update Tegra CSI driver accordingly 
>>>>> to retrieve lanes using csi2 bus type.
>>>>>
>>>>> Hans, Please confirm if you agree with this.
>>>>>
>>>>
>>>> I'm not sure if I agree with this. Shouldn't a device tree reflect the
>>>> hardware? And how would you represent the use case where the ganging
>>>> mode stitches together two synced sensors (left and right) into a single
>>>> 3D side-by-side image? Then you would have:
>>>>
>>>>  Left Sensor TX -> CSI RX0 (x4 port)
>>>> Right Sensor TX -> CSI RX1 (x4 port)
>>>>
>>>> And for the tc358840 something similar might be true: in the case of the
>>>> Tegra you have this nice ganging mode available, but for other SoCs each
>>>> half would have to go to a separate CSI port and captured via a separate
>>>> video DMA channel, and software or a GPU is needed to combine the two
>>>> halves.
>>>>
>>>> In both these examples it is my understanding that you have to model this
>>>> in the DT as separate x4 ports.
>>>
>>> Do note that a "port" as such is a logical concept. On modern hardware, a
>>> port consists of two or more lanes --- one clock, plus at least one data
>>> lane. Perhaps an example could be useful. For instance, if you have ten
>>> lanes on a device, this could be split into following configurations, based
>>> on the board design:
>>>
>>> configuration \ data lanes	port 0	port 1	port 2	port 3
>>>
>>> 1:				4	4
>>> 2:				4	2	1
>>> 3:				2	2	2
>>> 4:				2	2	1	1
>>>
>>> So if you add one more, say:
>>>
>>> 5:				8
>>>
>>> So what we're discussing is just how the lanes are distributed across the
>>> ports.
>>>
>>> There are usually hardware specific limitations how the lanes can be
>>> distributed. The interface we have in DT (data-lanes + clock-lanes
>>> properties) allows describing the hardware in general case, so what the
>>> interface allows may not be possible in hardware, but what hardware
>>> implements is supported by the interface.
>>>
>>
>> So for this particular instance using a single logical 8-lane port would
>> make sense, but in the two other scenarios (left/right sensor or supporting
>> tc358840 in a SoC that doesn't support ganging) I described you would still
>> have to model it as two 4-lane ports.
> 
> If you have two sensors, yes, then it'll be two separate ports; one sensor
> connected to each of them. The streams are usually separate, but other
> kinds of implementations exist. Still, they generally have no effect on
> CSI-2 bus configuration.
> 

OK, then we go with increasing the number of data lanes to 8.

Regards,

	Hans