v4l2: Adding support for multiple MIPI CSI-2 virtual channels

v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Thomas Axelsson]

Hi,

I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual Channels. I want to configure each virtual channel individually (e.g. set_fmt), but the v4l2 interface does not seem to have a clear way to access configuration on a virtual channel level, but only the v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual channels by extending the "reg" tag to be an array looks like the correct way to do it, based on the mipi-dsi-bus.txt document and current device tree endpoint structure.

However, I cannot figure out how to extend e.g. set_fmt/get_fmt subdev ioctls to specify which virtual channel the call applies to. Does anyone have any advice on how to handle this case?

Previous thread: "Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?"


Best Regards,
Thomas Axelsson

PS. First e-mail seems to have gotten caught in the spam filter. I apologize if this is a duplicate.

Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Ajay kumar]

Hi Everyone,

On Fri, Feb 17, 2017 at 7:27 PM, Thomas Axelsson
<Thomas.Axelsson@cybercom.com> wrote:
> Hi,
>
> I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual Channels. I want to configure each virtual channel individually (e.g. set_fmt), but the v4l2 interface does not seem to have a clear way to access configuration on a virtual channel level, but only the v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual channels by extending the "reg" tag to be an array looks like the correct way to do it, based on the mipi-dsi-bus.txt document and current device tree endpoint structure.
>
> However, I cannot figure out how to extend e.g. set_fmt/get_fmt subdev ioctls to specify which virtual channel the call applies to. Does anyone have any advice on how to handle this case?
This would be helpful for my project as well since even I need to
support multiple streams using Virtual Channels.
Can anyone point out to some V4L2 driver, if this kind of support is
already implemented?

Thanks.
>
> Previous thread: "Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?"
>
>
> Best Regards,
> Thomas Axelsson
>
> PS. First e-mail seems to have gotten caught in the spam filter. I apologize if this is a duplicate.

Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Guennadi Liakhovetski]

Hi,

On Tue, 21 Feb 2017, Ajay kumar wrote:

> Hi Everyone,
> 
> On Fri, Feb 17, 2017 at 7:27 PM, Thomas Axelsson
> <Thomas.Axelsson@cybercom.com> wrote:
> > Hi,
> >
> > I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual 
> > Channels. I want to configure each virtual channel individually (e.g. 
> > set_fmt), but the v4l2 interface does not seem to have a clear way to 
> > access configuration on a virtual channel level, but only the 
> > v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual 
> > channels by extending the "reg" tag to be an array looks like the 
> > correct way to do it, based on the mipi-dsi-bus.txt document and 
> > current device tree endpoint structure.
> >
> > However, I cannot figure out how to extend e.g. set_fmt/get_fmt subdev 
> > ioctls to specify which virtual channel the call applies to. Does 
> > anyone have any advice on how to handle this case?
> This would be helpful for my project as well since even I need to
> support multiple streams using Virtual Channels.
> Can anyone point out to some V4L2 driver, if this kind of support is
> already implemented?

My understanding is, that MIPI CSI virtual channel handling requires 
extensions to the V4L2 subdev API. These extensions have been discussed at 
a media mini-summit almost a year ago, slides are available at [1], but as 
my priorities shifted away from this work, don't think those extensions 
ever got implemented.

Thanks
Guennadi

[1] https://linuxtv.org/downloads/presentations/media_summit_2016_san_diego/v4l2-multistream.pdf

> 
> Thanks.
> >
> > Previous thread: "Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?"
> >
> >
> > Best Regards,
> > Thomas Axelsson
> >
> > PS. First e-mail seems to have gotten caught in the spam filter. I apologize if this is a duplicate.
> 

Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Laurent Pinchart]

Hi Guennadi,

On Wednesday 22 Feb 2017 18:54:20 Guennadi Liakhovetski wrote:
> On Tue, 21 Feb 2017, Ajay kumar wrote:
> > On Fri, Feb 17, 2017 at 7:27 PM, Thomas Axelsson wrote:
> >> Hi,
> >> 
> >> I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual
> >> Channels. I want to configure each virtual channel individually (e.g.
> >> set_fmt), but the v4l2 interface does not seem to have a clear way to
> >> access configuration on a virtual channel level, but only the
> >> v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual
> >> channels by extending the "reg" tag to be an array looks like the
> >> correct way to do it, based on the mipi-dsi-bus.txt document and
> >> current device tree endpoint structure.
> >> 
> >> However, I cannot figure out how to extend e.g. set_fmt/get_fmt subdev
> >> ioctls to specify which virtual channel the call applies to. Does
> >> anyone have any advice on how to handle this case?
> > 
> > This would be helpful for my project as well since even I need to
> > support multiple streams using Virtual Channels.
> > Can anyone point out to some V4L2 driver, if this kind of support is
> > already implemented?
> 
> My understanding is, that MIPI CSI virtual channel handling requires
> extensions to the V4L2 subdev API. These extensions have been discussed at
> a media mini-summit almost a year ago, slides are available at [1], but as
> my priorities shifted away from this work, don't think those extensions
> ever got implemented.

We've also discussed the topic last week in a face to face meeting with Niklas 
(CC'ed) and Sakari. Niklas will start working on upstreaming the necessary 
V4L2 API extensions for CSI-2 virtual channel support. The current plan is to 
start the work at the beginning of April.

> [1]
> https://linuxtv.org/downloads/presentations/media_summit_2016_san_diego/v4l
> 2-multistream.pdf
>
> >> Previous thread: "Device Tree formatting for multiple virtual channels
> >> in ti-vpe/cal driver?"
> >> 
> >> Best Regards,
> >> Thomas Axelsson
> >> 
> >> PS. First e-mail seems to have gotten caught in the spam filter. I
> >> apologize if this is a duplicate.

-- 
Regards,

Laurent Pinchart

RE: v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Thomas Axelsson]

Hi Laurent and Niklas,

> -----Original Message-----
> From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> Sent: den 22 februari 2017 20:33
> To: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> Cc: Ajay kumar <ajaynumb@gmail.com>; linux-media@vger.kernel.org;
> Marek Szyprowski <m.szyprowski@samsung.com>; Sylwester Nawrocki
> <sylvester.nawrocki@gmail.com>; Andrzej Hajda <a.hajda@samsung.com>;
> Thomas Axelsson <Thomas.Axelsson@cybercom.com>; Sakari Ailus
> <sakari.ailus@iki.fi>; Niklas Söderlund <niklas.soderlund@ragnatech.se>
> Subject: Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels
> 
> Hi Guennadi,
> 
> On Wednesday 22 Feb 2017 18:54:20 Guennadi Liakhovetski wrote:
> > On Tue, 21 Feb 2017, Ajay kumar wrote:
> > > On Fri, Feb 17, 2017 at 7:27 PM, Thomas Axelsson wrote:
> > >> Hi,
> > >>
> > >> I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual
> > >> Channels. I want to configure each virtual channel individually (e.g.
> > >> set_fmt), but the v4l2 interface does not seem to have a clear way
> > >> to access configuration on a virtual channel level, but only the
> > >> v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual
> > >> channels by extending the "reg" tag to be an array looks like the
> > >> correct way to do it, based on the mipi-dsi-bus.txt document and
> > >> current device tree endpoint structure.
> > >>
> > >> However, I cannot figure out how to extend e.g. set_fmt/get_fmt
> > >> subdev ioctls to specify which virtual channel the call applies to.
> > >> Does anyone have any advice on how to handle this case?
> > >
> > > This would be helpful for my project as well since even I need to
> > > support multiple streams using Virtual Channels.
> > > Can anyone point out to some V4L2 driver, if this kind of support is
> > > already implemented?
> >
> > My understanding is, that MIPI CSI virtual channel handling requires
> > extensions to the V4L2 subdev API. These extensions have been
> > discussed at a media mini-summit almost a year ago, slides are
> > available at [1], but as my priorities shifted away from this work,
> > don't think those extensions ever got implemented.
> 
> We've also discussed the topic last week in a face to face meeting with Niklas
> (CC'ed) and Sakari. Niklas will start working on upstreaming the necessary
> V4L2 API extensions for CSI-2 virtual channel support. The current plan is to
> start the work at the beginning of April.

I understand that the code is not ready for prime time, but it would not be 
optimal for me trying to implement a solution in parallel. Do you have some 
sketches or drafts that you can share, so that any temporary solution I make 
will hopefully not be too hard to adjust to match the final implementation?

Do you plan to use "pad", as suggested by Guennadi, to map virtual channels?

Regarding the ioctls, I had embarrassingly looked at the wrong struct, and I 
see that e.g. VIDIOC_SUBDEV_S_FMT accepts v4l2_subdev_format, which has a pad 
member. (Which I assume can be used to index in the v4l2_subdev_pad_config *cfg 
provided to the ioctl handler).

> 
> > [1]
> >
> https://linuxtv.org/downloads/presentations/media_summit_2016_san_die
> g
> > o/v4l
> > 2-multistream.pdf
> >
> > >> Previous thread: "Device Tree formatting for multiple virtual
> > >> channels in ti-vpe/cal driver?"
> > >>
> > >> Best Regards,
> > >> Thomas Axelsson
> > >>
> > >> PS. First e-mail seems to have gotten caught in the spam filter. I
> > >> apologize if this is a duplicate.
> 
> --
> Regards,
> 
> Laurent Pinchart

Best Regards,
Thomas Axelsson

Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels

[Niklas Söderlund]

Hi Thomas,

On 2017-02-23 10:07:14 +0000, Thomas Axelsson wrote:
> Hi Laurent and Niklas,
> 
> > -----Original Message-----
> > From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com]
> > Sent: den 22 februari 2017 20:33
> > To: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > Cc: Ajay kumar <ajaynumb@gmail.com>; linux-media@vger.kernel.org;
> > Marek Szyprowski <m.szyprowski@samsung.com>; Sylwester Nawrocki
> > <sylvester.nawrocki@gmail.com>; Andrzej Hajda <a.hajda@samsung.com>;
> > Thomas Axelsson <Thomas.Axelsson@cybercom.com>; Sakari Ailus
> > <sakari.ailus@iki.fi>; Niklas Söderlund <niklas.soderlund@ragnatech.se>
> > Subject: Re: v4l2: Adding support for multiple MIPI CSI-2 virtual channels
> > 
> > Hi Guennadi,
> > 
> > On Wednesday 22 Feb 2017 18:54:20 Guennadi Liakhovetski wrote:
> > > On Tue, 21 Feb 2017, Ajay kumar wrote:
> > > > On Fri, Feb 17, 2017 at 7:27 PM, Thomas Axelsson wrote:
> > > >> Hi,
> > > >>
> > > >> I have a v4l2_subdev that provides multiple MIPI CSI-2 Virtual
> > > >> Channels. I want to configure each virtual channel individually (e.g.
> > > >> set_fmt), but the v4l2 interface does not seem to have a clear way
> > > >> to access configuration on a virtual channel level, but only the
> > > >> v4l2_subdev as a whole. Using one v4l2_subdev for multiple virtual
> > > >> channels by extending the "reg" tag to be an array looks like the
> > > >> correct way to do it, based on the mipi-dsi-bus.txt document and
> > > >> current device tree endpoint structure.
> > > >>
> > > >> However, I cannot figure out how to extend e.g. set_fmt/get_fmt
> > > >> subdev ioctls to specify which virtual channel the call applies to.
> > > >> Does anyone have any advice on how to handle this case?
> > > >
> > > > This would be helpful for my project as well since even I need to
> > > > support multiple streams using Virtual Channels.
> > > > Can anyone point out to some V4L2 driver, if this kind of support is
> > > > already implemented?
> > >
> > > My understanding is, that MIPI CSI virtual channel handling requires
> > > extensions to the V4L2 subdev API. These extensions have been
> > > discussed at a media mini-summit almost a year ago, slides are
> > > available at [1], but as my priorities shifted away from this work,
> > > don't think those extensions ever got implemented.
> > 
> > We've also discussed the topic last week in a face to face meeting with Niklas
> > (CC'ed) and Sakari. Niklas will start working on upstreaming the necessary
> > V4L2 API extensions for CSI-2 virtual channel support. The current plan is to
> > start the work at the beginning of April.
> 
> I understand that the code is not ready for prime time, but it would not be 
> optimal for me trying to implement a solution in parallel. Do you have some 
> sketches or drafts that you can share, so that any temporary solution I make 
> will hopefully not be too hard to adjust to match the final implementation?

As Laurent said the plan is to start work on this in April so not much 
code exists yet that can be shared. We do have an idea on how to try and 
do this but I have yet to write it up in a easy to consume text. I'm 
currently on the road but hope to do write it up next week.

> 
> Do you plan to use "pad", as suggested by Guennadi, to map virtual channels?

In short the plan is to use pads that can mux/demux multiple streams 
to/from a single pad and then use routing inside each subdevice to 
connect 'normal' pads to the muxed pad.

On the CSi-2 transmitter subdev each sink pad receive its own stream 
which can be routed to the multiplexed source pad. On the CSI-2 receiver 
the sink pad is multiplexed and each stream coming in is demuxed to a 
separate source pad which once again will once more only carry a single 
stream.

The multiplexed source pad of the transmitter and the multiplexed sink 
pad of the receiver will have no format and user-space will have to look 
at the routing information inside the two subdevices to figure out which 
transmitter sink pad is connected to which receivers source pad to get a 
complete view of the formats in the media pipeline.

> 
> Regarding the ioctls, I had embarrassingly looked at the wrong struct, and I 
> see that e.g. VIDIOC_SUBDEV_S_FMT accepts v4l2_subdev_format, which has a pad 
> member. (Which I assume can be used to index in the v4l2_subdev_pad_config *cfg 
> provided to the ioctl handler).
> 
> > 
> > > [1]
> > >
> > https://linuxtv.org/downloads/presentations/media_summit_2016_san_die
> > g
> > > o/v4l
> > > 2-multistream.pdf
> > >
> > > >> Previous thread: "Device Tree formatting for multiple virtual
> > > >> channels in ti-vpe/cal driver?"
> > > >>
> > > >> Best Regards,
> > > >> Thomas Axelsson
> > > >>
> > > >> PS. First e-mail seems to have gotten caught in the spam filter. I
> > > >> apologize if this is a duplicate.
> > 
> > --
> > Regards,
> > 
> > Laurent Pinchart
> 
> Best Regards,
> Thomas Axelsson

-- 
Regards,
Niklas Söderlund