How to express planar formats with mediabus format code?

How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi,

I know the title looks crazy, but here is our problem:

In our SoC based ISP, the hardware can be divide to several blocks.
Some blocks can do color space conversion(raw to YUV
interleave/planar), others can do the pixel
re-order(interleave/planar/semi-planar conversion, UV planar switch).
We use one subdev to describe each of them, then came the problem: How
can we express the planar formats with mediabus format code?

I understand at beginning, media-bus was designed to describe the data
link between camera sensor and camera controller, where sensor is
described in subdev. So interleave formats looks good enough at that
time. But now as Media-controller is introduced, subdev can describe a
much wider range of hardware, which is not limited to camera sensor.
So now planar formats are possible to be passed between subdevs.

I think the problem we meet can be very common for SoC based ISP
solutions, what do you think about it?

there are many possible solution for it:

1> change the definition of v4l2_subdev_format::format, use v4l2_format;

2> extend the mediabus format code, add planar format code;

3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is
it in mediabus-format or in fourcc

 Do you have any suggestions?

 Thanks a lot!

Re: How to express planar formats with mediabus format code?

[Guennadi Liakhovetski]

Hi Su Jiaquan

On Sun, 4 Aug 2013, Su Jiaquan wrote:

> Hi,
> 
> I know the title looks crazy, but here is our problem:
> 
> In our SoC based ISP, the hardware can be divide to several blocks.
> Some blocks can do color space conversion(raw to YUV
> interleave/planar), others can do the pixel
> re-order(interleave/planar/semi-planar conversion, UV planar switch).
> We use one subdev to describe each of them, then came the problem: How
> can we express the planar formats with mediabus format code?

Could you please explain more exactly what you mean? How are those your 
blocks connected? How do they exchange data? If they exchange data over a 
serial bus, then I don't think planar formats make sense, right? Or do 
your blocks really output planes one after another, reordering data 
internally? That would be odd... If OTOH your blocks output data to RAM, 
and the next block takes data from there, then you use V4L2_PIX_FMT_* 
formats to describe them and any further processing block should be a 
mem2mem device. Wouldn't this work?

Thanks
Guennadi

> I understand at beginning, media-bus was designed to describe the data
> link between camera sensor and camera controller, where sensor is
> described in subdev. So interleave formats looks good enough at that
> time. But now as Media-controller is introduced, subdev can describe a
> much wider range of hardware, which is not limited to camera sensor.
> So now planar formats are possible to be passed between subdevs.
> 
> I think the problem we meet can be very common for SoC based ISP
> solutions, what do you think about it?
> 
> there are many possible solution for it:
> 
> 1> change the definition of v4l2_subdev_format::format, use v4l2_format;
> 
> 2> extend the mediabus format code, add planar format code;
> 
> 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is
> it in mediabus-format or in fourcc
> 
>  Do you have any suggestions?
> 
>  Thanks a lot!
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi Guennadi,

Thanks for the reply! Please see my description inline.

On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski
<g.liakhovetski@gmx.de> wrote:
> Hi Su Jiaquan
>
> On Sun, 4 Aug 2013, Su Jiaquan wrote:
>
>> Hi,
>>
>> I know the title looks crazy, but here is our problem:
>>
>> In our SoC based ISP, the hardware can be divide to several blocks.
>> Some blocks can do color space conversion(raw to YUV
>> interleave/planar), others can do the pixel
>> re-order(interleave/planar/semi-planar conversion, UV planar switch).
>> We use one subdev to describe each of them, then came the problem: How
>> can we express the planar formats with mediabus format code?
>
> Could you please explain more exactly what you mean? How are those your
> blocks connected? How do they exchange data? If they exchange data over a
> serial bus, then I don't think planar formats make sense, right? Or do
> your blocks really output planes one after another, reordering data
> internally? That would be odd... If OTOH your blocks output data to RAM,
> and the next block takes data from there, then you use V4L2_PIX_FMT_*
> formats to describe them and any further processing block should be a
> mem2mem device. Wouldn't this work?

These two hardware blocks are both located inside of ISP, and is
connected by a hardware data bus.

Actually, there are three blocks inside ISP: One is close to sensor,
and can do color space conversion(RGB->YUV), we call it IPC; The other
two are at back end, which are basically DMA Engine, and they are
identical. When data flow out of IPC, it can go into each one of these
DMA Engines and finally into RAM. Whether the DMA Engine is turned
on/off and the output format can be controlled independently. Since
they are DMA Engines, they have some basic pixel reordering
ability(i.e. interleave->planar/semi-planar).

In our H/W design, when we want to get YUV semi-planar format, the IPC
output should be configured to interleave, and the DMA engine will do
the interleave->semi-planar job. If we want planar / interleave
format, the IPC will output planar format directly, DMA engine simply
send the data to RAM, and don't do any re-order. So in the planar
output case, media-bus formats can't express the format of the data
between IPC and DMA Engine, that's the problem we meet.

We want to adopt a formal solution before we send our patch to the
community, that's where our headache comes.
>
> Thanks
> Guennadi
>
>> I understand at beginning, media-bus was designed to describe the data
>> link between camera sensor and camera controller, where sensor is
>> described in subdev. So interleave formats looks good enough at that
>> time. But now as Media-controller is introduced, subdev can describe a
>> much wider range of hardware, which is not limited to camera sensor.
>> So now planar formats are possible to be passed between subdevs.
>>
>> I think the problem we meet can be very common for SoC based ISP
>> solutions, what do you think about it?
>>
>> there are many possible solution for it:
>>
>> 1> change the definition of v4l2_subdev_format::format, use v4l2_format;
>>
>> 2> extend the mediabus format code, add planar format code;
>>
>> 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is
>> it in mediabus-format or in fourcc
>>
>>  Do you have any suggestions?
>>
>>  Thanks a lot!
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-media" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/

Thanks!

Jiaquan

Re: How to express planar formats with mediabus format code?

[Guennadi Liakhovetski]

Hi Su Jiaquan,

On Tue, 6 Aug 2013, Su Jiaquan wrote:

> Hi Guennadi,
> 
> Thanks for the reply! Please see my description inline.
> 
> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski
> <g.liakhovetski@gmx.de> wrote:
> > Hi Su Jiaquan
> >
> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
> >
> >> Hi,
> >>
> >> I know the title looks crazy, but here is our problem:
> >>
> >> In our SoC based ISP, the hardware can be divide to several blocks.
> >> Some blocks can do color space conversion(raw to YUV
> >> interleave/planar), others can do the pixel
> >> re-order(interleave/planar/semi-planar conversion, UV planar switch).
> >> We use one subdev to describe each of them, then came the problem: How
> >> can we express the planar formats with mediabus format code?
> >
> > Could you please explain more exactly what you mean? How are those your
> > blocks connected? How do they exchange data? If they exchange data over a
> > serial bus, then I don't think planar formats make sense, right? Or do
> > your blocks really output planes one after another, reordering data
> > internally? That would be odd... If OTOH your blocks output data to RAM,
> > and the next block takes data from there, then you use V4L2_PIX_FMT_*
> > formats to describe them and any further processing block should be a
> > mem2mem device. Wouldn't this work?
> 
> These two hardware blocks are both located inside of ISP, and is
> connected by a hardware data bus.
> 
> Actually, there are three blocks inside ISP: One is close to sensor,
> and can do color space conversion(RGB->YUV), we call it IPC; The other
> two are at back end, which are basically DMA Engine, and they are
> identical. When data flow out of IPC, it can go into each one of these
> DMA Engines and finally into RAM. Whether the DMA Engine is turned
> on/off and the output format can be controlled independently. Since
> they are DMA Engines, they have some basic pixel reordering
> ability(i.e. interleave->planar/semi-planar).
> 
> In our H/W design, when we want to get YUV semi-planar format, the IPC
> output should be configured to interleave, and the DMA engine will do
> the interleave->semi-planar job. If we want planar / interleave
> format, the IPC will output planar format directly, DMA engine simply
> send the data to RAM, and don't do any re-order. So in the planar
> output case, media-bus formats can't express the format of the data
> between IPC and DMA Engine, that's the problem we meet.

Ok, so, do I understand you correctly, that in the case, where IPC outputs 
planar data you have:

1. your sensor is sending data to IPC

Then one of the following happens

2a. IPC stores the complete frame first, and only when the frame is 
complete, it first outputs the Y plane amd then the UV plane.

A slight optimisation of this would be

2b. it outputs Y components as pixels arrive and stores UV data 
internally, and at the end of the frame it sends out UV

If this is indeed the case, well, then I'm on the same page with you - I 
don't know a standard solution for this, sorry. It seems to me, that you 
will indeed need a new mediabus pixel code for this - either a generic one 
or a vendor-specific one. Let's see what others say.

Thanks
Guennadi

> We want to adopt a formal solution before we send our patch to the
> community, that's where our headache comes.
> >
> > Thanks
> > Guennadi
> >
> >> I understand at beginning, media-bus was designed to describe the data
> >> link between camera sensor and camera controller, where sensor is
> >> described in subdev. So interleave formats looks good enough at that
> >> time. But now as Media-controller is introduced, subdev can describe a
> >> much wider range of hardware, which is not limited to camera sensor.
> >> So now planar formats are possible to be passed between subdevs.
> >>
> >> I think the problem we meet can be very common for SoC based ISP
> >> solutions, what do you think about it?
> >>
> >> there are many possible solution for it:
> >>
> >> 1> change the definition of v4l2_subdev_format::format, use v4l2_format;
> >>
> >> 2> extend the mediabus format code, add planar format code;
> >>
> >> 3> use a extra bit to tell the meaning of v4l2_mbus_framefmt::code, is
> >> it in mediabus-format or in fourcc
> >>
> >>  Do you have any suggestions?
> >>
> >>  Thanks a lot!
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >
> > ---
> > Guennadi Liakhovetski, Ph.D.
> > Freelance Open-Source Software Developer
> > http://www.open-technology.de/
> 
> Thanks!
> 
> Jiaquan
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: How to express planar formats with mediabus format code?

[Laurent Pinchart]

Hi,

On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
> Hi Guennadi,
> 
> Thanks for the reply! Please see my description inline.
> 
> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
> >> Hi,
> >> 
> >> I know the title looks crazy, but here is our problem:
> >> 
> >> In our SoC based ISP, the hardware can be divide to several blocks.
> >> Some blocks can do color space conversion(raw to YUV interleave/planar),
> >> others can do the pixel re-order(interleave/planar/semi-planar
> >> conversion, UV planar switch). We use one subdev to describe each of
> >> them, then came the problem: How can we express the planar formats with
> >> mediabus format code?
> > 
> > Could you please explain more exactly what you mean? How are those your
> > blocks connected? How do they exchange data? If they exchange data over a
> > serial bus, then I don't think planar formats make sense, right? Or do
> > your blocks really output planes one after another, reordering data
> > internally? That would be odd... If OTOH your blocks output data to RAM,
> > and the next block takes data from there, then you use V4L2_PIX_FMT_*
> > formats to describe them and any further processing block should be a
> > mem2mem device. Wouldn't this work?
> 
> These two hardware blocks are both located inside of ISP, and is connected
> by a hardware data bus.
> 
> Actually, there are three blocks inside ISP: One is close to sensor, and can
> do color space conversion(RGB->YUV), we call it IPC; The other two are at
> back end, which are basically DMA Engine, and they are identical. When data
> flow out of IPC, it can go into each one of these DMA Engines and finally
> into RAM. Whether the DMA Engine is turned on/off and the output format can
> be controlled independently. Since they are DMA Engines, they have some
> basic pixel reordering ability(i.e. interleave->planar/semi-planar).
> 
> In our H/W design, when we want to get YUV semi-planar format, the IPC
> output should be configured to interleave, and the DMA engine will do the
> interleave->semi-planar job. If we want planar / interleave format, the IPC
> will output planar format directly, DMA engine simply send the data to RAM,
> and don't do any re-order. So in the planar output case, media-bus formats
> can't express the format of the data between IPC and DMA Engine, that's the
> problem we meet.

If the format between the two subdevs is really planar, I don't see any 
problem defining a media bus pixel code for it. You will have to properly 
document the format of course.

I'm a bit surprised that the IPC could output planar data. It would need to 
buffer a whole image to do so, do you need to give it a temporary system RAM 
buffer ?

> We want to adopt a formal solution before we send our patch to the
> community, that's where our headache comes.

-- 
Regards,

Laurent Pinchart

Re: How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi Laurent / Guennadi,

On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> Hi,
>
> On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
>> Hi Guennadi,
>>
>> Thanks for the reply! Please see my description inline.
>>
>> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
>> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
>> >> Hi,
>> >>
>> >> I know the title looks crazy, but here is our problem:
>> >>
>> >> In our SoC based ISP, the hardware can be divide to several blocks.
>> >> Some blocks can do color space conversion(raw to YUV interleave/planar),
>> >> others can do the pixel re-order(interleave/planar/semi-planar
>> >> conversion, UV planar switch). We use one subdev to describe each of
>> >> them, then came the problem: How can we express the planar formats with
>> >> mediabus format code?
>> >
>> > Could you please explain more exactly what you mean? How are those your
>> > blocks connected? How do they exchange data? If they exchange data over a
>> > serial bus, then I don't think planar formats make sense, right? Or do
>> > your blocks really output planes one after another, reordering data
>> > internally? That would be odd... If OTOH your blocks output data to RAM,
>> > and the next block takes data from there, then you use V4L2_PIX_FMT_*
>> > formats to describe them and any further processing block should be a
>> > mem2mem device. Wouldn't this work?
>>
>> These two hardware blocks are both located inside of ISP, and is connected
>> by a hardware data bus.
>>
>> Actually, there are three blocks inside ISP: One is close to sensor, and can
>> do color space conversion(RGB->YUV), we call it IPC; The other two are at
>> back end, which are basically DMA Engine, and they are identical. When data
>> flow out of IPC, it can go into each one of these DMA Engines and finally
>> into RAM. Whether the DMA Engine is turned on/off and the output format can
>> be controlled independently. Since they are DMA Engines, they have some
>> basic pixel reordering ability(i.e. interleave->planar/semi-planar).
>>
>> In our H/W design, when we want to get YUV semi-planar format, the IPC
>> output should be configured to interleave, and the DMA engine will do the
>> interleave->semi-planar job. If we want planar / interleave format, the IPC
>> will output planar format directly, DMA engine simply send the data to RAM,
>> and don't do any re-order. So in the planar output case, media-bus formats
>> can't express the format of the data between IPC and DMA Engine, that's the
>> problem we meet.
>
> If the format between the two subdevs is really planar, I don't see any
> problem defining a media bus pixel code for it. You will have to properly
> document the format of course.
>
> I'm a bit surprised that the IPC could output planar data. It would need to
> buffer a whole image to do so, do you need to give it a temporary system RAM
> buffer ?
>
>> We want to adopt a formal solution before we send our patch to the
>> community, that's where our headache comes.
>
> --
> Regards,
>
> Laurent Pinchart
>

Thanks for the reply!

Actually, we don't need to buffer the frame inside IPC, there are
three channels in the data bus. When transfering interleave format,
only one channel is used, for planar formats, three channels send one
planar each, and to difference address(Let me confirm this with our
H/W team and get back to you later). So the planars is not sent one
after an other, but in parallel.

This may be a bit different from the planar formats as people think it
should be. Can we use planar format to describe it? Since this won't
cause any misunderstanding given it's used in this special case.
Please advice.

Thanks a lot!

Jiaquan

Re: How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi Laurent / Guennadi,

On Sat, Aug 10, 2013 at 1:06 AM, Su Jiaquan <jiaquan.lnx@gmail.com> wrote:
> Hi Laurent / Guennadi,
>
> On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
>> Hi,
>>
>> On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
>>> Hi Guennadi,
>>>
>>> Thanks for the reply! Please see my description inline.
>>>
>>> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
>>> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
>>> >> Hi,
>>> >>
>>> >> I know the title looks crazy, but here is our problem:
>>> >>
>>> >> In our SoC based ISP, the hardware can be divide to several blocks.
>>> >> Some blocks can do color space conversion(raw to YUV interleave/planar),
>>> >> others can do the pixel re-order(interleave/planar/semi-planar
>>> >> conversion, UV planar switch). We use one subdev to describe each of
>>> >> them, then came the problem: How can we express the planar formats with
>>> >> mediabus format code?
>>> >
>>> > Could you please explain more exactly what you mean? How are those your
>>> > blocks connected? How do they exchange data? If they exchange data over a
>>> > serial bus, then I don't think planar formats make sense, right? Or do
>>> > your blocks really output planes one after another, reordering data
>>> > internally? That would be odd... If OTOH your blocks output data to RAM,
>>> > and the next block takes data from there, then you use V4L2_PIX_FMT_*
>>> > formats to describe them and any further processing block should be a
>>> > mem2mem device. Wouldn't this work?
>>>
>>> These two hardware blocks are both located inside of ISP, and is connected
>>> by a hardware data bus.
>>>
>>> Actually, there are three blocks inside ISP: One is close to sensor, and can
>>> do color space conversion(RGB->YUV), we call it IPC; The other two are at
>>> back end, which are basically DMA Engine, and they are identical. When data
>>> flow out of IPC, it can go into each one of these DMA Engines and finally
>>> into RAM. Whether the DMA Engine is turned on/off and the output format can
>>> be controlled independently. Since they are DMA Engines, they have some
>>> basic pixel reordering ability(i.e. interleave->planar/semi-planar).
>>>
>>> In our H/W design, when we want to get YUV semi-planar format, the IPC
>>> output should be configured to interleave, and the DMA engine will do the
>>> interleave->semi-planar job. If we want planar / interleave format, the IPC
>>> will output planar format directly, DMA engine simply send the data to RAM,
>>> and don't do any re-order. So in the planar output case, media-bus formats
>>> can't express the format of the data between IPC and DMA Engine, that's the
>>> problem we meet.
>>
>> If the format between the two subdevs is really planar, I don't see any
>> problem defining a media bus pixel code for it. You will have to properly
>> document the format of course.
>>
>> I'm a bit surprised that the IPC could output planar data. It would need to
>> buffer a whole image to do so, do you need to give it a temporary system RAM
>> buffer ?
>>
>>> We want to adopt a formal solution before we send our patch to the
>>> community, that's where our headache comes.
>>
>> --
>> Regards,
>>
>> Laurent Pinchart
>>
>
> Thanks for the reply!
>
> Actually, we don't need to buffer the frame inside IPC, there are
> three channels in the data bus. When transfering interleave format,
> only one channel is used, for planar formats, three channels send one
> planar each, and to difference address(Let me confirm this with our
> H/W team and get back to you later). So the planars is not sent one
> after an other, but in parallel.
>
> This may be a bit different from the planar formats as people think it
> should be. Can we use planar format to describe it? Since this won't
> cause any misunderstanding given it's used in this special case.
> Please advice.
>
> Thanks a lot!
>
> Jiaquan

I have to say I'm sorry for the wrong information. I just double
checked with hardware team, and turns out there is only one channel
for the data bus between IPC and ispdma.
If ispdma output planar format, the data format between IPC and ispdma
should be configured to a special format, that is not the same with
any know media-bus format.
So I think what we need is to define vendor-specific media-bus code.
Since others any want to do the same thing, shall we define a base
address for the vendor specific formats? For example:

enum v4l2_mbus_pixelcode {
    V4L2_MBUS_FMT_FIXED = 0x0001,

    ...

/* JPEG compressed formats - next is 0x4002 */
    V4L2_MBUS_FMT_JPEG_1X8 = 0x4001,
+   V4L2_MBUS_FMT_PRIVATE_BASE = 0xF001,
};

If you are OK with this, I'll prepare a patch to add it

Thanks!

Jiaquan

Re: How to express planar formats with mediabus format code?

[Laurent Pinchart]

Hi Jiaquan,

On Thursday 15 August 2013 16:27:43 Su Jiaquan wrote:
> On Sat, Aug 10, 2013 at 1:06 AM, Su Jiaquan <jiaquan.lnx@gmail.com> wrote:
> > On Fri, Aug 9, 2013 at 5:12 AM, Laurent Pinchart wrote:
> >> On Tuesday 06 August 2013 17:18:14 Su Jiaquan wrote:
> >>> On Mon, Aug 5, 2013 at 5:02 AM, Guennadi Liakhovetski wrote:
> >>> > On Sun, 4 Aug 2013, Su Jiaquan wrote:
> >>> >> Hi,
> >>> >> 
> >>> >> I know the title looks crazy, but here is our problem:
> >>> >> 
> >>> >> In our SoC based ISP, the hardware can be divide to several blocks.
> >>> >> Some blocks can do color space conversion(raw to YUV
> >>> >> interleave/planar), others can do the pixel re-
> >>> >> order(interleave/planar/semi-planar conversion, UV planar switch). We
> >>> >> use one subdev to describe each of them, then came the problem: How
> >>> >> can we express the planar formats with mediabus format code?
> >>> > 
> >>> > Could you please explain more exactly what you mean? How are those
> >>> > your blocks connected? How do they exchange data? If they exchange
> >>> > data over a serial bus, then I don't think planar formats make sense,
> >>> > right? Or do your blocks really output planes one after another,
> >>> > reordering data internally? That would be odd... If OTOH your blocks
> >>> > output data to RAM, and the next block takes data from there, then you
> >>> > use V4L2_PIX_FMT_* formats to describe them and any further processing
> >>> > block should be a mem2mem device. Wouldn't this work?
> >>> 
> >>> These two hardware blocks are both located inside of ISP, and is
> >>> connected by a hardware data bus.
> >>> 
> >>> Actually, there are three blocks inside ISP: One is close to sensor, and
> >>> can do color space conversion(RGB->YUV), we call it IPC; The other two
> >>> are at back end, which are basically DMA Engine, and they are
> >>> identical. When data flow out of IPC, it can go into each one of these
> >>> DMA Engines and finally into RAM. Whether the DMA Engine is turned
> >>> on/off and the output format can be controlled independently. Since
> >>> they are DMA Engines, they have some basic pixel reordering
> >>> ability(i.e. interleave->planar/semi-planar).
> >>> 
> >>> In our H/W design, when we want to get YUV semi-planar format, the IPC
> >>> output should be configured to interleave, and the DMA engine will do
> >>> the interleave->semi-planar job. If we want planar / interleave format,
> >>> the IPC will output planar format directly, DMA engine simply send the
> >>> data to RAM, and don't do any re-order. So in the planar output case,
> >>> media-bus formats can't express the format of the data between IPC and
> >>> DMA Engine, that's the problem we meet.
> >> 
> >> If the format between the two subdevs is really planar, I don't see any
> >> problem defining a media bus pixel code for it. You will have to properly
> >> document the format of course.
> >> 
> >> I'm a bit surprised that the IPC could output planar data. It would need
> >> to buffer a whole image to do so, do you need to give it a temporary
> >> system RAM buffer ?
> >> 
> >>> We want to adopt a formal solution before we send our patch to the
> >>> community, that's where our headache comes.
> > 
> > Thanks for the reply!
> > 
> > Actually, we don't need to buffer the frame inside IPC, there are
> > three channels in the data bus. When transfering interleave format,
> > only one channel is used, for planar formats, three channels send one
> > planar each, and to difference address(Let me confirm this with our
> > H/W team and get back to you later). So the planars is not sent one
> > after an other, but in parallel.
> > 
> > This may be a bit different from the planar formats as people think it
> > should be. Can we use planar format to describe it? Since this won't
> > cause any misunderstanding given it's used in this special case.
> > Please advice.
> > 
> 
> I have to say I'm sorry for the wrong information. I just double checked
> with hardware team, and turns out there is only one channel for the data bus
> between IPC and ispdma.
> If ispdma output planar format, the data format between IPC and ispdma
> should be configured to a special format, that is not the same with any know
> media-bus format.
> So I think what we need is to define vendor-specific media-bus code.
> Since others any want to do the same thing, shall we define a base address
> for the vendor specific formats? For example:
> 
> enum v4l2_mbus_pixelcode {
>     V4L2_MBUS_FMT_FIXED = 0x0001,
> 
>     ...
> 
> /* JPEG compressed formats - next is 0x4002 */
>     V4L2_MBUS_FMT_JPEG_1X8 = 0x4001,
> +   V4L2_MBUS_FMT_PRIVATE_BASE = 0xF001,
> };
> 
> If you are OK with this, I'll prepare a patch to add it

I'm not sure if that's needed here. Vendor-specific formats still need to be 
documented, so we could just create a custom YUV format for your case. Let's 
start with the beginning, could you describe what gets transmitted on the bus 
when that special format is selected ?

-- 
Regards,

Laurent Pinchart

Re: How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi Laurent,

Thanks for the replay, I've removed earlier mail content and only keep
you question:

On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> Hi Jiaquan,
>
> I'm not sure if that's needed here. Vendor-specific formats still need to be
> documented, so we could just create a custom YUV format for your case. Let's
> start with the beginning, could you describe what gets transmitted on the bus
> when that special format is selected ?
>
> --
> Regards,
>
> Laurent Pinchart
>

For YUV420P format, the data format sent from IPC is similar to
V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different:
For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV...
then DMA engine send them to RAM in planar format.

For YUV420SP format, the data format sent from IPC is YYUV YYUV
YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV
every other line, then send them to RAM as semi-planar.

Well, the first data format is too odd, I don't have a clue how to
call it, do you have suggestion?

Thanks a lot!

Jiaquan

Re: How to express planar formats with mediabus format code?

[Laurent Pinchart]

Hi Jiaquan,

On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote:
> On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote:
> > Hi Jiaquan,
> > 
> > I'm not sure if that's needed here. Vendor-specific formats still need to
> > be documented, so we could just create a custom YUV format for your case.
> > Let's start with the beginning, could you describe what gets transmitted
> > on the bus when that special format is selected ?
> 
> For YUV420P format, the data format sent from IPC is similar to
> V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different:
> For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV...
> then DMA engine send them to RAM in planar format.
> 
> For YUV420SP format, the data format sent from IPC is YYUV YYUV
> YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV
> every other line, then send them to RAM as semi-planar.

V4L2_MBUS_FMT_YYUV8_2X8 looks good to me.

> Well, the first data format is too odd, I don't have a clue how to
> call it, do you have suggestion?

Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty 
creative for these issues.

-- 
Regards,

Laurent Pinchart

Re: How to express planar formats with mediabus format code?

[Su Jiaquan]

Hi Sakari,

On Thu, Aug 22, 2013 at 7:29 PM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> Hi Jiaquan,
>
> On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote:
>> On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote:
>> > Hi Jiaquan,
>> >
>> > I'm not sure if that's needed here. Vendor-specific formats still need to
>> > be documented, so we could just create a custom YUV format for your case.
>> > Let's start with the beginning, could you describe what gets transmitted
>> > on the bus when that special format is selected ?
>>
>> For YUV420P format, the data format sent from IPC is similar to
>> V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different:
>> For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV...
>> then DMA engine send them to RAM in planar format.
>>
>> For YUV420SP format, the data format sent from IPC is YYUV YYUV
>> YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV
>> every other line, then send them to RAM as semi-planar.
>
> V4L2_MBUS_FMT_YYUV8_2X8 looks good to me.
>
>> Well, the first data format is too odd, I don't have a clue how to
>> call it, do you have suggestion?
>
> Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty
> creative for these issues.
>
> --
> Regards,
>
> Laurent Pinchart
>

Does the format V4L2_MBUS_FMT_YU8_YV8_1_5X8 sounds good to you? Do you
have better idea how we should describe this format?

If there is no further concern, I'll prepare a patch

Thanks

Re: How to express planar formats with mediabus format code?

[Sakari Ailus]

Hi Jiaquan and Laurent,

Apologies for my delayed reply.

Su Jiaquan wrote:
> Hi Sakari,
>
> On Thu, Aug 22, 2013 at 7:29 PM, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
>> Hi Jiaquan,
>>
>> On Wednesday 21 August 2013 18:14:50 Su Jiaquan wrote:
>>> On Tue, Aug 20, 2013 at 8:53 PM, Laurent Pinchart wrote:
>>>> Hi Jiaquan,
>>>>
>>>> I'm not sure if that's needed here. Vendor-specific formats still need to
>>>> be documented, so we could just create a custom YUV format for your case.
>>>> Let's start with the beginning, could you describe what gets transmitted
>>>> on the bus when that special format is selected ?
>>>
>>> For YUV420P format, the data format sent from IPC is similar to
>>> V4L2_MBUS_FMT_YUYV8_1_5X8, but the content for each line is different:
>>> For odd line, it's YYU YYU YYU... For even line, it's YYV YYV YYV...
>>> then DMA engine send them to RAM in planar format.
>>>
>>> For YUV420SP format, the data format sent from IPC is YYUV YYUV
>>> YYUV(maybe called V4L2_MBUS_FMT_YYUV8_2X8?), but DMA engine drop UV
>>> every other line, then send them to RAM as semi-planar.
>>
>> V4L2_MBUS_FMT_YYUV8_2X8 looks good to me.
>>
>>> Well, the first data format is too odd, I don't have a clue how to
>>> call it, do you have suggestion?
>>
>> Maybe V4L2_MBUS_FMT_YU8_YV8_1_5X8 ? I've CC'ed Sakari Ailus, he's often pretty
>> creative for these issues.
>>
>> --
>> Regards,
>>
>> Laurent Pinchart
>>
>
> Does the format V4L2_MBUS_FMT_YU8_YV8_1_5X8 sounds good to you? Do you
> have better idea how we should describe this format?

If bus (or DMA) transfers 8 bits at a time, then yes. Otherwise perhaps
...1X12.

The documentation should be extended to cover different components on 
alternating lines; I don't think we've had such cases before. I think 
just a note telling to do exactly as above should suffice.

-- 
Kind regards,

Sakari Ailus
sakari.ailus@iki.fi