V4L2 runs out of memory when OMAP3 ISP parallel pixel clock is high

V4L2 runs out of memory when OMAP3 ISP parallel pixel clock is high

[Adam Ford]

I am running a DM3730 connected to an mt9p031 sensor, and the ISP is
running in 8-bit parallel mode.

I have the sensor endpoint configured as:

mt9p031_out: endpoint {
     input-clock-frequency = <24000000>;
     pixel-clock-frequency = <72000000>;
     remote-endpoint = <&ccdc_ep>;
};

I was looking through the datasheet, and it appears as if the pixel
clock frequency can go up to 96MHz, so I tried to increase the
pixel-clock-frequency to 96MHz, but v4l2 seems to get an out of memory
error.

libv4l2: error turning on stream: No space left on device
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
to allocate required memory.
Additional debug info:
gstv4l2src.c(658): gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.019073486

Through trial and error, I was able to get push the sensor's
pixel-clock-frequency to work at 90MHz, but no higher.  I have also
tried experimenting with the input clock frequency without success.

If I can get the clock to run at 96MHz, which the ISP and sensor
documentation appears to permit, I am hoping to be able to achieve a
little higher frame rate.

Is there something I need to do to allocate more memory to V4L2 or is
there some other limitation causing the out of memory at higher pixel
clock frequencies?


thank you,

adam

Re: V4L2 runs out of memory when OMAP3 ISP parallel pixel clock is high

[Laurent Pinchart]

Hi Adam,

On Sun, Oct 20, 2019 at 09:45:25AM -0500, Adam Ford wrote:
> I am running a DM3730 connected to an mt9p031 sensor, and the ISP is
> running in 8-bit parallel mode.
> 
> I have the sensor endpoint configured as:
> 
> mt9p031_out: endpoint {
>      input-clock-frequency = <24000000>;
>      pixel-clock-frequency = <72000000>;
>      remote-endpoint = <&ccdc_ep>;
> };
> 
> I was looking through the datasheet, and it appears as if the pixel
> clock frequency can go up to 96MHz, so I tried to increase the
> pixel-clock-frequency to 96MHz, but v4l2 seems to get an out of memory
> error.
> 
> libv4l2: error turning on stream: No space left on device
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
> to allocate required memory.
> Additional debug info:
> gstv4l2src.c(658): gst_v4l2src_decide_allocation ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Buffer pool activation failed
> Execution ended after 0:00:00.019073486

The error code may be misleading. ENOSPC is used here to report that the
maximum CCDC bandwidth has been exceeded, not that the driver is running
out of memory.

The check is performed in isp_video_check_external_subdevs(), and the
maximum CCDC bandwidth is calculated by omap3isp_ccdc_max_rate():

	/*
	 * TRM says that for parallel sensors the maximum data rate
	 * should be 90% form L3/2 clock, otherwise just L3/2.
	 */
	if (ccdc->input == CCDC_INPUT_PARALLEL)
		rate = pipe->l3_ick / 2 * 9 / 10;
	else
		rate = pipe->l3_ick / 2;

Could you point me to the part of the OMAP3 datasheet that you think
allows for 96 MHz ?

> Through trial and error, I was able to get push the sensor's
> pixel-clock-frequency to work at 90MHz, but no higher.  I have also
> tried experimenting with the input clock frequency without success.
> 
> If I can get the clock to run at 96MHz, which the ISP and sensor
> documentation appears to permit, I am hoping to be able to achieve a
> little higher frame rate.
> 
> Is there something I need to do to allocate more memory to V4L2 or is
> there some other limitation causing the out of memory at higher pixel
> clock frequencies?

-- 
Regards,

Laurent Pinchart

Re: V4L2 runs out of memory when OMAP3 ISP parallel pixel clock is high

[Adam Ford]

On Sun, Oct 20, 2019 at 1:02 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Adam,
>
> On Sun, Oct 20, 2019 at 09:45:25AM -0500, Adam Ford wrote:
> > I am running a DM3730 connected to an mt9p031 sensor, and the ISP is
> > running in 8-bit parallel mode.
> >
> > I have the sensor endpoint configured as:
> >
> > mt9p031_out: endpoint {
> >      input-clock-frequency = <24000000>;
> >      pixel-clock-frequency = <72000000>;
> >      remote-endpoint = <&ccdc_ep>;
> > };
> >
> > I was looking through the datasheet, and it appears as if the pixel
> > clock frequency can go up to 96MHz, so I tried to increase the
> > pixel-clock-frequency to 96MHz, but v4l2 seems to get an out of memory
> > error.
> >
> > libv4l2: error turning on stream: No space left on device
> > ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
> > to allocate required memory.
> > Additional debug info:
> > gstv4l2src.c(658): gst_v4l2src_decide_allocation ():
> > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> > Buffer pool activation failed
> > Execution ended after 0:00:00.019073486
>
> The error code may be misleading. ENOSPC is used here to report that the
> maximum CCDC bandwidth has been exceeded, not that the driver is running
> out of memory.
>
> The check is performed in isp_video_check_external_subdevs(), and the
> maximum CCDC bandwidth is calculated by omap3isp_ccdc_max_rate():
>
>         /*
>          * TRM says that for parallel sensors the maximum data rate
>          * should be 90% form L3/2 clock, otherwise just L3/2.
>          */
>         if (ccdc->input == CCDC_INPUT_PARALLEL)
>                 rate = pipe->l3_ick / 2 * 9 / 10;
>         else
>                 rate = pipe->l3_ick / 2;
>
> Could you point me to the part of the OMAP3 datasheet that you think
> allows for 96 MHz ?

The DM3730 TRM (SPRUGN4R – May 2010 – Revised September 2012) doesn't
use the 90% rule mentioned above from what I can see. Maybe it's
somewhere else, but I didn't see it.  It does state the folllowing in
Section 6.3.1.1:
Camera ISP Clocks Parallel interface clock domain. This frequency
depends on the imaging sensor type and size, its frame rate and its
blanking time. The functional clock is required to be at least 2x
faster than the pixel clock when the bridge is disabled and a least
equal when it is enabled.

When I queried the cam_ick, it returned 100MHz, so I expected 96MHz to
be tolerated.

# cat cat /sys/kernel/debug/clk/l3_ick/clk_rate
200000000

# cat /sys/kernel/debug/clk/cam_ick/clk_rate
100000000

For what it's worth, I removed the 90% calculation and just use the L3
/ 2.  With that done, I was able to push the camera to 96MHz, and the
frame rate increased

Can you point me to where this 90% requirement is located?

adam
>
> > Through trial and error, I was able to get push the sensor's
> > pixel-clock-frequency to work at 90MHz, but no higher.  I have also
> > tried experimenting with the input clock frequency without success.
> >
> > If I can get the clock to run at 96MHz, which the ISP and sensor
> > documentation appears to permit, I am hoping to be able to achieve a
> > little higher frame rate.
> >
> > Is there something I need to do to allocate more memory to V4L2 or is
> > there some other limitation causing the out of memory at higher pixel
> > clock frequencies?
>
> --
> Regards,
>
> Laurent Pinchart

Re: V4L2 runs out of memory when OMAP3 ISP parallel pixel clock is high

[Adam Ford]

On Sun, Oct 20, 2019 at 2:16 PM Adam Ford <aford173@gmail.com> wrote:
>
> On Sun, Oct 20, 2019 at 1:02 PM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Adam,
> >
> > On Sun, Oct 20, 2019 at 09:45:25AM -0500, Adam Ford wrote:
> > > I am running a DM3730 connected to an mt9p031 sensor, and the ISP is
> > > running in 8-bit parallel mode.
> > >
> > > I have the sensor endpoint configured as:
> > >
> > > mt9p031_out: endpoint {
> > >      input-clock-frequency = <24000000>;
> > >      pixel-clock-frequency = <72000000>;
> > >      remote-endpoint = <&ccdc_ep>;
> > > };
> > >
> > > I was looking through the datasheet, and it appears as if the pixel
> > > clock frequency can go up to 96MHz, so I tried to increase the
> > > pixel-clock-frequency to 96MHz, but v4l2 seems to get an out of memory
> > > error.
> > >
> > > libv4l2: error turning on stream: No space left on device
> > > ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
> > > to allocate required memory.
> > > Additional debug info:
> > > gstv4l2src.c(658): gst_v4l2src_decide_allocation ():
> > > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> > > Buffer pool activation failed
> > > Execution ended after 0:00:00.019073486
> >
> > The error code may be misleading. ENOSPC is used here to report that the
> > maximum CCDC bandwidth has been exceeded, not that the driver is running
> > out of memory.
> >
> > The check is performed in isp_video_check_external_subdevs(), and the
> > maximum CCDC bandwidth is calculated by omap3isp_ccdc_max_rate():
> >
> >         /*
> >          * TRM says that for parallel sensors the maximum data rate
> >          * should be 90% form L3/2 clock, otherwise just L3/2.
> >          */
> >         if (ccdc->input == CCDC_INPUT_PARALLEL)
> >                 rate = pipe->l3_ick / 2 * 9 / 10;
> >         else
> >                 rate = pipe->l3_ick / 2;
> >
> > Could you point me to the part of the OMAP3 datasheet that you think
> > allows for 96 MHz ?
>
> The DM3730 TRM (SPRUGN4R – May 2010 – Revised September 2012) doesn't
> use the 90% rule mentioned above from what I can see. Maybe it's
> somewhere else, but I didn't see it.  It does state the folllowing in
> Section 6.3.1.1:
> Camera ISP Clocks Parallel interface clock domain. This frequency
> depends on the imaging sensor type and size, its frame rate and its
> blanking time. The functional clock is required to be at least 2x
> faster than the pixel clock when the bridge is disabled and a least
> equal when it is enabled.
>
> When I queried the cam_ick, it returned 100MHz, so I expected 96MHz to
> be tolerated.
>
> # cat cat /sys/kernel/debug/clk/l3_ick/clk_rate
> 200000000
>
> # cat /sys/kernel/debug/clk/cam_ick/clk_rate
> 100000000
>
> For what it's worth, I removed the 90% calculation and just use the L3
> / 2.  With that done, I was able to push the camera to 96MHz, and the
> frame rate increased
>
> Can you point me to where this 90% requirement is located?
>

Laurent,

I think I figured out where the 90% calculation is located, but I can
only see it in the OMAP3530 TRM and not in the DM3730.

In the OMAP3530, Section 12.4.6.1.1 CCDC Features:

– Data up to 8-bit at 130 MHz can be transferred to memory.
– Data up to 10-bit at 75 MHz can be processed by the image pipeline
or transferred to memory.
– Data up to 12-bit at 75 MHz can be transferred to memory or
internally converted into 10-bit data to be processed by the image
pipeline.

If we assume the L3 is running at 166, then 166/2 = 83 and 83 * .9 = 74.7.

Does this seem like the right place?  If so,  then my impression is
that we should check for 10 or 12-bit mode before doing this math.  If
we're running at 8-bit, it seems to me like we can run at 130MHz for
the OMAP35, and potentially at 166MHz for the DM3730.

The Resizer has some of its own limitations, so when its invoked it
appears to have it's own limitations per section 6.4.7.2.1:

The horizontal resizer output rate must not exceed half the functional
clock; Moreover, the horizontal resizer output rate must not exceed
100M pixels/s. This limitation applies only for the on-the-fly
processing input source.

Either way, this is still 1/2 the clock rate of L3 and not 90% of that.

If you'll permit me, I'd like to add some code to if we're an
omap36xx.  Is so, I'd like to skip the 90% calculation for those
devices.  I have already been able to show that with it removed, I can
still capture paralell images at 96MHz.

adam
> adam
> >
> > > Through trial and error, I was able to get push the sensor's
> > > pixel-clock-frequency to work at 90MHz, but no higher.  I have also
> > > tried experimenting with the input clock frequency without success.
> > >
> > > If I can get the clock to run at 96MHz, which the ISP and sensor
> > > documentation appears to permit, I am hoping to be able to achieve a
> > > little higher frame rate.
> > >
> > > Is there something I need to do to allocate more memory to V4L2 or is
> > > there some other limitation causing the out of memory at higher pixel
> > > clock frequencies?
> >
> > --
> > Regards,
> >
> > Laurent Pinchart