downscaling YUV fails

downscaling YUV fails

[Peter Meerwald]

Hello,

downscaling a YUV video from /dev/fb1 silently fails and results in 
incorrectly rendered data (each line is shifted a bit more to the right, 
turning vertical lines into diagonals) -- observed with Linux 3.6.11 on 
omap3/dm3730


test data (an image with vertical bars) is produced by:
gst-launch-0.10 videotestsrc ! video/x-raw-yuv, width=400, height=240 ! freeze ! omapfbsink

cd /sys/devices/platform/omapdss/overlay1
echo 400,240 > output_size
echo 100,100 > position
echo 220,140 > output_size
this downscales the YUV data from 400x240->220x140 and results in 
distorted output data (see http://pmeerw.net/overlay-distorted.jpg)

note that upscaling works

more observations:
setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK=4 results in correct downscaling 
with x_predecim=1/y_predecim=1

CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK=0 leads to distorted output with 
x_predecim=2/y_predecim=1



I think there is a bug downscaling YUV data when resorting to 
pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround 
-- any ideas how this can be fixed?

thanks, regards, p.

-- 

Peter Meerwald
+43-664-2444418 (mobile)

Re: downscaling YUV fails

[Archit Taneja]

Hi,

On Wednesday 19 December 2012 03:21 PM, Peter Meerwald wrote:
> Hello,
>
> downscaling a YUV video from /dev/fb1 silently fails and results in
> incorrectly rendered data (each line is shifted a bit more to the right,
> turning vertical lines into diagonals) -- observed with Linux 3.6.11 on
> omap3/dm3730
>
>
> test data (an image with vertical bars) is produced by:
> gst-launch-0.10 videotestsrc ! video/x-raw-yuv, width=400, height=240 ! freeze ! omapfbsink
>
> cd /sys/devices/platform/omapdss/overlay1
> echo 400,240 > output_size
> echo 100,100 > position
> echo 220,140 > output_size
> this downscales the YUV data from 400x240->220x140 and results in
> distorted output data (see http://pmeerw.net/overlay-distorted.jpg)
>
> note that upscaling works
>
> more observations:
> setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK=4 results in correct downscaling
> with x_predecim=1/y_predecim=1
>
> CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK=0 leads to distorted output with
> x_predecim=2/y_predecim=1
>

Configuring CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK to 0 will make the driver 
try to derive a closer accurate pixel clock, but it will sacrifice on 
the DISPC_FCLK used by scalar.

Configuring it to 4 will ensure that DISPC_FCLK is at least 4 times the 
pixel clock.

The DISPC HW can achieve more downscaling when the DISPC_FCLK is higher.

In the case we set it to 4, the DISPC_FCLK is fast enough to do the 
required downscaling, but in the case when it's set to 0, it might need 
to pre decimate rather than try to scale.

>
>
> I think there is a bug downscaling YUV data when resorting to
> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround
> -- any ideas how this can be fixed?

Pre-decimation should work for YUV formats also. Could you share DISPC 
reg dumps when this happens:

mount -t debugfs debugfs /sys/kernel/debug
cat /sys/kernel/debug/omapdss/dispc

Thanks,
Archit

Re: downscaling YUV fails

[Peter Meerwald]

Hello,

> In the case we set it to 4, the DISPC_FCLK is fast enough to do the required
> downscaling, but in the case when it's set to 0, it might need to pre decimate
> rather than try to scale.

this is my understanding as well

> > I think there is a bug downscaling YUV data when resorting to
> > pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround
> > -- any ideas how this can be fixed?

> Pre-decimation should work for YUV formats also. Could you share DISPC reg
> dumps when this happens:
> 
> mount -t debugfs debugfs /sys/kernel/debug
> cat /sys/kernel/debug/omapdss/dispc

please find the files http://pmeerw.net/ok.dispc (with FCK_PER_PCK=4) and 
http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)

thanks, p.

-- 

Peter Meerwald
+43-664-2444418 (mobile)

Re: downscaling YUV fails

[Tomi Valkeinen]

[-- Attachment #1: Type: text/plain, Size: 1312 bytes --]

On 2012-12-19 12:51, Peter Meerwald wrote:
> Hello,
> 
>> In the case we set it to 4, the DISPC_FCLK is fast enough to do the required
>> downscaling, but in the case when it's set to 0, it might need to pre decimate
>> rather than try to scale.
> 
> this is my understanding as well
> 
>>> I think there is a bug downscaling YUV data when resorting to
>>> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround
>>> -- any ideas how this can be fixed?
> 
>> Pre-decimation should work for YUV formats also. Could you share DISPC reg
>> dumps when this happens:
>>
>> mount -t debugfs debugfs /sys/kernel/debug
>> cat /sys/kernel/debug/omapdss/dispc
> 
> please find the files http://pmeerw.net/ok.dispc (with FCK_PER_PCK=4) and 
> http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)

Something funny is going on here. I can reproduce on omap3 with a few
hacks that reduce the clocks. The PICTURE_SIZE's width field is
different for working and non-working cases, but I think it should be
the same for both.

Even more interesting, with my tests I first setup the ovl with a test
picture, and see the issue (with RGB mode also). If I then use fbset to
first set the mode to YUV and then back to RGB, predecimation is not
used and the issue is not visible...

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: downscaling YUV fails

[Tomi Valkeinen]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On 2012-12-19 13:19, Tomi Valkeinen wrote:
> On 2012-12-19 12:51, Peter Meerwald wrote:
>> Hello,
>>
>>> In the case we set it to 4, the DISPC_FCLK is fast enough to do the required
>>> downscaling, but in the case when it's set to 0, it might need to pre decimate
>>> rather than try to scale.
>>
>> this is my understanding as well
>>
>>>> I think there is a bug downscaling YUV data when resorting to
>>>> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround
>>>> -- any ideas how this can be fixed?
>>
>>> Pre-decimation should work for YUV formats also. Could you share DISPC reg
>>> dumps when this happens:
>>>
>>> mount -t debugfs debugfs /sys/kernel/debug
>>> cat /sys/kernel/debug/omapdss/dispc
>>
>> please find the files http://pmeerw.net/ok.dispc (with FCK_PER_PCK=4) and 
>> http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)
> 
> Something funny is going on here. I can reproduce on omap3 with a few
> hacks that reduce the clocks. The PICTURE_SIZE's width field is
> different for working and non-working cases, but I think it should be
> the same for both.

Ah, never mind. Of course the picture_size needs to be modified if
predecimation is used...

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: downscaling YUV fails

[Archit Taneja]

On Wednesday 19 December 2012 04:51 PM, Tomi Valkeinen wrote:
> On 2012-12-19 13:19, Tomi Valkeinen wrote:
>> On 2012-12-19 12:51, Peter Meerwald wrote:
>>> Hello,
>>>
>>>> In the case we set it to 4, the DISPC_FCLK is fast enough to do the required
>>>> downscaling, but in the case when it's set to 0, it might need to pre decimate
>>>> rather than try to scale.
>>>
>>> this is my understanding as well
>>>
>>>>> I think there is a bug downscaling YUV data when resorting to
>>>>> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a workaround
>>>>> -- any ideas how this can be fixed?
>>>
>>>> Pre-decimation should work for YUV formats also. Could you share DISPC reg
>>>> dumps when this happens:
>>>>
>>>> mount -t debugfs debugfs /sys/kernel/debug
>>>> cat /sys/kernel/debug/omapdss/dispc
>>>
>>> please find the files http://pmeerw.net/ok.dispc (with FCK_PER_PCK=4) and
>>> http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)
>>
>> Something funny is going on here. I can reproduce on omap3 with a few
>> hacks that reduce the clocks. The PICTURE_SIZE's width field is
>> different for working and non-working cases, but I think it should be
>> the same for both.
>
> Ah, never mind. Of course the picture_size needs to be modified if
> predecimation is used...

You said that you see the issue with RGB too. Did you mean the 
picture_size, or visually also?

One thing I'm guessing is that DMA fetching with predecimation isn't 
really optimised for omap3, if the pixel clock in Peter's setup is high, 
the DISPC DMA might not be able to fetch pixels fast enough in 
predecimation case. That sort of supports the possibility of a skewed 
image seen.

Peter,

Do you see any SYNC_LOST or UNDERFLOW prints when this happens?

Archit

Re: downscaling YUV fails

[Tomi Valkeinen]

[-- Attachment #1: Type: text/plain, Size: 2544 bytes --]

On 2012-12-19 13:52, Archit Taneja wrote:
> On Wednesday 19 December 2012 04:51 PM, Tomi Valkeinen wrote:
>> On 2012-12-19 13:19, Tomi Valkeinen wrote:
>>> On 2012-12-19 12:51, Peter Meerwald wrote:
>>>> Hello,
>>>>
>>>>> In the case we set it to 4, the DISPC_FCLK is fast enough to do the
>>>>> required
>>>>> downscaling, but in the case when it's set to 0, it might need to
>>>>> pre decimate
>>>>> rather than try to scale.
>>>>
>>>> this is my understanding as well
>>>>
>>>>>> I think there is a bug downscaling YUV data when resorting to
>>>>>> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a
>>>>>> workaround
>>>>>> -- any ideas how this can be fixed?
>>>>
>>>>> Pre-decimation should work for YUV formats also. Could you share
>>>>> DISPC reg
>>>>> dumps when this happens:
>>>>>
>>>>> mount -t debugfs debugfs /sys/kernel/debug
>>>>> cat /sys/kernel/debug/omapdss/dispc
>>>>
>>>> please find the files http://pmeerw.net/ok.dispc (with
>>>> FCK_PER_PCK=4) and
>>>> http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)
>>>
>>> Something funny is going on here. I can reproduce on omap3 with a few
>>> hacks that reduce the clocks. The PICTURE_SIZE's width field is
>>> different for working and non-working cases, but I think it should be
>>> the same for both.
>>
>> Ah, never mind. Of course the picture_size needs to be modified if
>> predecimation is used...
> 
> You said that you see the issue with RGB too. Did you mean the
> picture_size, or visually also?

Visually. I see skewed image for both RGB and YUV modes on omap3.

It was a bit difficult to reproduce, but I first forced the clocks down,
and then I was able to get predecimation for RGB, but not YUV. The
reason was this:

		if (color_mode == OMAP_DSS_COLOR_RGB24U)
			core_clk <<= 1;

So for some reason, the core_clk requirement for RGB24U is x2 than for
others. I have _no_ idea why that is, the line has been there from the
start...

After I removed the if, so that the core_clk requirement is x2 always, I
also got predecimation for YUV, and I see the same skew.

> One thing I'm guessing is that DMA fetching with predecimation isn't
> really optimised for omap3, if the pixel clock in Peter's setup is high,
> the DISPC DMA might not be able to fetch pixels fast enough in
> predecimation case. That sort of supports the possibility of a skewed
> image seen.

The image I get is stable, and clearly something that happens when, say,
row_inc is one pixel too much, or similar.

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: downscaling YUV fails

[Archit Taneja]

On Wednesday 19 December 2012 05:29 PM, Tomi Valkeinen wrote:
> On 2012-12-19 13:52, Archit Taneja wrote:
>> On Wednesday 19 December 2012 04:51 PM, Tomi Valkeinen wrote:
>>> On 2012-12-19 13:19, Tomi Valkeinen wrote:
>>>> On 2012-12-19 12:51, Peter Meerwald wrote:
>>>>> Hello,
>>>>>
>>>>>> In the case we set it to 4, the DISPC_FCLK is fast enough to do the
>>>>>> required
>>>>>> downscaling, but in the case when it's set to 0, it might need to
>>>>>> pre decimate
>>>>>> rather than try to scale.
>>>>>
>>>>> this is my understanding as well
>>>>>
>>>>>>> I think there is a bug downscaling YUV data when resorting to
>>>>>>> pre-decimation; setting CONFIG_OMAP2_DSS_MIN_FCK_PER_PCK is a
>>>>>>> workaround
>>>>>>> -- any ideas how this can be fixed?
>>>>>
>>>>>> Pre-decimation should work for YUV formats also. Could you share
>>>>>> DISPC reg
>>>>>> dumps when this happens:
>>>>>>
>>>>>> mount -t debugfs debugfs /sys/kernel/debug
>>>>>> cat /sys/kernel/debug/omapdss/dispc
>>>>>
>>>>> please find the files http://pmeerw.net/ok.dispc (with
>>>>> FCK_PER_PCK=4) and
>>>>> http://pmeerw.net/fail.dispc (FCK_PER_PCK=0)
>>>>
>>>> Something funny is going on here. I can reproduce on omap3 with a few
>>>> hacks that reduce the clocks. The PICTURE_SIZE's width field is
>>>> different for working and non-working cases, but I think it should be
>>>> the same for both.
>>>
>>> Ah, never mind. Of course the picture_size needs to be modified if
>>> predecimation is used...
>>
>> You said that you see the issue with RGB too. Did you mean the
>> picture_size, or visually also?
>
> Visually. I see skewed image for both RGB and YUV modes on omap3.
>
> It was a bit difficult to reproduce, but I first forced the clocks down,
> and then I was able to get predecimation for RGB, but not YUV. The
> reason was this:
>
> 		if (color_mode == OMAP_DSS_COLOR_RGB24U)
> 			core_clk <<= 1;
>
> So for some reason, the core_clk requirement for RGB24U is x2 than for
> others. I have _no_ idea why that is, the line has been there from the
> start...
>
> After I removed the if, so that the core_clk requirement is x2 always, I
> also got predecimation for YUV, and I see the same skew.
>
>> One thing I'm guessing is that DMA fetching with predecimation isn't
>> really optimised for omap3, if the pixel clock in Peter's setup is high,
>> the DISPC DMA might not be able to fetch pixels fast enough in
>> predecimation case. That sort of supports the possibility of a skewed
>> image seen.
>
> The image I get is stable, and clearly something that happens when, say,
> row_inc is one pixel too much, or similar.

Ok, about the width in this case. The original width is 400, and what we 
finally want is 220. After pre-decimation, the width would become 200, 
and we would need our scalar to actually upscale to 220.

I am wondering if the driver gets confused when such a scenario occurs, 
or the DSS HW is weird when we upscale some predecimated content. If you 
look at the hinc(1024  * width/out_width) value in the FIR register, in 
the "ok" regdumps, the value is 1861, which points to downscaling. And 
in the fail case, it's 92, which is upscaling.

Archit

Re: downscaling YUV fails

[Tomi Valkeinen]

[-- Attachment #1: Type: text/plain, Size: 1922 bytes --]

On 2012-12-19 14:19, Archit Taneja wrote:

>> The image I get is stable, and clearly something that happens when, say,
>> row_inc is one pixel too much, or similar.
> 
> Ok, about the width in this case. The original width is 400, and what we
> finally want is 220. After pre-decimation, the width would become 200,
> and we would need our scalar to actually upscale to 220.
> 
> I am wondering if the driver gets confused when such a scenario occurs,
> or the DSS HW is weird when we upscale some predecimated content. If you
> look at the hinc(1024  * width/out_width) value in the FIR register, in
> the "ok" regdumps, the value is 1861, which points to downscaling. And
> in the fail case, it's 92, which is upscaling.

I can reproduce this with a very simple case. 400x200 -> 200x200, and
forcing horizontal predecimation. So this will cause no scaling to be
used, only the pixel_inc is used to downscale.

The VID1 registers are as follows. They look right to me...

DISPC_OVL_BA0(VID1)                                8d900000
DISPC_OVL_BA1(VID1)                                8d900000
DISPC_OVL_POSITION(VID1)                           00000000
DISPC_OVL_SIZE(VID1)                               00c700c7
DISPC_OVL_ATTRIBUTES(VID1)                         00608011
DISPC_OVL_FIFO_THRESHOLD(VID1)                     03ff03c0
DISPC_OVL_FIFO_SIZE_STATUS(VID1)                   00000400
DISPC_OVL_ROW_INC(VID1)                            00000001
DISPC_OVL_PIXEL_INC(VID1)                          00000005
DISPC_OVL_PRELOAD(VID1)                            00000100
DISPC_OVL_FIR(VID1)                                04000400
DISPC_OVL_PICTURE_SIZE(VID1)                       00c700c7
DISPC_OVL_ACCU0(VID1)                              00000000
DISPC_OVL_ACCU1(VID1)                              00000000
DISPC_OVL_PRELOAD(VID1)                            00000100

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: downscaling YUV fails

[Archit Taneja]

On Wednesday 19 December 2012 06:07 PM, Tomi Valkeinen wrote:
> On 2012-12-19 14:19, Archit Taneja wrote:
>
>>> The image I get is stable, and clearly something that happens when, say,
>>> row_inc is one pixel too much, or similar.
>>
>> Ok, about the width in this case. The original width is 400, and what we
>> finally want is 220. After pre-decimation, the width would become 200,
>> and we would need our scalar to actually upscale to 220.
>>
>> I am wondering if the driver gets confused when such a scenario occurs,
>> or the DSS HW is weird when we upscale some predecimated content. If you
>> look at the hinc(1024  * width/out_width) value in the FIR register, in
>> the "ok" regdumps, the value is 1861, which points to downscaling. And
>> in the fail case, it's 92, which is upscaling.
>
> I can reproduce this with a very simple case. 400x200 -> 200x200, and
> forcing horizontal predecimation. So this will cause no scaling to be
> used, only the pixel_inc is used to downscale.

Ah okay. My guess was wrong then :)

>
> The VID1 registers are as follows. They look right to me...
>
> DISPC_OVL_BA0(VID1)                                8d900000
> DISPC_OVL_BA1(VID1)                                8d900000
> DISPC_OVL_POSITION(VID1)                           00000000
> DISPC_OVL_SIZE(VID1)                               00c700c7
> DISPC_OVL_ATTRIBUTES(VID1)                         00608011
> DISPC_OVL_FIFO_THRESHOLD(VID1)                     03ff03c0
> DISPC_OVL_FIFO_SIZE_STATUS(VID1)                   00000400
> DISPC_OVL_ROW_INC(VID1)                            00000001
> DISPC_OVL_PIXEL_INC(VID1)                          00000005
> DISPC_OVL_PRELOAD(VID1)                            00000100
> DISPC_OVL_FIR(VID1)                                04000400
> DISPC_OVL_PICTURE_SIZE(VID1)                       00c700c7
> DISPC_OVL_ACCU0(VID1)                              00000000
> DISPC_OVL_ACCU1(VID1)                              00000000
> DISPC_OVL_PRELOAD(VID1)                            00000100

The only thing which seems not so nice is that vertical taps and vid 
line buffer split are enabled in OVL_ATTRIBUTES. These shouldn't be 
harmful though.

Archit

Re: downscaling YUV fails

[Tomi Valkeinen]

[-- Attachment #1: Type: text/plain, Size: 2692 bytes --]

On 2012-12-19 15:00, Archit Taneja wrote:
> On Wednesday 19 December 2012 06:07 PM, Tomi Valkeinen wrote:
>> On 2012-12-19 14:19, Archit Taneja wrote:
>>
>>>> The image I get is stable, and clearly something that happens when,
>>>> say,
>>>> row_inc is one pixel too much, or similar.
>>>
>>> Ok, about the width in this case. The original width is 400, and what we
>>> finally want is 220. After pre-decimation, the width would become 200,
>>> and we would need our scalar to actually upscale to 220.
>>>
>>> I am wondering if the driver gets confused when such a scenario occurs,
>>> or the DSS HW is weird when we upscale some predecimated content. If you
>>> look at the hinc(1024  * width/out_width) value in the FIR register, in
>>> the "ok" regdumps, the value is 1861, which points to downscaling. And
>>> in the fail case, it's 92, which is upscaling.
>>
>> I can reproduce this with a very simple case. 400x200 -> 200x200, and
>> forcing horizontal predecimation. So this will cause no scaling to be
>> used, only the pixel_inc is used to downscale.
> 
> Ah okay. My guess was wrong then :)
> 
>>
>> The VID1 registers are as follows. They look right to me...
>>
>> DISPC_OVL_BA0(VID1)                                8d900000
>> DISPC_OVL_BA1(VID1)                                8d900000
>> DISPC_OVL_POSITION(VID1)                           00000000
>> DISPC_OVL_SIZE(VID1)                               00c700c7
>> DISPC_OVL_ATTRIBUTES(VID1)                         00608011
>> DISPC_OVL_FIFO_THRESHOLD(VID1)                     03ff03c0
>> DISPC_OVL_FIFO_SIZE_STATUS(VID1)                   00000400
>> DISPC_OVL_ROW_INC(VID1)                            00000001
>> DISPC_OVL_PIXEL_INC(VID1)                          00000005
>> DISPC_OVL_PRELOAD(VID1)                            00000100
>> DISPC_OVL_FIR(VID1)                                04000400
>> DISPC_OVL_PICTURE_SIZE(VID1)                       00c700c7
>> DISPC_OVL_ACCU0(VID1)                              00000000
>> DISPC_OVL_ACCU1(VID1)                              00000000
>> DISPC_OVL_PRELOAD(VID1)                            00000100
> 
> The only thing which seems not so nice is that vertical taps and vid
> line buffer split are enabled in OVL_ATTRIBUTES. These shouldn't be
> harmful though.

Yep, I've tested disabling five_taps also, but it doesn't have any effect.

I'm at loss here =). If I force pix inc to 1, I get what I expect: an
image without skew, but the height is doubled as the left and right
sides are shows on two lines. Then setting pix inc to 5 causes the skew.
Colors are still correct, so the byte alignment is right.

 Tomi



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: downscaling YUV fails

[Peter Meerwald]

> One thing I'm guessing is that DMA fetching with predecimation isn't really
> optimised for omap3, if the pixel clock in Peter's setup is high, the DISPC
> DMA might not be able to fetch pixels fast enough in predecimation case. That
> sort of supports the possibility of a skewed image seen.

> Do you see any SYNC_LOST or UNDERFLOW prints when this happens?

not that I am aware of

the issue is fully reproducible here; there seem to be two extra pixels 
per line

p.

-- 

Peter Meerwald
+43-664-2444418 (mobile)