I.MX6 HDMI support in v4.2

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Hi,

I'd like to ask if the HDMI output on I.MX6-based boards is possible.
I'm unable to get it to work. What I'm trying to do is getting
a) DRM HDMI output, b) X.org output using etnaviv, libdrm-armada and
xf86-video-armada, c) GLX and Xvideo. Basically, latest versions.

I'm working with Gateworks Ventana GW54xx boards and with Sabre-lite,
using newly built imx6q-gw54xx.dtb and imx6q-sabrelite.dtb. In both
cases, required kernel modules are inserted and (somehow) initialized,
but the HDMI device isn't created.

The first problem seems to be this:
- dw_hdmi_imx_probe() is called (and does component_add()), but
- dw_hdmi_imx_bind() is never called.

Now if I enable LVDS (CONFIG_DRM_IMX_LDB - I don't have any LVDS
hardware connected), the HDMI device is created (as well as LVDS).

This used to detect the monitor as "unknown" but now it's "connected"
most of the time - not sure what have changed. EDID is empty and I get
the following entries in /sys/devices/soc0/display-subsystem/drm/card0:

dev                                            226:0
card0-HDMI-A-1/edid                            (empty)
card0-HDMI-A-1/dpms                            On
card0-HDMI-A-1/modes
card0-HDMI-A-1/power/control                   auto
card0-HDMI-A-1/power/runtime_active_time       0
card0-HDMI-A-1/power/autosuspend_delay_ms      (Input/output error)
card0-HDMI-A-1/power/runtime_status            unsupported
card0-HDMI-A-1/power/runtime_suspended_time    0
card0-HDMI-A-1/enabled                         enabled
card0-HDMI-A-1/status                          used to be "unknown", now: "connected"
card0-HDMI-A-1/uevent                          power/control auto
power/runtime_active_time                      0
power/autosuspend_delay_ms                     (Input/output error)
power/runtime_status                           unsupported
power/runtime_suspended_time                   0
card0-LVDS-1/edid
card0-LVDS-1/dpms                              On
card0-LVDS-1/modes
card0-LVDS-1/power/control                     auto
card0-LVDS-1/power/runtime_active_time         0
card0-LVDS-1/power/autosuspend_delay_ms        (Input/output error)
card0-LVDS-1/power/runtime_status              unsupported
card0-LVDS-1/power/runtime_suspended_time      0
card0-LVDS-1/enabled                           disabled
card0-LVDS-1/status                            unknown
card0-LVDS-1/uevent
uevent                                         MAJOR=226
                                               MINOR=0
                                               DEVNAME=dri/card0
                                               DEVTYPE=drm_minor

Have to "touch /etc/xorg.conf" (otherwise the X server segfaults).

Now, somehow the X.org server sets the resolution to 1024x768, though
nothing is displayed on the monitor (it's in stand-by). Files in
/sys/.../card0-HDMI-A/ now have the actual EDID, mode list etc.

# xrandr
Screen 0: minimum 320 x 200, current 1024 x 768, maximum 4096 x 4096
HDMI-0 connected 1024x768+0+0 (normal left inverted right x axis y axis) 531mm x 299mm
   1920x1080     60.00 +
   1680x1050     59.88
   1280x1024     75.02    60.02
   1440x900      74.98    59.90
   1280x720      60.00
   1024x768      75.08    60.00*
   800x600       75.00    60.32
   640x480       75.00    72.81    66.67    60.00
   720x400       70.08
LVDS-0 connected 1024x768+0+0 (normal left inverted right x axis y axis) 0mm x 0mm
   1024x768      60.00*+

This also causes:
imx-ipuv3 2400000.ipu: DC stop timeout after 50 ms
imx-ipuv3 2400000.ipu: Timeout waiting for DMFC FIFOs to clear

What am I doing wrong?
- kernel command line? video=mxcfb0:dev=hdmi,1920x1080M at 60,if=RGB24
- wrong SDMA firmware? MD5 is:
  5d4584134cc4cba62e1be2f382cd6f3a  /lib/firmware/imx/sdma/sdma-imx6q.bin
- wrong X.org stuff?

CPU identified as i.MX6Q, silicon rev 1.2

imx_ipuv3_crtc         20480  0
ahci_imx               16384  0
libahci_platform       16384  1 ahci_imx
libahci                28672  2 ahci_imx,libahci_platform
imx_ipu_v3             49152  1 imx_ipuv3_crtc
fec                    45056  0
dw_hdmi_imx            16384  0
dw_hdmi                20480  1 dw_hdmi_imx
imx_ldb                16384  0
imxdrm                 16384  3 dw_hdmi_imx,imx_ipuv3_crtc,imx_ldb
drm_kms_helper         90112  4 dw_hdmi,imxdrm,imx_ipuv3_crtc,imx_ldb
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
coda                   45056  0
videobuf2_vmalloc      16384  1 coda
videobuf2_dma_contig   20480  1 coda
flexcan                20480  0
videobuf2_memops       16384  2 videobuf2_vmalloc,videobuf2_dma_contig
v4l2_mem2mem           16384  1 coda
videobuf2_core         40960  2 coda,v4l2_mem2mem
sysimgblt              16384  1 drm_kms_helper
sky2                   57344  0
gpmi_nand              28672  0
drm                   249856  7 dw_hdmi,drm_kms_helper,dw_hdmi_imx,imxdrm,imx_ipuv3_crtc,imx_ldb

CONFIG_SOC_IMX6=y
CONFIG_SOC_IMX6Q=y
CONFIG_I2C_IMX=y
CONFIG_SPI_IMX=y
CONFIG_PINCTRL_IMX=y
CONFIG_PINCTRL_IMX6Q=y
CONFIG_GPIO_MXC=y
CONFIG_DRM_IMX=m
# CONFIG_DRM_IMX_FB_HELPER is not set
# CONFIG_DRM_IMX_PARALLEL_DISPLAY is not set
# CONFIG_DRM_IMX_TVE is not set
# CONFIG_DRM_IMX_LDB is not set
CONFIG_DRM_IMX_IPUV3=m
CONFIG_DRM_IMX_HDMI=m
CONFIG_MX3_IPU=y
CONFIG_MX3_IPU_IRQS=4
CONFIG_IMX_SDMA=y
# CONFIG_IMX_DMA is not set
CONFIG_MXS_DMA=y
CONFIG_CLKSRC_IMX_GPT=y

Machine model: Gateworks Ventana i.MX6 Dual/Quad GW54XX
Kernel command line: console=ttymxc1,115200 video=mxcfb0:dev=hdmi,1920x1080M at 60,if=RGB24 earlyprintk
--
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Mon, Sep 07, 2015 at 12:55:54PM +0200, Krzysztof Ha?asa wrote:
> Hi,
> 
> I'd like to ask if the HDMI output on I.MX6-based boards is possible.
> I'm unable to get it to work. What I'm trying to do is getting
> a) DRM HDMI output, b) X.org output using etnaviv, libdrm-armada and
> xf86-video-armada, c) GLX and Xvideo. Basically, latest versions.
> 
> I'm working with Gateworks Ventana GW54xx boards and with Sabre-lite,
> using newly built imx6q-gw54xx.dtb and imx6q-sabrelite.dtb. In both
> cases, required kernel modules are inserted and (somehow) initialized,
> but the HDMI device isn't created.
> 
> The first problem seems to be this:
> - dw_hdmi_imx_probe() is called (and does component_add()), but
> - dw_hdmi_imx_bind() is never called.
> 
> Now if I enable LVDS (CONFIG_DRM_IMX_LDB - I don't have any LVDS
> hardware connected), the HDMI device is created (as well as LVDS).

Are you telling the kernel in your device tree file that LDB is required?
DRM doesn't support hot-plugging outputs, all specified output modules
must be present before DRM can bring up the display subsystem.

> This used to detect the monitor as "unknown" but now it's "connected"
> most of the time - not sure what have changed. EDID is empty and I get
> the following entries in /sys/devices/soc0/display-subsystem/drm/card0:

"used to" - when was this?  I don't think dw_hdmi has ever reported
a connected status of "unknown", always explicitly stating connected
or disconnected.

> dev                                            226:0
> card0-HDMI-A-1/edid                            (empty)
> card0-HDMI-A-1/dpms                            On
> card0-HDMI-A-1/modes
> card0-HDMI-A-1/power/control                   auto
> card0-HDMI-A-1/power/runtime_active_time       0
> card0-HDMI-A-1/power/autosuspend_delay_ms      (Input/output error)
> card0-HDMI-A-1/power/runtime_status            unsupported
> card0-HDMI-A-1/power/runtime_suspended_time    0
> card0-HDMI-A-1/enabled                         enabled
> card0-HDMI-A-1/status                          used to be "unknown", now: "connected"
> card0-HDMI-A-1/uevent                          power/control auto

Looks fine apart from the lack of EDID.  Are you sure you have the
pinctrl setup correct for this?  (We don't use the DDC I2C built
into the HDMI interface.)

> Now, somehow the X.org server sets the resolution to 1024x768, though
> nothing is displayed on the monitor (it's in stand-by). Files in
> /sys/.../card0-HDMI-A/ now have the actual EDID, mode list etc.

By default, 1024x768 is selected when there's nothing else available.
The DPMS "on" and enabled above tends to suggest you should be seeing
output though.

It's possible that having both LVDS and HDMI enabled causes a pixel
clocking conflict if both are routed to the same CRTC and PLL.  (The
clocking side of it is something I really hate, and the decision was
taken to control this statically.)

> This also causes:
> imx-ipuv3 2400000.ipu: DC stop timeout after 50 ms
> imx-ipuv3 2400000.ipu: Timeout waiting for DMFC FIFOs to clear

I don't have an answer for that... that's deep internal IPU stuff.

> What am I doing wrong?
> - kernel command line? video=mxcfb0:dev=hdmi,1920x1080M at 60,if=RGB24
> - wrong SDMA firmware? MD5 is:
>   5d4584134cc4cba62e1be2f382cd6f3a  /lib/firmware/imx/sdma/sdma-imx6q.bin
> - wrong X.org stuff?

Probably none of the above.

I should point out that virtually every -rc kernel gets tested here on
iMX6 with HDMI output - onto my Panasonic TV, and I've seen no evidence
of any regressions.  For me, it Just Works(tm).

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>> Now if I enable LVDS (CONFIG_DRM_IMX_LDB - I don't have any LVDS
>> hardware connected), the HDMI device is created (as well as LVDS).
>
> Are you telling the kernel in your device tree file that LDB is required?
> DRM doesn't support hot-plugging outputs, all specified output modules
> must be present before DRM can bring up the display subsystem.

It seems to be the case, I'll test with the LDB portion removed. Though
I don't mind LVDS if it doesn't break HDMI.

>> This used to detect the monitor as "unknown" but now it's "connected"
>> most of the time - not sure what have changed. EDID is empty and I get
>> the following entries in /sys/devices/soc0/display-subsystem/drm/card0:
>
> "used to" - when was this?

Well... before I made unspecified changes to something :-)
I mean, I don't think I made any related changes, but something must
have changed since it always prints "connected" now. Most of the time.
I mean, from time to time :-)
Seems low level.

> I don't think dw_hdmi has ever reported
> a connected status of "unknown", always explicitly stating connected
> or disconnected.

The "unknown" must be an uninitialized variable (neither connected = 1
or disconnected = 2):

--- dmesg-unknown
+++ dmesg-connected
-imx-ipuv3 2400000.ipu: IPUv3H probed
@@
-XXX dw_hdmi_imx_probe[261]
+imx-ipuv3 2400000.ipu: IPUv3H probed
 imx-ipuv3 2800000.ipu: IPUv3H probed
+XXX dw_hdmi_imx_probe[261]
 [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
 [drm] No driver support for vblank timestamp query.
 imx-drm display-subsystem: bound imx-ipuv3-crtc.0 (ops ipu_crtc_ops [imx_ipuv3_crtc])
@@ -323,8 +323,9 @@
 XXX dw_hdmi_hardirq[1479]
 XXX dw_hdmi_irq[1493]
 imx-drm display-subsystem: bound 2000000.aips-bus:ldb at 020e0008 (ops imx_ldb_driver_exit [imx_ldb])
 [drm] Initialized imx-drm 1.0.0 20120507 on minor 0
+XXX dw_hdmi_connector_detect[1389]: CONNECTED

In the "unknown" case, the dw_hdmi_connector_detect() wasn't called.
Maybe the problem happens when dw_hdmi_imx_probe() is called before
"imx-ipuv3 2400000.ipu: IPUv3H probed" is done. Probably an interrupt
isn't generated or something like this, maybe it should poll it once.
I'll check this later.

> Looks fine apart from the lack of EDID.  Are you sure you have the
> pinctrl setup correct for this?  (We don't use the DDC I2C built
> into the HDMI interface.)

How do I check it? I'm simply using the (v4.2) imx6q-gw54xx.dts file.

>> Now, somehow the X.org server sets the resolution to 1024x768, though
>> nothing is displayed on the monitor (it's in stand-by). Files in
>> /sys/.../card0-HDMI-A/ now have the actual EDID, mode list etc.
>
> By default, 1024x768 is selected when there's nothing else available.

How do I select e.g. 1920x1080? The EDID supports this mode, but
# xrandr --output HDMI1 --auto

X Error:  BadMatch
  Request Major code 139 (RANDR)
  Request Minor code 7 ()
  Error Serial #34
  Current Serial #35

imx-drm display-subsystem: failed to allocate buffer with size 8294400
imx-drm display-subsystem: failed to allocate buffer with size 8294400
imx-ipuv3 2400000.ipu: DC stop timeout after 50 ms

I assume I have to reserve the (big linear region of) memory at boot
time, but don't know how.

> The DPMS "on" and enabled above tends to suggest you should be seeing
> output though.

I've applied the PLL5 patch and it now started to work. At least I have
output. I'm still seeing "DC stop timeout after 50 ms" but apparently
only when the X server is being closed, from time to time.

> I should point out that virtually every -rc kernel gets tested here on
> iMX6 with HDMI output - onto my Panasonic TV, and I've seen no evidence
> of any regressions.  For me, it Just Works(tm).

Ok. I'll try to reach the same state :-)

Now, having the HDMI output on the screen, I'm trying to get XVideo
working. It seems all XV attributes are set to their minimum values and
I can't change that. Is it normal? For this or other reason, I can see
a black video window only (I'm trying to use I420 overlay). Could be
unrelated problem, though.

# xvattr
Found Xv 2.2
Adaptor: 0
Name: Marvell Armada Overlay Video
 Port: 85
  Name: XV_ENCODING
   Flags: XvGettable XvSettable
   Min value: 0
   Max value: 0
   Current value: 0
  Name: XV_SATURATION
   Flags: XvGettable XvSettable
   Min value: -16384
   Max value: 16383
   Current value: -16384
  Name: XV_BRIGHTNESS
   Flags: XvGettable XvSettable
   Min value: -256
   Max value: 255
   Current value: -256
  Name: XV_CONTRAST
   Flags: XvGettable XvSettable
   Min value: -16384
   Max value: 16383
   Current value: -16384
  Name: XV_AUTOPAINT_COLORKEY
   Flags: XvGettable XvSettable
   Min value: 0
   Max value: 1
   Current value: 1
  Name: XV_COLORKEY
   Flags: XvGettable XvSettable
   Min value: 0
   Max value: 16777215
   Current value: 0
  Name: XV_PIPE
   Flags: XvGettable XvSettable
   Min value: -1
   Max value: 3
   Current value: -1

# xvattr -a XV_BRIGHTNESS -v 0 -p 85
Found Xv 2.2
XV_BRIGHTNESS set to -256

I'll continue with this stuff tomorrow.


Thanks for the help,
--
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Mon, Sep 07, 2015 at 04:04:30PM +0200, Krzysztof Ha?asa wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> 
> >> Now if I enable LVDS (CONFIG_DRM_IMX_LDB - I don't have any LVDS
> >> hardware connected), the HDMI device is created (as well as LVDS).
> >
> > Are you telling the kernel in your device tree file that LDB is required?
> > DRM doesn't support hot-plugging outputs, all specified output modules
> > must be present before DRM can bring up the display subsystem.
> 
> It seems to be the case, I'll test with the LDB portion removed. Though
> I don't mind LVDS if it doesn't break HDMI.
> 
> >> This used to detect the monitor as "unknown" but now it's "connected"
> >> most of the time - not sure what have changed. EDID is empty and I get
> >> the following entries in /sys/devices/soc0/display-subsystem/drm/card0:
> >
> > "used to" - when was this?
> 
> Well... before I made unspecified changes to something :-)
> I mean, I don't think I made any related changes, but something must
> have changed since it always prints "connected" now. Most of the time.
> I mean, from time to time :-)
> Seems low level.

I don't know - what normally happens is that a HPD IRQ fires at boot time
from the HDMI interface which causes us to run the ->detect functions,
and that would have updated the connect status before the thing has
finished probing.

> > I don't think dw_hdmi has ever reported
> > a connected status of "unknown", always explicitly stating connected
> > or disconnected.
> 
> The "unknown" must be an uninitialized variable (neither connected = 1
> or disconnected = 2):
> 
> --- dmesg-unknown
> +++ dmesg-connected
> -imx-ipuv3 2400000.ipu: IPUv3H probed
> @@
> -XXX dw_hdmi_imx_probe[261]
> +imx-ipuv3 2400000.ipu: IPUv3H probed
>  imx-ipuv3 2800000.ipu: IPUv3H probed
> +XXX dw_hdmi_imx_probe[261]

Do you have DRM configured as modules?  I can't think of anything else
that would change the probe order like this.

>  [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
>  [drm] No driver support for vblank timestamp query.
>  imx-drm display-subsystem: bound imx-ipuv3-crtc.0 (ops ipu_crtc_ops [imx_ipuv3_crtc])
> @@ -323,8 +323,9 @@
>  XXX dw_hdmi_hardirq[1479]
>  XXX dw_hdmi_irq[1493]
>  imx-drm display-subsystem: bound 2000000.aips-bus:ldb at 020e0008 (ops imx_ldb_driver_exit [imx_ldb])
>  [drm] Initialized imx-drm 1.0.0 20120507 on minor 0
> +XXX dw_hdmi_connector_detect[1389]: CONNECTED
> 
> In the "unknown" case, the dw_hdmi_connector_detect() wasn't called.
> Maybe the problem happens when dw_hdmi_imx_probe() is called before
> "imx-ipuv3 2400000.ipu: IPUv3H probed" is done.

The IPU and HDMI are separate entities and can't influence each others
IRQs.

> Probably an interrupt
> isn't generated or something like this, maybe it should poll it once.
> I'll check this later.

You seem to have the interrupt generated (though you don't print what the
IRQ status register contained).  It should cause drm_helper_hpd_irq_event()
to be called, which then polls the connector detect functions.

If that detects any connector having changed status, it goes on to call
drm_kms_helper_hotplug_event(), which then goes on to call (via a few
other functions) drm_fb_helper_hotplug_event() and
drm_fb_helper_probe_connector_modes().

> > Looks fine apart from the lack of EDID.  Are you sure you have the
> > pinctrl setup correct for this?  (We don't use the DDC I2C built
> > into the HDMI interface.)
> 
> How do I check it? I'm simply using the (v4.2) imx6q-gw54xx.dts file.

drm_fb_helper_probe_connector_modes() should cause the modes to be read
via the get_modes() callback into the HDMI driver.

> >> Now, somehow the X.org server sets the resolution to 1024x768, though
> >> nothing is displayed on the monitor (it's in stand-by). Files in
> >> /sys/.../card0-HDMI-A/ now have the actual EDID, mode list etc.
> >
> > By default, 1024x768 is selected when there's nothing else available.
> 
> How do I select e.g. 1920x1080? The EDID supports this mode, but
> # xrandr --output HDMI1 --auto
> 
> X Error:  BadMatch
>   Request Major code 139 (RANDR)
>   Request Minor code 7 ()
>   Error Serial #34
>   Current Serial #35
> 
> imx-drm display-subsystem: failed to allocate buffer with size 8294400
> imx-drm display-subsystem: failed to allocate buffer with size 8294400

Why are you getting those errors?  Do you have CMA disabled?  Maybe you
need to use cma=128M or something on the command line.

> Now, having the HDMI output on the screen, I'm trying to get XVideo
> working. It seems all XV attributes are set to their minimum values and
> I can't change that. Is it normal? For this or other reason, I can see
> a black video window only (I'm trying to use I420 overlay). Could be
> unrelated problem, though.

That's probably because of the restrictions in the IPU - the mainline
IPU code is unable to scale overlays at all - it's not supported.  The
problem is most Xorg drivers assume that overlays can be scaled.  I've
said about this before, and how this is broken, but I've no idea whether
it's going to get fixed.  As far as I know, this only affects platforms
using the IPU.

The only alternative there is to switch to using the GPU to blit the
XVideo frame onto the display (but then you need all the etnaviv bits
in place, including the etnaviv DRM driver.)  This works up to a point,
but suffers from tearing, because there's no sane good way to synchronise
the GPU blit with the video scanout (because they're two different
completely unrelated chunks of hardware: the reason Intel i965 can do
this is because it seems possible to put into the GPU stream "wait for
scan line X before continuing" thereby preventing the blit modifying
scanlines that are about to be displayed.

I have some experimental code for the Xorg driver that computes the
period of time after the vsync that the scanlines would be scanned out.
However, my measurements of the time taken for the GPU to blit the
frame show that it takes almost one scan-out to do the filter blit.
That makes the timings are _really_ tight there which means any jitter
in the scheduling and interrupts will throw any solution to this off,
and the tearing will be back.

As long as the video playback situation (as a whole, I'm talking about
the VPU) is poorly supported both in userspace and mainline kernels (it
only supports H264, not MPEG2/4, and requires bleeding edge gstreamer
support), video decode and overlay on iMX6 doesn't interest me.  My
preferred ARM platform for that is Dove right now, so I'm not motivated
to put much effort into the iMX6 video playback issues, basically because
I can't decently test them.

As for the overlay attributes, don't worry about them - imx-drm doesn't
support the properties on overlay at all.  They're exposed because once
the Xorg atoms exist, you can't then return a BadMatch when getting or
setting them - applications tend to have a dislike for that.  Maybe I
should arrange it to return a more sensible value though.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Trying to make some progress... I was missing the etnaviv DRM kernel
driver. Now I have both IMX and etnaviv drivers, Linux v4.0-stable:

git://git.pengutronix.de/git/lst/linux.git (etnaviv DRM driver)
PLL5 DTS patch (enables HDMI with LVDS driver loaded):

--- a/arch/arm/boot/dts/imx6q-gw54xx.dts
+++ b/arch/arm/boot/dts/imx6q-gw54xx.dts
@@ -21,3 +21,11 @@
 &sata {
 	status = "okay";
 };
+
+&clks {
+	assigned-clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>,
+			  <&clks IMX6QDL_CLK_LDB_DI1_SEL>;
+	assigned-clock-parents = <&clks IMX6QDL_CLK_PLL3_USB_OTG>,
+				 <&clks IMX6QDL_CLK_PLL3_USB_OTG>;
+};
+

also had to apply:

--- a/drivers/staging/etnaviv/Makefile
+++ b/drivers/staging/etnaviv/Makefile
@@ -11,7 +11,6 @@ etnaviv-y := \
 	etnaviv_gem_submit.o \
 	etnaviv_gpu.o \
 	etnaviv_iommu.o \
-	etnaviv_iommu_v2.o \
 	etnaviv_mmu.o \
 	etnaviv_buffer.o
 

X.org etc.:
	git://github.com/laanwj/etna_viv.git
        git://ftp.arm.linux.org.uk/~rmk/libdrm-armada.git/
        git://ftp.arm.linux.org.uk/~rmk/xf86-video-armada.git unstable-devel
(I can't use master branch of xf86-video-armada.git because it doesn't
support etnaviv - does it?)

X log (relevant(?) stuff only):

[ 18919.848] (II) armada(0): hardware: imx-drm
[ 18919.923] (--) armada(0): Vivante GC320 GPU revision 5007 (etnaviv) 2d PE2.0
[ 18919.925] (II) armada(0): [DRI2] Setup complete
[ 18919.925] (II) armada(0): [DRI2]   DRI driver: etnadrm
[ 18919.925] (II) armada(0): direct rendering: DRI2 enabled
[ 18919.925] (II) armada(0): RandR 1.2 enabled, ignore the following RandR disabled message.
[ 18919.927] (==) armada(0): DPMS enabled
[ 18919.927] (==) armada(0): hotplug detection enabled
[ 18919.928] (II) armada(0): etnaviv: Xv: using YUY2 tiled format intermediate YUV target
[ 18919.971] (EE) AIGLX error: dlopen of /lib/dri/etnadrm_dri.so failed (I don't have this yet)
[ 18920.072] (II) armada(0): etnaviv: A8 target not supported

If I try to use pengutronix/etnaviv-4.1-wip branch (etnaviv DRM - for
v4.2), dmesg contains a lot of:
 [drm:etnaviv_ioctl_gem_submit [etnaviv]] *ERROR* cmdstream bo has flag ETNA_BO_CMDSTREAM not set
and it doesn't work. I assume I shouldn't use it yet?

Changing to pengutronix/etnaviv-for-upstream, it works - sort of.

I now have two Xvideo adapters:
Adaptor: 0
Name: Marvell Armada Overlay Video
 Port: 88
(which unfortunately doesn't work, the window is black, the XV
attributes are set to minimums and I can't set them).

Is it expected?

Adaptor: 1
Name: Etnaviv Textured Video
 Port(s): 89 - 104

  Name: XV_SYNC_TO_VBLANK
   Flags: XvGettable XvSettable
   Min value: 0
   Max value: 1
   Current value: 1
  Name: XV_PIPE
   Flags: XvGettable XvSettable
   Min value: -1
   Max value: 3
   Current value: -1
  Name: XV_ENCODING
   Flags: XvGettable XvSettable
   Min value: 0
   Max value: 0
   Current value: 0

This adapter works, though there are some image distortions (horizontal
lines apparently caused by changes in color between frames).

Any input?
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Dienstag, den 08.09.2015, 12:45 +0200 schrieb Krzysztof Ha?asa:
> Trying to make some progress... I was missing the etnaviv DRM kernel
> driver. Now I have both IMX and etnaviv drivers, Linux v4.0-stable:
> 
> git://git.pengutronix.de/git/lst/linux.git (etnaviv DRM driver)
> PLL5 DTS patch (enables HDMI with LVDS driver loaded):
> 
> --- a/arch/arm/boot/dts/imx6q-gw54xx.dts
> +++ b/arch/arm/boot/dts/imx6q-gw54xx.dts
> @@ -21,3 +21,11 @@
>  &sata {
>  	status = "okay";
>  };
> +
> +&clks {
> +	assigned-clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>,
> +			  <&clks IMX6QDL_CLK_LDB_DI1_SEL>;
> +	assigned-clock-parents = <&clks IMX6QDL_CLK_PLL3_USB_OTG>,
> +				 <&clks IMX6QDL_CLK_PLL3_USB_OTG>;
> +};
> +
> 
> also had to apply:
> 
> --- a/drivers/staging/etnaviv/Makefile
> +++ b/drivers/staging/etnaviv/Makefile
> @@ -11,7 +11,6 @@ etnaviv-y := \
>  	etnaviv_gem_submit.o \
>  	etnaviv_gpu.o \
>  	etnaviv_iommu.o \
> -	etnaviv_iommu_v2.o \
>  	etnaviv_mmu.o \
>  	etnaviv_buffer.o
>  
> 
> X.org etc.:
> 	git://github.com/laanwj/etna_viv.git
>         git://ftp.arm.linux.org.uk/~rmk/libdrm-armada.git/
>         git://ftp.arm.linux.org.uk/~rmk/xf86-video-armada.git unstable-devel
> (I can't use master branch of xf86-video-armada.git because it doesn't
> support etnaviv - does it?)
> 
> X log (relevant(?) stuff only):
> 
> [ 18919.848] (II) armada(0): hardware: imx-drm
> [ 18919.923] (--) armada(0): Vivante GC320 GPU revision 5007 (etnaviv) 2d PE2.0
> [ 18919.925] (II) armada(0): [DRI2] Setup complete
> [ 18919.925] (II) armada(0): [DRI2]   DRI driver: etnadrm
> [ 18919.925] (II) armada(0): direct rendering: DRI2 enabled
> [ 18919.925] (II) armada(0): RandR 1.2 enabled, ignore the following RandR disabled message.
> [ 18919.927] (==) armada(0): DPMS enabled
> [ 18919.927] (==) armada(0): hotplug detection enabled
> [ 18919.928] (II) armada(0): etnaviv: Xv: using YUY2 tiled format intermediate YUV target
> [ 18919.971] (EE) AIGLX error: dlopen of /lib/dri/etnadrm_dri.so failed (I don't have this yet)
> [ 18920.072] (II) armada(0): etnaviv: A8 target not supported
> 
> If I try to use pengutronix/etnaviv-4.1-wip branch (etnaviv DRM - for
> v4.2), dmesg contains a lot of:
>  [drm:etnaviv_ioctl_gem_submit [etnaviv]] *ERROR* cmdstream bo has flag ETNA_BO_CMDSTREAM not set
> and it doesn't work. I assume I shouldn't use it yet?
> 
You need changes to the X.Org driver for that to work. I'm currently
working on a new version of the etnaviv-drm driver which will break the
kernel userspace interface another time - hopefully for the last time
now.

I'm almost done and will publish patches for both the kernel and the
X.Org driver today.

> Changing to pengutronix/etnaviv-for-upstream, it works - sort of.
> 
> I now have two Xvideo adapters:
> Adaptor: 0
> Name: Marvell Armada Overlay Video
>  Port: 88
> (which unfortunately doesn't work, the window is black, the XV
> attributes are set to minimums and I can't set them).
> 
> Is it expected?
> 
This is expected, it's the overlay adapter which doesn't really work
with imx-drm yet.

Regards,
Lucas


-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 08, 2015 at 12:56:18PM +0200, Lucas Stach wrote:
> This is expected, it's the overlay adapter which doesn't really work
> with imx-drm yet.

No, it's imx-drm which isn't working.  As I explained in my previous
email, overlay planes are expected to do scaling.  imx-drm errors out
attempts for that.  This is a kernel bug, not an Xorg driver bug.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

> Do you have DRM configured as modules?  I can't think of anything else
> that would change the probe order like this.

Yes, they are modules.

> You seem to have the interrupt generated

Right.

> (though you don't print what the
> IRQ status register contained).  It should cause drm_helper_hpd_irq_event()
> to be called, which then polls the connector detect functions.
>
> If that detects any connector having changed status, it goes on to call
> drm_kms_helper_hotplug_event(), which then goes on to call (via a few
> other functions) drm_fb_helper_hotplug_event() and
> drm_fb_helper_probe_connector_modes().

Ok, I will queue this for investigation.

>> imx-drm display-subsystem: failed to allocate buffer with size 8294400
>> imx-drm display-subsystem: failed to allocate buffer with size 8294400
>
> Why are you getting those errors?  Do you have CMA disabled?  Maybe you
> need to use cma=128M or something on the command line.

Yes, I had it disabled.

>> Now, having the HDMI output on the screen, I'm trying to get XVideo
>> working. It seems all XV attributes are set to their minimum values and
>> I can't change that. Is it normal? For this or other reason, I can see
>> a black video window only (I'm trying to use I420 overlay). Could be
>> unrelated problem, though.
>
> That's probably because of the restrictions in the IPU - the mainline
> IPU code is unable to scale overlays at all - it's not supported.  The
> problem is most Xorg drivers assume that overlays can be scaled.  I've
> said about this before, and how this is broken, but I've no idea whether
> it's going to get fixed.  As far as I know, this only affects platforms
> using the IPU.

I see. But I'm ok with unscaled Xvideo. I only need a (fast) way to send
YUV (preferably YUV420) image to the screen.

> The only alternative there is to switch to using the GPU to blit the
> XVideo frame onto the display (but then you need all the etnaviv bits
> in place, including the etnaviv DRM driver.)  This works up to a point,
> but suffers from tearing, because there's no sane good way to synchronise
> the GPU blit with the video scanout (because they're two different
> completely unrelated chunks of hardware: the reason Intel i965 can do
> this is because it seems possible to put into the GPU stream "wait for
> scan line X before continuing" thereby preventing the blit modifying
> scanlines that are about to be displayed.

This could be the horizontal lines that I observed. Something similar
to what the old S3 Trio64 were doing.

> As long as the video playback situation (as a whole, I'm talking about
> the VPU) is poorly supported both in userspace and mainline kernels (it
> only supports H264, not MPEG2/4, and requires bleeding edge gstreamer
> support), video decode and overlay on iMX6 doesn't interest me.  My
> preferred ARM platform for that is Dove right now, so I'm not motivated
> to put much effort into the iMX6 video playback issues, basically because
> I can't decently test them.

I see. At the moment I'm able to use the hardware decoder (with a bit
long latencies, though), I'm getting a YUV420 image out of it. Also, the
encoder works.

> As for the overlay attributes, don't worry about them - imx-drm doesn't
> support the properties on overlay at all.  They're exposed because once
> the Xorg atoms exist, you can't then return a BadMatch when getting or
> setting them - applications tend to have a dislike for that.  Maybe I
> should arrange it to return a more sensible value though.

I see.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Lucas Stach <l.stach@pengutronix.de> writes:

>>  [drm:etnaviv_ioctl_gem_submit [etnaviv]] *ERROR* cmdstream bo has flag ETNA_BO_CMDSTREAM not set
>> and it doesn't work. I assume I shouldn't use it yet?
>> 
> You need changes to the X.Org driver for that to work. I'm currently
> working on a new version of the etnaviv-drm driver which will break the
> kernel userspace interface another time - hopefully for the last time
> now.
>
> I'm almost done and will publish patches for both the kernel and the
> X.Org driver today.

Great, I'm looking forward to be the first guinea pig :-)
Thanks.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Dienstag, den 08.09.2015, 12:01 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 08, 2015 at 12:56:18PM +0200, Lucas Stach wrote:
> > This is expected, it's the overlay adapter which doesn't really work
> > with imx-drm yet.
> 
> No, it's imx-drm which isn't working.  As I explained in my previous
> email, overlay planes are expected to do scaling.  imx-drm errors out
> attempts for that.  This is a kernel bug, not an Xorg driver bug.
> 
I would argue that this is a bug of the interface between kernel and
userspace.
Scaling isn't something that can be expected to be usable on every
hardware (and in fact the IPU isn't able to do arbitrary scaling with
its 1024 in/out pixel constraints), but there is no clear way to
communicate this to userspace other than flat out rejecting the plane
update. Atomic may provide some better ways, but we are not there yet
for imx-drm.

Regards,
Lucas
-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 08, 2015 at 01:07:47PM +0200, Lucas Stach wrote:
> Am Dienstag, den 08.09.2015, 12:01 +0100 schrieb Russell King - ARM
> Linux:
> > On Tue, Sep 08, 2015 at 12:56:18PM +0200, Lucas Stach wrote:
> > > This is expected, it's the overlay adapter which doesn't really work
> > > with imx-drm yet.
> > 
> > No, it's imx-drm which isn't working.  As I explained in my previous
> > email, overlay planes are expected to do scaling.  imx-drm errors out
> > attempts for that.  This is a kernel bug, not an Xorg driver bug.
> > 
> I would argue that this is a bug of the interface between kernel and
> userspace.

Yes, only in so far as knowing beforehand whether scaling is possible.
The only time that you get to know is when the call to display the plane
fails.  That's a really poor interface.

> Scaling isn't something that can be expected to be usable on every
> hardware (and in fact the IPU isn't able to do arbitrary scaling with
> its 1024 in/out pixel constraints), but there is no clear way to
> communicate this to userspace other than flat out rejecting the plane
> update. Atomic may provide some better ways, but we are not there yet
> for imx-drm.

I think that depends on your point of view - I suspect x86 people would
be surprised by that comment. :)

There was talk a while back when the overlay plane support went in that
it was possible to do >1024 pixels, but it was complex, but the impression
I was left with was one day it would work - and I'm still waiting.  I
suspect that the iMX6 is going to be obsolete before we have a decent
working video playback story on this hardware.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Dienstag, den 08.09.2015, 12:29 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 08, 2015 at 01:07:47PM +0200, Lucas Stach wrote:
> > Am Dienstag, den 08.09.2015, 12:01 +0100 schrieb Russell King - ARM
> > Linux:
> > > On Tue, Sep 08, 2015 at 12:56:18PM +0200, Lucas Stach wrote:
> > > > This is expected, it's the overlay adapter which doesn't really work
> > > > with imx-drm yet.
> > > 
> > > No, it's imx-drm which isn't working.  As I explained in my previous
> > > email, overlay planes are expected to do scaling.  imx-drm errors out
> > > attempts for that.  This is a kernel bug, not an Xorg driver bug.
> > > 
> > I would argue that this is a bug of the interface between kernel and
> > userspace.
> 
> Yes, only in so far as knowing beforehand whether scaling is possible.
> The only time that you get to know is when the call to display the plane
> fails.  That's a really poor interface.
> 
> > Scaling isn't something that can be expected to be usable on every
> > hardware (and in fact the IPU isn't able to do arbitrary scaling with
> > its 1024 in/out pixel constraints), but there is no clear way to
> > communicate this to userspace other than flat out rejecting the plane
> > update. Atomic may provide some better ways, but we are not there yet
> > for imx-drm.
> 
> I think that depends on your point of view - I suspect x86 people would
> be surprised by that comment. :)
> 
I think some of the recent Intel chips had the same problem, in that
they lost the ability to scale planes and ended up not exposing them due
to the poor interface.

> There was talk a while back when the overlay plane support went in that
> it was possible to do >1024 pixels, but it was complex, but the impression
> I was left with was one day it would work - and I'm still waiting. 

Don't wait for that. We can't do >1024 without doing a copy in memory,
which would be a big stretch of the plane definition and benefit.

In addition the IPU scaler has pitch and alignment constraints that make
the needed tiling extremely complex to get right. We fiddled with this
extensively to get it to the point that it is usable as a V4L2 mem2mem
device. Moving this to the display path is not going to happen, ever.

As all relevant resolutions already require a copy you are much better
of doing it with the GPU and get all the testing effort focused on that
path.

> I
> suspect that the iMX6 is going to be obsolete before we have a decent
> working video playback story on this hardware.
> 
MX6Plus is just around the corner and will probably keep us busy for a
while. ;)

Regards,
Lucas

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 08, 2015 at 01:01:47PM +0200, Krzysztof Ha?asa wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> > That's probably because of the restrictions in the IPU - the mainline
> > IPU code is unable to scale overlays at all - it's not supported.  The
> > problem is most Xorg drivers assume that overlays can be scaled.  I've
> > said about this before, and how this is broken, but I've no idea whether
> > it's going to get fixed.  As far as I know, this only affects platforms
> > using the IPU.
> 
> I see. But I'm ok with unscaled Xvideo. I only need a (fast) way to send
> YUV (preferably YUV420) image to the screen.

Note that most (all?) video requires some scaling.  A 720x576 video on
square pixels needs to be scaled to 768x576, but at "modern" aspect
ratio of 16:9, it needs to be scaled horizontally to 1024x576.

This pretty much makes the video overlay which imx-drm publishes to
userspace as useful as a chocolate teapot.  I suppose you could eat the
chocolate teapot, but it's not useful for its intended purpose of making
tea.

> > The only alternative there is to switch to using the GPU to blit the
> > XVideo frame onto the display (but then you need all the etnaviv bits
> > in place, including the etnaviv DRM driver.)  This works up to a point,
> > but suffers from tearing, because there's no sane good way to synchronise
> > the GPU blit with the video scanout (because they're two different
> > completely unrelated chunks of hardware: the reason Intel i965 can do
> > this is because it seems possible to put into the GPU stream "wait for
> > scan line X before continuing" thereby preventing the blit modifying
> > scanlines that are about to be displayed.
> 
> This could be the horizontal lines that I observed. Something similar
> to what the old S3 Trio64 were doing.

They're probably not the tearing - tearing is where you scan out part of
the old frame and part of the new frame, which appears as a particularly
noticable effect on scenes with a lot of change.

Horizontal lines which are significantly different (eg, short black lines)
will be CPU cache corruption - both the main etnaviv DRM and Lucas' "fixed"
cache handling suffer from this.  My patch set doesn't.

 git://ftp.arm.linux.org.uk/~rmk/linux-arm.git drm-etnaviv-devel

contains my code.

Etnaviv DRM development has stalled because of Pengutronix, and their
attitude towards open source development - which is to develop their
patches behind closed doors, and publish part of a story once in a blue
moon.  (Sorry Lucas, but Pengutronix has this criticism coming.)  Both
myself and Christian have tried to get more of their work out in the
open, and failed.  I think Christian has stopped work on the project
completely because of this, as he doesn't want to waste time duplicating
effort.  I've certainly asked for Lucas' modifications to my Xorg driver,
and got nowhere (I've had to update it myself, to align the kernel/user
API with Pengutronix' efforts.)  I think Christian has virtually given
up on it because of the lack of communication.

Etnaviv DRM is not a happy place to be right now, because of Pengutronix'
involvement in it, and their lack of sharing.  The only thing that they've
shared so far is a set of kernel patches.  Everything else is kept behind
their closed doors.

I think part of this is because Pengutronix want to be ones to claim "oh
look what we've done for the community" all the time - I see that a lot
on google+, where they're always posting "this is what we did in Linux
4.whatever" and they don't want anyone taking away their "thunder".
Quite sad really, because it means that development is actually being
hindered by such an approach, and people are getting pissed off with it.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 08, 2015 at 02:43:53PM +0200, Lucas Stach wrote:
> Am Dienstag, den 08.09.2015, 12:29 +0100 schrieb Russell King - ARM
> Linux:
> > There was talk a while back when the overlay plane support went in that
> > it was possible to do >1024 pixels, but it was complex, but the impression
> > I was left with was one day it would work - and I'm still waiting. 
> 
> Don't wait for that. We can't do >1024 without doing a copy in memory,
> which would be a big stretch of the plane definition and benefit.
> 
> In addition the IPU scaler has pitch and alignment constraints that make
> the needed tiling extremely complex to get right. We fiddled with this
> extensively to get it to the point that it is usable as a V4L2 mem2mem
> device. Moving this to the display path is not going to happen, ever.
> 
> As all relevant resolutions already require a copy you are much better
> of doing it with the GPU and get all the testing effort focused on that
> path.

And at that point, you might as well use the filter blit and blit directly
to the screen.

I've run some performance analysis on this on iMX6 hardware, and the time
taken for the CPU to copy the Xv data passed to it is about the same time
that it takes the CPU to flush the caches, then ask the GPU to copy the
data, and then wait for the GPU to report that the copy has finished.  My
analysis shows that you actually lose performance by using the GPU to
merely copy and then pass it to Xv.

Also, you mention that the IPU scaler has pitch and alignment constraints.
So does the GPU.  The Xv interface can cope with that just fine, because
the Xv backend gets to define the pitch and buffer size for any given
format/width/height, and offsets in the buffer for planar image formats.

It isn't able to specify the alignment of the actual buffer though - for
SHM, that's going to be the size of a struct page due to the way shmem
works.  For raw XvPutImage, things are harder to cope with, but the GPU
can cope provided the offset is aligned to a pixel boundary.

> MX6Plus is just around the corner and will probably keep us busy for a
> while. ;)

You might be excited about that, but for the rest of us, we have to put
up with the hardware we have, which is basically iMX6.  We need solutions
now on existing iMX6 hardware, not the next set of likely to be
problematical hardware.

I know the grass is always greener on the other side - and I expect that
when MX6+ comes out, you'll then have no time to work on etnaviv DRM,
and we'll still be stuck without any progress on that.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Robert Nelson]

>> I
>> suspect that the iMX6 is going to be obsolete before we have a decent
>> working video playback story on this hardware.
>>
> MX6Plus is just around the corner and will probably keep us busy for a
> while. ;)

Lucas,

In which case, it "might" be a good idea to get the current version of
etnaviv in mainline for current imx6's before you guys disappear again
for mx6plus development..

As you guys are basically repeating what happened with the limadriver:
http://limadriver.org/

Remember, there is an etnaviv community.

Russell:
is there any reason to not just push your version for mainline and let
Pengutronix submit patches.

Regards,

-- 
Robert Nelson
https://rcn-ee.com/

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Robert Nelson <robertcnelson@gmail.com> writes:

> In which case, it "might" be a good idea to get the current version of
> etnaviv in mainline for current imx6's before you guys disappear again
> for mx6plus development..

Definitely. This also includes the userspace parts.
Bugs can always be squashed, but everything is much easier when the
whole environment is already set up.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Dienstag, den 08.09.2015, 09:17 -0500 schrieb Robert Nelson:
> >> I
> >> suspect that the iMX6 is going to be obsolete before we have a decent
> >> working video playback story on this hardware.
> >>
> > MX6Plus is just around the corner and will probably keep us busy for a
> > while. ;)
> 
> Lucas,
> 
> In which case, it "might" be a good idea to get the current version of
> etnaviv in mainline for current imx6's before you guys disappear again
> for mx6plus development..
> 
This wasn't to signal that we are moving away from MX6 just to say that
there might be a lot more life in MX6 than what the above implies.

> As you guys are basically repeating what happened with the limadriver:
> http://limadriver.org/
> 
We have no intention to repeat that.

> Remember, there is an etnaviv community.
> 
I would love to get review feedback for what is currently in the open
from that community.

Regards,
Lucas

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

> Note that most (all?) video requires some scaling.  A 720x576 video on
> square pixels needs to be scaled to 768x576, but at "modern" aspect
> ratio of 16:9, it needs to be scaled horizontally to 1024x576.

Sure. What I meant was the very application I'm currently working on
is, at least at the moment, ok with unscaled video (because it's all
1920x1080p).

> They're probably not the tearing - tearing is where you scan out part of
> the old frame and part of the new frame, which appears as a particularly
> noticable effect on scenes with a lot of change.

Right, basically a set of two "halves", moving slowly up or down.
This is BTW what can be observed on unsynchronized displays (e.g. on
multi-display machine usually only one display can be synchronized with
a video stream).

> Horizontal lines which are significantly different (eg, short black lines)
> will be CPU cache corruption - both the main etnaviv DRM and Lucas' "fixed"
> cache handling suffer from this.  My patch set doesn't.
>
>  git://ftp.arm.linux.org.uk/~rmk/linux-arm.git drm-etnaviv-devel
>
> contains my code.

Great, will switch to it tomorrow.
Thanks again.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 08, 2015 at 09:17:06AM -0500, Robert Nelson wrote:
> >> I
> >> suspect that the iMX6 is going to be obsolete before we have a decent
> >> working video playback story on this hardware.
> >>
> > MX6Plus is just around the corner and will probably keep us busy for a
> > while. ;)
> 
> Lucas,
> 
> In which case, it "might" be a good idea to get the current version of
> etnaviv in mainline for current imx6's before you guys disappear again
> for mx6plus development..
> 
> As you guys are basically repeating what happened with the limadriver:
> http://limadriver.org/
> 
> Remember, there is an etnaviv community.
> 
> Russell:
> is there any reason to not just push your version for mainline and let
> Pengutronix submit patches.

The basic problem is getting the kernel<->user API stable before it
goes into mainline, which is an absolute must.  We can't drop it into
mainline and then break the user API.

We also want to avoid putting it into the staging tree and have it
sit around there for years.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Jon Nettleton]

On Tue, Sep 8, 2015 at 5:55 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Tue, Sep 08, 2015 at 09:17:06AM -0500, Robert Nelson wrote:
>> >> I
>> >> suspect that the iMX6 is going to be obsolete before we have a decent
>> >> working video playback story on this hardware.
>> >>
>> > MX6Plus is just around the corner and will probably keep us busy for a
>> > while. ;)
>>
>> Lucas,
>>
>> In which case, it "might" be a good idea to get the current version of
>> etnaviv in mainline for current imx6's before you guys disappear again
>> for mx6plus development..
>>
>> As you guys are basically repeating what happened with the limadriver:
>> http://limadriver.org/
>>
>> Remember, there is an etnaviv community.
>>
>> Russell:
>> is there any reason to not just push your version for mainline and let
>> Pengutronix submit patches.
>
> The basic problem is getting the kernel<->user API stable before it
> goes into mainline, which is an absolute must.  We can't drop it into
> mainline and then break the user API.
>
> We also want to avoid putting it into the staging tree and have it
> sit around there for years.
>

If you don't have much time to work on it then lets get it pushed to
the github community repos where we can have everyone else submit
patches.  We have #etnaviv where we can coordinate efforts.  Even if
something is on a todo list lets get it out where we someone else can
work on it.  I think our community is strong enough that we can all
make progress without stepping on each others feet.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Now I'm trying to use the imxdrm Xv output.

$ mplayer -vo xv test.mp4 -x 720

VO Config (720x576->720x576,flags=0,'MPlayer',0x32315659)
VO: [xv] 720x576 => 720x576 Planar YV12
Xvideo image format: 0x34325241 (AR24) packed
Xvideo image format: 0x34325258 (XR24) packed
Xvideo image format: 0x34324241 (AB24) packed
Xvideo image format: 0x34324258 (XB24) packed
Xvideo image format: 0x32595559 (YUY2) packed
Xvideo image format: 0x30323449 (I420) planar
Xvideo image format: 0x32315659 (YV12) planar
Xvideo image format: 0x4f425658 (XVBO) planar
using Xvideo port 85 for hw scaling
*** [vo] Exporting mp_image_t, 720x576x12bpp YUV planar, 622080 bytes

Adaptor #0: "Marvell Armada Overlay Video"
number of ports: 1
port base: 85

The Y plane is ok, but the Xv window is green, like on old monochrome
CRTs - what could that be? It looks like the UV planes are all wrong.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Thu, Sep 10, 2015 at 12:25:32PM +0200, Krzysztof Ha?asa wrote:
> The Y plane is ok, but the Xv window is green, like on old monochrome
> CRTs - what could that be? It looks like the UV planes are all wrong.

>From a quick read of the imx-drm ipuv3-plane code, it looks like it
doesn't bother with the U/V planes for planar images.  Nothing looks
at fb->offsets[1,2] nor fb->pitches[1,2].  However, it claims to
support DRM_FORMAT_YUV420 and DRM_FORMAT_YVU420 formats - which is
wrong if it doesn't look at these other offsets and pitches.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>>From a quick read of the imx-drm ipuv3-plane code, it looks like it
> doesn't bother with the U/V planes for planar images.  Nothing looks
> at fb->offsets[1,2] nor fb->pitches[1,2].  However, it claims to
> support DRM_FORMAT_YUV420 and DRM_FORMAT_YVU420 formats - which is
> wrong if it doesn't look at these other offsets and pitches.

Well... I have to agree.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Another round of tests, I noticed the new git versions :-)
Testing Linux v4.2 + PLL5 DTS patch (for HDMI output with enabled LVDS).
Using mplayer with YUV420 (DRM Xvideo would probably work with packed
UYVY-alike formats but I need YUV420 because H.264 decoder produces it).
The driver is git://ftp.arm.linux.org.uk/~rmk/xf86-video-armada.git,
branch unstable-devel, and it uses
git://ftp.arm.linux.org.uk/~rmk/libdrm-armada.git/.

	IMX DRM Xvideo output:

Only unscaled video: no color (luminance is good but the color
components are green). The driver doesn't use color information.
I hope this is easily fixable.


	rmk/drm-etnaviv-devel:

With unscaled video, the only visible problem is tearing in the middle
of the screen (unability to sync with screen refresh).

With scaling, I'm getting horizontal lines (mostly visible when the
scene changes) and some sort of stalls - sometimes two frames are
alternating for few seconds then it goes forward.


	pengutronix/etnaviv-for-upstream:
No etnaviv Xvideo:
(==) armada(0): Backing store enabled
(==) armada(0): Silken mouse enabled
(EE) armada(0): etnaviv: unable to open: Not a directory
(WW) armada(0): [drm] Vivante initialization failed, running unaccelerated

I assume I need a newer something.


	pengutronix/v4.2/topic/etnaviv-for-rmk:
It requires:
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -216,6 +216,7 @@ phys_addr_t dma_get_contiguous_base(struct device *dev)
 {
 	return cma_get_base(dev_get_cma_area(dev));
 }
+EXPORT_SYMBOL(dma_get_contiguous_base);
 
 /*
  * Support for reserved memory regions defined in device tree


Likewise, no etnaviv XVideo:
(--) armada(0): Vivante GC320 GPU revision 5007 (etnaviv) 2d PE2.0
(EE) armada(0): etnaviv: unable to create context: (null)
(WW) armada(0): [drm] Vivante initialization failed, running unaccelerated


Comments?
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

> Testing Linux v4.2 + PLL5 DTS patch (for HDMI output with enabled LVDS).
> Using mplayer with YUV420 (DRM Xvideo would probably work with packed
> UYVY-alike formats but I need YUV420 because H.264 decoder produces it).
> The driver is git://ftp.arm.linux.org.uk/~rmk/xf86-video-armada.git,
> branch unstable-devel, and it uses
> git://ftp.arm.linux.org.uk/~rmk/libdrm-armada.git/.
>
> 	IMX DRM Xvideo output:
>
> Only unscaled video: no color (luminance is good but the color
> components are green). The driver doesn't use color information.

1024 x 768 for now. I noticed a strange thing: XvShmPutImage() usually
takes (much) less than 4 milliseconds, but every 315 frames, it takes
much longer (when started, it takes 1259 frames). The following is with
constant image, i.e., not altered between frames. Source attached (one
may need to change xv_port variable).

Now what?

XVideo: using adaptor #0, port 85
Image ID 30323449 1024x768 data_size 1179648 num_planes 3 pitches 1024 512 512 data 0x76b86000
Frame#  frame count from the last such event
  vvvv   vvvv
  1259   1259: Took 0.665640667 s, avg 0.000530765 s
  1574    315: Took 0.501529334 s, avg 0.000743380 s
  1889    315: Took 0.496656667 s, avg 0.000882432 s
  2204    315: Took 0.496726667 s, avg 0.000981782 s
  2519    315: Took 0.496672333 s, avg 0.001056277 s
  2834    315: Took 0.496910000 s, avg 0.001114301 s
  3149    315: Took 0.497260667 s, avg 0.001160832 s
  3464    315: Took 0.499651334 s, avg 0.001199600 s
  3779    315: Took 0.495202334 s, avg 0.001230718 s
  4094    315: Took 0.499449667 s, avg 0.001258081 s
  4409    315: Took 0.525910333 s, avg 0.001287535 s
  4724    315: Took 0.510400000 s, avg 0.001309786 s
  5039    315: Took 0.541753334 s, avg 0.001335467 s
  5354    315: Took 0.537526666 s, avg 0.001357350 s
  5669    315: Took 0.542006000 s, avg 0.001377594 s
  5984    315: Took 0.539530333 s, avg 0.001395288 s
  6299    315: Took 0.541836000 s, avg 0.001411578 s
  6614    315: Took 0.541556333 s, avg 0.001426275 s
  6929    315: Took 0.543209666 s, avg 0.001439874 s
  7244    315: Took 0.540764000 s, avg 0.001451956 s
  7559    315: Took 0.512616666 s, avg 0.001459303 s
  7874    315: Took 0.541380000 s, avg 0.001469720 s
  8189    315: Took 0.535251333 s, avg 0.001478584 s
  8504    315: Took 0.536773334 s, avg 0.001486975 s
  8819    315: Took 0.521379000 s, avg 0.001493018 s
  9134    315: Took 0.492112667 s, avg 0.001495438 s
  9449    315: Took 0.616766000 s, avg 0.001510886 s
  9764    315: Took 0.492945666 s, avg 0.001512657 s
 10079    315: Took 0.492615000 s, avg 0.001514286 s
 10394    315: Took 0.493068000 s, avg 0.001515858 s
 10709    315: Took 0.496838000 s, avg 0.001517693 s
 11024    315: Took 0.516352333 s, avg 0.001521193 s
 11339    315: Took 0.500899333 s, avg 0.001523138 s
 11654    315: Took 0.494034334 s, avg 0.001524386 s
 11969    315: Took 0.492401000 s, avg 0.001525430 s
 12284    315: Took 0.505440000 s, avg 0.001527485 s
 12599    315: Took 0.511242666 s, avg 0.001529898 s
 12914    315: Took 0.493489333 s, avg 0.001530816 s

With etnaviv Xvideo, the effect is similar but much worse:

> 	rmk/drm-etnaviv-devel:
>
> With unscaled video, the only visible problem is tearing in the middle
> of the screen (unability to sync with screen refresh).

XVideo: using adaptor #1, port 90
Image ID 30323449 1024x768 data_size 1179648 num_planes 3 pitches 1024 512 512 data 0x76b8d000
  1259   1259: Took 18.864969669 s, avg 0.014974047 s
  1574    315: Took 20.758470002 s, avg 0.025159601 s
  1889    315: Took 20.720458336 s, avg 0.031929827 s
  2204    315: Took 20.831572669 s, avg 0.036815999 s
  2519    315: Took 20.807073336 s, avg 0.040470906 s
  2834    315: Took 20.789431003 s, avg 0.043307485 s
  3149    315: Took 20.809190003 s, avg 0.045582932 s
  3464    315: Took 20.803870003 s, avg 0.047443189 s
  3779    315: Took 20.798422002 s, avg 0.048991905 s
  4094    315: Took 20.809182336 s, avg 0.050304985 s
  4409    315: Took 20.812659336 s, avg 0.051431326 s
  4724    315: Took 20.816828669 s, avg 0.052408363 s
  5039    315: Took 20.814890002 s, avg 0.053262894 s
  5354    315: Took 20.803809003 s, avg 0.054014822 s
  5669    315: Took 20.798233669 s, avg 0.054682176 s
  5984    315: Took 20.809503336 s, avg 0.055281168 s
  6299    315: Took 20.814339002 s, avg 0.055821065 s
  6614    315: Took 20.812699003 s, avg 0.056309294 s
  6929    315: Took 20.800363670 s, avg 0.056751361 s
  7244    315: Took 20.814518669 s, avg 0.057156941 s
  7559    315: Took 20.814902002 s, avg 0.057528773 s
  7874    315: Took 20.814914336 s, avg 0.057870859 s
  8189    315: Took 20.803919669 s, avg 0.058185288 s
  8504    315: Took 20.807010336 s, avg 0.058476760 s
  8819    315: Took 20.855880669 s, avg 0.058752983 s
  9134    315: Took 20.809301669 s, avg 0.059005029 s
  9449    315: Took 26.004653003 s, avg 0.059790071 s
  9764    315: Took 20.796989002 s, avg 0.059991163 s
 10079    315: Took 20.798379670 s, avg 0.060179803 s
 10394    315: Took 20.809240335 s, avg 0.060358055 s
 10709    315: Took 20.814655003 s, avg 0.060526351 s
 11024    315: Took 20.803836002 s, avg 0.060684046 s
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

#include <ctype.h>
#include <signal.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <time.h>
#include <sys/shm.h>
#include <sys/time.h>
#include <X11/Xlib.h>
#include <X11/Xutil.h>
#include <X11/Xatom.h>
#include <X11/extensions/Xv.h>
#include <X11/extensions/Xvlib.h>
#include <X11/extensions/XShm.h>

#define IMAGE_FORMAT_NAME "planar I420"
#define IMAGE_FORMAT      0x30323449
#define PIXEL_FORMAT      AV_PIX_FMT_YUV420P

static Display *display;
static Window window;
static XShmSegmentInfo shminfo;
static XvImage *image;
static GC gc;
static unsigned w, h;
static unsigned xv_port = 90;

static void test_xv(void)
{
	fprintf(stderr, "Image ID %X %ux%u data_size %u num_planes %u pitches %u %u %u data %p\n",
			image->id, image->width, image->height, image->data_size, image->num_planes, image->pitches[0], image->pitches[1], image->pitches[2], image->data);

	struct timespec ts;
	if (clock_gettime(CLOCK_MONOTONIC, &ts)) {
		fprintf(stderr, "clock_gettime(MONOTONIC) failed: %m\n");
		exit(1);
	}

	unsigned long long prev_t = (unsigned long long)ts.tv_sec * 1000 * 1000 * 1000 + ts.tv_nsec;
	unsigned long long start_t = prev_t;
	unsigned count = 0, prev_count = 0;

	while (1) {
		memset(image->data, count, image->data_size);
		XvShmPutImage(display, xv_port, window, gc, image, 0, 0, w, h, 0, 0, w, h, True);

		if (clock_gettime(CLOCK_MONOTONIC, &ts)) {
			fprintf(stderr, "clock_gettime(MONOTONIC) failed: %m\n");
			exit(1);
		}
		unsigned long long t = (unsigned long long)ts.tv_sec * 1000 * 1000 * 1000 + ts.tv_nsec;
		if (t - prev_t > 4 * 1000 * 1000) {
			fprintf(stdout, "%6u %6u: Took %llu.%09llu s, avg %llu.%09llu s\n",
					count, count - prev_count,
					(t - prev_t) / (1000 * 1000 * 1000),
					(t - prev_t) % (1000 * 1000 * 1000),
					(t - start_t) / (count + 1) / (1000 * 1000 * 1000),
					(t - start_t) / (count + 1) % (1000 * 1000 * 1000));
			prev_count = count;
		}
		prev_t = t;
		count++;
	}
}

int main(void)
{
	display = XOpenDisplay(NULL);
	if (!display) {
		fprintf(stderr, "Unable to open X11 display\n");
		exit(1);
	}

	unsigned int screen = DefaultScreen(display);
	Window rdw;
	unsigned rb, rd;
	int x, y;
	if (!XGetGeometry(display, XRootWindow(display, screen), &rdw, &x, &y, &w, &h, &rb, &rd)) {
		fprintf(stderr, "Unable to get root window geometry\n");
		exit(1);
	}

	fprintf(stderr, "Creating %ux%u window at %ux%u\n", w, h, x, y);
	window = XCreateSimpleWindow(display, XRootWindow(display, screen), x, y, w, h, 0, 0, 0);

	static XClassHint class_hints = {.res_name = "Xvideo test", .res_class = "XVideo test"};
	XSetClassHint(display, window, &class_hints);

	XMapWindow(display, window);
	XStoreName(display, window, "Xvideo test");
	XSetIconName(display, window, "Xvideo test");
	XFlush(display);

	// Search for Xvideo ports
	XvAdaptorInfo *info;
	unsigned adaptors;
	if (XvQueryAdaptors(display, window, &adaptors, &info) != Success) {
		fprintf(stderr, "No XVideo adaptors found\n");
		exit(1);
	}

	unsigned cnt, found = 0;
	for (cnt = 0; cnt < adaptors; cnt++) {
		XvAdaptorInfo *adaptor = &info[cnt];
#if 0
		fprintf(stderr, "Xv adaptor: %s, port base %lu, count %lu, type 0x%X, %lu format(s)\n",
				adaptor->name, adaptor->base_id, adaptor->num_ports, adaptor->type, adaptor->num_formats);
		for (unsigned cnt1 = 0; cnt1 < adaptor->num_formats; cnt1++) {
			XvFormat *format = &adaptor->formats[cnt1];
			fprintf(stderr, "   depth %u, visual ID 0x%lX\n", format->depth, format->visual_id);
		}
#endif

		for (unsigned cnt1 = 0; cnt1 < adaptor->num_ports; cnt1++) {
			int num_formats;
			XvImageFormatValues *image_formats = XvListImageFormats(display, adaptor->base_id + cnt1, &num_formats);
			for (int cnt2 = 0; cnt2 < num_formats; cnt2++) {
				XvImageFormatValues *image_format = &image_formats[cnt2];

#if 0
				fprintf(stderr, "   Port %lu: type %i ID 0x%X (%c%c%c%c) byte order %i\n",
						adaptor->base_id + cnt1, image_format->type, image_format->id,
						isprint( image_format->id        & 0xFF) ?  image_format->id        & 0xFF : '?',
						isprint((image_format->id >>  8) & 0xFF) ? (image_format->id >>  8) & 0xFF : '?',
						isprint((image_format->id >> 16) & 0xFF) ? (image_format->id >> 16) & 0xFF : '?',
						isprint((image_format->id >> 24) & 0xFF) ? (image_format->id >> 24) & 0xFF : '?',
						image_format->byte_order);
#endif

				if (!found) {
					if (xv_port) {
						if (xv_port >= adaptor->base_id && xv_port < adaptor->base_id + adaptor->num_ports) {
							if (!(adaptor->type & XvImageMask))
								fprintf(stderr, "XVideo port %u doesn't have XvImageMask capability\n", xv_port);
							else if (image_format->id == IMAGE_FORMAT)
								found = 1;
							else if (xv_port == adaptor->base_id + adaptor->num_ports - 1)
								fprintf(stderr, "XVideo port %u doesn't support " IMAGE_FORMAT_NAME "image format\n", xv_port);
						}
					} else {
						if (adaptor->type & XvImageMask && adaptor->num_ports && image_format->id == IMAGE_FORMAT) {
							xv_port = adaptor->base_id;
							found = 1;
						}
					}
					if (found)
						fprintf(stderr, "XVideo: using adaptor #%u, port %u\n", cnt, xv_port);
				}
			}

			XFree(image_formats);
		}
	}
	XvFreeAdaptorInfo(info);

	if (!found) {
		fprintf(stderr, "No valid XVideo adaptor found\n");
		exit(1);
	}

	image = XvShmCreateImage(display, xv_port, IMAGE_FORMAT, NULL, w, h, &shminfo);
	shminfo.shmid = shmget(IPC_PRIVATE, image->data_size, IPC_CREAT | 0777);
	if (shminfo.shmid < 0)
		exit(1);

	shminfo.shmaddr = image->data = shmat(shminfo.shmid, 0, 0);
	if (shminfo.shmaddr == (void*) -1)
		exit(1);

	shminfo.readOnly = False;

	if (!XShmAttach(display, &shminfo))
		exit(1);

	gc = XCreateGC(display, window, 0, 0);
	test_xv();
	exit(0);
}

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 15, 2015 at 10:24:55AM +0200, Krzysztof Ha?asa wrote:
> 1024 x 768 for now. I noticed a strange thing: XvShmPutImage() usually
> takes (much) less than 4 milliseconds, but every 315 frames, it takes
> much longer (when started, it takes 1259 frames). The following is with
> constant image, i.e., not altered between frames. Source attached (one
> may need to change xv_port variable).

Testing on Dove and Armada DRM, I've had to make several changes to your
program:

1. Don't hard code the Xv port ID (worth mentioning somewhere so people
   testing it don't spend something like a quarter of an hour trying to
   debug why it doesn't work.)

2. Added _GNU_SOURCE at the top to get various structs and functions
   that the program uses.

3. Added a line detailing the GCC flags to be used

4. A description of what the program is doing would be nice.  For the
   record, it appears to try and call XvShmPutImage() as quickly as
   possible, recording the absolute time between calls, and reporting
   when it is more than 4ms.

5. Added a call to XSync(display, 1) after the call to XvShmPutImage()
   to ensure that the XvShmPutImage() gets pushed to the X server in a
   timely manner, doesn't sit in the applications X queue, and ensure
   that we wait for the X server to respond.

6. Extended the 4ms timeout to something sane, like 17ms.  If we're
   talking about synchronising to the vertical sync, then the period
   when we want to complain is when it takes much longer than that.

   If the display is synchronised to the vertical sync to avoid tearing,
   then you can only update the display once per vertical sync, and at
   60Hz, that's about 17ms.  At 50Hz, that's 20ms.

Now, running on Dove overlay, I see about 17ms being the average time,
which is slightly longer than the vsync.

Further analysis with strace of the X server (which pushes things up to
20ms average time):
- The X server takes about 1ms between select() indicating that there's
  an event available, and reading the event.
- It takes 14ms from reading the event to the DDX calling the DRM ioctl
  to display the image - mostly spent memcpy()'ing the image.
- 900us to return the response to the X client
- The X server then takes about 4ms to get back to calling select().

The big chunk of that is the memcpy() - that's something which can't be
eliminated for several reasons:
- With XvShm, the image buffer is shared between the X client and the
  X server.  The X server has no control over how the X client manipulates
  that buffer after the X server has returned from its call - however the
  displayed image must remain stable and free from manipulation until the
  next PutImage call.
- The X shm buffer can't be passed directly into DRM - DRM needs the
  image to be displayed to be in DRM buffer objects, which are then
  wrapped as frame buffers (a frame buffer can contain several buffer
  objects, one for each plane), and lastly the overlay plane updated
  with the new framebuffer.

Let's not forget that Dove hardware is slower than iMX6, so I think that
iMX6 should manage to comfortably achieve the 16.6ms required for
frame-by-frame display from your program.

Now, as for using the GPU, X server analysis:
- Again about 1ms between select() and read() of the event
- 1ms to the first WAIT_VBLANK ioctl on the display adapter to read the
  last VSYNC time
- 200us later to map the user pointer
- 300us to the WAIT_VBLANK ioctl to wait for the vblank (which takes in this
  instance 3.5ms to fire)
- 130us to first attempt to submit the GPU queue, which returns -EAGAIN
  after 6.4ms
- second attempt takes 9.5ms to complete successfully
- 1ms (with intervening SIGALRM handler) to wait for the GPU to finish,
  which takes 11.5ms
- 450us to ask DRM to release the user pointer buffer, which takes 9.4ms
- 1ms (with intervening SIGALRM handler) to report completion of the operation
- The X server then takes about 4ms to get back to calling select().

So, you can see that using the GPU is a much heavier and complex operation
all round.  The long delays in mapping and releasing the buffer are of
course the DMA mapping/unmapping of the buffer object to ensure that the
GPU can see the data in those buffers.

What's also notable is that it takes the GPU on the order of 10ms to do the
operation - it's actually two operations: a vertical filter blit followed
by a horizontal filter blit.  None of the Vivante GPUs that I've access to
support a single-pass filter blit scaling in both directions (the feature
bit for that is clear on both Dove's GC600 and iMX6 GC320 GPUs.)  I
suspect a Vivante GPU supporting that would take around half the time,
or better.

Of course, the time each of these operations take scales with the image
size, so an image half the width and height (which will be a quarter of
the size) will take a quarter of the time, and will be comfortably
within 17ms.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>> 1024 x 768 for now. I noticed a strange thing: XvShmPutImage() usually
>> takes (much) less than 4 milliseconds, but every 315 frames, it takes
>> much longer (when started, it takes 1259 frames). The following is with
>> constant image, i.e., not altered between frames. Source attached (one
>> may need to change xv_port variable).
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> 1. Don't hard code the Xv port ID (worth mentioning somewhere so people
>    testing it don't spend something like a quarter of an hour trying to
>    debug why it doesn't work.)

Well, I actually mentioned it.


So this means the delay was only "grouped" by the X protocol, and it
won't be magically faster.

> Now, running on Dove overlay, I see about 17ms being the average time,
> which is slightly longer than the vsync.

> Let's not forget that Dove hardware is slower than iMX6, so I think that
> iMX6 should manage to comfortably achieve the 16.6ms required for
> frame-by-frame display from your program.

Interestingly, i.MX6 seems to achive about 3.2 ms, which is much faster.
If not for the missing color planes (and scaling), it would be perfect.
(and I only need 30 FPS).

> Now, as for using the GPU, X server analysis:
> - Again about 1ms between select() and read() of the event
> - 1ms to the first WAIT_VBLANK ioctl on the display adapter to read the
>   last VSYNC time
> - 200us later to map the user pointer
> - 300us to the WAIT_VBLANK ioctl to wait for the vblank (which takes in this
>   instance 3.5ms to fire)
> - 130us to first attempt to submit the GPU queue, which returns -EAGAIN
>   after 6.4ms
> - second attempt takes 9.5ms to complete successfully
> - 1ms (with intervening SIGALRM handler) to wait for the GPU to finish,
>   which takes 11.5ms
> - 450us to ask DRM to release the user pointer buffer, which takes 9.4ms
> - 1ms (with intervening SIGALRM handler) to report completion of the operation
> - The X server then takes about 4ms to get back to calling select().

Sure, I know it has to be slower.

So this takes (on Dove) about 50 ms (at 1024x768), thus is way too slow
to display video at this (or higher) resolution and frame rate.
On i.MX6q, it takes about 66 - 74 ms, depending on actual clock speed
(frequency scaling, up to 1 GHz).

Thanks for testing.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 15, 2015 at 01:01:25PM +0200, Krzysztof Ha?asa wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> > Let's not forget that Dove hardware is slower than iMX6, so I think that
> > iMX6 should manage to comfortably achieve the 16.6ms required for
> > frame-by-frame display from your program.
> 
> Interestingly, i.MX6 seems to achive about 3.2 ms, which is much faster.
> If not for the missing color planes (and scaling), it would be perfect.
> (and I only need 30 FPS).

That means iMX6 isn't synchronising to the vertical sync, so would be
subject to tearing.

> 
> > Now, as for using the GPU, X server analysis:
> > - Again about 1ms between select() and read() of the event
> > - 1ms to the first WAIT_VBLANK ioctl on the display adapter to read the
> >   last VSYNC time
> > - 200us later to map the user pointer
> > - 300us to the WAIT_VBLANK ioctl to wait for the vblank (which takes in this
> >   instance 3.5ms to fire)
> > - 130us to first attempt to submit the GPU queue, which returns -EAGAIN
> >   after 6.4ms
> > - second attempt takes 9.5ms to complete successfully
> > - 1ms (with intervening SIGALRM handler) to wait for the GPU to finish,
> >   which takes 11.5ms
> > - 450us to ask DRM to release the user pointer buffer, which takes 9.4ms
> > - 1ms (with intervening SIGALRM handler) to report completion of the operation
> > - The X server then takes about 4ms to get back to calling select().
> 
> Sure, I know it has to be slower.
> 
> So this takes (on Dove) about 50 ms (at 1024x768), thus is way too slow
> to display video at this (or higher) resolution and frame rate.
> On i.MX6q, it takes about 66 - 74 ms, depending on actual clock speed
> (frequency scaling, up to 1 GHz).

It's interesting that iMX6 takes longer - without analysing what's going
on, I'd guess that will be because the GC320's memory bandwidth is
throttled compared to Dove.

iMX6 seems to have a problem with the bus arbitration in that each master
which can access DDR is given a fixed timeslot, whether or not there are
any other masters wanting access.  Performance measurement with 64-bit
DDR has shown that out of iMX6Q,D,DL,S the iMX6DL performs the best for
raw IO as there are fewer masters (fewer CPUs and fewer peripherals
wanting DDR access), so the timeslots are bigger.

I've found that it is very difficult to get iMX6's GPUs to out-perform
neon optimised libpixman in terms of performance measurements - and I
think the reason for this is because both Neon optimised libpixman and
the GPUs are running up against the same bottleneck - the available
DDR bandwidth.  However, what you do get is CPU offload of that effort
- filling one of those non-CPU DDR access timeslots with useful work,
freeing up the CPU DDR timeslot to do other tasks.

I have no idea how much truth there is to this, this is all based upon
performance measurement, and drawing conclusions from those measurements.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Montag, den 14.09.2015, 10:39 +0200 schrieb Krzysztof Ha?asa:
> Another round of tests, I noticed the new git versions :-)
> Testing Linux v4.2 + PLL5 DTS patch (for HDMI output with enabled
> LVDS).
> Using mplayer with YUV420 (DRM Xvideo would probably work with packed
> UYVY-alike formats but I need YUV420 because H.264 decoder produces
> it).
> The driver is git://ftp.arm.linux.org.uk/~rmk/xf86-video-armada.git,
> branch unstable-devel, and it uses
> git://ftp.arm.linux.org.uk/~rmk/libdrm-armada.git/.
> 
> 	IMX DRM Xvideo output:
> 
> Only unscaled video: no color (luminance is good but the color
> components are green). The driver doesn't use color information.
> I hope this is easily fixable.
> 
There were patches for that some weeks ago already, I would hope they
got applied to some tree destinied for upstream. Will look at that and
ping people if necessary.

> 
> 	rmk/drm-etnaviv-devel:
> 
> With unscaled video, the only visible problem is tearing in the
> middle
> of the screen (unability to sync with screen refresh).
> 
> With scaling, I'm getting horizontal lines (mostly visible when the
> scene changes) and some sort of stalls - sometimes two frames are
> alternating for few seconds then it goes forward.
> 
> 
> 	pengutronix/etnaviv-for-upstream:
That is the right branch, containing the changes I sent out for review.

It includes a new kernel<->userspace interface, so needs a modified
xf86-video-armada to go with it. You can pick that up here:
git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk

Caution: the armada code is WIP and needs further cleanups. It should
work though.

> No etnaviv Xvideo:
> (==) armada(0): Backing store enabled
> (==) armada(0): Silken mouse enabled
> (EE) armada(0): etnaviv: unable to open: Not a directory
> (WW) armada(0): [drm] Vivante initialization failed, running
> unaccelerated
> 
> I assume I need a newer something.
> 
> 
> 	pengutronix/v4.2/topic/etnaviv-for-rmk:

Please don't use that branch. It just there to keep stability for
people that already know about it.

Regards,
Lucas

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 15, 2015 at 05:53:45PM +0200, Lucas Stach wrote:
> It includes a new kernel<->userspace interface, so needs a modified
> xf86-video-armada to go with it. You can pick that up here:
> git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk
> 
> Caution: the armada code is WIP and needs further cleanups. It should
> work though.

And it needs updating to the latest version - you seem to be missing
a whole load of changes, including the changes which update the
kernel API.

So... you seem to be talking about the kernel API having changed from
the version which you picked up years ago, rather than the current
version.  It's really unclear what you're talking about.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>> Interestingly, i.MX6 seems to achive about 3.2 ms, which is much faster.
>> If not for the missing color planes (and scaling), it would be perfect.
>> (and I only need 30 FPS).
>
> That means iMX6 isn't synchronising to the vertical sync, so would be
> subject to tearing.

Right. However the bigger problem ATM is the lack of color, or actually
wrong colors (such as solid green or magenta etc). I'm looking if the
support can be added easily, this whole IPU stuff is a bit complicated.

> It's interesting that iMX6 takes longer - without analysing what's going
> on, I'd guess that will be because the GC320's memory bandwidth is
> throttled compared to Dove.

Well, one would think the bandwidths should be an order of magnitude
larger than needed here. The boards I have here (Ventana GW5xxx) are
using e.g. 4 * MT41K128M16JT-125 DDR3 chips, for a total of (if
configured correctly) 12 GB/s. Now, a single burst read (or write) at
1024x768 32 bit requires 3 MB, for 30 FPS it means 90 MB/s.
I realize the buffers may be copied many times, and the timeslots you
describe don't make it easier. 66 ms for a frame is rather long, though.

I guess, in this case, it would be better to convert YUV420 planar ->
YUV422 packed (perhaps with NEON - the packed mode(s) work correctly),
and use the more traditional overlay instead of the textured adapter.
Or maybe I will make this YUV420 mode work.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Lucas Stach]

Hi Russell,

Am Dienstag, den 15.09.2015, 17:36 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 15, 2015 at 05:53:45PM +0200, Lucas Stach wrote:
> > It includes a new kernel<->userspace interface, so needs a modified
> > xf86-video-armada to go with it. You can pick that up here:
> > git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk
> > 
> > Caution: the armada code is WIP and needs further cleanups. It
> > should
> > work though.
> 
> And it needs updating to the latest version - you seem to be missing
> a whole load of changes, including the changes which update the
> kernel API.
> 
> So... you seem to be talking about the kernel API having changed from
> the version which you picked up years ago, rather than the current
> version.  It's really unclear what you're talking about.
> 
What do you mean with missing a lot of stuff? The "for-rmk" branch in
that repo is your upstream "unstable-devel" branch minus the few XvBO
commits with my changes on top.

Regards,
Lucas

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Lucas Stach <l.stach@pengutronix.de> writes:

>> Only unscaled video: no color (luminance is good but the color
>> components are green). The driver doesn't use color information.
>> I hope this is easily fixable.

> There were patches for that some weeks ago already, I would hope they
> got applied to some tree destinied for upstream. Will look at that and
> ping people if necessary.

Good, this probably resolves one of main problems.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Tue, Sep 15, 2015 at 12:53:54PM -0400, Lucas Stach wrote:
> Hi Russell,
> 
> Am Dienstag, den 15.09.2015, 17:36 +0100 schrieb Russell King - ARM
> Linux:
> > On Tue, Sep 15, 2015 at 05:53:45PM +0200, Lucas Stach wrote:
> > > It includes a new kernel<->userspace interface, so needs a modified
> > > xf86-video-armada to go with it. You can pick that up here:
> > > git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk
> > > 
> > > Caution: the armada code is WIP and needs further cleanups. It
> > > should
> > > work though.
> > 
> > And it needs updating to the latest version - you seem to be missing
> > a whole load of changes, including the changes which update the
> > kernel API.
> > 
> > So... you seem to be talking about the kernel API having changed from
> > the version which you picked up years ago, rather than the current
> > version.  It's really unclear what you're talking about.
> > 
> What do you mean with missing a lot of stuff? The "for-rmk" branch in
> that repo is your upstream "unstable-devel" branch minus the few XvBO
> commits with my changes on top.

Ah, probably because I haven't pushed the stuff out :)

Anyway, I've taken two of your patches, reworking the second slightly:

1fec728 etnaviv: fix compilation without DRI2 support
1663382 make sure that system strndup doesn't collide with X.Org server version

which are _both_ fixes, and would have been nice for them to have been
sent in a timely manner.

Looking at "etnaviv: fix XV resize for UYVY source format", this is wrong
to the documentation for the GC320.  Documentation which was around for
the GC320 indicates that the _only_ YUV format it supports as a target
is YUY2 with the vertical filter blit.  It's very specifically worded to
indicate that this is the only YUV target format which is supported.
What testing have you done to prove uyvy support, and on which GPUs?

"etnaviv: align vximage buffer size to pagesize" I have issues with as
well - mapping more memory than was passed to the X server is really not
permissible.  If we need the image size to be larger, we need to report
a larger buffer size - and in fact, that's already done.
QueryImageAttributes indicates that the image size is to be rounded
up to a page size.  The X server will reject smaller buffers at higher
levels.  So, this patch isn't required.

The last patch is your final one, I'll look at that at some point,
but for now I'm pushing out the tree as it now stands.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Dienstag, den 15.09.2015, 18:04 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 15, 2015 at 12:53:54PM -0400, Lucas Stach wrote:
> > Hi Russell,
> > 
> > Am Dienstag, den 15.09.2015, 17:36 +0100 schrieb Russell King - ARM
> > Linux:
> > > On Tue, Sep 15, 2015 at 05:53:45PM +0200, Lucas Stach wrote:
> > > > It includes a new kernel<->userspace interface, so needs a
> > > > modified
> > > > xf86-video-armada to go with it. You can pick that up here:
> > > > git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk
> > > > 
> > > > Caution: the armada code is WIP and needs further cleanups. It
> > > > should
> > > > work though.
> > > 
> > > And it needs updating to the latest version - you seem to be
> > > missing
> > > a whole load of changes, including the changes which update the
> > > kernel API.
> > > 
> > > So... you seem to be talking about the kernel API having changed
> > > from
> > > the version which you picked up years ago, rather than the
> > > current
> > > version.  It's really unclear what you're talking about.
> > > 
> > What do you mean with missing a lot of stuff? The "for-rmk" branch
> > in
> > that repo is your upstream "unstable-devel" branch minus the few
> > XvBO
> > commits with my changes on top.
> 
> Ah, probably because I haven't pushed the stuff out :)
> 
> Anyway, I've taken two of your patches, reworking the second
> slightly:
> 
> 1fec728 etnaviv: fix compilation without DRI2 support
> 1663382 make sure that system strndup doesn't collide with X.Org
> server version
> 
> which are _both_ fixes, and would have been nice for them to have
> been
> sent in a timely manner.
> 
Sorry about that. I'll try to single out the fixes more quickly going
forward.

> Looking at "etnaviv: fix XV resize for UYVY source format", this is
> wrong
> to the documentation for the GC320.  Documentation which was around
> for
> the GC320 indicates that the _only_ YUV format it supports as a
> target
> is YUY2 with the vertical filter blit.  It's very specifically worded
> to
> indicate that this is the only YUV target format which is supported.
> What testing have you done to prove uyvy support, and on which GPUs?
> 
I've tested this on i.MX6Q hardware with some GStreamer pipelines using
the videotestsrc. Without this commit the GPU will write out only zerosin the first step, so the video image will show up as fully green after CSC.

I don't think I have that documentation you are talking about. Is this
doc under NDA?

Chapter "31.4.1.6 Filter BLT" of the i.MX6 RM seems to agree with you,
so this commit depends on unspecified behavior. But it fixes a real bug
where the GPU writes out only zeros to the intermediate buffer if the
source is UYVY, so videos show up as fully green after CSC. So this
needs more investigation.

> "etnaviv: align vximage buffer size to pagesize" I have issues with
> as
> well - mapping more memory than was passed to the X server is really
> not
> permissible.  If we need the image size to be larger, we need to
> report
> a larger buffer size - and in fact, that's already done.
> QueryImageAttributes indicates that the image size is to be rounded
> up to a page size.  The X server will reject smaller buffers at
> higher
> levels.  So, this patch isn't required.
> 
This does not seem to be be working. I've certainly seen GStreamer
pushing non-aligned buffers which cause the GEM import to fail.
Ignoring the real size may well be a GStreamer bug (I'll look into that
code), but the server doesn't seem to reject that.

> The last patch is your final one, I'll look at that at some point,
> but for now I'm pushing out the tree as it now stands.
> 
If you're not going to look at this too soonish I can rebase that on
top of what you just pushed out and mybe clean it up some more. But
certainly not this week, as I'm physically away from any test hardware.

Regards,
Lucas

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Lucas Stach <l.stach@pengutronix.de> writes:

>> 	IMX DRM Xvideo output:
>> 
>> Only unscaled video: no color (luminance is good but the color
>> components are green). The driver doesn't use color information.
>> I hope this is easily fixable.
>> 
> There were patches for that some weeks ago already, I would hope they
> got applied to some tree destinied for upstream. Will look at that and
> ping people if necessary.

BTW: any pointer to the patches?
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Lucas Stach]

Am Mittwoch, den 16.09.2015, 09:57 +0200 schrieb Krzysztof Ha?asa:
> Lucas Stach <l.stach@pengutronix.de> writes:
> 
> > > 	IMX DRM Xvideo output:
> > > 
> > > Only unscaled video: no color (luminance is good but the color
> > > components are green). The driver doesn't use color information.
> > > I hope this is easily fixable.
> > > 
> > There were patches for that some weeks ago already, I would hope
> > they
> > got applied to some tree destinied for upstream. Will look at that
> > and
> > ping people if necessary.
> 
> BTW: any pointer to the patches?

I just digged a bit and it seems they didn't make it out. I've pinged
Philipp about that.

Regards,
Lucas

I.MX6 HDMI support in v4.2

[Philipp Zabel]

Am Donnerstag, den 10.09.2015, 11:49 +0100 schrieb Russell King - ARM
Linux:
> On Thu, Sep 10, 2015 at 12:25:32PM +0200, Krzysztof Ha?asa wrote:
> > The Y plane is ok, but the Xv window is green, like on old monochrome
> > CRTs - what could that be? It looks like the UV planes are all wrong.
> 
> From a quick read of the imx-drm ipuv3-plane code, it looks like it
> doesn't bother with the U/V planes for planar images.  Nothing looks
> at fb->offsets[1,2] nor fb->pitches[1,2].  However, it claims to
> support DRM_FORMAT_YUV420 and DRM_FORMAT_YVU420 formats - which is
> wrong if it doesn't look at these other offsets and pitches.

The IDMAC has a few funny restrictions for multiplanar formats, and the
current code silently assumes that the U and V planes follow right after
Y.

I have a patch to enforce the same base address:
http://lists.freedesktop.org/archives/dri-devel/2015-August/089293.html
but it still doesn't check the offset/pitch limitations properly.

best regards
Philipp

I.MX6 HDMI support in v4.2

[Krzysztof Hałasa]

Philipp Zabel <p.zabel@pengutronix.de> writes:

> The IDMAC has a few funny restrictions for multiplanar formats, and the
> current code silently assumes that the U and V planes follow right after
> Y.
>
> I have a patch to enforce the same base address:
> http://lists.freedesktop.org/archives/dri-devel/2015-August/089293.html
> but it still doesn't check the offset/pitch limitations properly.

Do you mean the current code as in e.g. v4.2?
I'm passing it (the i.MX6 IMX DRM overlay) a 1024x768 buffer:

ipu_plane_mode_set pitches 1024 512 512 0 offsets 0 786432 983040 0

so the U and V planes closely follow Y, and it doesn't work. The screen
is green to magenta, sometimes there are diagonal stripes (color only,
Y is fine).

Also, e.g. mplayer shows a greenish output.

I've posted a short test program:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-September/369988.html

You will need to set xv_port = 0 (it will then use the first
YUV420-capable adapter, which on my system is IMX DRM overlay) and you
may want to stick "XSync(display, 0);" after the XvShmPutImage() call.

Also, you may change the "memset(image->data, count, image->data_size)"
into something smaller, e.g. to a subset of Y plane, leaving UV alone.
For example:
	memset(image->data, count % 0x100, 12345);
	memset(image->data + image->pitches[0] * h, 0, (image->pitches[1] + image->pitches[2]) * h / 2);

will set the first 12345 bytes of Y plane, while the U and V will be
zero.

Thanks for looking into this.
-- 
Krzysztof Halasa

Industrial Research Institute for Automation and Measurements PIAP
Al. Jerozolimskie 202, 02-486 Warsaw, Poland

I.MX6 HDMI support in v4.2

[Russell King - ARM Linux]

On Thu, Sep 17, 2015 at 09:21:56AM +0200, Philipp Zabel wrote:
> Am Donnerstag, den 10.09.2015, 11:49 +0100 schrieb Russell King - ARM
> Linux:
> > On Thu, Sep 10, 2015 at 12:25:32PM +0200, Krzysztof Ha?asa wrote:
> > > The Y plane is ok, but the Xv window is green, like on old monochrome
> > > CRTs - what could that be? It looks like the UV planes are all wrong.
> > 
> > From a quick read of the imx-drm ipuv3-plane code, it looks like it
> > doesn't bother with the U/V planes for planar images.  Nothing looks
> > at fb->offsets[1,2] nor fb->pitches[1,2].  However, it claims to
> > support DRM_FORMAT_YUV420 and DRM_FORMAT_YVU420 formats - which is
> > wrong if it doesn't look at these other offsets and pitches.
> 
> The IDMAC has a few funny restrictions for multiplanar formats, and the
> current code silently assumes that the U and V planes follow right after
> Y.
> 
> I have a patch to enforce the same base address:
> http://lists.freedesktop.org/archives/dri-devel/2015-August/089293.html
> but it still doesn't check the offset/pitch limitations properly.

	int planes = drm_format_num_planes(fb->pixel_format);
	dma_addr_t base, end;

	for (i = 0; i < planes; i++) {
		cma_obj[i] = drm_fb_cma_get_gem_obj(fb, i);
		if (!cma_obj[i]) {
			DRM_DEBUG_KMS("plane %d entry is null.\n", i);
			return -EINVAL;
		}
	}

EFAULT is not appropriate there - EFAULT means "userspace passed a bad
address into kernel space".  That's not what you're trying to convey
there, so EINVAL or some other error code more suited to the error
condition would be more appropriate.

	base = cma_obj[0]->paddr + fb->offsets[0];
	end = base + fb->pitches[0] * fb->height;
	eba = base + fb->pitches[0] * y + (fb->bits_per_pixel >> 3) * x;

	for (i = 1; i < planes; i++) {
		dma_addr_t this_base;

		if (y || x) {
			DRM_DEBUG_KMS("can't support planar with offset\n");
			return -EINVAL;
		}

		this_base = cma_obj[i]->paddr + fb->offsets[i];
		if (this_base != end) {
			DRM_DEBUG_KMS("plane does follow previous.\n");
			return -EINVAL;
		}

		end += fb->pitches[i] * fb->height;
	}

should do the trick.  Note that if x or y is non-zero, it's specifying
an offset into the image, which means that there _will_ be a gap
between the planes (consisting of the undisplayed portion of the plane.)
As you said they need to be contiguous, they aren't contiguous in this
situation, so we have to error that out if there's more than one plane.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Lucas Stach]

Am Dienstag, den 15.09.2015, 18:04 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 15, 2015 at 12:53:54PM -0400, Lucas Stach wrote:
> > Hi Russell,
> > 
> > Am Dienstag, den 15.09.2015, 17:36 +0100 schrieb Russell King - ARM
> > Linux:
> > > On Tue, Sep 15, 2015 at 05:53:45PM +0200, Lucas Stach wrote:
> > > > It includes a new kernel<->userspace interface, so needs a modified
> > > > xf86-video-armada to go with it. You can pick that up here:
> > > > git://git.pengutronix.de/git/lst/xf86-video-armada.git for-rmk
> > > > 
> > > > Caution: the armada code is WIP and needs further cleanups. It
> > > > should
> > > > work though.
> > > 
> > > And it needs updating to the latest version - you seem to be missing
> > > a whole load of changes, including the changes which update the
> > > kernel API.
> > > 
> > > So... you seem to be talking about the kernel API having changed from
> > > the version which you picked up years ago, rather than the current
> > > version.  It's really unclear what you're talking about.
> > > 
> > What do you mean with missing a lot of stuff? The "for-rmk" branch in
> > that repo is your upstream "unstable-devel" branch minus the few XvBO
> > commits with my changes on top.
> 
> Ah, probably because I haven't pushed the stuff out :)
> 
> Anyway, I've taken two of your patches, reworking the second slightly:
> 
> 1fec728 etnaviv: fix compilation without DRI2 support
> 1663382 make sure that system strndup doesn't collide with X.Org server version
> 
> which are _both_ fixes, and would have been nice for them to have been
> sent in a timely manner.
> 
> Looking at "etnaviv: fix XV resize for UYVY source format", this is wrong
> to the documentation for the GC320.  Documentation which was around for
> the GC320 indicates that the _only_ YUV format it supports as a target
> is YUY2 with the vertical filter blit.  It's very specifically worded to
> indicate that this is the only YUV target format which is supported.
> What testing have you done to prove uyvy support, and on which GPUs?
> 
> "etnaviv: align vximage buffer size to pagesize" I have issues with as
> well - mapping more memory than was passed to the X server is really not
> permissible.  If we need the image size to be larger, we need to report
> a larger buffer size - and in fact, that's already done.
> QueryImageAttributes indicates that the image size is to be rounded
> up to a page size.  The X server will reject smaller buffers at higher
> levels.  So, this patch isn't required.
> 
I've looked at that again and the issue here seems to be that GStreamer
is handing us a pointer to a buffer that isn't aligned to a page. The
buffers size is properly rounded up to a pagesize, as requested by
QueryImageAttributes, but if the buffer start pointer isn't aligned to a
page boundary we end up with an non-mappable buffer anyway.

Unfortunately there is no obvious way for a driver to request a minimum
alignment for the buffer. The only possible fix is for the client to
always align the buffer to a page boundary in hopes that this is enough
for the hardware to map it directly and allow to skip any unwanted
copying.

Regards,
Lucas

-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Russell King - ARM Linux]

On Mon, Sep 28, 2015 at 04:48:08PM +0200, Lucas Stach wrote:
> I've looked at that again and the issue here seems to be that GStreamer
> is handing us a pointer to a buffer that isn't aligned to a page. The
> buffers size is properly rounded up to a pagesize, as requested by
> QueryImageAttributes, but if the buffer start pointer isn't aligned to a
> page boundary we end up with an non-mappable buffer anyway.
> 
> Unfortunately there is no obvious way for a driver to request a minimum
> alignment for the buffer. The only possible fix is for the client to
> always align the buffer to a page boundary in hopes that this is enough
> for the hardware to map it directly and allow to skip any unwanted
> copying.

We can't just "round up" the size.  We've no idea whether the buffer
came from shmem (for XvShmPutImage), or whether it's part of an internal
X buffer (for XvPutImage).  In the latter case, we've no idea whether
data in the remainder of the page will be read or written by the CPU
when, eg, a signal occurs - and X does use signals.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Lucas Stach]

Am Montag, den 28.09.2015, 16:24 +0100 schrieb Russell King - ARM Linux:
> On Mon, Sep 28, 2015 at 04:48:08PM +0200, Lucas Stach wrote:
> > I've looked at that again and the issue here seems to be that GStreamer
> > is handing us a pointer to a buffer that isn't aligned to a page. The
> > buffers size is properly rounded up to a pagesize, as requested by
> > QueryImageAttributes, but if the buffer start pointer isn't aligned to a
> > page boundary we end up with an non-mappable buffer anyway.
> > 
> > Unfortunately there is no obvious way for a driver to request a minimum
> > alignment for the buffer. The only possible fix is for the client to
> > always align the buffer to a page boundary in hopes that this is enough
> > for the hardware to map it directly and allow to skip any unwanted
> > copying.
> 
> We can't just "round up" the size.  We've no idea whether the buffer
> came from shmem (for XvShmPutImage), or whether it's part of an internal
> X buffer (for XvPutImage).  In the latter case, we've no idea whether
> data in the remainder of the page will be read or written by the CPU
> when, eg, a signal occurs - and X does use signals.
> 
I'm not talking about the driver rounding up the size. That is obviously
(contrary to what my patch did) the wrong thing to do.

I was talking of the client (as in VLC, GStreamer, whatever) aligning
the buffer to a page boundary. As there is no way for the client to
query the alignment restrictions, we may still need a fallback path in
the driver, so that if an unaligned buffer comes in we don't do a
buffer_from_userptr but actually copy client memory to a new buffer.

Regards,
Lucas
-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Russell King - ARM Linux]

On Mon, Sep 28, 2015 at 05:40:13PM +0200, Lucas Stach wrote:
> Am Montag, den 28.09.2015, 16:24 +0100 schrieb Russell King - ARM Linux:
> > On Mon, Sep 28, 2015 at 04:48:08PM +0200, Lucas Stach wrote:
> > > I've looked at that again and the issue here seems to be that GStreamer
> > > is handing us a pointer to a buffer that isn't aligned to a page. The
> > > buffers size is properly rounded up to a pagesize, as requested by
> > > QueryImageAttributes, but if the buffer start pointer isn't aligned to a
> > > page boundary we end up with an non-mappable buffer anyway.
> > > 
> > > Unfortunately there is no obvious way for a driver to request a minimum
> > > alignment for the buffer. The only possible fix is for the client to
> > > always align the buffer to a page boundary in hopes that this is enough
> > > for the hardware to map it directly and allow to skip any unwanted
> > > copying.
> > 
> > We can't just "round up" the size.  We've no idea whether the buffer
> > came from shmem (for XvShmPutImage), or whether it's part of an internal
> > X buffer (for XvPutImage).  In the latter case, we've no idea whether
> > data in the remainder of the page will be read or written by the CPU
> > when, eg, a signal occurs - and X does use signals.
> > 
> I'm not talking about the driver rounding up the size. That is obviously
> (contrary to what my patch did) the wrong thing to do.
> 
> I was talking of the client (as in VLC, GStreamer, whatever) aligning
> the buffer to a page boundary. As there is no way for the client to
> query the alignment restrictions, we may still need a fallback path in
> the driver, so that if an unaligned buffer comes in we don't do a
> buffer_from_userptr but actually copy client memory to a new buffer.

You really do _not_ want to do be copying image data.  With large images
(1080p) that will consume lots of CPU, and tie up the X server doing not
much other than copying data.  You might as well manually convert and
copy the data to the screen at that point.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Lucas Stach]

Am Montag, den 28.09.2015, 17:50 +0100 schrieb Russell King - ARM Linux:
> On Mon, Sep 28, 2015 at 05:40:13PM +0200, Lucas Stach wrote:
> > Am Montag, den 28.09.2015, 16:24 +0100 schrieb Russell King - ARM Linux:
> > > On Mon, Sep 28, 2015 at 04:48:08PM +0200, Lucas Stach wrote:
> > > > I've looked at that again and the issue here seems to be that GStreamer
> > > > is handing us a pointer to a buffer that isn't aligned to a page. The
> > > > buffers size is properly rounded up to a pagesize, as requested by
> > > > QueryImageAttributes, but if the buffer start pointer isn't aligned to a
> > > > page boundary we end up with an non-mappable buffer anyway.
> > > > 
> > > > Unfortunately there is no obvious way for a driver to request a minimum
> > > > alignment for the buffer. The only possible fix is for the client to
> > > > always align the buffer to a page boundary in hopes that this is enough
> > > > for the hardware to map it directly and allow to skip any unwanted
> > > > copying.
> > > 
> > > We can't just "round up" the size.  We've no idea whether the buffer
> > > came from shmem (for XvShmPutImage), or whether it's part of an internal
> > > X buffer (for XvPutImage).  In the latter case, we've no idea whether
> > > data in the remainder of the page will be read or written by the CPU
> > > when, eg, a signal occurs - and X does use signals.
> > > 
> > I'm not talking about the driver rounding up the size. That is obviously
> > (contrary to what my patch did) the wrong thing to do.
> > 
> > I was talking of the client (as in VLC, GStreamer, whatever) aligning
> > the buffer to a page boundary. As there is no way for the client to
> > query the alignment restrictions, we may still need a fallback path in
> > the driver, so that if an unaligned buffer comes in we don't do a
> > buffer_from_userptr but actually copy client memory to a new buffer.
> 
> You really do _not_ want to do be copying image data.  With large images
> (1080p) that will consume lots of CPU, and tie up the X server doing not
> much other than copying data.  You might as well manually convert and
> copy the data to the screen at that point.
> 
So what other possibilities do we have if the client hands us a non page
aligned buffer, other than failing the PutImage? Converting the image
manually and possibly doubling the amount of bytes to write out
(YUV->RGB) will tie up the X server even longer.

This all doesn't seem to be a problem as long as the client allocates
from XShm, as this seems to hand out page aligned memory. Only if the
client mallocs the image buffer we might end up with unaligned buffers.

Regards,
Lucas
-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Russell King - ARM Linux]

On Tue, Sep 29, 2015 at 10:28:19AM +0200, Lucas Stach wrote:
> Am Montag, den 28.09.2015, 17:50 +0100 schrieb Russell King - ARM Linux:
> > You really do _not_ want to do be copying image data.  With large images
> > (1080p) that will consume lots of CPU, and tie up the X server doing not
> > much other than copying data.  You might as well manually convert and
> > copy the data to the screen at that point.
> > 
> So what other possibilities do we have if the client hands us a non page
> aligned buffer, other than failing the PutImage? Converting the image
> manually and possibly doubling the amount of bytes to write out
> (YUV->RGB) will tie up the X server even longer.

Are you sure about that?

If you memcpy() the buffer, you have to read about 6MB of data, write
6MB of data, flush the caches of that, then ask the GPU to perform two
blits (which on my measurements just about fit in one field scan), wait
for the blit to finish before then returning to the client.

I can tell you that VLC uses memcpy() on images internally, and you can't
get away with it for 1080p video.

> This all doesn't seem to be a problem as long as the client allocates
> from XShm, as this seems to hand out page aligned memory. Only if the
> client mallocs the image buffer we might end up with unaligned buffers.

I don't see that.  How does a malloc()'d buffer get to the X server,
other than throuh XvPutImage?  If you're using XvPutImage(), you'll
be incurring an even _larger_ overhead because the kernel will have
to copy the image data as well.  AFAIK, you can't pass arbitary malloc()d
buffers to sysvipc and therefore via XShm.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

xf86-video-armada + etnaviv (Was: Re: I.MX6 HDMI support in v4.2)

[Lucas Stach]

Am Dienstag, den 29.09.2015, 09:41 +0100 schrieb Russell King - ARM
Linux:
> On Tue, Sep 29, 2015 at 10:28:19AM +0200, Lucas Stach wrote:
> > Am Montag, den 28.09.2015, 17:50 +0100 schrieb Russell King - ARM Linux:
> > > You really do _not_ want to do be copying image data.  With large images
> > > (1080p) that will consume lots of CPU, and tie up the X server doing not
> > > much other than copying data.  You might as well manually convert and
> > > copy the data to the screen at that point.
> > > 
> > So what other possibilities do we have if the client hands us a non page
> > aligned buffer, other than failing the PutImage? Converting the image
> > manually and possibly doubling the amount of bytes to write out
> > (YUV->RGB) will tie up the X server even longer.
> 
> Are you sure about that?
> 
> If you memcpy() the buffer, you have to read about 6MB of data, write
> 6MB of data, flush the caches of that, then ask the GPU to perform two
> blits (which on my measurements just about fit in one field scan), wait
> for the blit to finish before then returning to the client.
> 
The cache flush can easily be avoided if you copy into a WC buffer.

And I don't see why we would need to wait for the blit to finish before
returning to the client. If we copy the userdata there is no need to
block the client until the GPU is done with the data.

Or is there something in the spec I'm not aware of which would mandate
such blocking?

> I can tell you that VLC uses memcpy() on images internally, and you can't
> get away with it for 1080p video.
> 
I'm not disputing that it's a really bad idea if you care about
performance.

> > This all doesn't seem to be a problem as long as the client allocates
> > from XShm, as this seems to hand out page aligned memory. Only if the
> > client mallocs the image buffer we might end up with unaligned buffers.
> 
> I don't see that.  How does a malloc()'d buffer get to the X server,
> other than throuh XvPutImage?  If you're using XvPutImage(), you'll
> be incurring an even _larger_ overhead because the kernel will have
> to copy the image data as well.  AFAIK, you can't pass arbitary malloc()d
> buffers to sysvipc and therefore via XShm.
> 
Right I'm talking about XvPutImage. I'm not saying it is a good way to
hand off your image data to the X server, all I'm saying is that this is
a path that applications _might_ use.

We can just choose to ignore this for the time being, as it's a
non-issue if the client is using XShm.

Regards,
Lucas
-- 
Pengutronix e.K.             | Lucas Stach                 |
Industrial Linux Solutions   | http://www.pengutronix.de/  |