Re: Hi again

Re: Hi again

[Sylwester Nawrocki]

Hi Alexander,

Thanks for the update ;)

On 02/05/2013 11:50 PM, Alexander Nestorov wrote:
> Hi Sylwester :)
>
>
> I have been working wth 2 guys from Collabora (company behind
> GStreamer/Farstream) and there has been
> some progress.
>
> We saw that if I run gst-launch v4l2src ! dfbvideosink (without any
> caps) then gstreamer won't show anything at all.
> But as soon as I force caps, like v4l2src !
> video/x-raw-rgb,width=XXX,height=YYY,bpp=16 ! dfbvideosink everything
> starts working.
>
> Same way, if I launch my Farstream app without any parameters, it
> crashes. But if I specify bpp and depth then
> it starts working.
>
> I don't want to confuse you so I'll copy exactly what one of the guys said:
>
> [23:02]<alexandernst_>  KaKaRoTo: I have another question regarding
> the CMOS and the caps "hack"
> [23:03]<alexandernst_>  we're setting bpp and depth because the driver
> isn't returning: a) anything at all  b) invalid values  c) ???
> [23:06]<KaKaRoTo>  alexandernst_, because the driver is reporting big
> endian for 16 bits, and it could easily be a gstreamer issue not
> supporting big endian 16bits rgb.. or the driver is reporting the
> endianness wrong...
> [23:06]<KaKaRoTo>  alexandernst_, the other reason is because the
> driver isn't giving us the capabilities of the CMOS, so we don't knowh

Yes, the driver doesn't implement VIDIOC_ENUM_FRAMESIZES ioctl at
/dev/video? device node. These parameters are supposed to be retrieved
by a driver-specific plugin at the libv4l [1] (it still don't exists
yet..) library. And applications (e.g. v4l2src) should use v4l2_* calls
from this library, rather than plain ioctl(). Until the plugin for the
s3c-camif driver (or a generic Media Controller plugin) is added to
libv4l there is no way v4l2src can query the list of supported resolutions
by the CMOS sensor. Well, it could be hacked in the driver, but I don't
want to do that. I could send you a patch for your own use if you wish,
and if I find some time to write it. ;)

> which resolutions it supports, so we're forced to manually setting it,
> otherwise it tries to do 2048x4096 and it doesn't work..

This looks suspicious, let me check it...

Hmm, no, its GStreamer that forces the s3c-camif driver to use its maximum
supported resolution at /dev/video?.

root@mini2440:~ v4l2-ctl -d /dev/video0 --all
Driver Info (not using libv4l2):
         Driver name   : s3c-camif
         Card type     : s3c-camif
         Bus info      : platform:s3c2440-camif.0
         Driver version: 3.8.0
         Capabilities  : 0x84000001
                 Video Capture
                 Streaming
                 Device Capabilities
         Device Caps   : 0x04000001
                 Video Capture
                 Streaming
Format Video Capture:
         Width/Height  : 1280/1024
         Pixel Format  : '422P'
         Field         : None
         Bytes per Line: 1280
         Size Image    : 614400
         Colorspace    : JPEG (JFIF/ITU601)
Selection: compose, Left 0, Top 0, Width 1280, Height 1024
Selection: compose_default, Left 0, Top 0, Width 1280, Height 1024
Selection: compose_bounds, Left 0, Top 0, Width 1280, Height 1024
Video input : 0 (OV9650: ok)
Priority: 2

The default resolution is 1280/1024.

The problem is that there is not enough memory in your system to allocate
all 2048x4096 buffers (it's 16 MB each, 2 bytes per pixel for 422P format).
The driver doesn't enforce any policy on memory allocation, i.e. there is
no limit in the driver. I could add some module parameter for that if
required. But in general it's the v4l2 memory allocator that should return
an allocation error and it should be propagated up to v4l2 videobuf2 layer
and to GStreamer (e.g. through VIDIOC_REQBUFS ioctl).

IIRC there were some issues in the videobuf2 about reporting memory
allocation errors back to user space. Not sure if this is already fixed.
I'll investigate it and will let you know.

I have enabled debug in videobuf2 and got something like below, which
looks like something unexpected:

root@mini2440:~ echo 100 > /sys/module/videobuf2_core/parameters/debug

root@mini2440:~ gst-launch -v v4l2src device=/dev/video0 queue-size=2
  ! video/x-raw-yuv,format='(fourcc)'YV12 ! ffmpegcolorspace ! fbdevsink

root@mini2440:~ dmesg -c
[ 1907.110000] s3c-camif s3c2440-camif: dma_alloc_coherent of size 
12582912 failed
[ 1907.115000] vb2: Failed allocating memory for buffer 1

The error is ignored here, it should all fail at this point.

[ 1907.120000] vb2: Buffer 0, plane 0 offset 0x00000000
[ 1907.125000] vb2: Allocated 1 buffers, 1 plane(s) each
[ 1907.220000] vb2: Buffer 0, plane 0 successfully mapped
[ 1907.225000] vb2: qbuf of buffer 0 succeeded
[ 1907.230000] vb2: Streamon successful


> [23:07]<KaKaRoTo>  which is also why it freezes (doesn't output
> buffers) instead of giving an error when you try to set that
> resolution
>
> I made a bug report here, and there are 2 REs which might be really
> helpful: https://bugzilla.gnome.org/show_bug.cgi?id=693175
>
>
> On the other hand, I haven't noticed till now because I compile with
> -j9, but I get this when I compile the kernel:
>
> CC      drivers/media/platform/s3c-camif/camif-capture.o
> drivers/media/platform/s3c-camif/camif-capture.c: In function
> 's3c_camif_subdev_set_fmt':
> drivers/media/platform/s3c-camif/camif-capture.c:1281:25: warning:
> array subscript is below array bounds
>
> Maybe that is somehow related to the bug I'm having?

This looks like a real bug, but I doubt it is much related to your
issues as above. Thanks for reporting it. I will prepare a patch for
this for v3.9-rc.

And please note that you should set resolution and frame rate
with e.g. media-ctl tool for optimal results. The libv4l plugin
would let us to do these things from the GStreamer level.

Let me answer your previous e-mail, but can do it only tomorrow...
Need more sleeeeping... ;)

--

Regards,
Sylwester

[1] http://git.linuxtv.org/v4l-utils.git

Re: Hi again

[Sylwester Nawrocki]

On 02/07/2013 12:53 AM, Sylwester Nawrocki wrote:
> root@mini2440:~ echo 100 > /sys/module/videobuf2_core/parameters/debug
>
> root@mini2440:~ gst-launch -v v4l2src device=/dev/video0 queue-size=2
> ! video/x-raw-yuv,format='(fourcc)'YV12 ! ffmpegcolorspace ! fbdevsink
>
> root@mini2440:~ dmesg -c
> [ 1907.110000] s3c-camif s3c2440-camif: dma_alloc_coherent of size
> 12582912 failed
> [ 1907.115000] vb2: Failed allocating memory for buffer 1
>
> The error is ignored here, it should all fail at this point.
>
> [ 1907.120000] vb2: Buffer 0, plane 0 offset 0x00000000
> [ 1907.125000] vb2: Allocated 1 buffers, 1 plane(s) each
> [ 1907.220000] vb2: Buffer 0, plane 0 successfully mapped
> [ 1907.225000] vb2: qbuf of buffer 0 succeeded
> [ 1907.230000] vb2: Streamon successful

I've found there is a bug in the driver. It happens that only one buffer
out of two gets allocated and the minimum required for the driver to start
streaming is 2. I've pushed a patch fixing this bug [1] to the 3.7 based
testing/s3c-camif branch. I'll post it to LMML in a minute.

It won't really solve your problem, but at least there is now clear
indication what's wrong, and the application should not get stuck. Please
check you CMA memory region size. I suspect that you use the default 16 MB
and it allows you to allocate only one buffer. 16 MB might still be
sufficient for 1280x1024 - OV9650's maximum resolution. Anyway, it might
be worth to check what CONFIG_CMA_SIZE_MBYTES is set to.
(Device drivers -> Generic Driver Options -> Contiguous Memory Allocator)

>> [23:07]<KaKaRoTo> which is also why it freezes (doesn't output
>> buffers) instead of giving an error when you try to set that
>> resolution

Thanks,
Sylwester

Re: [oselas] Audio support on Mini6410 board

[Sylwester Nawrocki]

Cc: LMML

On 02/09/2013 07:21 PM, Sylwester Nawrocki wrote:
> Hi,
>
> On 01/20/2013 09:46 PM, Alexander Nestorov wrote:
>> I have been playing for a week with the board. Both audio and video
>> work correctly, but I haven't
>> been able to set the mic settings in alsamixer (so I can't test the mic).
>> The touchscreen isn't working, so I'll try to make it working and send
>> you some patches.
>>
>> Anyways, now there's another question/problem that I have. Video
>> playback is really slow because
>> I'm not using the device's cpu-decoder but rather doing everything in
>> software mode.
>>
>> Is there support for hardware acceleration in the kernel for this
>> device? Also, after talking with
>
> No, there is still no video codec (MFC) driver for s3c6410 upstream.
> Now, when there is support for the hardware video codec available in
> newer SoC (Exynos4/5) and some V4L2 infrastructure added together with
> the s5p-mfc driver, it should be much easier to write a driver for the
> s3c64xx MFC. Still it is relatively huge task and I didn't see any
> volunteers willing to add support upstream for the s3c64xx MFC, except
> Andrey who replied in this thread. I could provide some help, but
> I will likely won't find time to do any development work or testing.
>
> Also please note there is no support for the mem-to-mem features (color
> space conversion, scaling, rotation/flip) in the s3c-camif driver.
> It should be relatively simple to add it though. I'm not really sure
> if it is needed to run the codec on s3c64xx, but the plugin [1] uses
> FIMC (CAMIF) as a video post-processor. This plugin sets up processing
> pipeline like:
>
> memory (compressed data) -> MFC -> (YCbCr tiled) memory -> FIMC ->
> memory (display)
>
>> some people from #gstreamer they pointed me to a component[1] in
>> gstreamer, but I'm not really
>> sure how to I use it. Any ideas/experience with that?
>
> This component uses multi-planar V4L2 API [2], which also use the s5p-mfc
> and s5p-fimc driver. The s3c-camif driver uses the single-planar API
> at the camera capture video node. So if this existing plugin was to be
> used with the s3c64xx hardware, the drivers for it would have to
> support the multi-planar API, which I believe is not needed on s3c64xx
> hardware.
> The best is probably to make the drivers only single-plane API aware
> and adapt the plugin. The required changes at the plugin wouldn't be
> significant.
>
> Anyway, a real problem here is lack of the s3c64xx MFC driver. So
> first we need the codec driver, which could be tested with modified
> test application [3], or directly with modified plugin [1].
>
>> Regards!
>>
>> [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/mfc
>
> [2] http://linuxtv.org/downloads/v4l-dvb-apis/planar-apis.html
> [3]
> http://git.infradead.org/users/kmpark/public-apps/tree/9c057b001e8873861a70f7025214003837a0860b
>
>
> --
>
> Regards,
> Sylwester

Re: [oselas] Audio support on Mini6410 board

[Tomasz Figa]

Hi,

On Saturday 09 of February 2013 19:21:32 Sylwester Nawrocki wrote:
> Hi,
> 
> On 01/20/2013 09:46 PM, Alexander Nestorov wrote:
> > I have been playing for a week with the board. Both audio and video
> > work correctly, but I haven't
> > been able to set the mic settings in alsamixer (so I can't test the
> > mic). The touchscreen isn't working, so I'll try to make it working
> > and send you some patches.
> > 
> > Anyways, now there's another question/problem that I have. Video
> > playback is really slow because
> > I'm not using the device's cpu-decoder but rather doing everything in
> > software mode.
> > 
> > Is there support for hardware acceleration in the kernel for this
> > device? Also, after talking with
> 
> No, there is still no video codec (MFC) driver for s3c6410 upstream.
> Now, when there is support for the hardware video codec available in
> newer SoC (Exynos4/5) and some V4L2 infrastructure added together with
> the s5p-mfc driver, it should be much easier to write a driver for the
> s3c64xx MFC. Still it is relatively huge task and I didn't see any
> volunteers willing to add support upstream for the s3c64xx MFC, except
> Andrey who replied in this thread. I could provide some help, but
> I will likely won't find time to do any development work or testing.
> 
> Also please note there is no support for the mem-to-mem features (color
> space conversion, scaling, rotation/flip) in the s3c-camif driver.
> It should be relatively simple to add it though. I'm not really sure
> if it is needed to run the codec on s3c64xx, but the plugin [1] uses
> FIMC (CAMIF) as a video post-processor. This plugin sets up processing
> pipeline like:
> 
> memory (compressed data) -> MFC -> (YCbCr tiled) memory -> FIMC ->
> memory (display)

AFAIK the MFC (like rest of the media processing peripherals) on S3C6410 
does not support tiled buffers. It uses the standard planar Y + Cb + Cr 
format.

In addition, the MFC of S3C6410 supports built-in rotation and mirroring 
of decoded video.

For scaling, there is a video post-processor block. There is no upstreamed 
driver for it, but the hardware is reasonably simple, so it wouldn't be 
too hard to write a driver for it. (I might be able to do it, although 
don't count on me, as I have also much other work to do, part of which is 
also related to S3C64xx).

Best regards,
Tomasz

> > some people from #gstreamer they pointed me to a component[1] in
> > gstreamer, but I'm not really
> > sure how to I use it. Any ideas/experience with that?
> 
> This component uses multi-planar V4L2 API [2], which also use the
> s5p-mfc and s5p-fimc driver. The s3c-camif driver uses the
> single-planar API at the camera capture video node. So if this existing
> plugin was to be used with the s3c64xx hardware, the drivers for it
> would have to support the multi-planar API, which I believe is not
> needed on s3c64xx hardware.
> The best is probably to make the drivers only single-plane API aware
> and adapt the plugin. The required changes at the plugin wouldn't be
> significant.
> 
> Anyway, a real problem here is lack of the s3c64xx MFC driver. So
> first we need the codec driver, which could be tested with modified
> test application [3], or directly with modified plugin [1].
> 
> > Regards!
> > 
> > [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/mfc
> 
> [2] http://linuxtv.org/downloads/v4l-dvb-apis/planar-apis.html
> [3]
> http://git.infradead.org/users/kmpark/public-apps/tree/9c057b001e8873861
> a70f7025214003837a0860b
> 
> --
> 
> Regards,
> Sylwester
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-samsung-soc" in the body of a message to
> majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html