Re: [PATCH 00/15] Intel IPU6 and IPU6 input system drivers

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

Hi Claus,

On Mon, Aug 21, 2023 at 12:07:59PM +0200, Claus Stovgaard wrote:
> On Mon, 2023-08-21 at 08:55 +0200, Claus Stovgaard wrote:
> > On Mon, 2023-08-21 at 14:22 +0800, Bingbu Cao wrote:
> > > 
> > > Claus,
> > > 
> > > 
> > > On 8/21/23 11:14 AM, Bingbu Cao wrote:
> > > 
> > > 
> > > I see that the ivsc driver has not been in master branch. Before
> > > that,
> > > could you try several hack to check whether camera can work on
> > > master?
> > > 
> > > https://github.com/bingbucao/linux/commits/ipu_dev
> > > 
> > > 7ebff51284d9 media: ov01a10: hack ivsc to make camera can work
> > > 01cc9f3d1b61 i2c: ljca: Call acpi_dev_clear_dependencies()
> > > 92e5d122e105 vsc: Defer firmware loading to avoid long probing time
> > > 5f5d5f0df06b driver: ivsc: add intel ivsc driver
> > > 0f4819dec533 Revert "gpio: Add support for Intel LJCA USB GPIO
> > > driver"
> > 
> > Thanks for your quick reply.
> > 
> > I was missing understanding of ivsc when I wrote the mail yesterday.
> > Got some basic understanding yesterday after I wrote, and big thanks
> > for confirming it, and also thanks for your ipu_dev branch. Has just
> > cloned it, and is building as I write.
> > 
> > Just fyi, I was trying to hack something together yesterday, and got
> > further, but not yet working.
> > 
> > My hack was to combine the out-of-tree ivsc drivers and firmware from
> > 
> > * https://github.com/intel/ivsc-firmware.git
> > * https://github.com/intel/ivsc-driver.git
> > 
> > Though noticed that I need some changes to the sensor driver so was
> > also building all the drivers from ipu6-drivers (with minor changes
> > to
> > get_pages) as out-of-tree modules.
> > 
> > * https://github.com/intel/ipu6-drivers.git 
> > 
> > Here I used everything beside media/pci/*.ko files. I could see the
> > sensor and got further, but was missing the last.
> > 
> > Looking forward to try your branch. Looks much cleaner, and would be
> > nice to get working :)
> 
> I got it to work on Dell XPS 9320.

I'm glad to hear this ! Even if PSYS support will be needed to make the
IPU6 truly usable, it is a very nice step in the right direction.

Would you be interested in adding initial support for the IPU6 in
libcamera ? :-) Given that only the ISYS is currently available, and
given the simplicity of the hardware, it may be as easy as a single line
addition.

> With some minor changes compared to your guide in Documentation/admin-
> guide/media/ipu6-isys.rst
> 
> [root@xps-1 ]# uname -a
> Linux xps-1 6.5.0-rc7-g7ebff51284d9 #1 SMP PREEMPT_DYNAMIC Mon Aug 21
> 09:02:20 CEST 2023 x86_64 GNU/Linux
> 
> [root@xps-1 ]# media-ctl -d /dev/media0 -p | tail -n10
> 
> - entity 2149: ov01a10 16-0036 (1 pad, 1 link)
>                type V4L2 subdev subtype Sensor flags 0
>                device node name /dev/v4l-subdev4
>         pad0: Source
>                 [fmt:SBGGR10_1X10/1280x800 field:none colorspace:raw
>                  crop.bounds:(0,0)/1296x816
>                  crop:(8,8)/1280x800]
>                 -> "Intel IPU6 CSI2 2":0 []
> 
> So i2c is 16-0036 - and we use it for setup like your guide.
> 
> export MDEV=/dev/media0
> 
> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
> 2\":0[1]"
> 
> media-ctl -d $MDEV -V "\"ov01a10 17-0036\":0 [fmt:SBGGR10/1280x800]"
> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":0 [fmt:SBGGR10/1280x800]"
> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":1 [fmt:SBGGR10/1280x800]"
> 
> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
> 2\":0[1]"
> media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS
> Capture 0\":0[5]"
> 
> Though yavta does not work in the way as described in the guide.
> 
> [root@xps-1 ]# yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --
> file=/tmp/frame-#.bin -f SBGGR10 /dev/video0
> Device /dev/video0 opened.
> Device `ipu6' on `PCI:0000:00:05.0' (driver 'isys') supports video,
> capture, with mplanes.
> Video format set: SBGGR10 (30314742) 1280x800 field none, 1 planes: 
>  * Stride 2560, buffer size 2050560
> Video format: SBGGR10 (30314742) 1280x800 field none, 1 planes: 
>  * Stride 2560, buffer size 2050560
> Unable to request buffers: Invalid argument (22).
> 
> 
> So I changed to use v4l2-ctl
> 
> [root@xps-1 ]# v4l2-ctl -d /dev/video0 --set-fmt-video
> width=1280,height=800,pixelformat=BG10 --stream-mmap --stream-count=1 -
> -stream-to=frame.bin
> 
> With this I created raw data in BG10 format, and later used a small
> python script with numpy and opencv to look at the data.
> 
> #!/usr/bin/env python3
> # Demosaicing Bayer Raw image
> 
> import cv2
> import numpy as np
> 
> width = 1280
> height = 800
> 
> with open("frame.bin", "rb") as rawimg:
>     # Read the bayer data
>     data = np.fromfile(rawimg, np.uint16, width * height)
>     bayer = np.reshape(data, (height, width))
> 
>     # Just a offset gain to be able to see something
>     for x in range(0, len(bayer)):
>         for y in range(0, len(bayer[0])):
>             bayer[x, y] = (bayer[x,y] << 8)
> 
>     rgb = cv2.cvtColor(bayer, cv2.COLOR_BayerBGGR2RGB)
> 
>     cv2.imshow('rgb', rgb)
>     cv2.waitKey()
>     cv2.destroyAllWindows()
> 
> 
> Thanks for the help, and now we know what is needed to make it work on
> top of yesterdays rc7

-- 
Regards,

Laurent Pinchart