From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51929C433FE for ; Thu, 10 Dec 2020 09:25:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0903E23D6C for ; Thu, 10 Dec 2020 09:25:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728403AbgLJJYs (ORCPT ); Thu, 10 Dec 2020 04:24:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727217AbgLJJYs (ORCPT ); Thu, 10 Dec 2020 04:24:48 -0500 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8254AC0613CF for ; Thu, 10 Dec 2020 01:24:07 -0800 (PST) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DEDF825E; Thu, 10 Dec 2020 10:24:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1607592244; bh=VnVpoVCzjQofdzJM5LjB75ojt6ZX2nHo+4v+tm/Rh5c=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=W4o94JOXAUnKLwJc/tNuyHQLARnSn8kOWiEcvVChC6S4G4NLx2rMBqhgtLTEKsBTl WAb4DLAkVBuS7amfWobio/53vPBGrqko/9UZgsKBYqUSCUCg4iIla0NsjBhwPAjdKU 26QQXtMNJOK1IPnWfDpTAlSmnOvAQJ1DyHBn03rw= Date: Thu, 10 Dec 2020 11:23:59 +0200 From: Laurent Pinchart To: Martin Kepplinger Cc: "rogerio.silva@nxp.com" , Philipp Zabel , "slongerbeam@gmail.com" , Fabio Estevam , "linux-arm-kernel@lists.infradead.org" , linux-media@vger.kernel.org, NXP Linux Team , "kernel@puri.sm" , Pengutronix Kernel Team Subject: Re: question: CSI on imx8mq with (any) CSI2 camera / experience with mx6s_capture? Message-ID: References: <9e09ca8b-d1bc-eaa9-d296-c4ee98882c06@puri.sm> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <9e09ca8b-d1bc-eaa9-d296-c4ee98882c06@puri.sm> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Hi Martin, On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > hi, > > TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with other > cameras? I've recently worked on camera support for i.MX8MM (whose camera IP cores are, if not identical, very similar to the i.MX8MQ's). The least I can say is that it was painful :-( I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and YUV and has a CSI-2 interface. My first use case is to capture RAW10, which isn't supported by the mx6s_capture driver. > I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > Using NXP's CSI bridge driver > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > as well as the CSI driver itself: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > works fine when using the ov5640 camera with this driver: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > > (I realize there is a CSI bridge driver in staging, but that need more > work to be actually used. Of course after this the goal is to fix and > use it; and mainline a CSI phy driver too.) I have lots of patches for this driver, which I've developed on i.MX7D for a separate project. I'd like to mainline them, but this is blocked by one last issue that I haven't been able to solve yet. In a nutshell, the CSI writes two consecutive frames in each buffer, overflowing the allocated memory. The registers that control the buffer size seem to be programmed correctly as far as I can tell. I've reported this issue to NXP but haven't received any feedback yet. I've also added support for i.MX8MM to this driver, but haven't been able to capture RAW8, RAW10 or YUV successfully. YUV produces the "best images", but seems to drop 3 out of 4 consecutive pixels. The sensor driver has been tested successfully on i.MX6 so I don't think it's the most likely cause of issues, but I can't rule out bugs on that side either. The i.MX8 reference manuals describe the MIPI_CSI and CSI IP cores but doesn't tell much about how the two are connected, so I have lots of unanswered questions about the register fields that control the interface between those IP cores. I'm pretty sure there are issues in that area of the drivers, but I've tested all combinations I could think of, without luck. > Now I use said NXP's CSI drivers on mainline without problems with the > ov5640. For the hi846 (as an example), I want to use a different > pixelformat and CSI media bus format. Describing, say, > MEDIA_BUS_FMT_SBGGR10_1X10 in the driver lets mx6s_capture find it; but > as soon as I try to add a different pixelformat than V4L2_PIX_FMT_UYVY in: > https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er#n244 > streaming won't start anymore: "not negotiated, -4" and the > "mx6s_vidioc_enum_framesizes" ioctl is not called anymore. Why so? That will likely be the easy part of the problem. The NXP driver doesn't support RAW10 at all, so you will have to add that, and I don't think it will be easy. I'll push my development branch shortly if you're interested in trying the mainline driver. > I didn't find much when comparing strace from "gst-launch-1.0 v4l2src ! > video/x-raw,width=1280,height=720 ! videoconvert ! xvimagesink" (which > is how I test). > > When I simply use V4L2_PIX_FMT_UYVY I *do* get interrupts (in mx6s) but > only "FIELD0_INT" (and drawing at this point suggests that buffer 1 is > not yet full and "distorted" in a wrong format, which could explain why > DMA is never completed in order to create a full frame). > > Now this is details, but the hi846 camera uses a 10bit CSI format and > therefore I need to set PIXEL_BIT in cr1 too, but when I do that for > example, I don't get "FIELD0_INT" anymore (only SFF_OR_INT and of course > the "base address switching Change Err"). > > Do you have experience with using this driver with other cameras and can > point me in a direction that might help me? > > If you want to look at the work-in-progress driver: > https://source.puri.sm/martin.kepplinger/linux-next/-/blob/hi846/drivers/media/i2c/hi846.c -- Regards, Laurent Pinchart