From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3F5A7C3DA4A for ; Wed, 14 Aug 2024 16:00:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5xBX9XcWgNXlZ2p9lMYGImJ9L8gJGOrnlHEQMQ6i27s=; b=bCXAE7950KFiKu9WVYDrSC0qVb nfPUOO0A5UBgXBNz9F2RvxL6SBUXxTotSDRYSZ4empIMhsFKAfh0jbOhiReOLuouH1RioeqcGakAI WvXMZPLg6kFNnfbKySWfkGFteo2RywA1UT1TLCt/Xww6r96MXsXEFzhViEQcDP2JqgwlKy2sFsyGT 4wpnle9L/Q+x5FFg39ssgnStQMMIsC5+6zbbmJFBjbKy8bU/xtdZ20x72nOljEYB9fkmD8uXJByRn fXtvC6ZB9YzvDqiF/97V5rcmoJ4h8c/7gWnZX0C3DeYZ9zf5h8nHAgCVrfNj+AB+c3gZL+i/zKUML QAGdo5yg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1seGQY-00000007ZdM-35O4; Wed, 14 Aug 2024 16:00:46 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1seFHz-00000007MCH-3IkY for linux-arm-kernel@lists.infradead.org; Wed, 14 Aug 2024 14:47:53 +0000 Received: from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi [81.175.209.231]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D38162E0; Wed, 14 Aug 2024 16:46:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723646810; bh=Spp9/znjTXt6A1KCOS7O2DwLt2vipP8wNmQMc4Ks5ok=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=vwsHBVkHyO+D1GOBvcxeLlxoJBPMAOcquuGUGgTE4jTyf335Zh/xHbyyjvXn9m/Xi LCKZN5ws4Y+TO5iEf1blPDlNHSKrXxFCd/YyZ0HdoRdFdCgrNQrFYN8lmLvwfZKVh9 8YRSZ3dnKEVR3p6tCXp1zMX2Lxw22Do00EXvTZ/s= Date: Wed, 14 Aug 2024 17:47:22 +0300 From: Laurent Pinchart To: Adam Ford Cc: Alexander Stein , imx@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Paul Elder , Conor Dooley , Fabio Estevam , Krzysztof Kozlowski , Marek Vasut , Pengutronix Kernel Team , Rob Herring , Sascha Hauer , Shawn Guo , devicetree@vger.kernel.org, linux-media@vger.kernel.org, Peng Fan Subject: Re: [PATCH v2] arm64: dts: imx8mp: Add DT nodes for the two ISPs Message-ID: <20240814144722.GA22567@pendragon.ideasonboard.com> References: <20240325151339.19041-1-laurent.pinchart@ideasonboard.com> <4879631.GXAFRqVoOG@steina-w> <20240325204924.GY18799@pendragon.ideasonboard.com> <2929432.e9J7NaK4W3@steina-w> <20240813222204.GA21831@pendragon.ideasonboard.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240814_074752_143298_152A0A51 X-CRM114-Status: GOOD ( 67.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Adam, (CC'ing Peng Fan) On Wed, Aug 14, 2024 at 01:29:16AM -0500, Adam Ford wrote: > On Tue, Aug 13, 2024 at 5:22 PM Laurent Pinchart wrote: > > On Sun, Jun 09, 2024 at 03:21:19PM -0500, Adam Ford wrote: > > > On Tue, Mar 26, 2024 at 2:14 AM Alexander Stein wrote: > > > > Am Montag, 25. März 2024, 21:49:24 CET schrieb Laurent Pinchart: > > > > > On Mon, Mar 25, 2024 at 04:52:21PM +0100, Alexander Stein wrote: > > > > > > Am Montag, 25. März 2024, 16:13:39 CET schrieb Laurent Pinchart: > > > > > > > From: Paul Elder > > > > > > > > > > > > > > The ISP supports both CSI and parallel interfaces, where port 0 > > > > > > > corresponds to the former and port 1 corresponds to the latter. Since > > > > > > > the i.MX8MP's ISPs are connected by the parallel interface to the CSI > > > > > > > receiver, set them both to port 1. > > > > > > > > > > > > > > Signed-off-by: Paul Elder > > > > > > > Signed-off-by: Laurent Pinchart > > > > > > > --- > > > > > > > Changes since v1: > > > > > > > > > > > > > > - Fix clock ordering > > > > > > > - Add #address-cells and #size-cells to ports nodes > > > > > > > --- > > > > > > > arch/arm64/boot/dts/freescale/imx8mp.dtsi | 50 +++++++++++++++++++++++ > > > > > > > 1 file changed, 50 insertions(+) > > > > > > > > > > > > > > diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi > > > > > > > index bfc5c81a5bd4..1d2670b91b53 100644 > > > > > > > --- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi > > > > > > > +++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi > > > > > > > @@ -1616,6 +1616,56 @@ isi_in_1: endpoint { > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > + isp_0: isp@32e10000 { > > > > > > > + compatible = "fsl,imx8mp-isp"; > > > > > > > + reg = <0x32e10000 0x10000>; > > > > > > > + interrupts = ; > > > > > > > + clocks = <&clk IMX8MP_CLK_MEDIA_ISP_ROOT>, > > > > > > > + <&clk IMX8MP_CLK_MEDIA_AXI_ROOT>, > > > > > > > + <&clk IMX8MP_CLK_MEDIA_APB_ROOT>; > > > > > > > + clock-names = "isp", "aclk", "hclk"; > > > > > > > + assigned-clocks = <&clk IMX8MP_CLK_MEDIA_ISP>; > > > > > > > + assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_500M>; > > > > > > > + assigned-clock-rates = <500000000>; > > > > > > > + power-domains = <&media_blk_ctrl IMX8MP_MEDIABLK_PD_ISP>; > > > > > > > + fsl,blk-ctrl = <&media_blk_ctrl 0>; > > > > > > > + status = "disabled"; > > > > > > > + > > > > > > > + ports { > > > > > > > + #address-cells = <1>; > > > > > > > + #size-cells = <0>; > > > > > > > + > > > > > > > + port@1 { > > > > > > > + reg = <1>; > > > > > > > + }; > > > > > > > + }; > > > > > > > + }; > > > > > > > + > > > > > > > + isp_1: isp@32e20000 { > > > > > > > + compatible = "fsl,imx8mp-isp"; > > > > > > > + reg = <0x32e20000 0x10000>; > > > > > > > + interrupts = ; > > > > > > > + clocks = <&clk IMX8MP_CLK_MEDIA_ISP_ROOT>, > > > > > > > + <&clk IMX8MP_CLK_MEDIA_AXI_ROOT>, > > > > > > > + <&clk IMX8MP_CLK_MEDIA_APB_ROOT>; > > > > > > > + clock-names = "isp", "aclk", "hclk"; > > > > > > > + assigned-clocks = <&clk IMX8MP_CLK_MEDIA_ISP>; > > > > > > > + assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_500M>; > > > > > > > + assigned-clock-rates = <500000000>; > > > > > > > + power-domains = <&media_blk_ctrl IMX8MP_MEDIABLK_PD_ISP>; > > > > > > > + fsl,blk-ctrl = <&media_blk_ctrl 1>; > > > > > > > + status = "disabled"; > > > > > > > + > > > > > > > + ports { > > > > > > > + #address-cells = <1>; > > > > > > > + #size-cells = <0>; > > > > > > > + > > > > > > > + port@1 { > > > > > > > + reg = <1>; > > > > > > > + }; > > > > > > > + }; > > > > > > > + }; > > > > > > > + > > > > > > > > > > > > The patch itself is okay. But you might not be able to > > > > > > configure the parent of IMX8MP_CLK_MEDIA_ISP if dewarp is enabled before. > > > > > > This is due to IMX8MP_CLK_MEDIA_ISP_ROOT being enabled in 'pgc_ispdwp' > > > > > > power domain. Reparenting is not possible anymore in this case. > > > > > > > > > > Good point. > > > > > > > > > > > Something like > > > > > > ---8<--- > > > > > > --- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi > > > > > > +++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi > > > > > > @@ -1837,11 +1837,13 @@ media_blk_ctrl: blk-ctrl@32ec0000 { > > > > > > <&clk IMX8MP_CLK_MEDIA_APB>, > > > > > > <&clk IMX8MP_CLK_MEDIA_DISP1_PIX>, > > > > > > <&clk IMX8MP_CLK_MEDIA_DISP2_PIX>, > > > > > > + <&clk IMX8MP_CLK_MEDIA_ISP>, > > > > > > <&clk IMX8MP_VIDEO_PLL1>; > > > > > > assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_1000M>, > > > > > > <&clk IMX8MP_SYS_PLL1_800M>, > > > > > > <&clk IMX8MP_VIDEO_PLL1_OUT>, > > > > > > - <&clk IMX8MP_VIDEO_PLL1_OUT>; > > > > > > + <&clk IMX8MP_VIDEO_PLL1_OUT>, > > > > > > + <&clk IMX8MP_SYS_PLL2_500M>; > > > > > > assigned-clock-rates = <500000000>, <200000000>, > > > > > > <0>, <0>, <1039500000>; > > > > > > > > > > > According to the i.MX8MP Data sheet, the nominal speed for > > > MEDIA_ISP_CLOCL_ROOT is 400MHZ with 500MHz being allowed in overdrive > > > mode. > > > > > > I think this clock rate should drop to the nominal value of 400MHz > > > and those boards who support overdrive can increase it to 500MHz to > > > avoid stiability issues and/or running out of spec. I created an > > > imx8mm and imx8mn-overdrive.dtsi file. If there is interest, I can do > > > the same for the 8MP as well. > > > > I've experimented with this, and lowered the ISP clock to 400MHz by > > selecting IMX8MP_SYS_PLL1_400M as the parent. I couldn't capture images > > anymore :-S > > > > The camera sensor I'm using outputs a 74.25 Mpix/s pixel rate, which is > > way below the maximum in either normal or overdrive mode. The reference > > manual states, regarding the CSI-2 receivers, > > > > - When one ISP is used, MIPI CSI interface 1 supports: > > - Pixel clock up to 400MHz at nominal voltage and 500MHz at overdrive > > voltage > > - When two ISPs are used, both MIPI CSI interfaces supports: > > - Pixel clock up to 266MHz at nominal and overdrive voltage > > > > The datasheets for both the CEC and IEC variants give slightly > > different information: > > > > - For single Camera, MIPI CSI 1 can support up to 400/500 MHz pixel > > clock in the Nominal/Overdrive mode. > > - For single Camera, MIPI CSI 2 can support up to 277 MHz pixel clock. > > - For dual Camera, both MIPI CSI can support up to 266 MHz pixel clock. > > > > The also list the maximum clock frequencies as > > > > - MEDIA_ISP_CLK_ROOT: nominal 400 MHz, overdrive 500 MHz > > - MEDIA_CAM1_PIX_CLK_ROOT: nominal 400 MHz, overdrive 500 MHz > > - MEDIA_CAM2_PIX_CLK_ROOT: nominal 277 MHz, overdrive 277 MHz > > > > (On a side note, the revision history indicates the revision 2.1 of the > > datasheet changed MEDIA_CAM2_PIX_CLK_ROOT from 266/266 MHz to 277/277 > > MHz, but the 277 MHz frequency has been listed since revision 0.) > > > > The MIPI CSI-2 RX (MIPI_CSI) receives data for the camera sensor at the > > CSI-2 link frequency rate. It reclocks the data stream using an internal > > FIFO, with an output pixel rate selected by the > > IMX8MP_CLK_MEDIA_CAM1_PIX_ROOT (for the first CSI-2 receiver) or > > IMX8MP_CLK_MEDIA_CAM2_PIX_ROOT (for the second CSI-2 receiver). Those > > clocks support the following parents, with the following frequencies on > > my system: > > > > - 24M_REF_CLK 24 MHz > > - SYSTEM_PLL1_CLK 800 MHz > > - SYSTEM_PLL1_DIV3 266 MHz > > - SYSTEM_PLL2_CLK 1000 MHz > > - SYSTEM_PLL2_DIV4 250 MHz > > - SYSTEM_PLL3_CLK 600 MHz > > - AUDIO_PLL2_CLK 361.2672 MHz > > - VIDEO_PLL_CLK 400 MHz > > > > This makes it easy to configure the CAM PIX ROOT clock to 266 MHz or 400 > > MHz easily. Achieving 277 MHz or 500 MHz would I assume require > > modifying some of the PLL configurations. I'm not sure how the clock > > tree was designed to be configured for overdrive mode. > > > > imx8mp.dtsi configures the CAM1_PIX and CAM2_PIX clocks with > > > > assigned-clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>; > > assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_1000M>; > > assigned-clock-rates = <266000000>; > > > > and > > > > assigned-clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>; > > assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_1000M>; > > assigned-clock-rates = <266000000>; > > > > This is misleading as it selects the SYSTEM_PLL2_DIV4 parent, with a > > clock frequency of 250 MHz. I'll send a patch to set the assigned clock > > rate to 250 MHz to avoid misunderstandings, we can in parallel decide if > > the parent should be set to SYSTEM_PLL1_DIV3 instead. > > > > As the CSI-2 transmits, as far as I can tell, one pixel per clock cycle > > to the ISP for raw sensors, this is well within the hardware limits. > > > > The clock domains of the ISP are briefly described in the reference > > manual as follows: > > > > - Core Clock (clk) - Core clock is the main clock domain used by the ISP > > modules and is the biggest clock domain in the ISP. > > - Sensor Clock (sclk) - The sensor clock domain is selected from the > > sensor interface and is the data input clock. > > - AHB Clock (hclk) - The AHB Clock is the AHB Interface clock. > > - AXI Clock (m_hclk) - The AXI Clock is the AXI Interface clock. > > > > Additionally, the reference manual states that > > > > The communication between the different domains in ISP occurs mostly > > by way of asynchronous FIFOs. All clocks are asynchronous to each > > other. There is no communication between AHB and AXI clock domains. > > > > The sensor clock (sclk) is the clock received by the ISP, which is the > > MIPI_CSI output clock (MEDIA_CAM[12]_PIX_CLK_ROOT). This clock isn't > > listed in the ISP DT node, as it isn't controlled directly by the ISP. > > It isn't clear where the transition to the core clock (clk) happens, I > > would assume quite early in the ISP pipeline. That clock is the > > MEDIA_ISP_CLOCK_ROOT, which that patch configures to 500 MHz. My > > assumption is that the core clock frequency needs to be at least as high > > as the sensor clock, but as configuring the ISP clock to 400 MHz breaks > > image capture, there's something I'm missing. > > > > As the 500 MHz ISP clock frequency is exactly twice the sensor clock > > frequency in my setup, I wondered if there could be a x2 relationship > > between those two clocks. I have tried to increase the CAM1_PIX clock to > > 266 MHz, and the ISP then operates correctly with a 500 MHz ISP clock. > > There must thus be something else. > > > > I'm not sure how to move forward. There's no further information I could > > find in the datasheet or reference manual that would hint how to operate > > with a 400 MHz ISP clock frequency. We may require help from someone at > > NXP. In the meantime, could we merge this patch with a 500 MHz ISP > > clock frequency (setup in the blk-ctrl@32ec0000 node instead of the isp > > nodes) ? > > Thank you for your detailed analysis. If the ISP doesn't work at > 400MHz, it doesn't make sense to me to clock it at 400MHz. Maybe NXP > can clarify this. If 500MHz is the only operating point, I have no > objection, but I think having a comment in the device tree node where > the clock is set to 500MHz as what it is this way might be in order so > someone in the future doesn't go in a arbitrarily set it to 400MHz > based on a document without doing some testing. I wouldn't say that the ISP can't work at 400 MHz at all, but I think we'll need help from NXP to better understand the clocking schemes and constraints. Peng, do you think this is something you or someone else could have a look at ? I'll send a new version with a comment in the device tree. > > > I haven't gone through all the clocks to determine if/what clocks are > > > being overdriven. > > > > > > > > With an assigned clock rate here too then ? > > > > > > > > You are right. This posted diff is what I was using for a while now though. > > > > Apparently the clock frequency was still correct. > > > > > > > > > > #power-domain-cells = <1>; > > > > > > ---8<--- > > > > > > is needed. > > > > > > > > > > Sascha, are you OK with this approach ? > > > > > > This patch appears to have gone stale. Is there any way we can push > > > this forward? > > > > > > > > > > dewarp: dwe@32e30000 { > > > > > > > compatible = "nxp,imx8mp-dw100"; > > > > > > > reg = <0x32e30000 0x10000>; > > > > > > > > > > > > > > base-commit: 4cece764965020c22cff7665b18a012006359095 -- Regards, Laurent Pinchart