From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 354BEC4332F for ; Fri, 9 Dec 2022 11:27:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229911AbiLIL1u (ORCPT ); Fri, 9 Dec 2022 06:27:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229479AbiLIL1q (ORCPT ); Fri, 9 Dec 2022 06:27:46 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D306CD11C for ; Fri, 9 Dec 2022 03:27:44 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6F48462228 for ; Fri, 9 Dec 2022 11:27:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBC95C433D2; Fri, 9 Dec 2022 11:27:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1670585263; bh=R9rAfVVymVvpnxrvUlohdqpNC5TLNy/FhPqdee8K+DY=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Ww1N+yWQcZxpA7ENd51R3wCFQKy15TM8ZUZgqYH6PunVkpduqE4lkeq8sMom9rvZx Lhh6sdgM/iETa1e2k0kZpnj6NrWeiGQwt3UtXEmY/sQ/Hcw5bPp0NY50MVKIUG6FvP v6+d76yWg2NQULrKmL8lKqP82asg4WXfH2pFgDEEHIznyrOGuxF6Pk1FajYI6xToGb 0lUTMM26ezuj3EdDBgc5fN6k94dwwji5aYYaGhjQzsSd6rrE2l/uFfbMztELW2+RN0 LWEWVTR7iJWM/NIvdvYdzuVwwRucE8MsyAPMzraD0dJd6Ve4azz2xn7aoPyvYvjDAA baEBXsHg5/q6Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1p3bXZ-00BZkv-Sl; Fri, 09 Dec 2022 11:27:41 +0000 Date: Fri, 09 Dec 2022 11:27:41 +0000 Message-ID: <86edt8rf4i.wl-maz@kernel.org> From: Marc Zyngier To: Sebastian Reichel Cc: Harry Song , tglx@linutronix.de, linux-kernel@vger.kernel.org Subject: Re: [PATCH] irqchip/gic-v3-its: remove the shareability of ITS In-Reply-To: <20221208165820.5maej4we3mfdeprm@mercury.elektranox.org> References: <20221207135223.3938-1-jundongsong1@gmail.com> <86a63zkzru.wl-maz@kernel.org> <20221208165820.5maej4we3mfdeprm@mercury.elektranox.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: sebastian.reichel@collabora.com, jundongsong1@gmail.com, tglx@linutronix.de, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 08 Dec 2022 16:58:20 +0000, Sebastian Reichel wrote: > > [1 ] > Hi, > > On Thu, Dec 08, 2022 at 10:28:29AM +0800, Harry Song wrote: > > On Wed, Dec 7, 2022 at 11:19 PM Marc Zyngier wrote: > > > > > > On Wed, 07 Dec 2022 13:52:23 +0000, > > > Harry Song wrote: > > > > > > > > I know this is a very wrong patch, but my platform > > > > has an abnormal ITS problem caused by data consistency: > > > > My chip does not support Cache Coherent Interconnect (CCI). > > > > > > That doesn't mean much. Nothing mandates to have a CCI, and plenty of > > > systems have other ways to maintain coherency. > > > > > > > By default, ITS driver is the inner memory attribute. > > > > gits_write_cbaser() is used to write the inner memory > > > > attribute. But hw doesn't return the hardware's non-shareable > > > > property,so I don't think reading GITS_CBASER and GICR_PROPBASER > > > > here will get the real property of the current hardware: inner > > > > or outer shareable is not supported, so I would like to know > > > > whether ITS driver cannot be used on chips without CCI, or > > > > what method can be used to use ITS driver on chips without CCI? > > > > > > It's not about CCI or not CCI. It is about which shareability domain > > > your ITS is in. > > > > > > And it doesn't only affect the ITS. It also affects the > > > redistributors, and anything that accesses memory. > > > > > > > Yes, my current chip is Rockchip platform (rk3588), so is it because the chip > > is not designed as a proper shared domain for ITS, so the exception of > > ITS is caused? > > > > > > > > > > The current patch is designed to make ITS think that the current > > > > chip has no inner or outer memory properties, and then use > > > > its by flushing dcache. > > > > > > > > This is the log for bug reports without patches: > > > > > > > > [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000003460000 > > > > [ 0.000000] ITS [mem 0x03440000-0x0345ffff] > > > > [ 0.000000] ITS@0x0000000003440000: allocated 8192 Devices @41850000 (indirect, esz 8, psz 64K, shr 0) > > > > [ 0.000000] ITS@0x0000000003440000: allocated 32768 Interrupt Collections @41860000 (flat, esz 2, psz 64K, shr 0) > > > > [ 0.000000] GICv3: using LPI property table @0x0000000041870000 > > > > [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000041880000 > > > > [ 0.000000] ITS queue timeout (64 1) > > > > [ 0.000000] ITS cmd its_build_mapc_cmd failed > > > > [ 0.000000] ITS queue timeout (128 1) > > > > [ 0.000000] ITS cmd its_build_invall_cmd failed > > > > > > Ah, this suspiciously looks like a Rockchip machine... > > > > > > > > > > > Signed-off-by: Harry Song > > > > --- > > > > > > > > I am very sorry to bother you. This problem has been bothering me > > > > for several weeks. I am looking forward to your reply. > > > > > > If you have such issue, this needs to be handled as per-platform > > > quirk. I'm not putting such generic hacks in the driver. > > > > > > M. > > > > > > -- > > > Without deviation from the norm, progress is not possible. > > > > Are there currently other chip platforms that have this problem, and then ITS > > is already compatible with the problem? Please tell me how to submit > > hacks patch for rk3588 platform? > > > > If the hacks patch cannot be submitted, please tell me whether it > > requires the next generation > > chip to have any design requirements for ITS? > > > > Design ITS and CPU to be the same inner memory property? > > Previous rockchip generation has the same bug and it got discussed > here: > > https://lore.kernel.org/lkml/871rbdt4tu.wl-maz@kernel.org/T/#m5dbc70ff308d81e98dd0d797e23d3fbf9c353245 > > You can use this DT as base with mainline to avoid ITS: > > https://lore.kernel.org/all/20221205172350.75234-1-sebastian.reichel@collabora.com/ Using the MBI region really isn't the same thing. You're limited to a handful of MSI instead of 64k, and you don't get any device isolation. Also, your usage of the GIC binding is pretty broken. A single PPI partition covering all the CPUs, the two PMU having the same PPI and #interrupt-calls=3, that's all completely wrong. M. -- Without deviation from the norm, progress is not possible.