From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D4DEDC38145 for ; Wed, 7 Sep 2022 12:34:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Yo7FYnRsB3+UcAFu5d+a8d6wE8Ng/i3p3dTDC9yzjyw=; b=z6v6TWiV1CVzix IQ5wap/scLtxS95VRvC0JeFWEmHsRPIGkg70z+qNy8wDiqy8057Vybrjs3TDcUCW8wEqEbb2Wdr+c b1PfV5xkfS3aBqdra8kUlprPJbou50r+BAH3Lxg82LWdi1i//beXfGTOfGGfJtzYoXYLDqPOmwrUD z+nj8nMvfzUF7Oh/hKhgLfg4GcFYo/aFWc8RhYfywZJcR45vUGIbhgRrd5ZXgIUJs3JkflAZuKQGP ZQqTPcEd9UHVf5vrI7uFUHqsUn3Bd9sIyI41iMOCgXclgoeHhphbYjYJ8vLRZAfShG7twQcRa1q6+ earV7lM1KkcgeSxC0Qcg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVuEb-0067wF-CE; Wed, 07 Sep 2022 12:32:49 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVuEX-0067uK-RR for linux-arm-kernel@lists.infradead.org; Wed, 07 Sep 2022 12:32:47 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2193DB81CA9; Wed, 7 Sep 2022 12:32:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 64B63C433D7; Wed, 7 Sep 2022 12:32:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662553962; bh=qPWwVXDjn4Q55jco6FWXfEBz+1fJ7OcY6z2suhkH3Q8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=lKItM+eYz6i+1ovteA9GrTwOKpThCTpXrvg0yucKfkUydau2LjvnNeTcJUSknDhfx a6bpqXKgzUi89BWJT1i07irNkDPkgYhtW+JSk6CKs+Ed3kUkAO7lqofNh27x28gKIW j6vvhY4iNzt+wv4VqbPp3I5s7xs5dbrUkmqVtssfrHeoTXXIolYxU+32lltfzi0ymA nwXlkp903of3LJO2qUF+uLm0wEWbYiHcodZSDmOSNNa880ZoPyo7Cs4s61KL7b48h1 eqeIIrWNmpeViFzisRpb4FBiEY6bDfxBuFSI0OXxrcJO95CdkyIcvpDRczECUWcMaQ ZwFvG/IhEZdrA== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oVuES-008dkY-A3; Wed, 07 Sep 2022 13:32:40 +0100 Date: Wed, 07 Sep 2022 13:32:39 +0100 Message-ID: <87illzuzyw.wl-maz@kernel.org> From: Marc Zyngier To: "Radovanovic, Aleksandar" Cc: Jason Gunthorpe , "Gupta, Nipun" , "robh+dt@kernel.org" , "krzysztof.kozlowski+dt@linaro.org" , "gregkh@linuxfoundation.org" , "rafael@kernel.org" , "eric.auger@redhat.com" , "alex.williamson@redhat.com" , "cohuck@redhat.com" , "Gupta, Puneet (DCG-ENG)" , "song.bao.hua@hisilicon.com" , "mchehab+huawei@kernel.org" , "f.fainelli@gmail.com" , "jeffrey.l.hugo@gmail.com" , "saravanak@google.com" , "Michael.Srba@seznam.cz" , "mani@kernel.org" , "yishaih@nvidia.com" , "robin.murphy@arm.com" , "will@kernel.org" , "joro@8bytes.org" , "masahiroy@kernel.org" , "ndesaulniers@google.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kbuild@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "devicetree@vger.kernel.org" , "kvm@vger.kernel.org" , "okaya@kernel.org" , "Anand, Harpreet" , "Agarwal, Nikhil" , "Simek,\ Michal" , "git (AMD-Xilinx)" Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent In-Reply-To: References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> <20220906134801.4079497-5-nipun.gupta@amd.com> <87leqvv3g7.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: aleksandar.radovanovic@amd.com, jgg@nvidia.com, Nipun.Gupta@amd.com, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org, gregkh@linuxfoundation.org, rafael@kernel.org, eric.auger@redhat.com, alex.williamson@redhat.com, cohuck@redhat.com, puneet.gupta@amd.com, song.bao.hua@hisilicon.com, mchehab+huawei@kernel.org, f.fainelli@gmail.com, jeffrey.l.hugo@gmail.com, saravanak@google.com, Michael.Srba@seznam.cz, mani@kernel.org, yishaih@nvidia.com, robin.murphy@arm.com, will@kernel.org, joro@8bytes.org, masahiroy@kernel.org, ndesaulniers@google.com, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, kvm@vger.kernel.org, okaya@kernel.org, harpreet.anand@amd.com, nikhil.agarwal@amd.com, michal.simek@amd.com, git@amd.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220907_053246_201711_038C41FC X-CRM114-Status: GOOD ( 51.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 07 Sep 2022 12:35:54 +0100, "Radovanovic, Aleksandar" wrote: > > [AMD Official Use Only - General] > > > > > -----Original Message----- > > From: Marc Zyngier > > Sent: 07 September 2022 12:17 > > To: Jason Gunthorpe > > Cc: Gupta, Nipun ; robh+dt@kernel.org; > > krzysztof.kozlowski+dt@linaro.org; gregkh@linuxfoundation.org; > > rafael@kernel.org; eric.auger@redhat.com; alex.williamson@redhat.com; > > cohuck@redhat.com; Gupta, Puneet (DCG-ENG) > > ; song.bao.hua@hisilicon.com; > > mchehab+huawei@kernel.org; f.fainelli@gmail.com; > > jeffrey.l.hugo@gmail.com; saravanak@google.com; > > Michael.Srba@seznam.cz; mani@kernel.org; yishaih@nvidia.com; > > robin.murphy@arm.com; will@kernel.org; joro@8bytes.org; > > masahiroy@kernel.org; ndesaulniers@google.com; linux-arm- > > kernel@lists.infradead.org; linux-kbuild@vger.kernel.org; linux- > > kernel@vger.kernel.org; devicetree@vger.kernel.org; kvm@vger.kernel.org; > > okaya@kernel.org; Anand, Harpreet ; Agarwal, > > Nikhil ; Simek, Michal ; > > Radovanovic, Aleksandar ; git (AMD- > > Xilinx) > > Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its > > domain as parent > > > > [CAUTION: External Email] > > > > On Tue, 06 Sep 2022 18:19:06 +0100, > > Jason Gunthorpe wrote: > > > > > > On Tue, Sep 06, 2022 at 07:17:58PM +0530, Nipun Gupta wrote: > > > > > > > +static void cdx_msi_write_msg(struct irq_data *irq_data, > > > > + struct msi_msg *msg) { > > > > + /* > > > > + * Do nothing as CDX devices have these pre-populated > > > > + * in the hardware itself. > > > > + */ > > > > +} > > > > > > Huh? > > > > > > There is no way it can be pre-populated, the addr/data pair, > > > especially on ARM, is completely under SW control. > > > > There is nothing in the GIC spec that says that. > > > > > There is some commonly used IOVA base in Linux for the ITS page, but > > > no HW should hardwire that. > > > > That's not strictly true. It really depends on how this block is integrated, and > > there is a number of existing blocks that know *in HW* how to signal an LPI. > > > > See, as the canonical example, how the mbigen driver doesn't need to know > > about the address of GITS_TRANSLATER. > > > > Yes, this messes with translation (the access is downstream of the > > SMMU) if you relied on it to have some isolation, and it has a "black hole" > > effect as nobody can have an IOVA that overlaps with the physical address of > > the GITS_TRANSLATER register. > > > > But is it illegal as per the architecture? No. It's just stupid. > > > > M. > > > > -- > > Without deviation from the norm, progress is not possible. > > To give some context, CDX devices are specific to embedded ARM CPUs > on the FPGA and a lot of the CDX hardware core is under the control > of the system firmware, not the application CPUs. > > That being said, the MSI address is always going to be the GIC > GITS_TRANSLATER, which is known to the system firmware, as it is > fixed per FPGA platform. At present, we do not allow the application > CPU OS to change this - I believe this is for security reasons, but > this may or may not be a good idea in general. I'm sure that being downstream of the SMMU is a security feature... > As Marc mentions, CDX > MSI writes are downstream of the SMMU and, if SMMU does not provide > identity mapping for GITS_TRANSLATER, then we have a problem and may > need to allow the OS to write the address part. However, even if we > did, the CDX hardware is limited in that it can only take one > GITS_TRANSLATER register target address per system, not per CDX > device, nor per MSI vector. If the MSI generation is downstream of the SMMU, why should the SMMU provide a 1:1 mapping for GITS_TRANSLATER? I don't think it should provide a mapping at all in this case. But it looks like I don't really understand how these things are placed relative to each other... :-/ > > As for the data part (EventID in GIC parlance), this is always going > to be the CDX device-relative vector number - I believe this can't > be changed, it is a hardware limitation (but I need to > double-check). That should be OK, though, as I believe this is > exactly what Linux would write anyway, as each CDX device should be > in its own IRQ domain (i.e. have its own ITS device table). But that's really the worse part. You have hardcoded what is the *current* Linux behaviour. Things change. And baking SW behaviour into a piece of HW looks incredibly shortsighted... > The best I can propose is to pass the addr/data info to firmware > here, which will then decide what to do with it. At least, it can > assert that the values are what the hardware expects and fail loudly > if not, rather than having a silently misconfigured system. And then what? It means that by agreeing to support this bus, we are agreeing to *never* change the EventID allocation scheme. I'm not signing up to this. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel