From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 967BFECAAD5 for ; Thu, 8 Sep 2022 08:09:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Ph0fSIFKhvHhiHLoi1lews1XeLOU8+8P0uHfRwm5Ikc=; b=iEifzhYRyPOkqu hKcSlpDLL32jVRhHAxMZFfb0sFpcKKjW9imoQNTrnBfDPSdj7pF6labcEaCSKEkXqDBKrLfqtW6Qd WAoi7A+gTy9TgMvVsoj2BasRw7BNem41MLMZ7G8M/M6y5fVGuEv5tZicl1NC4Bj6VUTf58jTMyALd 0PrZ0SiuC++sjuhH67eC710RcSYzExT2mwX3/2yns9uDInPxTnUV8tqpiaJjPmlrQep3jvJhgI91d smK+ra1NRlNVKYwHI5YuKNRHsV1tTh+ZNpGCid6a9Yn75ITd5iGJglL+Voz0KXiLHUl/bwsYDOxNe fwUOKW53iYKEqgB/lG8g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oWCa2-000w7F-PJ; Thu, 08 Sep 2022 08:08:11 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oWCZy-000vv8-TO for linux-arm-kernel@lists.infradead.org; Thu, 08 Sep 2022 08:08:08 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 58DD3B8203E; Thu, 8 Sep 2022 08:08:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0E0FC433D7; Thu, 8 Sep 2022 08:08:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662624484; bh=GD6OLumDwO8umKLLrqr/6VkSMx2COiJ9Ap4vjY8VW1w=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=NRtRFHjhqm6mytN+d8LLXcfnX0UENfxbHqExuLjbXsE1YC7BW6pCWz5iUBENNIHz6 JMVRuSMwNvy/lmzAywjEv086zKYppphjoX9sTboehByb3O7XeMqAdE8ftxEnFCh740 YxbDxe6DG4bdbJO6Bwq2hYZPEdRzeYCy6/QZvG9r+zRAD/vZRJHXphuCO2usrJHIej ER6+xnlZmQFhslWr2ZBuPPEPOn/E8tdZz9o2srfkMi4YSMyIjhcX9brjCUS5DutUUG MXttpK7mEA4bcujcvwA+1evme6d1DB9adokSP/2ROW63OVO75c9pQctRNjiEEZZ13T E5GEPD86QfdBg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oWCZs-008qTO-UI; Thu, 08 Sep 2022 09:08:01 +0100 Date: Thu, 08 Sep 2022 09:08:00 +0100 Message-ID: <87edwmuw4f.wl-maz@kernel.org> From: Marc Zyngier To: "Radovanovic, Aleksandar" Cc: Jason Gunthorpe , "Gupta, Nipun" , "robh+dt@kernel.org" , "krzysztof.kozlowski+dt@linaro.org" , "gregkh@linuxfoundation.org" , "rafael@kernel.org" , "eric.auger@redhat.com" , "alex.williamson@redhat.com" , "cohuck@redhat.com" , "Gupta, Puneet (DCG-ENG)" , "song.bao.hua@hisilicon.com" , "mchehab+huawei@kernel.org" , "f.fainelli@gmail.com" , "jeffrey.l.hugo@gmail.com" , "saravanak@google.com" , "Michael.Srba@seznam.cz" , "mani@kernel.org" , "yishaih@nvidia.com" , "robin.murphy@arm.com" , "will@kernel.org" , "joro@8bytes.org" , "masahiroy@kernel.org" , "ndesaulniers@google.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kbuild@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "devicetree@vger.kernel.org" , "kvm@vger.kernel.org" , "okaya@kernel.org" , "Anand, Harpreet" , "Agarwal, Nikhil" , "Simek,\ Michal" , "git (AMD-Xilinx)" Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent In-Reply-To: References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> <20220906134801.4079497-5-nipun.gupta@amd.com> <87leqvv3g7.wl-maz@kernel.org> <87illzuzyw.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: aleksandar.radovanovic@amd.com, jgg@nvidia.com, Nipun.Gupta@amd.com, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org, gregkh@linuxfoundation.org, rafael@kernel.org, eric.auger@redhat.com, alex.williamson@redhat.com, cohuck@redhat.com, puneet.gupta@amd.com, song.bao.hua@hisilicon.com, mchehab+huawei@kernel.org, f.fainelli@gmail.com, jeffrey.l.hugo@gmail.com, saravanak@google.com, Michael.Srba@seznam.cz, mani@kernel.org, yishaih@nvidia.com, robin.murphy@arm.com, will@kernel.org, joro@8bytes.org, masahiroy@kernel.org, ndesaulniers@google.com, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, kvm@vger.kernel.org, okaya@kernel.org, harpreet.anand@amd.com, nikhil.agarwal@amd.com, michal.simek@amd.com, git@amd.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220908_010807_277540_4D44EC8F X-CRM114-Status: GOOD ( 40.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 07 Sep 2022 14:18:52 +0100, "Radovanovic, Aleksandar" wrote: > > > > > As Marc mentions, CDX > > > MSI writes are downstream of the SMMU and, if SMMU does not provide > > > identity mapping for GITS_TRANSLATER, then we have a problem and may > > > need to allow the OS to write the address part. However, even if we > > > did, the CDX hardware is limited in that it can only take one > > > GITS_TRANSLATER register target address per system, not per CDX > > > device, nor per MSI vector. > > > > If the MSI generation is downstream of the SMMU, why should the SMMU > > provide a 1:1 mapping for GITS_TRANSLATER? I don't think it should provide a > > mapping at all in this case. But it looks like I don't really understand how > > these things are placed relative to each other... :-/ > > > > Apologies, I got my streams confused. It is _upstream_ of the SMMU, > it does go through SMMU mapping. OK, so you definitely need a mapping, but it cannot be a translation, and it needs to be in all the possible address spaces. OMG. > > > As for the data part (EventID in GIC parlance), this is always going > > > to be the CDX device-relative vector number - I believe this can't be > > > changed, it is a hardware limitation (but I need to double-check). > > > That should be OK, though, as I believe this is exactly what Linux > > > would write anyway, as each CDX device should be in its own IRQ domain > > > (i.e. have its own ITS device table). > > > > But that's really the worse part. You have hardcoded what is the > > *current* Linux behaviour. Things change. And baking SW behaviour into a > > piece of HW looks incredibly shortsighted... > > For posterity, I'm not an RTL designer/architect, so share your > sentiment to a certain extent. That said, I expect the decision was > not based on Linux or any other SW behaviour, but because it is the > most straightforward and least expensive way to do it. Having an > EventID register for each and every MSI source just so you can > program it in any random order costs flops and all the associated > complexity of extra RTL logic (think timing closure, etc.), so > trade-offs are made. The fact that it matches current Linux > behaviour means we just got lucky... Yeah, but that's not the only problem: there is no guarantee that we have enough LPIs to allocate for the device, so we'll perform a partial allocation (8 instead of 32 LPIs, for example). How do you tell the device which events are valid? As far as I can see, you can't, and I guess it will fire on all of them, resulting in interrupts being dropped on the floor. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel