From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ED41C34022 for ; Wed, 19 Feb 2020 08:12:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 303492176D for ; Wed, 19 Feb 2020 08:12:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ngV3aMZh"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ahFy/0bN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 303492176D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gKV6MJbZlal93B/pxMFcDDRXS1kDnvjuS8tufjLDOGQ=; b=ngV3aMZhHTQtGj sz0vjIX/utWxrSeXZrZbH4F+VDCh2Y/asXG+MnxNLczkH6IQPEzlyl/SGFgyry0Jjk0vZ8vZjpFLF l0eGpyAgJZAp9WLVWDVumRxh2obATghfOkji8qqN0LSEFx+MXZu2c5B2AiviDywPbONIEDBgNHIE6 AlZ/nUr3miFItkfyUfzwQ5A8KPu7pU3DWfutPTqR141JpAIvhFn/pd00BCcgS8dkzzxRDANSDwQ+e mxX2RzxoOyJY60bdHo6OqPH8pb4gGAqSEx6GIuk7x9AtdmXhVewIBy+vbXf6dLL2n2/x01vnGa71G V6tOysqEuzHVoG12qvsw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j4KSZ-0005rY-Kg; Wed, 19 Feb 2020 08:11:55 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j4KSV-0005r1-Sp for linux-arm-kernel@lists.infradead.org; Wed, 19 Feb 2020 08:11:53 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5DCC32176D; Wed, 19 Feb 2020 08:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1582099911; bh=QfU8iG3TjKxX5yhqwL+LRUe53aZyIy+gvaXUuliPqbg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ahFy/0bNrSNT6TxkKoZyef6m1GZfFipzdpOtruXbovl7mcipDDL9gWBa7SAlf3ZKo 1W504MjHeHa3ipk9lMGCcKv3draY0I7miXUG1c+aAwCwjg5psgyKDyYK6lEczG031I DFqNxIrNsJGc/zqiW2t2ERpYCAJlGbfukfjo2itU= Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why) by disco-boy.misterjones.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j4KST-006SGG-J4; Wed, 19 Feb 2020 08:11:49 +0000 Date: Wed, 19 Feb 2020 08:11:48 +0000 From: Marc Zyngier To: Alan Mikhak Subject: Re: [PATCH 2/3] pci: designware: add separate driver for the MSI part of the RC Message-ID: <20200219081148.5307e30a@why> In-Reply-To: References: <20170821192907.8695-3-ard.biesheuvel@linaro.org> <1581728065-5862-1-git-send-email-alan.mikhak@sifive.com> <867e0o6ssr.wl-maz@kernel.org> Organization: Approximate X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: alan.mikhak@sifive.com, ardb@kernel.org, Joao.Pinto@synopsys.com, bhelgaas@google.com, graeme.gregory@linaro.org, jingoohan1@gmail.com, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, leif@nuviainc.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200219_001151_972890_4909E6E6 X-CRM114-Status: GOOD ( 30.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joao Pinto , Graeme Gregory , Jingoo Han , linux-pci , Bjorn Helgaas , Leif Lindholm , Ard Biesheuvel , linux-arm-kernel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 18 Feb 2020 11:09:10 -0800 Alan Mikhak wrote: > On Sat, Feb 15, 2020 at 2:36 AM Marc Zyngier wrote: > > > > On Sat, 15 Feb 2020 09:35:56 +0000, > > Ard Biesheuvel wrote: > > > > > > (updated some email addresses in cc, including my own) > > > > > > On Sat, 15 Feb 2020 at 01:54, Alan Mikhak wrote: > > > > > > > > Hi.. > > > > > > > > What is the right approach for adding MSI support for the generic > > > > Linux PCI host driver? > > > > > > > > I came across this patch which seems to address a similar > > > > situation. It seems to have been dropped in v3 of the patchset > > > > with the explanation "drop MSI patch [for now], since it > > > > turns out we may not need it". > > > > > > > > [PATCH 2/3] pci: designware: add separate driver for the MSI part of the RC > > > > https://lore.kernel.org/linux-pci/20170821192907.8695-3-ard.biesheuvel@linaro.org/ > > > > > > > > [PATCH v2 2/3] pci: designware: add separate driver for the MSI part of the RC > > > > https://lore.kernel.org/linux-pci/20170824184321.19432-3-ard.biesheuvel@linaro.org/ > > > > > > > > [PATCH v3 0/2] pci: add support for firmware initialized designware RCs > > > > https://lore.kernel.org/linux-pci/20170828180437.2646-1-ard.biesheuvel@linaro.org/ > > > > > > > > > > For the platform in question, it turned out that we could use the MSI > > > block of the core's GIC interrupt controller directly, which is a much > > > better solution. > > > > > > In general, turning MSIs into wired interrupts is not a great idea, > > > since the whole point of MSIs is that they are sufficiently similar to > > > other DMA transactions to ensure that the interrupt won't arrive > > > before the related memory transactions have completed. > > > > > > If your interrupt controller does not have this capability, then yes, > > > you are stuck with this little widget that decodes an inbound write to > > > a magic address and turns it into a wired interrupt. > > > > I can only second this. It is much better to have a generic block > > implementing MSI *in a non multiplexed way*, for multiple reasons: > > > > - the interrupt vs DMA race that Ard mentions above, > > > > - MSIs are very often used to describe the state of per-CPU queues. If > > you multiplex MSIs behind a single multiplexing interrupt, it is > > always the same CPU that gets interrupted, and you don't benefit > > from having multiple queues at all. > > > > Even if you have to implement the support as a bunch of wired > > interrupts, there is still a lot of value in keeping a 1:1 mapping > > between MSIs and wires. > > > > Thanks, > > > > M. > > > > -- > > Jazz is not dead, it just smells funny. > > Ard and Marc, Thanks for you comments. I will take a look at the code > related to MSI block of GIC interrupt controller for some reference. GICv2m or GICv3 MBI are probably your best bets. Don't get anywhere near the GICv3 ITS, there lies madness. ;-) > I am looking into supporting MSI in a non-multiplexed way when using > ECAM and the generic Linux PCI host driver when Linux is booted > from U-Boot. I don't really get the relationship between ECAM and MSIs. They should be fairly independent, unless that has to do with the allowing the MSI doorbell to be reached from the PCIe endpoint. > Specifically, what is the right approach for sharing the physical > address of the MSI data block used in Linux with U-Boot? > > I imagine the Linux driver for MSI interrupt controller allocates > some DMA-able memory for use as the MSI data block. The > U-Boot PCIe driver would program an inbound ATU region to > map mem writes from endpoint devices to that MSI data block > before booting Linux. The "MSI block" is really a piece of HW, not memory. So whatever you have to program in the PCIe RC must allow an endpoint to reach that device with a 32bit write. M. -- Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel