From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 682CDC433FE for ; Thu, 24 Nov 2022 14:56:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229448AbiKXO4f (ORCPT ); Thu, 24 Nov 2022 09:56:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229613AbiKXO4b (ORCPT ); Thu, 24 Nov 2022 09:56:31 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1831B827DF; Thu, 24 Nov 2022 06:56:30 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AF0986219D; Thu, 24 Nov 2022 14:56:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1828DC433C1; Thu, 24 Nov 2022 14:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669301789; bh=4ZTYBdgCLzsXOsSCItv0xvxw64IzWm2GzGfVZWPLYl0=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Mvb47uwHPhgA+36dMpA/V8iqG5Wfglh6VjTrBTeYa5OhxWqwfYsLlZl26Xt7ux/Bf Sy8cEH453/FiuonpLlhcL3o1RLpTMn7FLDvGw1728E1gnntJyVq/PznvgGIFPSGrg3 ZMfKZci7wL7pQJhiKMZt5dVfTT09K6AiD/lXavZ9/QDYTwUpODwApqSfM6+wWG/LtW 7pcIzQedDfqw25MQykyJwAWWOlrIIjQEBx2v1iiSgpx7CfpnfHbqr+szD9MJAFEaj8 jFha/FZUtgY4HlndX1UmeeJrwCVyH8mhLkLZMAcAZBfaNy0o0a4PvMpR65uLX97XYp JApzWZd1R6sRw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oyDeM-008P1i-Pr; Thu, 24 Nov 2022 14:56:26 +0000 Date: Thu, 24 Nov 2022 14:56:25 +0000 Message-ID: <86sfi8mmee.wl-maz@kernel.org> From: Marc Zyngier To: Thomas Gleixner Cc: LKML , x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: Re: [patch V2 08/21] genirq/msi: Add pointers for per device irq domains In-Reply-To: <20221121083325.950255253@linutronix.de> References: <20221121083210.309161925@linutronix.de> <20221121083325.950255253@linutronix.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: tglx@linutronix.de, linux-kernel@vger.kernel.org, x86@kernel.org, joro@8bytes.org, will@kernel.org, linux-pci@vger.kernel.org, bhelgaas@google.com, lorenzo.pieralisi@arm.com, gregkh@linuxfoundation.org, jgg@mellanox.com, dave.jiang@intel.com, alex.williamson@redhat.com, kevin.tian@intel.com, dan.j.williams@intel.com, logang@deltatee.com, ashok.raj@intel.com, jdmason@kudzu.us, allenbh@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Mon, 21 Nov 2022 14:36:28 +0000, Thomas Gleixner wrote: > > With the upcoming per device MSI interrupt domain support it is necessary > to store the domain pointers per device. > > Instead of delegating that storage to device drivers or subsystems create a > storage array in struct msi_device_data which will also take care of > tearing down the irq domains when msi_device_data is cleaned up via devres. > > The interfaces into the MSI core will be changed from irqdomain pointer > based interfaces to domain id based interfaces to support multiple MSI > domains on a single device (e.g. PCI/MSI[-X] and PCI/IMS. > > Once the per device domain support is complete the irq domain pointer in > struct device::msi.domain will not longer contain a pointer to the "global" > MSI domain. It will contain a pointer to the MSI parent domain instead. > > It would be a horrible maze of conditionals to evaluate all over the place > which domain pointer should be used, i.e. the "global" one in > device::msi::domain or one from the internal pointer array. > > To avoid this evaluate in msi_setup_device_data() whether the irq domain > which is associated to a device is a "global" or a parent MSI domain. If it > is global then copy the pointer into the first entry in the irqdomain > pointer array. > > This allows to convert interfaces and implementation to domain ids while > keeping everything existing working. > > Signed-off-by: Thomas Gleixner > --- > include/linux/msi.h | 3 +++ > include/linux/msi_api.h | 8 ++++++++ > kernel/irq/msi.c | 14 ++++++++++++++ > 3 files changed, 25 insertions(+) > > --- a/include/linux/msi.h > +++ b/include/linux/msi.h > @@ -77,6 +77,7 @@ struct msi_desc; > struct pci_dev; > struct platform_msi_priv_data; > struct device_attribute; > +struct irq_domain; > > void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg); > #ifdef CONFIG_GENERIC_MSI_IRQ > @@ -180,6 +181,7 @@ enum msi_desc_filter { > * @mutex: Mutex protecting the MSI descriptor store > * @__store: Xarray for storing MSI descriptor pointers > * @__iter_idx: Index to search the next entry for iterators > + * @__irqdomains: Per device interrupt domains > */ > struct msi_device_data { > unsigned long properties; > @@ -187,6 +189,7 @@ struct msi_device_data { > struct mutex mutex; > struct xarray __store; > unsigned long __iter_idx; > + struct irq_domain *__irqdomains[MSI_MAX_DEVICE_IRQDOMAINS]; > }; > > int msi_setup_device_data(struct device *dev); > --- a/include/linux/msi_api.h > +++ b/include/linux/msi_api.h > @@ -10,6 +10,14 @@ > > struct device; > > +/* > + * Per device interrupt domain related constants. > + */ > +enum msi_domain_ids { > + MSI_DEFAULT_DOMAIN, > + MSI_MAX_DEVICE_IRQDOMAINS, > +}; > + > unsigned int msi_get_virq(struct device *dev, unsigned int index); > > #endif > --- a/kernel/irq/msi.c > +++ b/kernel/irq/msi.c > @@ -21,6 +21,18 @@ > > static inline int msi_sysfs_create_group(struct device *dev); > > +static inline void msi_setup_default_irqdomain(struct device *dev, struct msi_device_data *md) Do we really need this to be inline? I'm sure the compiler can figure it out. > +{ > + if (!dev->msi.domain) > + return; > + /* > + * If @dev::msi::domain is a global MSI domain, copy the pointer > + * into the domain array to avoid conditionals all over the place. > + */ > + if (!irq_domain_is_msi_parent(dev->msi.domain)) > + md->__irqdomains[MSI_DEFAULT_DOMAIN] = dev->msi.domain; > +} > + > /** > * msi_alloc_desc - Allocate an initialized msi_desc > * @dev: Pointer to the device for which this is allocated > @@ -213,6 +225,8 @@ int msi_setup_device_data(struct device > return ret; > } > > + msi_setup_default_irqdomain(dev, md); > + nit: if you move the setup below the msi.data assignment, you could only pass dev as a parameter. Or pass both and move the assignment in the function? > xa_init(&md->__store); > mutex_init(&md->mutex); > dev->msi.data = md; > > Irrespective of the above, Reviewed-by: Marc Zyngier M. -- Without deviation from the norm, progress is not possible.