From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0F3FC433DB for ; Fri, 26 Feb 2021 20:11:25 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BDB464F1F for ; Fri, 26 Feb 2021 20:11:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BDB464F1F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B17024F0AC; Fri, 26 Feb 2021 20:11:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gsREgNnxFvQ0; Fri, 26 Feb 2021 20:11:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id DF5944F0C1; Fri, 26 Feb 2021 20:11:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D8590C0016; Fri, 26 Feb 2021 20:11:21 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3279DC0013 for ; Fri, 26 Feb 2021 20:11:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 770434B799 for ; Fri, 26 Feb 2021 20:11:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GxIjfaAc6Zsd for ; Fri, 26 Feb 2021 20:11:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp4.osuosl.org (Postfix) with ESMTPS id DCE964F07A for ; Fri, 26 Feb 2021 20:11:15 +0000 (UTC) IronPort-SDR: JqQvioPIA2EcAHV5xcpc9eOfrMQfwLuvUl7tT0dVrW1pREavaw5nK0FvedlaWfoTT6raozMcJS qGSoodUrYegA== X-IronPort-AV: E=McAfee;i="6000,8403,9907"; a="186039461" X-IronPort-AV: E=Sophos;i="5.81,209,1610438400"; d="scan'208";a="186039461" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 12:11:15 -0800 IronPort-SDR: l8rKYvxAd3WhPCjppPB1z+QYNNOCIF1Jf/BzlyVS9i5076WJM/YC9rcYScxQhFw4ErgXPe4sxP uKISFwCdieeQ== X-IronPort-AV: E=Sophos;i="5.81,209,1610438400"; d="scan'208";a="405109460" Received: from megha-z97x-ud7-th.sc.intel.com ([143.183.85.154]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 26 Feb 2021 12:11:14 -0800 From: Megha Dey To: tglx@linutronix.de Subject: [Patch V2 12/13] irqchip: Add IMS (Interrupt Message Store) driver Date: Fri, 26 Feb 2021 12:11:16 -0800 Message-Id: <1614370277-23235-13-git-send-email-megha.dey@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1614370277-23235-1-git-send-email-megha.dey@intel.com> References: <1614370277-23235-1-git-send-email-megha.dey@intel.com> Cc: alex.williamson@redhat.com, kevin.tian@intel.com, tony.luck@intel.com, dave.jiang@intel.com, ashok.raj@intel.com, kvm@vger.kernel.org, ravi.v.shankar@intel.com, maz@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, jgg@mellanox.com, megha.dey@intel.com, linux-pci@vger.kernel.org, bhelgaas@google.com, dan.j.williams@intel.com, dwmw@amazon.co.uk X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Generic IMS(Interrupt Message Store) irq chips and irq domain implementations for IMS based devices which store the interrupt messages in an array in device memory. Allocation and freeing of interrupts happens via the generic msi_domain_alloc/free_irqs() interface. No special purpose IMS magic required as long as the interrupt domain is stored in the underlying device struct. The irq_set_auxdata() is used to program the pasid into the IMS entry. [Megha: Fixed compile time errors Added necessary dependencies to IMS_MSI_ARRAY config Fixed polarity of IMS_VECTOR_CTRL Added reads after writes to flush writes to device Added set_desc ops to IMS msi domain ops Tested the IMS infrastructure with the IDXD driver] Reviewed-by: Tony Luck Signed-off-by: Thomas Gleixner Signed-off-by: Megha Dey --- drivers/irqchip/Kconfig | 14 +++ drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-ims-msi.c | 211 ++++++++++++++++++++++++++++++++++++ include/linux/irqchip/irq-ims-msi.h | 68 ++++++++++++ 4 files changed, 294 insertions(+) create mode 100644 drivers/irqchip/irq-ims-msi.c create mode 100644 include/linux/irqchip/irq-ims-msi.h diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index e74fa20..2fb0c24 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -586,4 +586,18 @@ config MST_IRQ help Support MStar Interrupt Controller. +config IMS_MSI + depends on PCI + select DEVICE_MSI + bool + +config IMS_MSI_ARRAY + bool "IMS Interrupt Message Store MSI controller for device memory storage arrays" + depends on PCI + select IMS_MSI + select GENERIC_MSI_IRQ_DOMAIN + help + Support for IMS Interrupt Message Store MSI controller + with IMS slot storage in a slot array in device memory + endmenu diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index c59b95a..e903201 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -113,3 +113,4 @@ obj-$(CONFIG_LOONGSON_PCH_MSI) += irq-loongson-pch-msi.o obj-$(CONFIG_MST_IRQ) += irq-mst-intc.o obj-$(CONFIG_SL28CPLD_INTC) += irq-sl28cpld.o obj-$(CONFIG_MACH_REALTEK_RTL) += irq-realtek-rtl.o +obj-$(CONFIG_IMS_MSI) += irq-ims-msi.o diff --git a/drivers/irqchip/irq-ims-msi.c b/drivers/irqchip/irq-ims-msi.c new file mode 100644 index 0000000..fa23207 --- /dev/null +++ b/drivers/irqchip/irq-ims-msi.c @@ -0,0 +1,211 @@ +// SPDX-License-Identifier: GPL-2.0 +// (C) Copyright 2021 Thomas Gleixner +/* + * Shared interrupt chips and irq domains for IMS devices + */ +#include +#include +#include +#include +#include + +#include + +#ifdef CONFIG_IMS_MSI_ARRAY + +struct ims_array_data { + struct ims_array_info info; + unsigned long map[0]; +}; + +static inline void iowrite32_and_flush(u32 value, void __iomem *addr) +{ + iowrite32(value, addr); + ioread32(addr); +} + +static void ims_array_mask_irq(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 __iomem *ctrl = &slot->ctrl; + + iowrite32_and_flush(ioread32(ctrl) | IMS_CTRL_VECTOR_MASKBIT, ctrl); +} + +static void ims_array_unmask_irq(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 __iomem *ctrl = &slot->ctrl; + + iowrite32_and_flush(ioread32(ctrl) & ~IMS_CTRL_VECTOR_MASKBIT, ctrl); +} + +static void ims_array_write_msi_msg(struct irq_data *data, struct msi_msg *msg) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + + iowrite32(msg->address_lo, &slot->address_lo); + iowrite32(msg->address_hi, &slot->address_hi); + iowrite32_and_flush(msg->data, &slot->data); +} + +static int ims_array_set_auxdata(struct irq_data *data, unsigned int which, + u64 auxval) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->device_msi.priv_iomem; + u32 val, __iomem *ctrl = &slot->ctrl; + + if (which != IMS_AUXDATA_CONTROL_WORD) + return -EINVAL; + if (auxval & ~(u64)IMS_CONTROL_WORD_AUXMASK) + return -EINVAL; + + val = ioread32(ctrl) & IMS_CONTROL_WORD_IRQMASK; + iowrite32_and_flush(val | (u32)auxval, ctrl); + return 0; +} + +static const struct irq_chip ims_array_msi_controller = { + .name = "IMS", + .irq_mask = ims_array_mask_irq, + .irq_unmask = ims_array_unmask_irq, + .irq_write_msi_msg = ims_array_write_msi_msg, + .irq_set_auxdata = ims_array_set_auxdata, + .irq_retrigger = irq_chip_retrigger_hierarchy, + .flags = IRQCHIP_SKIP_SET_WAKE, +}; + +static void ims_array_reset_slot(struct ims_slot __iomem *slot) +{ + iowrite32(0, &slot->address_lo); + iowrite32(0, &slot->address_hi); + iowrite32(0, &slot->data); + iowrite32_and_flush(IMS_CTRL_VECTOR_MASKBIT, &slot->ctrl); +} + +static void ims_array_free_msi_store(struct irq_domain *domain, + struct device *dev) +{ + struct msi_domain_info *info = domain->host_data; + struct ims_array_data *ims = info->data; + struct msi_desc *entry; + + for_each_msi_entry(entry, dev) { + if (entry->device_msi.priv_iomem) { + clear_bit(entry->device_msi.hwirq, ims->map); + ims_array_reset_slot(entry->device_msi.priv_iomem); + entry->device_msi.priv_iomem = NULL; + entry->device_msi.hwirq = 0; + } + } +} + +static int ims_array_alloc_msi_store(struct irq_domain *domain, + struct device *dev, int nvec) +{ + struct msi_domain_info *info = domain->host_data; + struct ims_array_data *ims = info->data; + struct msi_desc *entry; + + for_each_msi_entry(entry, dev) { + unsigned int idx; + + idx = find_first_zero_bit(ims->map, ims->info.max_slots); + if (idx >= ims->info.max_slots) + goto fail; + set_bit(idx, ims->map); + entry->device_msi.priv_iomem = &ims->info.slots[idx]; + ims_array_reset_slot(entry->device_msi.priv_iomem); + entry->device_msi.hwirq = idx; + } + return 0; + +fail: + ims_array_free_msi_store(domain, dev); + return -ENOSPC; +} + +struct ims_array_domain_template { + struct msi_domain_ops ops; + struct msi_domain_info info; +}; + +static void ims_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc) +{ + arg->desc = desc; + arg->hwirq = desc->device_msi.hwirq; +} + +static const struct ims_array_domain_template ims_array_domain_template = { + .ops = { + .msi_alloc_store = ims_array_alloc_msi_store, + .msi_free_store = ims_array_free_msi_store, + .set_desc = ims_set_desc, + }, + .info = { + .flags = MSI_FLAG_USE_DEF_DOM_OPS | + MSI_FLAG_USE_DEF_CHIP_OPS, + .handler = handle_edge_irq, + .handler_name = "edge", + }, +}; + +struct irq_domain * +pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev, + struct ims_array_info *ims_info) +{ + struct ims_array_domain_template *info; + struct ims_array_data *data; + struct irq_domain *domain; + struct irq_chip *chip; + unsigned int size; + + /* Allocate new domain storage */ + info = kmemdup(&ims_array_domain_template, + sizeof(ims_array_domain_template), GFP_KERNEL); + if (!info) + return NULL; + /* Link the ops */ + info->info.ops = &info->ops; + + /* Allocate ims_info along with the bitmap */ + size = sizeof(*data); + size += BITS_TO_LONGS(ims_info->max_slots) * sizeof(unsigned long); + data = kzalloc(size, GFP_KERNEL); + if (!data) + goto err_info; + + data->info = *ims_info; + info->info.data = data; + + /* + * Allocate an interrupt chip because the core needs to be able to + * update it with default callbacks. + */ + chip = kmemdup(&ims_array_msi_controller, + sizeof(ims_array_msi_controller), GFP_KERNEL); + if (!chip) + goto err_data; + info->info.chip = chip; + + domain = pci_subdevice_msi_create_irq_domain(pdev, &info->info); + if (!domain) + goto err_chip; + + return domain; + +err_chip: + kfree(chip); +err_data: + kfree(data); +err_info: + kfree(info); + return NULL; +} +EXPORT_SYMBOL_GPL(pci_ims_array_create_msi_irq_domain); + +#endif /* CONFIG_IMS_MSI_ARRAY */ diff --git a/include/linux/irqchip/irq-ims-msi.h b/include/linux/irqchip/irq-ims-msi.h new file mode 100644 index 0000000..9ba767f --- /dev/null +++ b/include/linux/irqchip/irq-ims-msi.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* (C) Copyright 2021 Thomas Gleixner */ + +#ifndef _LINUX_IRQCHIP_IRQ_IMS_MSI_H +#define _LINUX_IRQCHIP_IRQ_IMS_MSI_H + +#include +#include + +/** + * ims_hw_slot - The hardware layout of an IMS based MSI message + * @address_lo: Lower 32bit address + * @address_hi: Upper 32bit address + * @data: Message data + * @ctrl: Control word + * + * This structure is used by both the device memory array and the queue + * memory variants of IMS. + */ +struct ims_slot { + u32 address_lo; + u32 address_hi; + u32 data; + u32 ctrl; +} __packed; + +/* + * The IMS control word utilizes bit 0-2 for interrupt control. The remaining + * bits can contain auxiliary data. + */ +#define IMS_CONTROL_WORD_IRQMASK GENMASK(2, 0) +#define IMS_CONTROL_WORD_AUXMASK GENMASK(31, 3) + +/* Auxiliary control word data related defines */ +enum { + IMS_AUXDATA_CONTROL_WORD, +}; + +/* Bit to mask the interrupt in ims_hw_slot::ctrl */ +#define IMS_CTRL_VECTOR_MASKBIT BIT(0) +#define IMS_CTRL_PASID_ENABLE BIT(3) +#define IMS_CTRL_PASID_SHIFT 12 + +/* Set pasid and enable bit for the IMS entry */ +static inline u32 ims_ctrl_pasid_aux(unsigned int pasid, bool enable) +{ + u32 auxval = pasid << IMS_CTRL_PASID_SHIFT; + + return enable ? auxval | IMS_CTRL_PASID_ENABLE : auxval; +} + +/** + * struct ims_array_info - Information to create an IMS array domain + * @slots: Pointer to the start of the array + * @max_slots: Maximum number of slots in the array + */ +struct ims_array_info { + struct ims_slot __iomem *slots; + unsigned int max_slots; +}; + +struct pci_dev; +struct irq_domain; + +struct irq_domain *pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev, + struct ims_array_info *ims_info); + +#endif -- 2.7.4 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu