From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58611C433E4 for ; Fri, 21 Aug 2020 13:16:30 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D2CF20724 for ; Fri, 21 Aug 2020 13:16:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="N+lXLh4H" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D2CF20724 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 1A8F988198; Fri, 21 Aug 2020 13:16:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wiwG9A3JUdZ6; Fri, 21 Aug 2020 13:16:29 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 4E0FA8805E; Fri, 21 Aug 2020 13:16:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 45A0BC0889; Fri, 21 Aug 2020 13:16:29 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0E922C0051 for ; Fri, 21 Aug 2020 13:16:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EB0AB20428 for ; Fri, 21 Aug 2020 13:16:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KqLuxoh-Xj6q for ; Fri, 21 Aug 2020 13:16:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-ed1-f66.google.com (mail-ed1-f66.google.com [209.85.208.66]) by silver.osuosl.org (Postfix) with ESMTPS id 9BDE920442 for ; Fri, 21 Aug 2020 13:16:14 +0000 (UTC) Received: by mail-ed1-f66.google.com with SMTP id cq28so1378066edb.10 for ; Fri, 21 Aug 2020 06:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NxlhAJ1KOAer/ks+xcHyw//iJAXespq5AefJ6dSktes=; b=N+lXLh4HfaQ0PykHmJWAY0wIFAYXmf/kkyLxM6n+tR4lng3Wep0f01oGUHkb3cVCKf gFYq/d+EjZ+hkllmQv9OYJ4Rf39hAngnL69AsRS5PXRNowVzwZ7PLFDUw0C/0W89lqgA BCIm4CBQFIEAxT5+ERAEH6aCBuytNmJtF1ppIsUMzlQZs0XJ0IZpiduM9cW3MT9h5oUC ZmvnjVz4bUwMBR9lKVv5+QrmR9q/q2yMOGYf1NTWWZhWCbU6T7Ln0iMpaG/luqAZE2Fk 6TTvwj1S+mcCYqM9wYawQ3inV9lJ1qqimzG9Po1gngxKQYiECEmXLXQctrXj+WSTwDPK eVfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NxlhAJ1KOAer/ks+xcHyw//iJAXespq5AefJ6dSktes=; b=o2qOFkqF/GQ6suFLnf11R0LAJDD5Bn0+Hd8CVfMuAQjIXO6nQ1F9DG/6fY4fJgul0h wuQUTTdplUcIjsBM3km2y00UzLXQhY/WFt4G0+J9X96arsruC9WtY3u+WnUzoLy6aiGL 7fI3DJUJxYF+TjOV7lmT5CmiWlhOoecHdOWZ6YQ3sma1ftlCChF11ZoLHkyjx6Jpf4pg I1njlnGbgunJDfr1m7Dq87eMpYdaiMmqMbfJ0RScg8sJuVzxShcVGkAo2hcIBG/f99U8 M81XOT/dVtaz5IalmH1otYnFlEpLMwWJe/Ct2k/yvQVhWf7jxzoCCOjjRTJYcKBdLoYv H/tw== X-Gm-Message-State: AOAM530sb4DiUrMHe+JwJ8oT2Jl4v//+Dw2XyfDcoVfJkY11V3jHgJpX sGMeTgUkGuqSukwGL0myEhWWKWaqezwTJDPO X-Google-Smtp-Source: ABdhPJyZPNka7zmqI+7SgdHXEjXp0y2ZbjsHxMhgluktwq4n/jUSPkWoqJlR0OsXxAp22OOeuv0G3A== X-Received: by 2002:aa7:c1ca:: with SMTP id d10mr2750322edp.261.1598015772958; Fri, 21 Aug 2020 06:16:12 -0700 (PDT) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id v4sm1299748eje.39.2020.08.21.06.16.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Aug 2020 06:16:12 -0700 (PDT) From: Jean-Philippe Brucker To: iommu@lists.linux-foundation.org, virtualization@lists.linux-foundation.org, virtio-dev@lists.oasis-open.org, linux-pci@vger.kernel.org Subject: [PATCH v3 5/6] iommu/virtio: Support topology description in config space Date: Fri, 21 Aug 2020 15:15:39 +0200 Message-Id: <20200821131540.2801801-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200821131540.2801801-1-jean-philippe@linaro.org> References: <20200821131540.2801801-1-jean-philippe@linaro.org> MIME-Version: 1.0 Cc: Jean-Philippe Brucker , lorenzo.pieralisi@arm.com, mst@redhat.com, joro@8bytes.org, eric.auger@redhat.com, sebastien.boeuf@intel.com, bhelgaas@google.com X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" Platforms without device-tree nor ACPI can provide a topology description embedded into the virtio config space. Parse it. Use PCI FIXUP to probe the config space early, because we need to discover the topology before any DMA configuration takes place, and the virtio driver may be loaded much later. Since we discover the topology description when probing the PCI hierarchy, the virtual IOMMU cannot manage other platform devices discovered earlier. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Kconfig | 12 ++ drivers/iommu/virtio/Makefile | 1 + drivers/iommu/virtio/topology.c | 259 ++++++++++++++++++++++++++++++++ 3 files changed, 272 insertions(+) create mode 100644 drivers/iommu/virtio/topology.c diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index e29ae50f7100..98d28fdbc19a 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -394,4 +394,16 @@ config VIRTIO_IOMMU config VIRTIO_IOMMU_TOPOLOGY_HELPERS bool +config VIRTIO_IOMMU_TOPOLOGY + bool "Handle topology properties from the virtio-iommu" + depends on VIRTIO_IOMMU + depends on PCI + default y + select VIRTIO_IOMMU_TOPOLOGY_HELPERS + help + Enable early probing of virtio-iommu devices to detect the built-in + topology description. + + Say Y here if you intend to run this kernel as a guest. + endif # IOMMU_SUPPORT diff --git a/drivers/iommu/virtio/Makefile b/drivers/iommu/virtio/Makefile index b42ad47eac7e..1eda8ca1cbbf 100644 --- a/drivers/iommu/virtio/Makefile +++ b/drivers/iommu/virtio/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o +obj-$(CONFIG_VIRTIO_IOMMU_TOPOLOGY) += topology.o obj-$(CONFIG_VIRTIO_IOMMU_TOPOLOGY_HELPERS) += topology-helpers.o diff --git a/drivers/iommu/virtio/topology.c b/drivers/iommu/virtio/topology.c new file mode 100644 index 000000000000..4923eec618b9 --- /dev/null +++ b/drivers/iommu/virtio/topology.c @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: GPL-2.0 +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "topology-helpers.h" + +struct viommu_cap_config { + u8 bar; + u32 length; /* structure size */ + u32 offset; /* structure offset within the bar */ +}; + +struct viommu_topo_header { + u8 type; + u8 reserved; + u16 length; +}; + +static struct virt_topo_endpoint * +viommu_parse_node(void __iomem *buf, size_t len) +{ + int ret = -EINVAL; + union { + struct viommu_topo_header hdr; + struct virtio_iommu_topo_pci_range pci; + struct virtio_iommu_topo_mmio mmio; + } __iomem *cfg = buf; + struct virt_topo_endpoint *spec; + + spec = kzalloc(sizeof(*spec), GFP_KERNEL); + if (!spec) + return ERR_PTR(-ENOMEM); + + switch (ioread8(&cfg->hdr.type)) { + case VIRTIO_IOMMU_TOPO_PCI_RANGE: + if (len < sizeof(cfg->pci)) + goto err_free; + + spec->dev_id.type = VIRT_TOPO_DEV_TYPE_PCI; + spec->dev_id.segment = ioread16(&cfg->pci.segment); + spec->dev_id.bdf_start = ioread16(&cfg->pci.bdf_start); + spec->dev_id.bdf_end = ioread16(&cfg->pci.bdf_end); + spec->endpoint_id = ioread32(&cfg->pci.endpoint_start); + break; + case VIRTIO_IOMMU_TOPO_MMIO: + if (len < sizeof(cfg->mmio)) + goto err_free; + + spec->dev_id.type = VIRT_TOPO_DEV_TYPE_MMIO; + spec->dev_id.base = ioread64(&cfg->mmio.address); + spec->endpoint_id = ioread32(&cfg->mmio.endpoint); + break; + default: + pr_warn("unhandled format 0x%x\n", ioread8(&cfg->hdr.type)); + ret = 0; + goto err_free; + } + return spec; + +err_free: + kfree(spec); + return ERR_PTR(ret); +} + +static int viommu_parse_topology(struct device *dev, + struct virtio_iommu_config __iomem *cfg, + size_t max_len) +{ + int ret; + u16 len; + size_t i; + LIST_HEAD(endpoints); + size_t offset, count; + struct virt_topo_iommu *viommu; + struct virt_topo_endpoint *ep, *next; + struct viommu_topo_header __iomem *cur; + + offset = ioread16(&cfg->topo_config.offset); + count = ioread16(&cfg->topo_config.count); + if (!offset || !count) + return 0; + + viommu = kzalloc(sizeof(*viommu), GFP_KERNEL); + if (!viommu) + return -ENOMEM; + + viommu->dev = dev; + + for (i = 0; i < count; i++, offset += len) { + if (offset + sizeof(*cur) > max_len) { + ret = -EOVERFLOW; + goto err_free; + } + + cur = (void __iomem *)cfg + offset; + len = ioread16(&cur->length); + if (offset + len > max_len) { + ret = -EOVERFLOW; + goto err_free; + } + + ep = viommu_parse_node((void __iomem *)cur, len); + if (!ep) { + continue; + } else if (IS_ERR(ep)) { + ret = PTR_ERR(ep); + goto err_free; + } + + ep->viommu = viommu; + list_add(&ep->list, &endpoints); + } + + list_for_each_entry_safe(ep, next, &endpoints, list) + /* Moves ep to the helpers list */ + virt_topo_add_endpoint(ep); + virt_topo_add_iommu(viommu); + + return 0; +err_free: + list_for_each_entry_safe(ep, next, &endpoints, list) + kfree(ep); + kfree(viommu); + return ret; +} + +#define VPCI_FIELD(field) offsetof(struct virtio_pci_cap, field) + +static inline int viommu_pci_find_capability(struct pci_dev *dev, u8 cfg_type, + struct viommu_cap_config *cap) +{ + int pos; + u8 bar; + + for (pos = pci_find_capability(dev, PCI_CAP_ID_VNDR); + pos > 0; + pos = pci_find_next_capability(dev, pos, PCI_CAP_ID_VNDR)) { + u8 type; + + pci_read_config_byte(dev, pos + VPCI_FIELD(cfg_type), &type); + if (type != cfg_type) + continue; + + pci_read_config_byte(dev, pos + VPCI_FIELD(bar), &bar); + + /* Ignore structures with reserved BAR values */ + if (type != VIRTIO_PCI_CAP_PCI_CFG && bar > 0x5) + continue; + + cap->bar = bar; + pci_read_config_dword(dev, pos + VPCI_FIELD(length), + &cap->length); + pci_read_config_dword(dev, pos + VPCI_FIELD(offset), + &cap->offset); + + return pos; + } + return 0; +} + +static int viommu_pci_reset(struct virtio_pci_common_cfg __iomem *cfg) +{ + u8 status; + ktime_t timeout = ktime_add_ms(ktime_get(), 100); + + iowrite8(0, &cfg->device_status); + while ((status = ioread8(&cfg->device_status)) != 0 && + ktime_before(ktime_get(), timeout)) + msleep(1); + + return status ? -ETIMEDOUT : 0; +} + +static void viommu_pci_parse_topology(struct pci_dev *dev) +{ + int ret; + u32 features; + void __iomem *regs, *common_regs; + struct viommu_cap_config cap = {0}; + struct virtio_pci_common_cfg __iomem *common_cfg; + + /* + * The virtio infrastructure might not be loaded at this point. We need + * to access the BARs ourselves. + */ + ret = viommu_pci_find_capability(dev, VIRTIO_PCI_CAP_COMMON_CFG, &cap); + if (!ret) { + pci_warn(dev, "common capability not found\n"); + return; + } + + if (pci_enable_device_mem(dev)) + return; + + common_regs = pci_iomap(dev, cap.bar, 0); + if (!common_regs) + return; + + common_cfg = common_regs + cap.offset; + + /* Perform the init sequence before we can read the config */ + ret = viommu_pci_reset(common_cfg); + if (ret < 0) { + pci_warn(dev, "unable to reset device\n"); + goto out_unmap_common; + } + + iowrite8(VIRTIO_CONFIG_S_ACKNOWLEDGE, &common_cfg->device_status); + iowrite8(VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER, + &common_cfg->device_status); + + /* Find out if the device supports topology description */ + iowrite32(0, &common_cfg->device_feature_select); + features = ioread32(&common_cfg->device_feature); + + if (!(features & BIT(VIRTIO_IOMMU_F_TOPOLOGY))) { + pci_dbg(dev, "device doesn't have topology description"); + goto out_reset; + } + + ret = viommu_pci_find_capability(dev, VIRTIO_PCI_CAP_DEVICE_CFG, &cap); + if (!ret) { + pci_warn(dev, "device config capability not found\n"); + goto out_reset; + } + + regs = pci_iomap(dev, cap.bar, 0); + if (!regs) + goto out_reset; + + pci_info(dev, "parsing virtio-iommu topology\n"); + ret = viommu_parse_topology(&dev->dev, regs + cap.offset, + pci_resource_len(dev, 0) - cap.offset); + if (ret) + pci_warn(dev, "failed to parse topology: %d\n", ret); + + pci_iounmap(dev, regs); +out_reset: + ret = viommu_pci_reset(common_cfg); + if (ret) + pci_warn(dev, "unable to reset device\n"); +out_unmap_common: + pci_iounmap(dev, common_regs); +} + +/* + * Catch a PCI virtio-iommu implementation early to get the topology description + * before we start probing other endpoints. + */ +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_REDHAT_QUMRANET, 0x1040 + VIRTIO_ID_IOMMU, + viommu_pci_parse_topology); -- 2.28.0 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization