From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DEA5C433E7 for ; Tue, 1 Sep 2020 03:40:15 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F34BA206FA for ; Tue, 1 Sep 2020 03:40:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F34BA206FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 941D120483; Tue, 1 Sep 2020 03:40:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7PVsjYr7i9Z9; Tue, 1 Sep 2020 03:40:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 482B520424; Tue, 1 Sep 2020 03:40:13 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2D333C088B; Tue, 1 Sep 2020 03:40:13 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id BFCEEC0052 for ; Tue, 1 Sep 2020 03:40:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A7A9386FBB for ; Tue, 1 Sep 2020 03:40:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wN9reU3bXtnx for ; Tue, 1 Sep 2020 03:40:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by hemlock.osuosl.org (Postfix) with ESMTPS id 9D6298643B for ; Tue, 1 Sep 2020 03:40:10 +0000 (UTC) IronPort-SDR: EYutX5n5l1moNKHH0q9vixjT2GopmCa5n1WqtRuGJnrdYMzRgbn/ru+iv7nKtXlg9OacvA3KHn BeCiCsLu7Fkw== X-IronPort-AV: E=McAfee;i="6000,8403,9730"; a="144843766" X-IronPort-AV: E=Sophos;i="5.76,377,1592895600"; d="scan'208";a="144843766" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Aug 2020 20:40:09 -0700 IronPort-SDR: xGboYKzvsJlkHn6PaFg2gJdD6Ho9X+J4xWldMcYK4l8ocnu41LSY2oGnWtKaxhn0DKvjOlZD+S 0KuPivwRFKtQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,377,1592895600"; d="scan'208";a="325180845" Received: from allen-box.sh.intel.com ([10.239.159.139]) by fmsmga004.fm.intel.com with ESMTP; 31 Aug 2020 20:40:06 -0700 From: Lu Baolu To: Joerg Roedel , Alex Williamson Subject: [PATCH v4 0/5] iommu aux-domain APIs extensions Date: Tue, 1 Sep 2020 11:34:17 +0800 Message-Id: <20200901033422.22249-1-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 Cc: Jean-Philippe Brucker , Kevin Tian , Dave Jiang , Ashok Raj , kvm@vger.kernel.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" This series aims to extend the IOMMU aux-domain API set so that it could be more friendly to vfio/mdev usage. The interactions between vfio/mdev and iommu during mdev creation and passthr are: 1. Create a group for mdev with iommu_group_alloc(); 2. Add the device to the group with group = iommu_group_alloc(); if (IS_ERR(group)) return PTR_ERR(group); ret = iommu_group_add_device(group, &mdev->dev); if (!ret) dev_info(&mdev->dev, "MDEV: group_id = %d\n", iommu_group_id(group)); 3. Allocate an aux-domain with iommu_domain_alloc(); 4. Attach the aux-domain to the iommu_group. iommu_group_for_each_dev { if (iommu_dev_feature_enabled(iommu_device, IOMMU_DEV_FEAT_AUX)) return iommu_aux_attach_device(domain, iommu_device); else return iommu_attach_device(domain, iommu_device); } where, iommu_device is the aux-domain-capable device. The mdev's in the group are all derived from it. In the whole process, an iommu group was allocated for the mdev and an iommu domain was attached to the group, but the group->domain leaves NULL. As the result, iommu_get_domain_for_dev() (or other similar interfaces) doesn't work anymore. The iommu_get_domain_for_dev() is a necessary interface for device drivers that want to support vfio/mdev based aux-domain. For example, unsigned long pasid; struct iommu_domain *domain; struct device *dev = mdev_dev(mdev); struct device *iommu_device = vfio_mdev_get_iommu_device(dev); domain = iommu_aux_get_domain_for_dev(dev); if (!domain) return -ENODEV; pasid = iommu_aux_get_pasid(domain, iommu_device); if (pasid <= 0) return -EINVAL; /* Program the device context */ .... We tried to address this by extending iommu_aux_at(de)tach_device() so that the users could pass in an optional device pointer (for example vfio/mdev). (v2 of this series) https://lore.kernel.org/linux-iommu/20200707013957.23672-1-baolu.lu@linux.intel.com/ But that will cause a lock issue as group->mutex has been applied in iommu_group_for_each_dev(), but has to be reapplied again in the iommu_aux_attach_device(). We also tried to implement an equivalent iommu_attch_group() for groups which includes subdevices derived from a single physical device. (v3 of this series) https://lore.kernel.org/linux-iommu/20200714055703.5510-1-baolu.lu@linux.intel.com/ But that's too harsh (requires that all subdevices in an iommu_group must be derived from a same physical device) and breaks some generic concept of iommmu_group. This version continues to address this by introducing some new APIs into the aux-domain API set according to comments during v3 reviewing period. /** * iommu_attach_subdev_group - attach domain to an iommu_group which * contains subdevices. * * @domain: domain * @group: iommu_group which contains subdevices * @fn: callback for each subdevice in the @iommu_group to retrieve the * physical device where the subdevice was created from. * * Returns 0 on success, or an error value. */ int iommu_attach_subdev_group(struct iommu_domain *domain, struct iommu_group *group, iommu_device_lookup_t fn) /** * iommu_detach_subdev_group - detach domain from an iommu_group which * contains subdevices * * @domain: domain * @group: iommu_group which contains subdevices * @fn: callback for each subdevice in the @iommu_group to retrieve the * physical device where the subdevice was created from. * * The domain must have been attached to @group via iommu_attach_subdev_group(). */ void iommu_detach_subdev_group(struct iommu_domain *domain, struct iommu_group *group, iommu_device_lookup_t fn) struct iommu_domain *iommu_aux_get_domain_for_dev(struct device *subdev) This version is evolved according to feedbacks from Robin, Alex and Kevin. I'm very appreciated to their contributions. Best regards, baolu --- Change log: - v1->v2: - https://lore.kernel.org/linux-iommu/20200627031532.28046-1-baolu.lu@linux.intel.com/ - Suggested by Robin. - v2->v3: - https://lore.kernel.org/linux-iommu/20200707013957.23672-1-baolu.lu@linux.intel.com/ - Suggested by Alex, Kevin. - v3->v4: - https://lore.kernel.org/linux-iommu/20200714055703.5510-1-baolu.lu@linux.intel.com/ - Evolve the aux_attach_group APIs to take an iommu_device lookup callback. - Add interface to check whether a domain is aux-domain for a device. - Return domain only if the domain is aux-domain in iommu_aux_get_domain_for_dev(). Lu Baolu (5): iommu: Add optional subdev in aux_at(de)tach ops iommu: Add iommu_at(de)tach_subdev_group() iommu: Add iommu_aux_get_domain_for_dev() vfio/type1: Use iommu_aux_at(de)tach_group() APIs iommu/vt-d: Add is_aux_domain support drivers/iommu/intel/iommu.c | 135 +++++++++++++++++++-------- drivers/iommu/iommu.c | 158 +++++++++++++++++++++++++++++++- drivers/vfio/vfio_iommu_type1.c | 43 ++------- include/linux/intel-iommu.h | 17 ++-- include/linux/iommu.h | 46 +++++++++- 5 files changed, 315 insertions(+), 84 deletions(-) -- 2.17.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu