From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jacob Pan Subject: Re: [PATCH 10/18] iommu/vt-d: Add custom allocator for IOASID Date: Mon, 15 Apr 2019 16:10:15 -0700 Message-ID: <20190415161015.564cb071@jacob-builder> References: <1554767973-30125-1-git-send-email-jacob.jun.pan@linux.intel.com> <1554767973-30125-11-git-send-email-jacob.jun.pan@linux.intel.com> <20190415143711.13c29308@x1.home> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8BIT Return-path: In-Reply-To: <20190415143711.13c29308@x1.home> Sender: linux-kernel-owner@vger.kernel.org To: Alex Williamson Cc: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , Yi Liu , "Tian, Kevin" , Raj Ashok , Christoph Hellwig , Lu Baolu , Andriy Shevchenko , jacob.jun.pan@linux.intel.com List-Id: iommu@lists.linux-foundation.org On Mon, 15 Apr 2019 14:37:11 -0600 Alex Williamson wrote: > On Mon, 8 Apr 2019 16:59:25 -0700 > Jacob Pan wrote: > > > When VT-d driver runs in the guest, PASID allocation must be > > performed via virtual command interface. This patch register a > > custom IOASID allocator which takes precedence over the default > > IDR based allocator. The resulting IOASID allocation will always > > come from the host. This ensures that PASID namespace is system- > > wide. > > > > Signed-off-by: Lu Baolu > > Signed-off-by: Liu, Yi L > > Signed-off-by: Jacob Pan > > --- > > drivers/iommu/intel-iommu.c | 50 > > +++++++++++++++++++++++++++++++++++++++++++++ > > include/linux/intel-iommu.h | 1 + 2 files changed, 51 insertions(+) > > > > diff --git a/drivers/iommu/intel-iommu.c > > b/drivers/iommu/intel-iommu.c index 28cb713..a38d774 100644 > > --- a/drivers/iommu/intel-iommu.c > > +++ b/drivers/iommu/intel-iommu.c > > @@ -4820,6 +4820,42 @@ static int __init > > platform_optin_force_iommu(void) return 1; > > } > > > > +static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max, > > void *data) +{ > > + struct intel_iommu *iommu = data; > > + ioasid_t ioasid; > > + > > + if (vcmd_alloc_pasid(iommu, &ioasid)) > > + return INVALID_IOASID; > > + return ioasid; > > How does this honor min/max? > > > +} > > + > > +static int intel_ioasid_free(ioasid_t ioasid, void *data) > > +{ > > + struct iommu_pasid_alloc_info *svm; > > + struct intel_iommu *iommu = data; > > + > > + if (!iommu || !cap_caching_mode(iommu->cap)) > > + return -EINVAL; > > + /* > > + * Sanity check the ioasid owner is done at upper layer, > > e.g. VFIO > > + * We can only free the PASID when all the devices are > > unbond. > > + */ > > + svm = ioasid_find(NULL, ioasid, NULL); > > + if (!svm) { > > + pr_warn("Freeing unbond IOASID %d\n", ioasid); > > + return -EBUSY; > > + } > > + vcmd_free_pasid(iommu, ioasid); > > + > > + return 0; > > +} > > + > > +static struct ioasid_allocator intel_iommu_ioasid_allocator = { > > + .alloc = intel_ioasid_alloc, > > + .free = intel_ioasid_free, > > +}; > > + > > int __init intel_iommu_init(void) > > { > > int ret = -ENODEV; > > @@ -4921,6 +4957,20 @@ int __init intel_iommu_init(void) > > "%s", iommu->name); > > iommu_device_set_ops(&iommu->iommu, > > &intel_iommu_ops); iommu_device_register(&iommu->iommu); > > + if (cap_caching_mode(iommu->cap) && > > sm_supported(iommu)) { > > + /* > > + * Register a custom ASID allocator if we > > are running > > + * in a guest, the purpose is to have a > > system wide PASID > > + * namespace among all PASID users. > > + * Note that only one vIOMMU in each guest > > is supported. > > Why one vIOMMU per guest? This would prevent guests with multiple PCI > domains aiui. > This is mainly for simplicity reasons. These are all virtual BDFs anyway. As long as guest BDF can be mapped to a host BDF, it should be sufficient, am I missing anything? >>From PASID allocation perspective, it is not tied to any PCI device until bind call. We only need to track PASID ownership per guest. virtio-IOMMU spec does support multiple PCI domains but I am not sure if that applies to all assigned devices, whether all assigned devices are under the same domain. Perhaps Jean can help to clarify how PASID allocation API looks like on virtio IOMMU. > > + */ > > + intel_iommu_ioasid_allocator.pdata = (void > > *)iommu; > > + ret = > > ioasid_set_allocator(&intel_iommu_ioasid_allocator); > > + if (ret == -EBUSY) { > > + pr_info("Custom IOASID allocator > > already registered\n"); > > + break; > > + } > > + } > > } > > > > bus_set_iommu(&pci_bus_type, &intel_iommu_ops); > > diff --git a/include/linux/intel-iommu.h > > b/include/linux/intel-iommu.h index b29c85c..bc09d80 100644 > > --- a/include/linux/intel-iommu.h > > +++ b/include/linux/intel-iommu.h > > @@ -31,6 +31,7 @@ > > #include > > #include > > #include > > +#include > > > > #include > > #include > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 768D9C10F0E for ; Mon, 15 Apr 2019 23:07:40 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A8682084B for ; Mon, 15 Apr 2019 23:07:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A8682084B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id F0824AF0; Mon, 15 Apr 2019 23:07:39 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 3F287A67 for ; Mon, 15 Apr 2019 23:07:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B6FDF2C3 for ; Mon, 15 Apr 2019 23:07:38 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Apr 2019 16:07:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,355,1549958400"; d="scan'208";a="223819111" Received: from jacob-builder.jf.intel.com (HELO jacob-builder) ([10.7.199.155]) by orsmga001.jf.intel.com with ESMTP; 15 Apr 2019 16:07:37 -0700 Date: Mon, 15 Apr 2019 16:10:15 -0700 From: Jacob Pan To: Alex Williamson Subject: Re: [PATCH 10/18] iommu/vt-d: Add custom allocator for IOASID Message-ID: <20190415161015.564cb071@jacob-builder> In-Reply-To: <20190415143711.13c29308@x1.home> References: <1554767973-30125-1-git-send-email-jacob.jun.pan@linux.intel.com> <1554767973-30125-11-git-send-email-jacob.jun.pan@linux.intel.com> <20190415143711.13c29308@x1.home> Organization: OTC X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Cc: "Tian, Kevin" , Raj Ashok , Jean-Philippe Brucker , LKML , iommu@lists.linux-foundation.org, Andriy Shevchenko , David Woodhouse X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org Message-ID: <20190415231015.eUGqBB9wYjth5Ltt5MpeOi4tgGdDsJyMKn9oswR14Bk@z> On Mon, 15 Apr 2019 14:37:11 -0600 Alex Williamson wrote: > On Mon, 8 Apr 2019 16:59:25 -0700 > Jacob Pan wrote: > > > When VT-d driver runs in the guest, PASID allocation must be > > performed via virtual command interface. This patch register a > > custom IOASID allocator which takes precedence over the default > > IDR based allocator. The resulting IOASID allocation will always > > come from the host. This ensures that PASID namespace is system- > > wide. > > > > Signed-off-by: Lu Baolu > > Signed-off-by: Liu, Yi L > > Signed-off-by: Jacob Pan > > --- > > drivers/iommu/intel-iommu.c | 50 > > +++++++++++++++++++++++++++++++++++++++++++++ > > include/linux/intel-iommu.h | 1 + 2 files changed, 51 insertions(+) > > > > diff --git a/drivers/iommu/intel-iommu.c > > b/drivers/iommu/intel-iommu.c index 28cb713..a38d774 100644 > > --- a/drivers/iommu/intel-iommu.c > > +++ b/drivers/iommu/intel-iommu.c > > @@ -4820,6 +4820,42 @@ static int __init > > platform_optin_force_iommu(void) return 1; > > } > > > > +static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max, > > void *data) +{ > > + struct intel_iommu *iommu = data; > > + ioasid_t ioasid; > > + > > + if (vcmd_alloc_pasid(iommu, &ioasid)) > > + return INVALID_IOASID; > > + return ioasid; > > How does this honor min/max? > > > +} > > + > > +static int intel_ioasid_free(ioasid_t ioasid, void *data) > > +{ > > + struct iommu_pasid_alloc_info *svm; > > + struct intel_iommu *iommu = data; > > + > > + if (!iommu || !cap_caching_mode(iommu->cap)) > > + return -EINVAL; > > + /* > > + * Sanity check the ioasid owner is done at upper layer, > > e.g. VFIO > > + * We can only free the PASID when all the devices are > > unbond. > > + */ > > + svm = ioasid_find(NULL, ioasid, NULL); > > + if (!svm) { > > + pr_warn("Freeing unbond IOASID %d\n", ioasid); > > + return -EBUSY; > > + } > > + vcmd_free_pasid(iommu, ioasid); > > + > > + return 0; > > +} > > + > > +static struct ioasid_allocator intel_iommu_ioasid_allocator = { > > + .alloc = intel_ioasid_alloc, > > + .free = intel_ioasid_free, > > +}; > > + > > int __init intel_iommu_init(void) > > { > > int ret = -ENODEV; > > @@ -4921,6 +4957,20 @@ int __init intel_iommu_init(void) > > "%s", iommu->name); > > iommu_device_set_ops(&iommu->iommu, > > &intel_iommu_ops); iommu_device_register(&iommu->iommu); > > + if (cap_caching_mode(iommu->cap) && > > sm_supported(iommu)) { > > + /* > > + * Register a custom ASID allocator if we > > are running > > + * in a guest, the purpose is to have a > > system wide PASID > > + * namespace among all PASID users. > > + * Note that only one vIOMMU in each guest > > is supported. > > Why one vIOMMU per guest? This would prevent guests with multiple PCI > domains aiui. > This is mainly for simplicity reasons. These are all virtual BDFs anyway. As long as guest BDF can be mapped to a host BDF, it should be sufficient, am I missing anything? >From PASID allocation perspective, it is not tied to any PCI device until bind call. We only need to track PASID ownership per guest. virtio-IOMMU spec does support multiple PCI domains but I am not sure if that applies to all assigned devices, whether all assigned devices are under the same domain. Perhaps Jean can help to clarify how PASID allocation API looks like on virtio IOMMU. > > + */ > > + intel_iommu_ioasid_allocator.pdata = (void > > *)iommu; > > + ret = > > ioasid_set_allocator(&intel_iommu_ioasid_allocator); > > + if (ret == -EBUSY) { > > + pr_info("Custom IOASID allocator > > already registered\n"); > > + break; > > + } > > + } > > } > > > > bus_set_iommu(&pci_bus_type, &intel_iommu_ops); > > diff --git a/include/linux/intel-iommu.h > > b/include/linux/intel-iommu.h index b29c85c..bc09d80 100644 > > --- a/include/linux/intel-iommu.h > > +++ b/include/linux/intel-iommu.h > > @@ -31,6 +31,7 @@ > > #include > > #include > > #include > > +#include > > > > #include > > #include > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu