From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jacob Pan Subject: Re: [PATCH 18/18] iommu/vt-d: Add svm/sva invalidate function Date: Tue, 9 Apr 2019 10:43:39 -0700 Message-ID: <20190409104339.311f3ca0@jacob-builder> References: <1554767973-30125-1-git-send-email-jacob.jun.pan@linux.intel.com> <1554767973-30125-19-git-send-email-jacob.jun.pan@linux.intel.com> <20190409145712.GR9224@smile.fi.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20190409145712.GR9224@smile.fi.intel.com> Sender: linux-kernel-owner@vger.kernel.org To: Andriy Shevchenko Cc: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , David Woodhouse , Alex Williamson , Jean-Philippe Brucker , Yi Liu , "Tian, Kevin" , Raj Ashok , Christoph Hellwig , Lu Baolu , Liu@smile.fi.intel.com, Yi L , jacob.jun.pan@linux.intel.com List-Id: iommu@lists.linux-foundation.org On Tue, 9 Apr 2019 17:57:12 +0300 Andriy Shevchenko wrote: > On Mon, Apr 08, 2019 at 04:59:33PM -0700, Jacob Pan wrote: > > When Shared Virtual Address (SVA) is enabled for a guest OS via > > vIOMMU, we need to provide invalidation support at IOMMU API and > > driver level. This patch adds Intel VT-d specific function to > > implement iommu passdown invalidate API for shared virtual address. > > > > The use case is for supporting caching structure invalidation > > of assigned SVM capable devices. Emulated IOMMU exposes queue > > invalidation capability and passes down all descriptors from the > > guest to the physical IOMMU. > > > > The assumption is that guest to host device ID mapping should be > > resolved prior to calling IOMMU driver. Based on the device handle, > > host IOMMU driver can replace certain fields before submit to the > > invalidation queue. > > > +static int intel_iommu_sva_invalidate(struct iommu_domain *domain, > > + struct device *dev, struct > > iommu_cache_invalidate_info *inv_info) +{ > > + struct dmar_domain *dmar_domain = to_dmar_domain(domain); > > + struct device_domain_info *info; > > + struct intel_iommu *iommu; > > + unsigned long flags; > > + int cache_type; > > + u8 bus, devfn; > > + u16 did, sid; > > + int ret = 0; > > + u64 granu; > > + u64 size; > > + > > + if (!inv_info || !dmar_domain || > > + inv_info->version != > > IOMMU_CACHE_INVALIDATE_INFO_VERSION_1) > > + return -EINVAL; > > + > > + iommu = device_to_iommu(dev, &bus, &devfn); > > + if (!iommu) > > + return -ENODEV; > > + > > > + if (!dev || !dev_is_pci(dev)) > > + return -ENODEV; > > How dev is used in above call? Can be dev NULL there optional and > give non-NULL iommu? > Good catch, dev cannot be NULL. I will move the check before device_to_iommu(). > > + switch (1 << cache_type) { > > BIT() ? > > > + case IOMMU_CACHE_INV_TYPE_IOTLB: > > + if (size && (inv_info->addr_info.addr & > > ((1 << (VTD_PAGE_SHIFT + size)) - 1))) { > > BIT() ? > Sounds good for the two BITs. Thanks > > + pr_err("Address out of range, > > 0x%llx, size order %llu\n", > > + inv_info->addr_info.addr, > > size); > > + ret = -ERANGE; > > + goto out_unlock; > > + } > > + > > + qi_flush_piotlb(iommu, did, > > mm_to_dma_pfn(inv_info->addr_info.addr), > > + inv_info->addr_info.pasid, > > + size, granu); > > + > > + /* > > + * Always flush device IOTLB if ATS is > > enabled since guest > > + * vIOMMU exposes CM = 1, no device IOTLB > > flush will be passed > > + * down. REVISIT: cannot assume Linux guest > > + */ > > + if (info->ats_enabled) { > > + qi_flush_dev_piotlb(iommu, sid, > > info->pfsid, > > + > > inv_info->addr_info.pasid, info->ats_qdep, > > + > > inv_info->addr_info.addr, size, > > + granu); > > + } > > + break; > > + case IOMMU_CACHE_INV_TYPE_DEV_IOTLB: > > + if (info->ats_enabled) { > > + qi_flush_dev_piotlb(iommu, sid, > > info->pfsid, > > + > > inv_info->addr_info.pasid, info->ats_qdep, > > + > > inv_info->addr_info.addr, size, > > + granu); > > + } else > > + pr_warn("Passdown device IOTLB > > flush w/o ATS!\n"); + > > + break; > > + case IOMMU_CACHE_INV_TYPE_PASID: > > + qi_flush_pasid_cache(iommu, did, granu, > > inv_info->pasid); + > > + break; > > + default: > > + dev_err(dev, "Unsupported IOMMU > > invalidation type %d\n", > > + cache_type); > > + ret = -EINVAL; > > + } > > > +out_unlock: > > + spin_unlock(&iommu->lock); > > + spin_unlock_irqrestore(&device_domain_lock, flags); > > + > > + return ret; > > +} > [Jacob Pan] From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72901C282DA for ; Tue, 9 Apr 2019 17:41:08 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4E92320857 for ; Tue, 9 Apr 2019 17:41:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E92320857 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 14EF8CD4; Tue, 9 Apr 2019 17:41:08 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 09EEFCCE for ; Tue, 9 Apr 2019 17:41:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 16721866 for ; Tue, 9 Apr 2019 17:41:05 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Apr 2019 10:41:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,330,1549958400"; d="scan'208";a="132822390" Received: from jacob-builder.jf.intel.com (HELO jacob-builder) ([10.7.199.155]) by orsmga008.jf.intel.com with ESMTP; 09 Apr 2019 10:41:04 -0700 Date: Tue, 9 Apr 2019 10:43:39 -0700 From: Jacob Pan To: Andriy Shevchenko Subject: Re: [PATCH 18/18] iommu/vt-d: Add svm/sva invalidate function Message-ID: <20190409104339.311f3ca0@jacob-builder> In-Reply-To: <20190409145712.GR9224@smile.fi.intel.com> References: <1554767973-30125-1-git-send-email-jacob.jun.pan@linux.intel.com> <1554767973-30125-19-git-send-email-jacob.jun.pan@linux.intel.com> <20190409145712.GR9224@smile.fi.intel.com> Organization: OTC X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Cc: Yi L , "Tian, Kevin" , Raj Ashok , Jean-Philippe Brucker , iommu@lists.linux-foundation.org, LKML , Alex Williamson , Liu@smile.fi.intel.com, David Woodhouse X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org Message-ID: <20190409174339.JwL-SbgDhucb-YOAxzhBKLciolRfECV8NjLoAkxSiXk@z> On Tue, 9 Apr 2019 17:57:12 +0300 Andriy Shevchenko wrote: > On Mon, Apr 08, 2019 at 04:59:33PM -0700, Jacob Pan wrote: > > When Shared Virtual Address (SVA) is enabled for a guest OS via > > vIOMMU, we need to provide invalidation support at IOMMU API and > > driver level. This patch adds Intel VT-d specific function to > > implement iommu passdown invalidate API for shared virtual address. > > > > The use case is for supporting caching structure invalidation > > of assigned SVM capable devices. Emulated IOMMU exposes queue > > invalidation capability and passes down all descriptors from the > > guest to the physical IOMMU. > > > > The assumption is that guest to host device ID mapping should be > > resolved prior to calling IOMMU driver. Based on the device handle, > > host IOMMU driver can replace certain fields before submit to the > > invalidation queue. > > > +static int intel_iommu_sva_invalidate(struct iommu_domain *domain, > > + struct device *dev, struct > > iommu_cache_invalidate_info *inv_info) +{ > > + struct dmar_domain *dmar_domain = to_dmar_domain(domain); > > + struct device_domain_info *info; > > + struct intel_iommu *iommu; > > + unsigned long flags; > > + int cache_type; > > + u8 bus, devfn; > > + u16 did, sid; > > + int ret = 0; > > + u64 granu; > > + u64 size; > > + > > + if (!inv_info || !dmar_domain || > > + inv_info->version != > > IOMMU_CACHE_INVALIDATE_INFO_VERSION_1) > > + return -EINVAL; > > + > > + iommu = device_to_iommu(dev, &bus, &devfn); > > + if (!iommu) > > + return -ENODEV; > > + > > > + if (!dev || !dev_is_pci(dev)) > > + return -ENODEV; > > How dev is used in above call? Can be dev NULL there optional and > give non-NULL iommu? > Good catch, dev cannot be NULL. I will move the check before device_to_iommu(). > > + switch (1 << cache_type) { > > BIT() ? > > > + case IOMMU_CACHE_INV_TYPE_IOTLB: > > + if (size && (inv_info->addr_info.addr & > > ((1 << (VTD_PAGE_SHIFT + size)) - 1))) { > > BIT() ? > Sounds good for the two BITs. Thanks > > + pr_err("Address out of range, > > 0x%llx, size order %llu\n", > > + inv_info->addr_info.addr, > > size); > > + ret = -ERANGE; > > + goto out_unlock; > > + } > > + > > + qi_flush_piotlb(iommu, did, > > mm_to_dma_pfn(inv_info->addr_info.addr), > > + inv_info->addr_info.pasid, > > + size, granu); > > + > > + /* > > + * Always flush device IOTLB if ATS is > > enabled since guest > > + * vIOMMU exposes CM = 1, no device IOTLB > > flush will be passed > > + * down. REVISIT: cannot assume Linux guest > > + */ > > + if (info->ats_enabled) { > > + qi_flush_dev_piotlb(iommu, sid, > > info->pfsid, > > + > > inv_info->addr_info.pasid, info->ats_qdep, > > + > > inv_info->addr_info.addr, size, > > + granu); > > + } > > + break; > > + case IOMMU_CACHE_INV_TYPE_DEV_IOTLB: > > + if (info->ats_enabled) { > > + qi_flush_dev_piotlb(iommu, sid, > > info->pfsid, > > + > > inv_info->addr_info.pasid, info->ats_qdep, > > + > > inv_info->addr_info.addr, size, > > + granu); > > + } else > > + pr_warn("Passdown device IOTLB > > flush w/o ATS!\n"); + > > + break; > > + case IOMMU_CACHE_INV_TYPE_PASID: > > + qi_flush_pasid_cache(iommu, did, granu, > > inv_info->pasid); + > > + break; > > + default: > > + dev_err(dev, "Unsupported IOMMU > > invalidation type %d\n", > > + cache_type); > > + ret = -EINVAL; > > + } > > > +out_unlock: > > + spin_unlock(&iommu->lock); > > + spin_unlock_irqrestore(&device_domain_lock, flags); > > + > > + return ret; > > +} > [Jacob Pan] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu