From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4A26C433DB for ; Wed, 13 Jan 2021 02:59:34 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7704223133 for ; Wed, 13 Jan 2021 02:59:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7704223133 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=D45YEw9J22EEeqcpWyyTer6xoPZDiWh98WjtSGonM8A=; b=qLrruzVWmDN6RvdXShK8+a7SI T2FuGZDdjWaSKKMLP/esYWzpldpI7pQH8gAhhZ1hrTIwdzL/GSPys8sYH5dnUBFewktUQBWs7Orth FScxuQA4DgZFYYpxGxrVqA68fuUWpMzhfFJT+zWrlBsvIwzhcbvAA0Ci8agChnqOeNKFz7h0VykKC XFDzymIRpeP6quW01as0EI6da4DfMyr7ELHm1IxmrKMYqI+mBGFKzN0p4TOsh+jZNfQt5vpk8s2AV wDd/Sqww2fmJmgCjNuvv5s3g0AD7EvraMUse2NqoQVPvhN3WkQec2QbpKdOGBgt6MF6z2xBLvbK9K x3DGYspHQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kzWM9-0001zp-Jw; Wed, 13 Jan 2021 02:57:57 +0000 Received: from mga09.intel.com ([134.134.136.24]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kzWM5-0001xs-E6 for linux-arm-kernel@lists.infradead.org; Wed, 13 Jan 2021 02:57:55 +0000 IronPort-SDR: PyQS8F1im9xBMEDsOqIbIfxnOYgM9OG5RYI+CaZtivpgyIfYKIKg2UHr4IAwbWdGtcg57WZUzQ 7KIIdmycPj7Q== X-IronPort-AV: E=McAfee;i="6000,8403,9862"; a="178289974" X-IronPort-AV: E=Sophos;i="5.79,343,1602572400"; d="scan'208";a="178289974" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jan 2021 18:57:50 -0800 IronPort-SDR: +X9T3MaFFO67nRlp0dxZ8TxMXkHxjJOAiOBc3En2s6MlrikQP6movdXK64uL82EtWckUi7GFJu VUrSsRsxGT6g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,343,1602572400"; d="scan'208";a="464751072" Received: from allen-box.sh.intel.com (HELO [10.239.159.28]) ([10.239.159.28]) by fmsmga001.fm.intel.com with ESMTP; 12 Jan 2021 18:57:44 -0800 Subject: Re: [PATCH v9 03/10] iommu: Separate IOMMU_DEV_FEAT_IOPF from IOMMU_DEV_FEAT_SVA To: Jean-Philippe Brucker References: <20210108145217.2254447-1-jean-philippe@linaro.org> <20210108145217.2254447-4-jean-philippe@linaro.org> <4de8ef03-a2ed-316e-d3e3-6b8474e20113@linux.intel.com> From: Lu Baolu Message-ID: Date: Wed, 13 Jan 2021 10:49:39 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210112_215753_930250_971A9AAA X-CRM114-Status: GOOD ( 31.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Greg Kroah-Hartman , vivek.gautam@arm.com, guohanjun@huawei.com, will@kernel.org, lorenzo.pieralisi@arm.com, joro@8bytes.org, Zhou Wang , linux-acpi@vger.kernel.org, zhangfei.gao@linaro.org, lenb@kernel.org, devicetree@vger.kernel.org, "Tian, Kevin" , Arnd Bergmann , eric.auger@redhat.com, robh+dt@kernel.org, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org, David Woodhouse , rjw@rjwysocki.net, shameerali.kolothum.thodi@huawei.com, iommu@lists.linux-foundation.org, sudeep.holla@arm.com, robin.murphy@arm.com, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Jean, On 1/12/21 5:16 PM, Jean-Philippe Brucker wrote: > Hi Baolu, > > On Tue, Jan 12, 2021 at 12:31:23PM +0800, Lu Baolu wrote: >> Hi Jean, >> >> On 1/8/21 10:52 PM, Jean-Philippe Brucker wrote: >>> Some devices manage I/O Page Faults (IOPF) themselves instead of relying >>> on PCIe PRI or Arm SMMU stall. Allow their drivers to enable SVA without >>> mandating IOMMU-managed IOPF. The other device drivers now need to first >>> enable IOMMU_DEV_FEAT_IOPF before enabling IOMMU_DEV_FEAT_SVA. >>> >>> Signed-off-by: Jean-Philippe Brucker >>> --- >>> Cc: Arnd Bergmann >>> Cc: David Woodhouse >>> Cc: Greg Kroah-Hartman >>> Cc: Joerg Roedel >>> Cc: Lu Baolu >>> Cc: Will Deacon >>> Cc: Zhangfei Gao >>> Cc: Zhou Wang >>> --- >>> include/linux/iommu.h | 20 +++++++++++++++++--- >>> 1 file changed, 17 insertions(+), 3 deletions(-) >>> >>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >>> index 583c734b2e87..701b2eeb0dc5 100644 >>> --- a/include/linux/iommu.h >>> +++ b/include/linux/iommu.h >>> @@ -156,10 +156,24 @@ struct iommu_resv_region { >>> enum iommu_resv_type type; >>> }; >>> -/* Per device IOMMU features */ >>> +/** >>> + * enum iommu_dev_features - Per device IOMMU features >>> + * @IOMMU_DEV_FEAT_AUX: Auxiliary domain feature >>> + * @IOMMU_DEV_FEAT_SVA: Shared Virtual Addresses >>> + * @IOMMU_DEV_FEAT_IOPF: I/O Page Faults such as PRI or Stall. Generally using >>> + * %IOMMU_DEV_FEAT_SVA requires %IOMMU_DEV_FEAT_IOPF, but >>> + * some devices manage I/O Page Faults themselves instead >>> + * of relying on the IOMMU. When supported, this feature >>> + * must be enabled before and disabled after >>> + * %IOMMU_DEV_FEAT_SVA. >> >> Is this only for SVA? We may see more scenarios of using IOPF. For >> example, when passing through devices to user level, the user's pages >> could be managed dynamically instead of being allocated and pinned >> statically. > > Hm, isn't that precisely what SVA does? I don't understand the > difference. That said FEAT_IOPF doesn't have to be only for SVA. It could > later be used as a prerequisite some another feature. For special cases > device drivers can always use the iommu_register_device_fault_handler() > API and handle faults themselves. From the perspective of IOMMU, there is a little difference between these two. For SVA, the page table is from CPU side, so IOMMU only needs to call handle_mm_fault(); For above pass-through case, the page table is from IOMMU side, so the device driver (probably VFIO) needs to register a fault handler and call iommu_map/unmap() to serve the page faults. If we think about the nested mode (or dual-stage translation), it's more complicated since the kernel (probably VFIO) handles the second level page faults, while the first level page faults need to be delivered to user-level guest. Obviously, this hasn't been fully implemented in any IOMMU driver. > >> If @IOMMU_DEV_FEAT_IOPF is defined as generic iopf support, the current >> vendor IOMMU driver support may not enough. > > IOMMU_DEV_FEAT_IOPF on its own doesn't do anything useful, it's mainly a > way for device drivers to probe the IOMMU capability. Granted in patch > 10 the SMMU driver registers the IOPF queue on enable() but that could be > done by FEAT_SVA enable() instead, if we ever repurpose FEAT_IOPF. I have no objection to split IOPF from SVA. Actually we must have this eventually. My concern is that at this stage, the IOMMU drivers only support SVA type of IOPF, a generic IOMMU_DEV_FEAT_IOPF feature might confuse the device drivers which want to add other types of IOPF usage. > > Thanks, > Jean > Best regards, baolu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel