From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48D9EC4332F for ; Fri, 17 Dec 2021 22:02:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0A47084D62; Fri, 17 Dec 2021 22:02:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iyirVcIBZJL9; Fri, 17 Dec 2021 22:02:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9AB4184D06; Fri, 17 Dec 2021 22:02:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A81B0C002F; Fri, 17 Dec 2021 22:02:02 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 24A83C0070 for ; Fri, 17 Dec 2021 22:02:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B1B0140C36 for ; Fri, 17 Dec 2021 22:01:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id acP6ZcysRANo for ; Fri, 17 Dec 2021 22:01:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp2.osuosl.org (Postfix) with ESMTPS id BCA6740192 for ; Fri, 17 Dec 2021 22:01:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639778518; x=1671314518; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jOdSxuEtbkAQOM8wsD7B/5P8akQ7YsOBzOXhYbomw+g=; b=Xekkpdkp/UBGxz2JA14VMMiV/zszXYxb8AmWFYOG25hiRB08pf0sqFDp ye2xk2ba93ckCgl2BKFGLQsZhg6t8DLmJGOUCNrnOIQ19AV5Y03eviS+A TFieikkaE+Us7gShAhBKJ16HcDkbPOp3beuSwVz2EsWlKoahZBIhsBBCD hXzEcz1oOoHLcol7RIShmYijcqEqVTImBx/3s7OPJit6726GYmROIo2Is txg9PrvpmUiAp3LtgiZxZKsswtCjuajhYe8rWW9rEUdiimtizPOB9L0qD ptHyxbuG82RSUwSxliKEns/R0m1LoGYZJUX5zz4cpV+bI3LLEwMUF1e9v g==; X-IronPort-AV: E=McAfee;i="6200,9189,10201"; a="239793603" X-IronPort-AV: E=Sophos;i="5.88,214,1635231600"; d="scan'208";a="239793603" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Dec 2021 14:01:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,214,1635231600"; d="scan'208";a="506928113" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga007.jf.intel.com with ESMTP; 17 Dec 2021 14:01:51 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Dave Hansen" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Subject: [PATCH v2 11/11] docs: x86: Change documentation for SVA (Shared Virtual Addressing) Date: Fri, 17 Dec 2021 22:01:36 +0000 Message-Id: <20211217220136.2762116-12-fenghua.yu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20211217220136.2762116-1-fenghua.yu@intel.com> References: <20211217220136.2762116-1-fenghua.yu@intel.com> MIME-Version: 1.0 Cc: Fenghua Yu , iommu@lists.linux-foundation.org, x86 , linux-kernel X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Since allocating, freeing and fixing up PASID are changed, the documentation is updated to reflect the changes. Originally-by: Ashok Raj Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck --- v2: - Update life cycle info (Tony and Thomas). - Update initial PASID value to INVALID_IOASID on fork(). Documentation/x86/sva.rst | 58 +++++++++++++++++++++++++++++++-------- 1 file changed, 46 insertions(+), 12 deletions(-) diff --git a/Documentation/x86/sva.rst b/Documentation/x86/sva.rst index 076efd51ef1f..92341f26e525 100644 --- a/Documentation/x86/sva.rst +++ b/Documentation/x86/sva.rst @@ -104,18 +104,52 @@ The MSR must be configured on each logical CPU before any application thread can interact with a device. Threads that belong to the same process share the same page tables, thus the same MSR value. -PASID is cleared when a process is created. The PASID allocation and MSR -programming may occur long after a process and its threads have been created. -One thread must call iommu_sva_bind_device() to allocate the PASID for the -process. If a thread uses ENQCMD without the MSR first being populated, a #GP -will be raised. The kernel will update the PASID MSR with the PASID for all -threads in the process. A single process PASID can be used simultaneously -with multiple devices since they all share the same address space. - -One thread can call iommu_sva_unbind_device() to free the allocated PASID. -The kernel will clear the PASID MSR for all threads belonging to the process. - -New threads inherit the MSR value from the parent. +PASID Life Cycle Management +========================== + +PASID is initialized as INVALID_IOASID (-1) when a process is created. + +Only processes that access SVA-capable devices need to have a PASID +allocated. This allocation happens when a process opens/binds an SVA-capable +device but finds no PASID for this process. Subsequent binds of the same, or +other devices will share the same PASID. + +Although the PASID is allocated to the process by opening a device, +it is not active in any of the threads of that process. It's loaded to the +IA32_PASID MSR lazily when a thread tries to submit a work descriptor +to a device using the ENQCMD. + +That first access will trigger a #GP fault because the IA32_PASID MSR +has not been initialized with the PASID value assigned to the process +when the device was opened. The Linux #GP handler notes that a PASID has +been allocated for the process, and so initializes the IA32_PASID MSR +and returns so that the ENQCMD instruction is re-executed. + +On fork(2) or exec(2) the PASID is removed from the process as it no +longer has the same address space that it had when the device was opened. + +On clone(2) the new task shares the same address space, so will be +able to use the PASID allocated to the process. The IA32_PASID is not +preemptively initialized as the PASID value might not be allocated yet or +the kernel does not know whether this thread is going to access the device +and the cleared IA32_PASID MSR reduces context switch overhead by xstate +init optimization. Since #GP faults have to be handled on any threads that +were created before the PASID was assigned to the mm of the process, newly +created threads might as well be treated in a consistent way. + +Due to complexity of freeing the PASID and clearing all IA32_PASID MSRs in +all threads in unbind, free the PASID lazily only on mm exit. Track the +PASID's reference count in the following way: + +- Initialize the PASID's reference to 1 when the PASID is first allocated + to the mm. The reference is held for the rest life time of the mm until + it's dropped to 0 and the PASID is freed on mm exit. + +If a process does a close(2) of the device file descriptor and munmap(2) +of the device MMIO portal, then the driver will unbind the device. The +PASID is still marked VALID in the PASID_MSR for any threads in the +process that accessed the device. But this is harmless as without the +MMIO portal they cannot submit new work to the device. Relationships ============= -- 2.34.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu