From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A70638C410 for ; Mon, 18 May 2026 20:32:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779136368; cv=none; b=m4DbZ4f19zqQN5/P0u2/rX6IPioK+1jHChQmR/TUyAS9WGIeOO2qCut60Tc+7q3ju+RVMe0ZQwjdefOgxMZOJ2QDbfdUonPjeYlfZ/nXe+MVGUVNcFShw8Mo5yz6WGoqzTfRoqMNgoWu8xIXJkqbkfdo9KvVY1s6o61lN7wbqCQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779136368; c=relaxed/simple; bh=ueB0hMp+FeQZ2rw6x9BVNLt7mojoUSebvdMXIaLRWYA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=iabgzob8jW8WtqCsdqBQaZtJY3O24Q19jEW0ldc4uP9Bcx5PhrkM/nKLi8WARrGYwETFtwbqGhzQLvYHHbIjoR/riYIyJM/CUQJBg+8PH6KePq7EYZRRkN3oL9WPUHWoehcb61FjpYveXIKQZSUrn075cLk0E0RiDD24sB9iY/Q= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VrhT63k5; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VrhT63k5" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2b2e8b95bdbso1045ad.0 for ; Mon, 18 May 2026 13:32:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779136367; x=1779741167; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=TNmFlCcVDbEeItJqA64fw5/4nyffyvxzQppQsiHP4dc=; b=VrhT63k5UIyhuFVcy/F9BQ9j29+SBh/bRxyVsiJkFSEbjimpMN1bUNU5nuu8dZkFg2 H4+ZIBJEW9RlaAJ4lGhubIWoJahDhQQ96JLI06gixutGHqK63bUNYyrbw78xXgJW/Uhb DEk0VDhn+gc6R+3uDh78g7MIUTDB4akMWY9IJIKlySLzw3rn4VI5XNF/5sUoeIaF1gVL BnhqtHGd6ic3sDNzaG+DtYU3DXFvKXHy+PW75A1qkad5rNNOyCagYq2PlDmKyJAfTolg xNVTDwTqbYUG1nAWaiMJmuzn4ieEgjI7mDTqAV5+FXwNG4ezqlogLRPe4Re+8W3KCoaa 2pqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779136367; x=1779741167; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TNmFlCcVDbEeItJqA64fw5/4nyffyvxzQppQsiHP4dc=; b=QfkV0bGxn9571xzDAeearBnLFXctR9cDroED8jIBh+ztze+76CXXBWg7+f/pmiIGZ5 +sc5RRbkT1IPfCz5dAh3dL80btBsn3wYV4lIxHmP+n1uUC7fXK2wSahDhBsViNUF6xAB wJdWJSaWCp9axWHSYtC3ujJwP1y5+UG5ASytNliDj6hPSh5B/YhdobOeDyCbUnyUdIAE mjg9bKI9pliusd/GZ2luS4fX2rmS1M75/2xnKfdCqI8JR4JX+NFY8at7u3HxugV346tq l6CI6iqHqk3ZOrwP1bdzZpnoLlCK8GE4dL6IH9lP84qKdnepQQvu7wfVLVN0sp72pMxV X18w== X-Forwarded-Encrypted: i=1; AFNElJ+J2t46iyssJbEylXxCrOKS0I6huab1D7gE6C9EwGCoVpQSaWeUKtEFBp259mMCsMEepbiFLZQzkaMtI/o=@vger.kernel.org X-Gm-Message-State: AOJu0YyKplzOXp8eeS4UNrGNGDGasIKEgYVl0ZKiSuKKHB/3ndmDPSHC F1sEfeeZvIFeUMxxVpRoCmFYl/x3N0VNdW+t4m0oTDgAmP+glMvLEWqst8adJ/QJ8A== X-Gm-Gg: Acq92OE2DFleIX4xqHeNi+TLJnwrhqACX9v6o5PQlnPxxDFA5WK81ko+5BL7/KJ2o2A 9wnviJX9IXqBFcc3caB8IHQEgfLnLIqAlVmDpL3CjY057dNe4gzBsBkM1EfplwoBCE/NAOf3bub xqG1ocYXZNctQXivp+VVnp3peEXucE9S5dBFRw1z9bO9xU8dNUnAP9q8l1BWVA4cVp05ZVtpCZS KLVdMQGlls7LCbVYuc1MJLjm50QJZs17ew2Yyhl7M2U/hrWyzeANxEfaC/CIad/txBG3t3fZ89f Woi52my+DfkEdO/nbVRGjW7PeUE7fs2naz7OWQcduoN4SVNob0+nAKUNAEcYh2qrEVhntbYVkJe Z7z8Ng2CI/GYeZr6fVoL4LHbw36FjywylgJ7uWaXKOo6kDpIDK2YHae97boXVswOzA+c5u2kae9 MrRNx0bG+ZNszyoJu97DuLwXEefYjj+4piEj245/wdjxvDA8opYksR5oJiNUzdFwVf+t/pkg== X-Received: by 2002:a17:903:17c3:b0:2ba:4749:c9a2 with SMTP id d9443c01a7336-2bdb03a17e1mr3609045ad.2.1779136365901; Mon, 18 May 2026 13:32:45 -0700 (PDT) Received: from google.com (153.46.83.34.bc.googleusercontent.com. [34.83.46.153]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-369514361b6sm11821958a91.12.2026.05.18.13.32.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 May 2026 13:32:45 -0700 (PDT) Date: Mon, 18 May 2026 20:32:42 +0000 From: Samiullah Khawaja To: Baolu Lu Cc: David Woodhouse , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Pranjal Shrivastava , Vipin Sharma , YiFei Zhu Subject: Re: [PATCH v2 07/16] iommu/vt-d: Implement device and iommu preserve/unpreserve ops Message-ID: References: <20260427175633.1978233-1-skhawaja@google.com> <20260427175633.1978233-8-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: On Fri, May 08, 2026 at 02:36:56AM +0000, Samiullah Khawaja wrote: >On Thu, May 07, 2026 at 02:25:14PM +0800, Baolu Lu wrote: >>On 4/28/26 01:56, Samiullah Khawaja wrote: >>>Add implementation of the device and iommu presevation in a separate >>>file. Also set the device and iommu preserve/unpreserve ops in the >>>struct iommu_ops. >>> >>>During normal shutdown the iommu translation is disabled. Since the root >>>table is preserved during live update, it needs to be cleaned up and the >>>context entries of the unpreserved devices need to be cleared. >> >>This is not related to preserve/unpreserve ops and could be made in a >>separated patch? > >Agreed. I will move this stuff to a separate patch. >> >>> >>>Signed-off-by: Samiullah Khawaja >>>--- >>> MAINTAINERS | 1 + >>> drivers/iommu/intel/Makefile | 1 + >>> drivers/iommu/intel/iommu.c | 52 +++++++++++- >>> drivers/iommu/intel/iommu.h | 28 +++++++ >>> drivers/iommu/intel/liveupdate.c | 139 +++++++++++++++++++++++++++++++ >>> drivers/iommu/iommu.c | 18 ++++ >>> include/linux/iommu-liveupdate.h | 10 +++ >>> include/linux/iommu.h | 14 ++++ >>> include/linux/kho/abi/iommu.h | 18 ++++ >>> 9 files changed, 277 insertions(+), 4 deletions(-) >>> create mode 100644 drivers/iommu/intel/liveupdate.c >>> [snip] >> >>>+{ >>>+ struct context_entry *context; >>>+ int ret; >>>+ int i; >>>+ >>>+ for (i = 0; i < ROOT_ENTRY_NR; i++) { >>>+ /* >>>+ * Alloc the context tables now to make sure the iommu unit is >>>+ * properly preserved. These might stay unused and wastes around >>>+ * 32MB max in scalable mode. >>>+ */ >> >>Instead of allocating and preserving context tables for all root entries >>(as noted, can waste up to 32MB), could we restrict this only to the >>entries possibly in use by active PCI devices? > >I think the hotplug devices or VFs created through SR-IOV will be missed >that way. Lets say device A is preserved and the associated iommu is >also preserved. And then a new device B is hotplugged and preserved, >then the context table for that will be missed. Ok I thought about it a little more and basically we have following things to consider when we preserve context tables, - The devices can be hotplugged and preserved, so the context tables of those need to be preserved if we don't allocate all of them first time we preserve iommu, as done here. - New context tables can be added (after hotplug) for unpreserved devices. And if we don't get another iommu preserve call after these are added, those remain unpreserved, so during shutdown those entries need to be removed from root table or preserved for simplicity. To solve this we can, 1. Either preserve the new context table when it is added for a preserved iommu. This can be done in iommu_context_addr(). This is simpler and no tracking needed. 2. Or track the preserved context tables using a bitmap and then preserve them incremently whenever a device is preserved. On shutdown during cleanup, we can clear the entries for unpreserved context tables from root table. I am inclined towards second option. WDYT? I think we will have to do similar stuff for PASID also down the road to preserve pasid_tables in PASID directory. > >Since we don't track the context_tables that are preserved, there is no >way to incremently preserve the new-ones. Let me look into the behaviour >of KHO, maybe we can make the preserve call idempotent and do these >incrementally. >> >>>+ spin_lock(&iommu->lock); >>>+ context = iommu_context_addr(iommu, i, 0, 1); >>>+ spin_unlock(&iommu->lock); >>>+ if (!context) { >>>+ ret = -ENOMEM; >>>+ goto error; >>>+ } [snip] >> >>Thanks, >>baolu >> > >Thanks, >Sami Sami