From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15A0D37472F for ; Fri, 20 Mar 2026 18:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774030357; cv=none; b=DPv1Zz/G6ucfWOpmNsk9k+jBTwBAtSnahptgOgsBmbd0e3jg4eAwnQ6rTM3D9RbcHqkQdWL4qef920A1msaVy+lUL+wbu3DyxNzw6gNQ5fZ9PX5S9UXzqJ5k8HN9pdLPpSq/xCSrNzjmdiySVHzmr+SqgMjIXLxyyqQ8VrzdZMI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774030357; c=relaxed/simple; bh=QS1WWhfnhlxuhnsuWa2H0Rvb7wQFQVJGDESzCKXlKw0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=PaF/mYq7LZlqVyeKgfvZUAXUjlJrDY7PoCArjYdcieW7BHDKveugiVALI7H4hKO2sqTxu2sQVBxUyECGf6wO6QOXtT0DmAErVxS+7FHYeQPOXPL9xb9Ox3Bm4C9oFQFuAueoVBtkXecpsqmy4S8kbHo3+rEArNDsAW0iQFteIGk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=P60v6DeD; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="P60v6DeD" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2b052562254so11585ad.0 for ; Fri, 20 Mar 2026 11:12:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774030354; x=1774635154; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cLXxR402vA3/2ADlyJ4N6o23ie2gXoPSWXjjE5PFaBA=; b=P60v6DeDQz86XBDoxVUqaQ84m3F7OYDbGmAH7FmPECKFN/wKzaL6U7Aa0yluRlL0cN rGAiEXNTX9fWlBRRBm7ZFIzhLf5KykEcjTIpuNqYGXQJzG76K9wnsY+Z47jnS6828nzv 2/J9mY9dRbbeS8GxKR0bE3VVit0F2K3iEjEm6B797UaBwoQIZg56qi8iYWZyy2LMKfGo abaAj1ajhYY6ysotnyk6JKZkBqa9pvtTzMF6ysnoJyBna0jURJuV08QTRW9nD2jaZj0W OcaA5Pi1V4g62uWHIFt9WKICtjsXNSUCZHMvjnrKM0m/Zt1EviF8qrq+1EoykQqUPQgF BM1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774030354; x=1774635154; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cLXxR402vA3/2ADlyJ4N6o23ie2gXoPSWXjjE5PFaBA=; b=khwuMuG4QBCn7PKSUbbMZYz91A8Qpt5oP0mtwM6uJsa3MpGg3xaRDkziysOGOyjLt9 LI7cAoXOFziOVRgtuhx1G/Ct/RiLm2c0D1k95NgKYlzNkRH6OQSnBM1D7CCTsiVbycSl kKgSAfeuTmDjLR+cVam5Ezx5eVEoi9RahxBL84aoWbxT0SGy9dbY5FiyTUEsEWw5hQVf IxG2Vh4BRHZLX1BhK7y+gk9oOtuTEW9+HNHKI/cUOxKRFk8sz3viwtTPLbTbOxbDB7B5 zweCkFQtiSqaPcmZ2aw28uw/1uTIh1cVOHzpDStFSlKAGFFR8VOlvo/YPM87/WZH7giy l8Mg== X-Forwarded-Encrypted: i=1; AJvYcCWoaLXH/rCXCC8tVFyxPkKxhLlNdtusbwzb5uDCLLI23xmy20ErvCYtdSDTsi/i+JgYEfpvtntf3Y0X1+o=@vger.kernel.org X-Gm-Message-State: AOJu0Ywk9VPesj8kS99Ew4QjZLfEpvmpsTTtxsD4R8bY6UE3BV3Jy6ju LoOIfThM7QLhpm+OagcXMw9ENbI5Ltw4sNuYuQv+XYeijfEmWxFGYQHkQC7q/2rrNA== X-Gm-Gg: ATEYQzxu8AtT+h1jvZrD5DmUGKdd/JdfULZwbh5v32y749Xgg3scAAg2uo64lTCKQOq xowDTW26PNHZ3S2VmalposYsAUC0KONT/2NaeTPnVA1w2tS9u1l4lRuswdOkr0mCv8m4xkKu0zv oRXQpEmH0t5xc93VPR/m72fAE5QIVPiev9lkyvkCjaE5KQFiZvWsIFs6mjxTFNWtfPHyr+MKt2i IdSmCSlgDidHHNN4+2QlupQ+KHobl6U0h7E599SqH+MImnFImJWYVobOBDXutTSNGtmxShcGYbR rHYo0UV+lFbsWB7o/Wvv94MavJEpJKyPWcR9N3ClgbovUjmHAA5aDjl8qLNZrKkiPWXfm6YZg3p 23I+nBEWBK44gMp1q5iNcQh3Eu92UjYu8alYA9pYY3gke1r0JMKg6d05xGdmB0GcJDnaXt4fhpL EPY5xZXfjNMf6mAyq9DPH2rXhNIFhGcS3pFtsFpbeAYwzo9LvUldTuo1QV53ETfeOXm30oZft1 X-Received: by 2002:a17:902:cf45:b0:2b0:5193:1212 with SMTP id d9443c01a7336-2b08b40a7b1mr217595ad.4.1774030353862; Fri, 20 Mar 2026 11:12:33 -0700 (PDT) Received: from google.com (168.136.83.34.bc.googleusercontent.com. [34.83.136.168]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35bd36b169esm1069896a91.12.2026.03.20.11.12.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Mar 2026 11:12:33 -0700 (PDT) Date: Fri, 20 Mar 2026 18:12:28 +0000 From: Samiullah Khawaja To: Pranjal Shrivastava Cc: Ankit Soni , David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Vipin Sharma , YiFei Zhu Subject: Re: [PATCH 04/14] iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages Message-ID: References: <20260203220948.2176157-1-skhawaja@google.com> <20260203220948.2176157-5-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: On Fri, Mar 20, 2026 at 05:27:46PM +0000, Pranjal Shrivastava wrote: >On Tue, Mar 03, 2026 at 06:41:26PM +0000, Samiullah Khawaja wrote: >> On Tue, Mar 03, 2026 at 04:42:02PM +0000, Ankit Soni wrote: >> > On Tue, Feb 03, 2026 at 10:09:38PM +0000, Samiullah Khawaja wrote: >> > > IOMMU pages are allocated/freed using APIs using struct ioptdesc. For >> > > the proper preservation and restoration of ioptdesc add helper >> > > functions. >> > > >> > > Signed-off-by: Samiullah Khawaja >> > > --- >> > > drivers/iommu/iommu-pages.c | 74 +++++++++++++++++++++++++++++++++++++ >> > > drivers/iommu/iommu-pages.h | 30 +++++++++++++++ >> > > 2 files changed, 104 insertions(+) >> > > >> > > diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c >> > > index 3bab175d8557..588a8f19b196 100644 >> > > --- a/drivers/iommu/iommu-pages.c >> > > +++ b/drivers/iommu/iommu-pages.c >> > > @@ -6,6 +6,7 @@ >> > > #include "iommu-pages.h" >> > > #include >> > > #include >> > > +#include >> > > #include >> > > >> > > #define IOPTDESC_MATCH(pg_elm, elm) \ >> > > @@ -131,6 +132,79 @@ void iommu_put_pages_list(struct iommu_pages_list *list) >> > > } >> > > EXPORT_SYMBOL_GPL(iommu_put_pages_list); >> > > >> > > +#if IS_ENABLED(CONFIG_IOMMU_LIVEUPDATE) >> > > +void iommu_unpreserve_page(void *virt) >> > > +{ >> > > + kho_unpreserve_folio(ioptdesc_folio(virt_to_ioptdesc(virt))); >> > > +} >> > > +EXPORT_SYMBOL_GPL(iommu_unpreserve_page); >> > > + >> > > +int iommu_preserve_page(void *virt) >> > > +{ >> > > + return kho_preserve_folio(ioptdesc_folio(virt_to_ioptdesc(virt))); >> > > +} >> > > +EXPORT_SYMBOL_GPL(iommu_preserve_page); >> > > + >> > > +void iommu_unpreserve_pages(struct iommu_pages_list *list, int count) >> > > +{ >> > > + struct ioptdesc *iopt; >> > > + >> > > + if (!count) >> > > + return; >> > > + >> > > + /* If less than zero then unpreserve all pages. */ >> > > + if (count < 0) >> > > + count = 0; >> > > + >> > > + list_for_each_entry(iopt, &list->pages, iopt_freelist_elm) { >> > > + kho_unpreserve_folio(ioptdesc_folio(iopt)); >> > > + if (count > 0 && --count == 0) >> > > + break; >> > > + } >> > > +} >> > > +EXPORT_SYMBOL_GPL(iommu_unpreserve_pages); >> > > + >> > > +void iommu_restore_page(u64 phys) >> > > +{ >> > > + struct ioptdesc *iopt; >> > > + struct folio *folio; >> > > + unsigned long pgcnt; >> > > + unsigned int order; >> > > + >> > > + folio = kho_restore_folio(phys); >> > > + BUG_ON(!folio); >> > > + >> > > + iopt = folio_ioptdesc(folio); >> > >> > iopt->incoherent = false; should be here? >> > >> >> Yes this should be set here. I will update this. > >I'm wondering if we are silently losing state here. What if the >preserved page was actually incoherent in the previous kernel? >I understand we likely need to initialize it to false here because we >don't have a dev pointer for DMA sync operations at this low level (though >x86 uses clflush). > >But when is it set back to "incoherent" again? I don't see that >happening during the driver re-attach phase? This can be done during restore_domain as the domain has a reference to the dev when it is recreated. I will updated the walker and add this in the next revision. > >Should we at least mention that this API intentionally overwrites the >preserved coherency state and that these pages must explicitly be marked >incoherent again later by the driver based on its preserved HW state OR >by the IOMMUFD re-attach? Agreed. I will add a Note about this. > >> > > + >> > > + order = folio_order(folio); >> > > + pgcnt = 1UL << order; >> > > + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, pgcnt); >> > > + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, pgcnt); >> > > +} >> > > +EXPORT_SYMBOL_GPL(iommu_restore_page); > >[------ snip >8 -------] > >Thanks, >Praan Thanks, Sami