From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3662438B7BF for ; Wed, 25 Mar 2026 21:23:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774473806; cv=none; b=mMZ6R225xO0m0seZLtWO/VXsYEtV4rEksVo1O6eqUHxBdJYkenUqvU7k4Mx0n9GDIkiMkV6fvOs5nsaAz9LSGL8JT2R6+35JXHefXbgxt6QCrqQ9g7t+AZtLu8OCaCLyImy1MKK0N6LHusJePYTM9ubHdEKagc4YSrC2xmBw1Ww= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774473806; c=relaxed/simple; bh=xonbs3FxEAukhIqtIxD52TSJ8OC6RqHwbD83ZCiBA/I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=g8x1XGxVJXujCck3gmuQMaRPOTpzAj9L/1n8ZNcvMOpYuYHOwLDbd67QnnH4Bxc+aH5jgNU/27us8x/HOoSi67DmPSlN8Xocu4zBlEe5eNgsae9suhzOwHN3mHUsbDlsYdSv3tWW1WSU/Wsynwl3tyQKa7D0G4pTsHJOPVFfruY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EAtCkdhr; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EAtCkdhr" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2b0b260d309so12025ad.1 for ; Wed, 25 Mar 2026 14:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774473802; x=1775078602; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EVG0u1tNMBgT8gCqMtRqwY74fPi9b/gLjEZ2D+Du3es=; b=EAtCkdhrM6y0QEb2oDKQAYRI9pYZTIpjcnMfv5wnn3dZH2EanTMo5CAKeTy3/gm/Ae 7i7nFEFgCfrEaT0khQoEdAxAS4GVI1MwjKC4GuVNw2oZRNr0R+EUnDtSLifLg92zmPHX pOaGXhEanZBTQd5lcEOBKP3slfkJgUIVtBk0RUKfpZ6iPZg1VWNceKRI9kxmL69u+K4v 3hy7dBDPKVFZ7YYiFS/RpPjgerUjjdknFKl9p0wFJfeicLKucOLCMEIpHCxHwelu8Zub hSy3uu/LkETFMkaahB4Oq5U9Z235r/pjyCioC2XgxB0NRpSWhLGhR65Iy9+KTD3sKQ3Y LqMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774473802; x=1775078602; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EVG0u1tNMBgT8gCqMtRqwY74fPi9b/gLjEZ2D+Du3es=; b=NVZG9Xl8sCCySVV2oCXLwfH26qnrZ/j4leB6hOxS8HCX59fHzZYBxWvqOvuMIpsIUw cFXlx16Lxy2fZZNljUunb4P87yQ5PIsPMBUzxRXL7OB9Y/ubjkJsBT1JAWweliCvWQro 85lHCLR8xAS03HkqVaAk+kqUaFPT///v63eEqT6/Tyz8MArG1ovHQsIlRXjlRNNrMzol ahpi4SnC4XnBCWWLmI9bpyWWuvr4t6laLEe4pdbaHN/hwyqfHWtRHVBjs+tjBqhwmLZO 8JK7LOjFXDJSGB7zHRgJ6gWGKvzdRNb9x+w9xxQfhLmNJCwmZ3+SCNOGmI+3FpIRUzWG fh/w== X-Forwarded-Encrypted: i=1; AJvYcCWlNR49B4jYOCsitz2yCYGbxnIKa9FW+yVTC2oK/xaI6Zt0y22E/cqePdFepHewaleumW1YwQ92Ohz3NLI=@vger.kernel.org X-Gm-Message-State: AOJu0YxHaRTWfXvgj9tLiOpp2W8xkjZNW87Ol5CtghTVBi2BpV+zdpKc Hjsco1bS6kjUU2Sr4zwDMECRL1JChcz1OtvGz12DOSGib4UktfnEXco18Hk/4qeRzA== X-Gm-Gg: ATEYQzx5Cf7puv/n7p8BDar5cVAiZNkc3QpgDQuGxix9m9tqULBNPExnM1vdnsfKN1I KxxwZgUjDCp67N7p1+NsaeUxpbhRix7ywuxLRSOckHEziw3Hda56AETHI0iVzmbYueYPJH8zKxv GW2ekPm69bkEVRVPH54yP9KfApt3k4o7fItPuDIl8Di+8MvizV4HjX2JGxwKQGUFwxVAQE/v6LV 8/2Czv5RkevqWlO56hjcULnPnqWlD/i+ytMCe5lN0IcTU8/zxcAa8HgWykQ+9hOHnlOp1XOpSH9 92oy8JI/i35NmcRRi+VRuUGXVwOQcFUBGtONW9D1kv0JlM03xcx43BsA0wY8ED+aDF6pPpBxfKt vmhpRZ9vsOUysxX5usFRA6g1M50oote5sgLbCSbDWHXt5cZNheOREuo6eLfftdQjjC+G5XO7g46 iGPl7zk4jIITdaa69HkjwDUFyDQ1IP2GnzyZkLDef33P9bm/gOye+kaZsNsg== X-Received: by 2002:a17:902:c947:b0:2b0:5193:1212 with SMTP id d9443c01a7336-2b0bf755ef9mr236845ad.4.1774473801622; Wed, 25 Mar 2026 14:23:21 -0700 (PDT) Received: from google.com (10.129.124.34.bc.googleusercontent.com. [34.124.129.10]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc76f1ccsm8429245ad.12.2026.03.25.14.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Mar 2026 14:23:21 -0700 (PDT) Date: Wed, 25 Mar 2026 21:23:13 +0000 From: Pranjal Shrivastava To: Samiullah Khawaja Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Vipin Sharma , YiFei Zhu Subject: Re: [PATCH 12/14] iommufd: Add APIs to preserve/unpreserve a vfio cdev Message-ID: References: <20260203220948.2176157-1-skhawaja@google.com> <20260203220948.2176157-13-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Wed, Mar 25, 2026 at 08:41:46PM +0000, Samiullah Khawaja wrote: > On Wed, Mar 25, 2026 at 08:24:24PM +0000, Pranjal Shrivastava wrote: > > On Tue, Feb 03, 2026 at 10:09:46PM +0000, Samiullah Khawaja wrote: > > > Add APIs that can be used to preserve and unpreserve a vfio cdev. Use > > > the APIs exported by the IOMMU core to preserve/unpreserve device. Pass > > > the LUO preservation token of the attached iommufd into IOMMU preserve > > > device API. This establishes the ownership of the device with the > > > preserved iommufd. > > > > > > Signed-off-by: Samiullah Khawaja > > > --- > > > drivers/iommu/iommufd/device.c | 69 ++++++++++++++++++++++++++++++++++ > > > include/linux/iommufd.h | 23 ++++++++++++ > > > 2 files changed, 92 insertions(+) > > > > > > diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c > > > index 4c842368289f..30cb5218093b 100644 > > > --- a/drivers/iommu/iommufd/device.c > > > +++ b/drivers/iommu/iommufd/device.c > > > @@ -2,6 +2,7 @@ > > > /* Copyright (c) 2021-2022, NVIDIA CORPORATION & AFFILIATES > > > */ > > > #include > > > +#include > > > #include > > > #include > > > #include > > > @@ -1661,3 +1662,71 @@ int iommufd_get_hw_info(struct iommufd_ucmd *ucmd) > > > iommufd_put_object(ucmd->ictx, &idev->obj); > > > return rc; > > > } > > > + > > > +#ifdef CONFIG_IOMMU_LIVEUPDATE > > > +int iommufd_device_preserve(struct liveupdate_session *s, > > > + struct iommufd_device *idev, > > > + u64 *tokenp) > > > +{ > > > + struct iommufd_group *igroup = idev->igroup; > > > + struct iommufd_hwpt_paging *hwpt_paging; > > > + struct iommufd_hw_pagetable *hwpt; > > > + struct iommufd_attach *attach; > > > + int ret; > > > + > > > + mutex_lock(&igroup->lock); > > > + attach = xa_load(&igroup->pasid_attach, IOMMU_NO_PASID); > > > > By explicitly looking up IOMMU_NO_PASID, we skip any PASID attachments > > the device might have. Since PASID live update is NOT supported in this > > series, should we check if the pasid_attach xarray contains anything > > other than IOMMU_NO_PASID and return -EOPNOTSUPP? > > > > Otherwise, we silently fail to preserve those domains without informing > > the VMM? > > VMM should be able to preserve the NO_PASID domains even if it has PASID > attachments. This is the intended behaviour, I will document it in the > uAPI docs. I think I'm miscommunicating here. My concern isn't about whether the kernel can mechanically preserve the NO_PASID domain when PASID attachments exist. I agree that part works fine. My concern is purely about silent state loss. If a VMM asks the kernel to preserve a device, it expects the entire IOMMU state for that device to be safely handed over. If the kernel silently skips the PASID attachments and returns success (0), the VMM on the new kernel will wake up assuming those PASIDs are still perfectly intact. When the guest attempts a PASID-tagged DMA, it will unexpectedly fault. So the question is: how strictly should the kernel protect userspace from this footgun? A few options that I can see: 1. Rely on uAPI docs 2. Fail the preserve ioctl (-EOPNOTSUPP) if active PASID attachments are detected. 3. Add an opt-in flag: We could add a flag to the ioctl (IOMMU_LU_FLAG_IGNORE_PASID) so userspace has to explicitly acknowledge the state drop? Options 2 or 3 are especially important when we consider backwards compatibility. If this series is merged in 7.2 with the "silent drop" behavior now, when full PASID live update support is eventually added in a future kernel, userspace will have no robust way to know if it's running on a kernel that preserves PASIDs or silently drops them. By returning an error or requiring a flag now, we reserve the right to cleanly implement the feature later without breaking the UAPI contract. This is an open question from me, I'm okay with any of the 3 options I'd like to know what the maintainers think about this as well. [ ---- >8 ----- ] Thanks, Praan