From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35E11382F39 for ; Wed, 25 Mar 2026 21:23:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774473807; cv=none; b=OjdOeCeGG4lfI9fXF7+OoM5m44uAGLE8fnFvSAEq+pA9s9m/8BbdayVt/0Zc+OrsL0rci48VNb0KCUQAwVpK8MAZZyhvHpRW7TBQhu2pwE3nuL32mAmUZrBg/b6ewJQeJuGOrcIuemHQ+MtT1xmQEMofHLVFVl1EqD4USjTwjTw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774473807; c=relaxed/simple; bh=xonbs3FxEAukhIqtIxD52TSJ8OC6RqHwbD83ZCiBA/I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=G7a8xpv4r63ZlPH6xRhfYO+D1eZgIkyzh/POGyEPB6PuD2a4/GGfY2oSjvz5Yv/w5X5iU+45VIq34TqgkeaJ4jlZu0MxzEDYBa+2cyKL5SP27XSoxEZPdfg7NOzuc3kinNReSu7SXP1faK2j3evA9aly2/vgZKuHAuCfL3T58g0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EAtCkdhr; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EAtCkdhr" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2b0b260d309so12005ad.1 for ; Wed, 25 Mar 2026 14:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774473802; x=1775078602; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EVG0u1tNMBgT8gCqMtRqwY74fPi9b/gLjEZ2D+Du3es=; b=EAtCkdhrM6y0QEb2oDKQAYRI9pYZTIpjcnMfv5wnn3dZH2EanTMo5CAKeTy3/gm/Ae 7i7nFEFgCfrEaT0khQoEdAxAS4GVI1MwjKC4GuVNw2oZRNr0R+EUnDtSLifLg92zmPHX pOaGXhEanZBTQd5lcEOBKP3slfkJgUIVtBk0RUKfpZ6iPZg1VWNceKRI9kxmL69u+K4v 3hy7dBDPKVFZ7YYiFS/RpPjgerUjjdknFKl9p0wFJfeicLKucOLCMEIpHCxHwelu8Zub hSy3uu/LkETFMkaahB4Oq5U9Z235r/pjyCioC2XgxB0NRpSWhLGhR65Iy9+KTD3sKQ3Y LqMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774473802; x=1775078602; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EVG0u1tNMBgT8gCqMtRqwY74fPi9b/gLjEZ2D+Du3es=; b=SUJCGHkSzyMoTBNQ8WuBktp+NOLWrIgkrbHWTZlTqZcOCiXR4uiDuR0LoyExsybYln j4PWuq/r78XjVXbAB4uc4GsC4k9rccSc4mY+KkwRtcw8eSJz9LdFK0iQpq2ZkqOp4Hn1 jfyu8WpgWNKqS5MH3YZSiVT7lVUEHN0w3W7MldPTayRnevqZu+BUVLCsgOzsdU3xrOsG 7HfQKnM70l1evciKrguFD1R82Kd8I+ceoEqRpnVQfY8ov6XZzC98cCfWFg5lsqhAsoW+ iZb3A/2+G54vkqM5exRRQBpZaN006Q4GotSOx51kAxGdf6eXydFFOSMxV0N/7QVwKZnu xEWQ== X-Forwarded-Encrypted: i=1; AJvYcCXTBxFUaOalk0feSjo5S1W1cpBvLQ3uce0tkBTiHWlaPTa8zowfm2t6MEtYuGSzJ+YSuZM=@vger.kernel.org X-Gm-Message-State: AOJu0YzSbpSCfYk6HWKVNA4RbUTrN9rekdFx2qnHltVpmd4bBi7aZLMO KDUsoZCeacsMYGPdPAg4774Qq6Dv9oOSLgnSWfDsoQxvbj1WPs7iMHosYWAnUWEUSw== X-Gm-Gg: ATEYQzxIQ88E0DuZdNGFP6YjFR/FzYS/RcsfpGTONB8wYYKPhaMMrU4/BN0rERvq0yr OH1AspPGQqZYH6JZSMKOZuFnIql7xM4CJPoKzFTO8QIY4NVaNwHByuuukK/va+RYWDUhhvbLxKH WZydM+TEH+5mlvJ6d8o4Q/y55QcAp9SHBreR40w5SVK+pUUEfpniAuw6+qmoF0b1Gug19Ae3svW pYtH1Gy+obi/PjndTLic6XxOjUs8sYzArZSaLKW9Nh98+jJYNUYLFdjeoBCg6iaMzrbS04lBJxm 5QRZDzWCo2q41vwWeM2VrdgiNwZCv0G/A6z583h/waRlhqqByCvntJqaXzx1hHYQAFJMy5tDG88 OsvU4t7QlZgETniQSZmuIxcIs7DYsYoOL0xZFqO5OzRlSFzrp5cf7wpM5QOx/IOd2WMZ9XEk3hE n3yTK5eEvsaBfMORiaqNfiApI+h5i5tIzt/8kLAguRpR85J97KYt7xYibjTA== X-Received: by 2002:a17:902:c947:b0:2b0:5193:1212 with SMTP id d9443c01a7336-2b0bf755ef9mr236845ad.4.1774473801622; Wed, 25 Mar 2026 14:23:21 -0700 (PDT) Received: from google.com (10.129.124.34.bc.googleusercontent.com. [34.124.129.10]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc76f1ccsm8429245ad.12.2026.03.25.14.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Mar 2026 14:23:21 -0700 (PDT) Date: Wed, 25 Mar 2026 21:23:13 +0000 From: Pranjal Shrivastava To: Samiullah Khawaja Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Vipin Sharma , YiFei Zhu Subject: Re: [PATCH 12/14] iommufd: Add APIs to preserve/unpreserve a vfio cdev Message-ID: References: <20260203220948.2176157-1-skhawaja@google.com> <20260203220948.2176157-13-skhawaja@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Wed, Mar 25, 2026 at 08:41:46PM +0000, Samiullah Khawaja wrote: > On Wed, Mar 25, 2026 at 08:24:24PM +0000, Pranjal Shrivastava wrote: > > On Tue, Feb 03, 2026 at 10:09:46PM +0000, Samiullah Khawaja wrote: > > > Add APIs that can be used to preserve and unpreserve a vfio cdev. Use > > > the APIs exported by the IOMMU core to preserve/unpreserve device. Pass > > > the LUO preservation token of the attached iommufd into IOMMU preserve > > > device API. This establishes the ownership of the device with the > > > preserved iommufd. > > > > > > Signed-off-by: Samiullah Khawaja > > > --- > > > drivers/iommu/iommufd/device.c | 69 ++++++++++++++++++++++++++++++++++ > > > include/linux/iommufd.h | 23 ++++++++++++ > > > 2 files changed, 92 insertions(+) > > > > > > diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c > > > index 4c842368289f..30cb5218093b 100644 > > > --- a/drivers/iommu/iommufd/device.c > > > +++ b/drivers/iommu/iommufd/device.c > > > @@ -2,6 +2,7 @@ > > > /* Copyright (c) 2021-2022, NVIDIA CORPORATION & AFFILIATES > > > */ > > > #include > > > +#include > > > #include > > > #include > > > #include > > > @@ -1661,3 +1662,71 @@ int iommufd_get_hw_info(struct iommufd_ucmd *ucmd) > > > iommufd_put_object(ucmd->ictx, &idev->obj); > > > return rc; > > > } > > > + > > > +#ifdef CONFIG_IOMMU_LIVEUPDATE > > > +int iommufd_device_preserve(struct liveupdate_session *s, > > > + struct iommufd_device *idev, > > > + u64 *tokenp) > > > +{ > > > + struct iommufd_group *igroup = idev->igroup; > > > + struct iommufd_hwpt_paging *hwpt_paging; > > > + struct iommufd_hw_pagetable *hwpt; > > > + struct iommufd_attach *attach; > > > + int ret; > > > + > > > + mutex_lock(&igroup->lock); > > > + attach = xa_load(&igroup->pasid_attach, IOMMU_NO_PASID); > > > > By explicitly looking up IOMMU_NO_PASID, we skip any PASID attachments > > the device might have. Since PASID live update is NOT supported in this > > series, should we check if the pasid_attach xarray contains anything > > other than IOMMU_NO_PASID and return -EOPNOTSUPP? > > > > Otherwise, we silently fail to preserve those domains without informing > > the VMM? > > VMM should be able to preserve the NO_PASID domains even if it has PASID > attachments. This is the intended behaviour, I will document it in the > uAPI docs. I think I'm miscommunicating here. My concern isn't about whether the kernel can mechanically preserve the NO_PASID domain when PASID attachments exist. I agree that part works fine. My concern is purely about silent state loss. If a VMM asks the kernel to preserve a device, it expects the entire IOMMU state for that device to be safely handed over. If the kernel silently skips the PASID attachments and returns success (0), the VMM on the new kernel will wake up assuming those PASIDs are still perfectly intact. When the guest attempts a PASID-tagged DMA, it will unexpectedly fault. So the question is: how strictly should the kernel protect userspace from this footgun? A few options that I can see: 1. Rely on uAPI docs 2. Fail the preserve ioctl (-EOPNOTSUPP) if active PASID attachments are detected. 3. Add an opt-in flag: We could add a flag to the ioctl (IOMMU_LU_FLAG_IGNORE_PASID) so userspace has to explicitly acknowledge the state drop? Options 2 or 3 are especially important when we consider backwards compatibility. If this series is merged in 7.2 with the "silent drop" behavior now, when full PASID live update support is eventually added in a future kernel, userspace will have no robust way to know if it's running on a kernel that preserves PASIDs or silently drops them. By returning an error or requiring a flag now, we reserve the right to cleanly implement the feature later without breaking the UAPI contract. This is an open question from me, I'm okay with any of the 3 options I'd like to know what the maintainers think about this as well. [ ---- >8 ----- ] Thanks, Praan