From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A04EC3E869A for ; Mon, 27 Apr 2026 17:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777312625; cv=none; b=cIAUD5I/UQTofUaBYYmkSmV3p/D3YHi/1lribyMn3WmbkOAeATQtHGQC+TsX1gjTvqSfMxntsYAS9gHQRuWPwUVKpAdOwnkKS0QYx1uFsQFCeIpa+Y+HGRl3bUA0xlx37OfxijK/bX9YBhVIoZ8WF/URxB68iS3VI9kjZq0GVx0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777312625; c=relaxed/simple; bh=534PXqsRH0ZNWtMk7TNPuiF5gAIRjYEwUzbP3uo7gV8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Cgxm6MxrSfjsEPsB7MsQsWsdighPfD0TOy5Y4FpkdGL41UmWGVH+YzLJJjuwhmsYbOOFjr1k2IMtxiJ8zSjlOpz+g8sGTgJtaoykv0wQYEO8k1gfwM+MpnkqMcSp6Q5oA2x7vpgIbgTpdYrl3vdSs8P6GVc4sxRVoU8uxkwnq3E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dw2ZW84A; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dw2ZW84A" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2b2e91add2aso8408425ad.1 for ; Mon, 27 Apr 2026 10:57:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777312621; x=1777917421; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=t4OVoEgV8q2yYCFvxS4iHEMfYA31+l0LuQw3fH3C5DI=; b=dw2ZW84AqbCqKWdwwdJ1xyRjb3tfDaWpErezjyAFL/T0QsmuVL//WLO3XL1NLDlHv8 Z/pD/U43DTLtV2/gFp6AivdZh1v4zVPRQtiZYS6myeqaRsI/GDiHPRe0LB7+W0Bp08/F BBSOS86NblzzRa/hTAP8Yo9APiYyL7jLn0iTRN/NDq++kLWxDZNkNex4temFAtbqFq/d bg0iG3KYC/rNL83cHuLZd3SH4ugllTFaC1JqLkdLaxJjaaINHtA6uw6/mApU+BvvyB5H mTCTRALmBDgR7zZW7ayZpspYP6J7rlZ8UeOGRwAQnvkK9C0LFeZsJ/l9vfT3p5iR+vdp xn0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777312621; x=1777917421; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=t4OVoEgV8q2yYCFvxS4iHEMfYA31+l0LuQw3fH3C5DI=; b=dos6/VaRRcgXYo6DJ5Go+lPzZsKbEl64Uica5SFSXXz7f58oKETSCUciF0tqIigITz uWJM8Qw6Yj3oTPwQblcTVStfCsalwaUkdfOWS+w8T6qXJ7MuDnOb7ceMtC1P76GsL6To ThdAwQl8oXWNThpFh+4TGH+k7PpNXlK9E90QPTBfsAsGM2m0NpXEmRnTyxz2co0re+6g MOGnixsrm3I3J46aqWSKmr2pJktDVxnOMWVgXCBXUdREVU5tH0VPTKlt/UHGY+iameE9 bGFAzr665sdC4kNQECSTadMAhG7/XZMqh5L9YpzSizlCuYoSfeAr6M9C+w/oAwQtRyny VK9A== X-Forwarded-Encrypted: i=1; AFNElJ/13eVfoDQv3YUSlQbCeGvudfINdktdRQgTjQw/UYyRRfLhodIWhm/DnM5MO0hhKWD3Co70uNCAevORZRc=@vger.kernel.org X-Gm-Message-State: AOJu0YyMnPinLl1R5gF+TJeUjFWpNv6oiDZbJZgOQosv0L5WbKbEViCc of3zkxJbfIlvt6yOsoAeLfxxo4N6Mbr0K8oMFoA4CB10UEv0flxs+5WCrnjlJMuWG/29x3/eOpz 2A1vV73VHwKnu9Q== X-Received: from plgr6.prod.google.com ([2002:a17:902:ea46:b0:2b2:a704:94d8]) (user=skhawaja job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ce0b:b0:2b2:65db:8c51 with SMTP id d9443c01a7336-2b97a7aaa57mr2542145ad.8.1777312620699; Mon, 27 Apr 2026 10:57:00 -0700 (PDT) Date: Mon, 27 Apr 2026 17:56:32 +0000 In-Reply-To: <20260427175633.1978233-1-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260427175633.1978233-1-skhawaja@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260427175633.1978233-16-skhawaja@google.com> Subject: [PATCH v2 15/16] vfio/pci: Preserve the iommufd state of the vfio cdev From: Samiullah Khawaja To: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe Cc: Samiullah Khawaja , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Pranjal Shrivastava , Vipin Sharma , YiFei Zhu Content-Type: text/plain; charset="UTF-8" If the vfio cdev is attached to an iommufd, preserve the state of the attached iommufd also. Basically preserve the iommu specific state of the device and also the attach iommu HW unit. Once the device and its iommufd attachment is preserved, it cannot be detached or attached to another IOAS until it is unpreserved. Signed-off-by: Samiullah Khawaja --- drivers/vfio/device_cdev.c | 10 ++++++++ drivers/vfio/pci/vfio_pci_liveupdate.c | 33 +++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/device_cdev.c b/drivers/vfio/device_cdev.c index 6844684a3d8e..23f083c0891a 100644 --- a/drivers/vfio/device_cdev.c +++ b/drivers/vfio/device_cdev.c @@ -271,6 +271,11 @@ int vfio_df_ioctl_attach_pt(struct vfio_device_file *df, } mutex_lock(&device->dev_set->lock); + if (iommufd_device_is_preserved(device->iommufd_device)) { + ret = -EBUSY; + goto out_unlock; + } + if (attach.flags & VFIO_DEVICE_ATTACH_PASID) ret = device->ops->pasid_attach_ioas(device, attach.pasid, @@ -329,6 +334,11 @@ int vfio_df_ioctl_detach_pt(struct vfio_device_file *df, } mutex_lock(&device->dev_set->lock); + if (iommufd_device_is_preserved(device->iommufd_device)) { + mutex_unlock(&device->dev_set->lock); + return -EBUSY; + } + if (detach.flags & VFIO_DEVICE_DETACH_PASID) device->ops->pasid_detach_ioas(device, detach.pasid); else diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio_pci_liveupdate.c index 976ef17e6103..b56d80379ffc 100644 --- a/drivers/vfio/pci/vfio_pci_liveupdate.c +++ b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -108,10 +108,13 @@ #include #include #include +#include #include #include "vfio_pci_priv.h" +MODULE_IMPORT_NS("IOMMUFD"); + static bool vfio_pci_liveupdate_can_preserve(struct liveupdate_file_handler *handler, struct file *file) { @@ -153,9 +156,26 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *args) vdev = container_of(device, struct vfio_pci_core_device, vdev); pdev = vdev->pdev; +#ifdef CONFIG_IOMMU_LIVEUPDATE + /* If iommufd is attached, preserve the underlying domain */ + mutex_lock(&device->dev_set->lock); + if (device->iommufd_attached) { + u64 token, preserved_state; + + ret = iommufd_device_preserve(args->session, + device->iommufd_device, + &token, &preserved_state); + if (ret) { + mutex_unlock(&device->dev_set->lock); + return ret; + } + } + mutex_unlock(&device->dev_set->lock); +#endif + ret = pci_liveupdate_preserve(pdev); if (ret) - return ret; + goto err_iommufd_unpreserve; ser = kho_alloc_preserve(sizeof(*ser)); if (IS_ERR(ser)) { @@ -170,6 +190,9 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *args) args->serialized_data = virt_to_phys(ser); return 0; +err_iommufd_unpreserve: + iommufd_device_unpreserve(args->session, device->iommufd_device); + err_unpreserve: pci_liveupdate_unpreserve(pdev); return ret; @@ -178,6 +201,14 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *args) static void vfio_pci_liveupdate_unpreserve(struct liveupdate_file_op_args *args) { struct vfio_device *device = vfio_device_from_file(args->file); + struct vfio_pci_core_device_ser *ser; + + ser = phys_to_virt(args->serialized_data); + mutex_lock(&device->dev_set->lock); + if (device->iommufd_attached) + iommufd_device_unpreserve(args->session, + device->iommufd_device); + mutex_unlock(&device->dev_set->lock); pci_liveupdate_unpreserve(to_pci_dev(device->dev)); kho_unpreserve_free(phys_to_virt(args->serialized_data)); -- 2.54.0.545.g6539524ca2-goog