All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jacob Pan <jacob.pan@linux.microsoft.com>
To: "Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org>
Cc: linux-coco@lists.linux.dev, Alex Williamson <alex@shazbot.org>,
	Alexey Kardashevskiy <aik@amd.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Jason Gunthorpe <jgg@ziepe.ca>, Joerg Roedel <joro@8bytes.org>,
	Jonathan Cameron <jic23@kernel.org>,
	Jonathan Hunter <jonathanh@nvidia.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Krishna Reddy <vdumpa@nvidia.com>, Lukas Wunner <lukas@wunner.de>,
	Nicolin Chen <nicolinc@nvidia.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Samuel Ortiz <sameo@rivosinc.com>,
	Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
	Steven Price <steven.price@arm.com>,
	Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Thierry Reding <thierry.reding@kernel.org>,
	Vasant Hegde <vasant.hegde@amd.com>,
	Will Deacon <will@kernel.org>,
	Xu Yilun <yilun.xu@linux.intel.com>,
	Ashish Kalra <ashish.kalra@amd.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	iommu@lists.linux.dev,
	Jonathan Cameron <jonathan.cameron@huawei.com>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-pci@vger.kernel.org, linux-tegra@vger.kernel.org,
	patches@lists.linux.dev, jacob.pan@linux.microsoft.com
Subject: Re: [PATCH v9 10/13] iommufd/viommu: Allow PCI TSM backends to provide vIOMMU operations
Date: Thu, 19 Nov 2026 11:50:41 -0800	[thread overview]
Message-ID: <20261119115041.00007215@linux.microsoft.com> (raw)
In-Reply-To: <20261116081935.85372-11-aneesh.kumar@kernel.org>

Hi Aneesh,

On Mon, 16 Nov 2026 13:49:32 +0530
"Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org> wrote:

> In confidential-computing configurations, a vIOMMU may be implemented
> by a TSM driver rather than by the physical IOMMU driver. Such an
> implementation also needs the TSM identity when creating and
> validating vdevices.
> 
> Allow VIOMMU_ALLOC to first query the TSM associated with the target
> PCI device for vIOMMU operations. Fall back to the physical IOMMU
> driver when the TSM does not provide operations for the requested
> vIOMMU type.
> 
> Resolve the PCI TSM association under pci_tsm_rwsem, take a TSM device
> reference, and pin the module providing the operations. Store the TSM
> device in the vIOMMU. Release the device reference and module pin on
> allocation failure, fallback to the physical IOMMU driver, or vIOMMU
> destruction. Drop the device reference before releasing the module
> pin, since the final put may call into the backend module.
> 
> The device reference preserves the TSM allocation. Backends providing
> vIOMMU operations must not unregister their TSM while the module is
> pinned, since the reference does not prevent backend resource
> teardown.
> 
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Will Deacon <will@kernel.org>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Lukas Wunner <lukas@wunner.de>
> Cc: Xu Yilun <yilun.xu@linux.intel.com>
> Cc: iommu@lists.linux.dev
> Cc: linux-pci@vger.kernel.org
> Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
> ---
>  drivers/iommu/iommufd/viommu.c | 35 +++++++++++++++++++++++++++-----
>  drivers/pci/tsm/core.c         | 22 ++++++++++++++++++++
>  drivers/virt/coco/tsm-core.c   | 37
> ++++++++++++++++++++++++++++++++++ include/linux/iommufd.h        |
> 2 ++ include/linux/pci-tsm.h        | 24 ++++++++++++++++++++++
>  include/linux/tsm.h            | 25 +++++++++++++++++++++++
>  6 files changed, 140 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/iommu/iommufd/viommu.c
> b/drivers/iommu/iommufd/viommu.c index 2eba1562a704..7e74d3992ae4
> 100644 --- a/drivers/iommu/iommufd/viommu.c
> +++ b/drivers/iommu/iommufd/viommu.c
> @@ -15,6 +15,8 @@ void iommufd_viommu_destroy(struct iommufd_object
> *obj) 
>  	if (viommu->ops && viommu->ops->destroy)
>  		viommu->ops->destroy(viommu);
> +	if (viommu->tsm_dev)
> +		tsm_put_device(viommu->tsm_dev);
>  	refcount_dec(&viommu->hwpt->common.obj.users);
>  	if (viommu->kvm_file)
>  		fput(viommu->kvm_file);
> @@ -30,6 +32,7 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd
> *ucmd) .len = cmd->data_len,
>  	};
>  	struct iommufd_hwpt_paging *hwpt_paging;
> +	struct tsm_dev *tsm_dev = NULL;
>  	struct iommufd_viommu *viommu;
>  	struct iommufd_device *idev;
>  	struct iommu_device *iommu_dev;
> @@ -49,15 +52,33 @@ int iommufd_viommu_alloc_ioctl(struct
> iommufd_ucmd *ucmd) rc = -EOPNOTSUPP;
>  		goto out_put_idev;
>  	}
> -	if (!iommu_dev->ops->get_viommu_ops) {
> -		rc = -EOPNOTSUPP;
> +	tsm_dev = tsm_get_device(idev->dev);
> +	if (IS_ERR(tsm_dev)) {
> +		rc = PTR_ERR(tsm_dev);
> +		tsm_dev = NULL;
>  		goto out_put_idev;
>  	}
> -	ops = iommu_dev->ops->get_viommu_ops(idev->dev, cmd->type);
> -	if (!ops) {
> -		rc = -EOPNOTSUPP;
> +	ops = tsm_dev ? tsm_viommu_get_ops(tsm_dev, idev->dev,
> cmd->type) : NULL;
> +	if (IS_ERR(ops)) {
> +		rc = PTR_ERR(ops);
>  		goto out_put_idev;
>  	}
> +	if (!ops) {
> +		if (tsm_dev) {
> +			tsm_put_device(tsm_dev);
nit: Since tsm_get_device() may legitimately return NULL, maybe you can
make tsm_put_device NULL safe, then you can avoid this check and other
places.

> +			tsm_dev = NULL;
nit: maybe a candidate for cleanup helper? if tsm_put_device is null
safe. then:
           tsm_put_device(no_free_ptr(tsm_dev));

> +		}
> +		if (!iommu_dev->ops->get_viommu_ops) {
> +			rc = -EOPNOTSUPP;
> +			goto out_put_idev;
> +		}
> +		ops = iommu_dev->ops->get_viommu_ops(idev->dev,
> cmd->type);
> +		if (!ops) {
> +			rc = -EOPNOTSUPP;
> +			goto out_put_idev;
> +		}
> +	}
> +
>  	/*
>  	 * It is a driver bug to omit the required operations or
> provide a size
>  	 * smaller than the core vIOMMU structure.
> @@ -95,6 +116,8 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd
> *ucmd) viommu->kvm_file = get_file(idev->kvm_file);
>  	viommu->type = cmd->type;
>  	viommu->ictx = ucmd->ictx;
> +	viommu->tsm_dev = tsm_dev;
> +	tsm_dev = NULL;
   viommu->tsm_dev = no_free_ptr(tsm_dev)

>  	viommu->hwpt = hwpt_paging;
>  	refcount_inc(&viommu->hwpt->common.obj.users);
>  	INIT_LIST_HEAD(&viommu->veventqs);
> @@ -119,6 +142,8 @@ int iommufd_viommu_alloc_ioctl(struct
> iommufd_ucmd *ucmd) out_put_hwpt:
>  	iommufd_put_object(ucmd->ictx, &hwpt_paging->common.obj);
>  out_put_idev:
> +	if (tsm_dev)
> +		tsm_put_device(tsm_dev);
ditto, can be unconditional.

>  	iommufd_put_object(ucmd->ictx, &idev->obj);
>  	return rc;
>  }
> diff --git a/drivers/pci/tsm/core.c b/drivers/pci/tsm/core.c
> index 8f0e5af808a5..4107ff443228 100644
> --- a/drivers/pci/tsm/core.c
> +++ b/drivers/pci/tsm/core.c
> @@ -9,6 +9,8 @@
>  #define dev_fmt(fmt) "PCI/TSM: " fmt
>  
>  #include <linux/bitfield.h>
> +#include <linux/iommufd.h>
> +#include <linux/module.h>
>  #include <linux/pci.h>
>  #include <linux/pci-doe.h>
>  #include <linux/pci-tsm.h>
> @@ -36,6 +38,26 @@ static const struct pci_tsm_ops
> *to_pci_tsm_ops(struct pci_tsm *tsm) return tsm->tsm_dev->pci_ops;
>  }
>  
> +struct tsm_dev *pci_tsm_get_device(struct pci_dev *pdev)
> +{
> +	const struct pci_tsm_ops *ops;
> +	struct tsm_dev *tsm_dev;
> +
> +	guard(rwsem_read)(&pci_tsm_rwsem);
> +	if (!pdev->tsm)
> +		return NULL;
> +
> +	tsm_dev = pdev->tsm->tsm_dev;
> +	ops = tsm_dev->pci_ops;
> +	if (!ops->viommu_get_ops)
> +		return NULL;
> +	if (!try_module_get(ops->owner))
> +		return ERR_PTR(-ENODEV);
> +	get_device(&tsm_dev->dev);
what if tsm_unregister() is called directly? without protection from
tsm_dev->users, unregister can succeed tearing down backend while this
reference is held.

For bisectable result, maybe you can move tsm_get() and wait patch
before this one?

> +	return tsm_dev;
> +}
> +EXPORT_SYMBOL_GPL(pci_tsm_get_device);
> +
>  static inline bool is_dsm(struct pci_dev *pdev)
>  {
>  	return pdev->tsm && pdev->tsm->dsm_dev == pdev;
> diff --git a/drivers/virt/coco/tsm-core.c
> b/drivers/virt/coco/tsm-core.c index 181d6114086f..766f4ad67ed7 100644
> --- a/drivers/virt/coco/tsm-core.c
> +++ b/drivers/virt/coco/tsm-core.c
> @@ -9,6 +9,16 @@
>  #include <linux/cleanup.h>
>  #include <linux/pci-tsm.h>
>  
> +/* The caller must hold a tsm_get_device() reference for the entire
> use. */ +const struct iommufd_viommu_ops *tsm_viommu_get_ops(struct
> tsm_dev *tsm_dev,
> +		struct device *dev, enum iommu_viommu_type type)
> +{
> +	if (!tsm_dev->pci_ops->viommu_get_ops)
> +		return NULL;
> +	return tsm_dev->pci_ops->viommu_get_ops(dev, type);
> +}
> +EXPORT_SYMBOL_GPL(tsm_viommu_get_ops);
> +
>  static void tsm_release(struct device *);
>  static const struct class tsm_class = {
>  	.name		= "tsm",
> @@ -16,6 +26,33 @@ static const struct class tsm_class = {
>  };
>  static DEFINE_IDA(tsm_ida);
>  
> +/**
> + * tsm_get_device() - Pin the TSM attached to a device
> + * @dev: device whose TSM is to be pinned
> + *
> + * A successful lookup takes a TSM device reference and pins the
> backend
> + * module. Release both with tsm_put_device(). Backends providing
> vIOMMU
> + * operations must not unregister their TSM independently of module
> unload.
> + * Return: NULL if no vIOMMU-capable TSM is attached, an error if
> the module
> + * cannot be pinned, or the TSM device associated with the pinned
> backend.
> + */
> +struct tsm_dev *tsm_get_device(struct device *dev)
> +{
> +	if (!dev_is_pci(dev))
> +		return NULL;
> +	return pci_tsm_get_device(to_pci_dev(dev));
> +}
> +EXPORT_SYMBOL_GPL(tsm_get_device);
> +
> +void tsm_put_device(struct tsm_dev *tsm_dev)
> +{
> +	struct module *owner = tsm_dev->pci_ops->owner;
> +
> +	put_device(&tsm_dev->dev);
> +	module_put(owner);
> +}
> +EXPORT_SYMBOL_GPL(tsm_put_device);
> +
>  static int match_id(struct device *dev, const void *data)
>  {
>  	struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev,
> dev); diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h
> index 5f5c67b24134..a4b29915d0ca 100644
> --- a/include/linux/iommufd.h
> +++ b/include/linux/iommufd.h
> @@ -23,6 +23,7 @@ struct iommufd_access;
>  struct iommufd_ctx;
>  struct iommufd_device;
>  struct iommufd_viommu_ops;
> +struct tsm_dev;
>  struct page;
>  
>  enum iommufd_object_type {
> @@ -104,6 +105,7 @@ struct iommufd_viommu {
>  	struct iommu_device *iommu_dev;
>  	struct iommufd_hwpt_paging *hwpt;
>  	struct file *kvm_file;
> +	struct tsm_dev *tsm_dev;
>  
>  	const struct iommufd_viommu_ops *ops;
>  
> diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
> index 1458935234a4..27a76aeedf50 100644
> --- a/include/linux/pci-tsm.h
> +++ b/include/linux/pci-tsm.h
> @@ -3,7 +3,10 @@
>  #define __PCI_TSM_H
>  #include <linux/mutex.h>
>  #include <linux/pci.h>
> +#include <uapi/linux/iommufd.h>
>  
> +struct iommufd_viommu_ops;
> +struct module;
>  struct pci_tsm;
>  struct pci_tsm_context;
>  struct tsm_dev;
> @@ -16,12 +19,27 @@ struct tsm_dev;
>   * @devsec_ops: Lock, unlock, and interrogate the security state of
> the
>   *		function via the platform TSM (typically virtual
> function
>   *		operations).
> + * @viommu_get_ops: Return operations for a supported TSM-owned
> vIOMMU type,
> + *		  or NULL if unsupported
> + * @owner: Module providing the vIOMMU operations, if built as a
> module
> + *
> + * The vIOMMU owner holds a TSM device reference and pins the
> backend module
> + * before querying @viommu_get_ops, releasing both after vIOMMU
> destruction.
> + * A backend providing this callback must not unregister its TSM
> while the
> + * module is pinned.
>   *
>   * This operations are mutually exclusive either a tsm_dev instance
>   * manages physical link properties or it manages function security
>   * states like TDISP lock/unlock.
> + *
> + * @viommu_get_ops runs with the backend module pinned. The PCI/TSM
> + * association is checked during acquisition under pci_tsm_rwsem.
>   */
>  struct pci_tsm_ops {
> +	struct module *owner;
> +	const struct iommufd_viommu_ops *(*viommu_get_ops)(
> +		struct device *dev, enum iommu_viommu_type type);
> +
>  	/*
>  	 * struct pci_tsm_link_ops - Manage physical link and the
> TSM/DSM session
>  	 * @probe: establish context with the TSM (allocate / wrap
> 'struct @@ -151,6 +169,7 @@ struct pci_tsm_pf0
> *pci_tsm_context_pf0(struct pci_tsm_context *context); struct pci_dev
> *pci_tsm_context_dsm_dev(struct pci_tsm_context *context); bool
> pci_tsm_context_match_device(struct pci_tsm_context *context, struct
> pci_dev *pdev); +struct tsm_dev *pci_tsm_get_device(struct pci_dev
> *pdev); #else
>  static inline int pci_tsm_register(struct tsm_dev *tsm_dev)
>  {
> @@ -165,6 +184,11 @@ static inline bool pci_tsm_is_configured(struct
> pci_dev *pdev) return false;
>  }
>  
> +static inline struct tsm_dev *pci_tsm_get_device(struct pci_dev
> *pdev) +{
> +	return NULL;
> +}
> +
>  static inline struct pci_tsm_context *
>  pci_tsm_context_get(struct pci_dev *pdev, struct tsm_dev *tsm_dev)
>  {
> diff --git a/include/linux/tsm.h b/include/linux/tsm.h
> index 19dedf971f14..3d5286aeaa42 100644
> --- a/include/linux/tsm.h
> +++ b/include/linux/tsm.h
> @@ -110,6 +110,8 @@ struct tsm_report_ops {
>  };
>  
>  struct pci_tsm_ops;
> +struct iommufd_viommu_ops;
> +
>  struct tsm_dev {
>  	struct device dev;
>  	int id;
> @@ -124,4 +126,27 @@ int tsm_report_unregister(const struct
> tsm_report_ops *ops); struct tsm_dev *tsm_register(struct device
> *parent, struct pci_tsm_ops *ops); void tsm_unregister(struct tsm_dev
> *tsm_dev); struct tsm_dev *find_tsm_dev(int id);
> +
> +#ifdef CONFIG_TSM
> +struct tsm_dev *tsm_get_device(struct device *dev);
> +void tsm_put_device(struct tsm_dev *tsm_dev);
> +const struct iommufd_viommu_ops *tsm_viommu_get_ops(struct tsm_dev
> *tsm_dev,
> +		struct device *dev, enum iommu_viommu_type type);
> +#else
> +static inline struct tsm_dev *tsm_get_device(struct device *dev)
> +{
> +	return NULL;
> +}
> +
> +static inline void tsm_put_device(struct tsm_dev *tsm_dev)
> +{
> +}
> +
> +static inline const struct iommufd_viommu_ops *
> +tsm_viommu_get_ops(struct tsm_dev *tsm_dev, struct device *dev,
> +		enum iommu_viommu_type type)
> +{
> +	return NULL;
> +}
> +#endif
>  #endif /* __TSM_H */
With the above addressed,

Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>

  reply	other threads:[~2026-11-19 19:50 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  8:19 [PATCH v9 00/13] iommufd: TSM-backed virtual IOMMUs for confidential guests Aneesh Kumar K.V (Arm)
2026-11-16  8:19 ` [PATCH v9 01/13] virt: coco: tsm: Require PCI operations when registering a TSM Aneesh Kumar K.V (Arm)
2026-11-19  7:18   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 02/13] virt: coco: tsm: Remove the device from lookup before PCI teardown Aneesh Kumar K.V (Arm)
2026-11-18  6:43   ` Xu Yilun
2026-11-16  8:19 ` [PATCH v9 03/13] virt: coco: tsm: Complete PCI registration before publishing the device Aneesh Kumar K.V (Arm)
2026-11-18  7:11   ` Xu Yilun
2026-11-16  8:19 ` [PATCH v9 04/13] iommufd/device: Associate KVM file pointer with iommufd_device Aneesh Kumar K.V (Arm)
2026-11-19  8:13   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 05/13] iommufd/viommu: Keep a reference to the KVM file Aneesh Kumar K.V (Arm)
2026-11-19  8:20   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 06/13] iommufd: Add a vDEVICE operation ioctl framework Aneesh Kumar K.V (Arm)
2026-11-20  1:17   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 07/13] PCI/TSM: Remove the legacy guest request interface Aneesh Kumar K.V (Arm)
2026-11-20  1:37   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 08/13] PCI/TSM: Keep connections active while device contexts exist Aneesh Kumar K.V (Arm)
2026-11-20  2:58   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 09/13] iommufd/viommu: Select vIOMMU operations before allocation Aneesh Kumar K.V (Arm)
2026-11-20  4:44   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 10/13] iommufd/viommu: Allow PCI TSM backends to provide vIOMMU operations Aneesh Kumar K.V (Arm)
2026-11-19 19:50   ` Jacob Pan [this message]
2026-11-20  5:39   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 11/13] iommufd/viommu: Allow vIOMMUs without a parent HWPT Aneesh Kumar K.V (Arm)
2026-11-20  6:49   ` Baolu Lu
2026-11-16  8:19 ` [PATCH v9 12/13] PCI/TSM: Wait for device contexts before removing a DSM Aneesh Kumar K.V (Arm)
2026-11-16  8:19 ` [PATCH v9 13/13] PCI/TSM: Wait for users during unregister Aneesh Kumar K.V (Arm)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119115041.00007215@linux.microsoft.com \
    --to=jacob.pan@linux.microsoft.com \
    --cc=aik@amd.com \
    --cc=alex@shazbot.org \
    --cc=aneesh.kumar@kernel.org \
    --cc=ashish.kalra@amd.com \
    --cc=bhelgaas@google.com \
    --cc=catalin.marinas@arm.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=iommu@lists.linux.dev \
    --cc=jgg@ziepe.ca \
    --cc=jic23@kernel.org \
    --cc=jonathan.cameron@huawei.com \
    --cc=jonathanh@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=nicolinc@nvidia.com \
    --cc=patches@lists.linux.dev \
    --cc=robin.murphy@arm.com \
    --cc=sameo@rivosinc.com \
    --cc=shameerali.kolothum.thodi@huawei.com \
    --cc=steven.price@arm.com \
    --cc=suravee.suthikulpanit@amd.com \
    --cc=suzuki.poulose@arm.com \
    --cc=thierry.reding@kernel.org \
    --cc=thomas.lendacky@amd.com \
    --cc=vasant.hegde@amd.com \
    --cc=vdumpa@nvidia.com \
    --cc=will@kernel.org \
    --cc=yilun.xu@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.