From: Jacob Pan <jacob.pan@linux.microsoft.com>
To: "Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org>
Cc: linux-coco@lists.linux.dev, Alex Williamson <alex@shazbot.org>,
Alexey Kardashevskiy <aik@amd.com>,
Bjorn Helgaas <bhelgaas@google.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Jason Gunthorpe <jgg@ziepe.ca>, Joerg Roedel <joro@8bytes.org>,
Jonathan Cameron <jic23@kernel.org>,
Jonathan Hunter <jonathanh@nvidia.com>,
Kevin Tian <kevin.tian@intel.com>,
Krishna Reddy <vdumpa@nvidia.com>, Lukas Wunner <lukas@wunner.de>,
Nicolin Chen <nicolinc@nvidia.com>,
Robin Murphy <robin.murphy@arm.com>,
Samuel Ortiz <sameo@rivosinc.com>,
Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
Steven Price <steven.price@arm.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Thierry Reding <thierry.reding@kernel.org>,
Vasant Hegde <vasant.hegde@amd.com>,
Will Deacon <will@kernel.org>,
Xu Yilun <yilun.xu@linux.intel.com>,
Ashish Kalra <ashish.kalra@amd.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
iommu@lists.linux.dev,
Jonathan Cameron <jonathan.cameron@huawei.com>,
Tom Lendacky <thomas.lendacky@amd.com>,
kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
linux-pci@vger.kernel.org, linux-tegra@vger.kernel.org,
patches@lists.linux.dev, jacob.pan@linux.microsoft.com
Subject: Re: [PATCH v9 10/13] iommufd/viommu: Allow PCI TSM backends to provide vIOMMU operations
Date: Thu, 19 Nov 2026 11:50:41 -0800 [thread overview]
Message-ID: <20261119115041.00007215@linux.microsoft.com> (raw)
In-Reply-To: <20261116081935.85372-11-aneesh.kumar@kernel.org>
Hi Aneesh,
On Mon, 16 Nov 2026 13:49:32 +0530
"Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org> wrote:
> In confidential-computing configurations, a vIOMMU may be implemented
> by a TSM driver rather than by the physical IOMMU driver. Such an
> implementation also needs the TSM identity when creating and
> validating vdevices.
>
> Allow VIOMMU_ALLOC to first query the TSM associated with the target
> PCI device for vIOMMU operations. Fall back to the physical IOMMU
> driver when the TSM does not provide operations for the requested
> vIOMMU type.
>
> Resolve the PCI TSM association under pci_tsm_rwsem, take a TSM device
> reference, and pin the module providing the operations. Store the TSM
> device in the vIOMMU. Release the device reference and module pin on
> allocation failure, fallback to the physical IOMMU driver, or vIOMMU
> destruction. Drop the device reference before releasing the module
> pin, since the final put may call into the backend module.
>
> The device reference preserves the TSM allocation. Backends providing
> vIOMMU operations must not unregister their TSM while the module is
> pinned, since the reference does not prevent backend resource
> teardown.
>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Will Deacon <will@kernel.org>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Lukas Wunner <lukas@wunner.de>
> Cc: Xu Yilun <yilun.xu@linux.intel.com>
> Cc: iommu@lists.linux.dev
> Cc: linux-pci@vger.kernel.org
> Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
> ---
> drivers/iommu/iommufd/viommu.c | 35 +++++++++++++++++++++++++++-----
> drivers/pci/tsm/core.c | 22 ++++++++++++++++++++
> drivers/virt/coco/tsm-core.c | 37
> ++++++++++++++++++++++++++++++++++ include/linux/iommufd.h |
> 2 ++ include/linux/pci-tsm.h | 24 ++++++++++++++++++++++
> include/linux/tsm.h | 25 +++++++++++++++++++++++
> 6 files changed, 140 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/iommu/iommufd/viommu.c
> b/drivers/iommu/iommufd/viommu.c index 2eba1562a704..7e74d3992ae4
> 100644 --- a/drivers/iommu/iommufd/viommu.c
> +++ b/drivers/iommu/iommufd/viommu.c
> @@ -15,6 +15,8 @@ void iommufd_viommu_destroy(struct iommufd_object
> *obj)
> if (viommu->ops && viommu->ops->destroy)
> viommu->ops->destroy(viommu);
> + if (viommu->tsm_dev)
> + tsm_put_device(viommu->tsm_dev);
> refcount_dec(&viommu->hwpt->common.obj.users);
> if (viommu->kvm_file)
> fput(viommu->kvm_file);
> @@ -30,6 +32,7 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd
> *ucmd) .len = cmd->data_len,
> };
> struct iommufd_hwpt_paging *hwpt_paging;
> + struct tsm_dev *tsm_dev = NULL;
> struct iommufd_viommu *viommu;
> struct iommufd_device *idev;
> struct iommu_device *iommu_dev;
> @@ -49,15 +52,33 @@ int iommufd_viommu_alloc_ioctl(struct
> iommufd_ucmd *ucmd) rc = -EOPNOTSUPP;
> goto out_put_idev;
> }
> - if (!iommu_dev->ops->get_viommu_ops) {
> - rc = -EOPNOTSUPP;
> + tsm_dev = tsm_get_device(idev->dev);
> + if (IS_ERR(tsm_dev)) {
> + rc = PTR_ERR(tsm_dev);
> + tsm_dev = NULL;
> goto out_put_idev;
> }
> - ops = iommu_dev->ops->get_viommu_ops(idev->dev, cmd->type);
> - if (!ops) {
> - rc = -EOPNOTSUPP;
> + ops = tsm_dev ? tsm_viommu_get_ops(tsm_dev, idev->dev,
> cmd->type) : NULL;
> + if (IS_ERR(ops)) {
> + rc = PTR_ERR(ops);
> goto out_put_idev;
> }
> + if (!ops) {
> + if (tsm_dev) {
> + tsm_put_device(tsm_dev);
nit: Since tsm_get_device() may legitimately return NULL, maybe you can
make tsm_put_device NULL safe, then you can avoid this check and other
places.
> + tsm_dev = NULL;
nit: maybe a candidate for cleanup helper? if tsm_put_device is null
safe. then:
tsm_put_device(no_free_ptr(tsm_dev));
> + }
> + if (!iommu_dev->ops->get_viommu_ops) {
> + rc = -EOPNOTSUPP;
> + goto out_put_idev;
> + }
> + ops = iommu_dev->ops->get_viommu_ops(idev->dev,
> cmd->type);
> + if (!ops) {
> + rc = -EOPNOTSUPP;
> + goto out_put_idev;
> + }
> + }
> +
> /*
> * It is a driver bug to omit the required operations or
> provide a size
> * smaller than the core vIOMMU structure.
> @@ -95,6 +116,8 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd
> *ucmd) viommu->kvm_file = get_file(idev->kvm_file);
> viommu->type = cmd->type;
> viommu->ictx = ucmd->ictx;
> + viommu->tsm_dev = tsm_dev;
> + tsm_dev = NULL;
viommu->tsm_dev = no_free_ptr(tsm_dev)
> viommu->hwpt = hwpt_paging;
> refcount_inc(&viommu->hwpt->common.obj.users);
> INIT_LIST_HEAD(&viommu->veventqs);
> @@ -119,6 +142,8 @@ int iommufd_viommu_alloc_ioctl(struct
> iommufd_ucmd *ucmd) out_put_hwpt:
> iommufd_put_object(ucmd->ictx, &hwpt_paging->common.obj);
> out_put_idev:
> + if (tsm_dev)
> + tsm_put_device(tsm_dev);
ditto, can be unconditional.
> iommufd_put_object(ucmd->ictx, &idev->obj);
> return rc;
> }
> diff --git a/drivers/pci/tsm/core.c b/drivers/pci/tsm/core.c
> index 8f0e5af808a5..4107ff443228 100644
> --- a/drivers/pci/tsm/core.c
> +++ b/drivers/pci/tsm/core.c
> @@ -9,6 +9,8 @@
> #define dev_fmt(fmt) "PCI/TSM: " fmt
>
> #include <linux/bitfield.h>
> +#include <linux/iommufd.h>
> +#include <linux/module.h>
> #include <linux/pci.h>
> #include <linux/pci-doe.h>
> #include <linux/pci-tsm.h>
> @@ -36,6 +38,26 @@ static const struct pci_tsm_ops
> *to_pci_tsm_ops(struct pci_tsm *tsm) return tsm->tsm_dev->pci_ops;
> }
>
> +struct tsm_dev *pci_tsm_get_device(struct pci_dev *pdev)
> +{
> + const struct pci_tsm_ops *ops;
> + struct tsm_dev *tsm_dev;
> +
> + guard(rwsem_read)(&pci_tsm_rwsem);
> + if (!pdev->tsm)
> + return NULL;
> +
> + tsm_dev = pdev->tsm->tsm_dev;
> + ops = tsm_dev->pci_ops;
> + if (!ops->viommu_get_ops)
> + return NULL;
> + if (!try_module_get(ops->owner))
> + return ERR_PTR(-ENODEV);
> + get_device(&tsm_dev->dev);
what if tsm_unregister() is called directly? without protection from
tsm_dev->users, unregister can succeed tearing down backend while this
reference is held.
For bisectable result, maybe you can move tsm_get() and wait patch
before this one?
> + return tsm_dev;
> +}
> +EXPORT_SYMBOL_GPL(pci_tsm_get_device);
> +
> static inline bool is_dsm(struct pci_dev *pdev)
> {
> return pdev->tsm && pdev->tsm->dsm_dev == pdev;
> diff --git a/drivers/virt/coco/tsm-core.c
> b/drivers/virt/coco/tsm-core.c index 181d6114086f..766f4ad67ed7 100644
> --- a/drivers/virt/coco/tsm-core.c
> +++ b/drivers/virt/coco/tsm-core.c
> @@ -9,6 +9,16 @@
> #include <linux/cleanup.h>
> #include <linux/pci-tsm.h>
>
> +/* The caller must hold a tsm_get_device() reference for the entire
> use. */ +const struct iommufd_viommu_ops *tsm_viommu_get_ops(struct
> tsm_dev *tsm_dev,
> + struct device *dev, enum iommu_viommu_type type)
> +{
> + if (!tsm_dev->pci_ops->viommu_get_ops)
> + return NULL;
> + return tsm_dev->pci_ops->viommu_get_ops(dev, type);
> +}
> +EXPORT_SYMBOL_GPL(tsm_viommu_get_ops);
> +
> static void tsm_release(struct device *);
> static const struct class tsm_class = {
> .name = "tsm",
> @@ -16,6 +26,33 @@ static const struct class tsm_class = {
> };
> static DEFINE_IDA(tsm_ida);
>
> +/**
> + * tsm_get_device() - Pin the TSM attached to a device
> + * @dev: device whose TSM is to be pinned
> + *
> + * A successful lookup takes a TSM device reference and pins the
> backend
> + * module. Release both with tsm_put_device(). Backends providing
> vIOMMU
> + * operations must not unregister their TSM independently of module
> unload.
> + * Return: NULL if no vIOMMU-capable TSM is attached, an error if
> the module
> + * cannot be pinned, or the TSM device associated with the pinned
> backend.
> + */
> +struct tsm_dev *tsm_get_device(struct device *dev)
> +{
> + if (!dev_is_pci(dev))
> + return NULL;
> + return pci_tsm_get_device(to_pci_dev(dev));
> +}
> +EXPORT_SYMBOL_GPL(tsm_get_device);
> +
> +void tsm_put_device(struct tsm_dev *tsm_dev)
> +{
> + struct module *owner = tsm_dev->pci_ops->owner;
> +
> + put_device(&tsm_dev->dev);
> + module_put(owner);
> +}
> +EXPORT_SYMBOL_GPL(tsm_put_device);
> +
> static int match_id(struct device *dev, const void *data)
> {
> struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev,
> dev); diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h
> index 5f5c67b24134..a4b29915d0ca 100644
> --- a/include/linux/iommufd.h
> +++ b/include/linux/iommufd.h
> @@ -23,6 +23,7 @@ struct iommufd_access;
> struct iommufd_ctx;
> struct iommufd_device;
> struct iommufd_viommu_ops;
> +struct tsm_dev;
> struct page;
>
> enum iommufd_object_type {
> @@ -104,6 +105,7 @@ struct iommufd_viommu {
> struct iommu_device *iommu_dev;
> struct iommufd_hwpt_paging *hwpt;
> struct file *kvm_file;
> + struct tsm_dev *tsm_dev;
>
> const struct iommufd_viommu_ops *ops;
>
> diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
> index 1458935234a4..27a76aeedf50 100644
> --- a/include/linux/pci-tsm.h
> +++ b/include/linux/pci-tsm.h
> @@ -3,7 +3,10 @@
> #define __PCI_TSM_H
> #include <linux/mutex.h>
> #include <linux/pci.h>
> +#include <uapi/linux/iommufd.h>
>
> +struct iommufd_viommu_ops;
> +struct module;
> struct pci_tsm;
> struct pci_tsm_context;
> struct tsm_dev;
> @@ -16,12 +19,27 @@ struct tsm_dev;
> * @devsec_ops: Lock, unlock, and interrogate the security state of
> the
> * function via the platform TSM (typically virtual
> function
> * operations).
> + * @viommu_get_ops: Return operations for a supported TSM-owned
> vIOMMU type,
> + * or NULL if unsupported
> + * @owner: Module providing the vIOMMU operations, if built as a
> module
> + *
> + * The vIOMMU owner holds a TSM device reference and pins the
> backend module
> + * before querying @viommu_get_ops, releasing both after vIOMMU
> destruction.
> + * A backend providing this callback must not unregister its TSM
> while the
> + * module is pinned.
> *
> * This operations are mutually exclusive either a tsm_dev instance
> * manages physical link properties or it manages function security
> * states like TDISP lock/unlock.
> + *
> + * @viommu_get_ops runs with the backend module pinned. The PCI/TSM
> + * association is checked during acquisition under pci_tsm_rwsem.
> */
> struct pci_tsm_ops {
> + struct module *owner;
> + const struct iommufd_viommu_ops *(*viommu_get_ops)(
> + struct device *dev, enum iommu_viommu_type type);
> +
> /*
> * struct pci_tsm_link_ops - Manage physical link and the
> TSM/DSM session
> * @probe: establish context with the TSM (allocate / wrap
> 'struct @@ -151,6 +169,7 @@ struct pci_tsm_pf0
> *pci_tsm_context_pf0(struct pci_tsm_context *context); struct pci_dev
> *pci_tsm_context_dsm_dev(struct pci_tsm_context *context); bool
> pci_tsm_context_match_device(struct pci_tsm_context *context, struct
> pci_dev *pdev); +struct tsm_dev *pci_tsm_get_device(struct pci_dev
> *pdev); #else
> static inline int pci_tsm_register(struct tsm_dev *tsm_dev)
> {
> @@ -165,6 +184,11 @@ static inline bool pci_tsm_is_configured(struct
> pci_dev *pdev) return false;
> }
>
> +static inline struct tsm_dev *pci_tsm_get_device(struct pci_dev
> *pdev) +{
> + return NULL;
> +}
> +
> static inline struct pci_tsm_context *
> pci_tsm_context_get(struct pci_dev *pdev, struct tsm_dev *tsm_dev)
> {
> diff --git a/include/linux/tsm.h b/include/linux/tsm.h
> index 19dedf971f14..3d5286aeaa42 100644
> --- a/include/linux/tsm.h
> +++ b/include/linux/tsm.h
> @@ -110,6 +110,8 @@ struct tsm_report_ops {
> };
>
> struct pci_tsm_ops;
> +struct iommufd_viommu_ops;
> +
> struct tsm_dev {
> struct device dev;
> int id;
> @@ -124,4 +126,27 @@ int tsm_report_unregister(const struct
> tsm_report_ops *ops); struct tsm_dev *tsm_register(struct device
> *parent, struct pci_tsm_ops *ops); void tsm_unregister(struct tsm_dev
> *tsm_dev); struct tsm_dev *find_tsm_dev(int id);
> +
> +#ifdef CONFIG_TSM
> +struct tsm_dev *tsm_get_device(struct device *dev);
> +void tsm_put_device(struct tsm_dev *tsm_dev);
> +const struct iommufd_viommu_ops *tsm_viommu_get_ops(struct tsm_dev
> *tsm_dev,
> + struct device *dev, enum iommu_viommu_type type);
> +#else
> +static inline struct tsm_dev *tsm_get_device(struct device *dev)
> +{
> + return NULL;
> +}
> +
> +static inline void tsm_put_device(struct tsm_dev *tsm_dev)
> +{
> +}
> +
> +static inline const struct iommufd_viommu_ops *
> +tsm_viommu_get_ops(struct tsm_dev *tsm_dev, struct device *dev,
> + enum iommu_viommu_type type)
> +{
> + return NULL;
> +}
> +#endif
> #endif /* __TSM_H */
With the above addressed,
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
next prev parent reply other threads:[~2026-11-19 19:50 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 8:19 [PATCH v9 00/13] iommufd: TSM-backed virtual IOMMUs for confidential guests Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 01/13] virt: coco: tsm: Require PCI operations when registering a TSM Aneesh Kumar K.V (Arm)
2026-11-19 7:18 ` Baolu Lu
2026-11-16 8:19 ` [PATCH v9 02/13] virt: coco: tsm: Remove the device from lookup before PCI teardown Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 03/13] virt: coco: tsm: Complete PCI registration before publishing the device Aneesh Kumar K.V (Arm)
2026-11-18 7:11 ` Xu Yilun
2026-11-16 8:19 ` [PATCH v9 04/13] iommufd/device: Associate KVM file pointer with iommufd_device Aneesh Kumar K.V (Arm)
2026-11-19 8:13 ` Baolu Lu
2026-11-16 8:19 ` [PATCH v9 05/13] iommufd/viommu: Keep a reference to the KVM file Aneesh Kumar K.V (Arm)
2026-11-19 8:20 ` Baolu Lu
2026-11-16 8:19 ` [PATCH v9 06/13] iommufd: Add a vDEVICE operation ioctl framework Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 07/13] PCI/TSM: Remove the legacy guest request interface Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 08/13] PCI/TSM: Keep connections active while device contexts exist Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 09/13] iommufd/viommu: Select vIOMMU operations before allocation Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 10/13] iommufd/viommu: Allow PCI TSM backends to provide vIOMMU operations Aneesh Kumar K.V (Arm)
2026-11-19 19:50 ` Jacob Pan [this message]
2026-11-16 8:19 ` [PATCH v9 11/13] iommufd/viommu: Allow vIOMMUs without a parent HWPT Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 12/13] PCI/TSM: Wait for device contexts before removing a DSM Aneesh Kumar K.V (Arm)
2026-11-16 8:19 ` [PATCH v9 13/13] PCI/TSM: Wait for users during unregister Aneesh Kumar K.V (Arm)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119115041.00007215@linux.microsoft.com \
--to=jacob.pan@linux.microsoft.com \
--cc=aik@amd.com \
--cc=alex@shazbot.org \
--cc=aneesh.kumar@kernel.org \
--cc=ashish.kalra@amd.com \
--cc=bhelgaas@google.com \
--cc=catalin.marinas@arm.com \
--cc=herbert@gondor.apana.org.au \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=jic23@kernel.org \
--cc=jonathan.cameron@huawei.com \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-pci@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=lukas@wunner.de \
--cc=nicolinc@nvidia.com \
--cc=patches@lists.linux.dev \
--cc=robin.murphy@arm.com \
--cc=sameo@rivosinc.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=steven.price@arm.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=suzuki.poulose@arm.com \
--cc=thierry.reding@kernel.org \
--cc=thomas.lendacky@amd.com \
--cc=vasant.hegde@amd.com \
--cc=vdumpa@nvidia.com \
--cc=will@kernel.org \
--cc=yilun.xu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox