[PATCH v2 00/23] Add SMMUv3 Stage 1 Support for Xen guests

[PATCH v2 00/23] Add SMMUv3 Stage 1 Support for Xen guests

[Milan Djokic]

This patch series provides emulated SMMUv3 support in Xen, enabling stage-1
translation for the guest OS.

Stage 1 translation support is required to provide isolation between different
devices within OS. Xen already supports Stage 2 translation but there is no
support for Stage 1 translation. The goal of this work is to support Stage 1
translation for Xen guests.

This patch series represents a continuation of work from Rahul Singh:
https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/
Original patch series is aligned with the newest Xen structure, with the addition
of translation layer which provides 1:N vIOMMU->pIOMMU mapping, in order to
support passthrough of the devices attached to different physical IOMMUs.

We cannot trust the guest OS to control the SMMUv3 hardware directly as
compromised guest OS can corrupt the SMMUv3 configuration and make the system
vulnerable. The guest gets the ownership of the stage 1 page tables and also
owns stage 1 configuration structures. The Xen handles the root configuration
structure (for security reasons), including the stage 2 configuration.

XEN will emulate the SMMUv3 hardware and expose the virtual SMMUv3 to the
guest. Guest can use the native SMMUv3 driver to configure the stage 1
translation. When the guest configures the SMMUv3 for Stage 1, XEN will trap
the access and configure hardware.

SMMUv3 Driver(Guest OS) -> Configure the Stage-1 translation ->
XEN trap access -> XEN SMMUv3 driver configure the HW.

The final patch series commit provides a design document for the emulated
IOMMU (arm-viommu.rst), which was previously discussed with the maintainers.
Details regarding implementation, future work and security risks are outlined
in this document.

---
Changes in v2:
 - Updated design and implementation with vIOMMU->pIOMMU mapping layer
 - Addressed security risks in the design, provided initial performance
   measurements
 - Addressed comments from previous version
 - Tested on Renesas R-Car platform, initial performance measurements for
   stage-1 vs stage-1-less guests
---

Jean-Philippe Brucker (1):
  xen/arm: smmuv3: Maintain a SID->device structure

Milan Djokic (3):
  xen/arm: vIOMMU vSID->pSID mapping layer
  libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping
  doc/arm: vIOMMU design document

Rahul Singh (19):
  xen/arm: smmuv3: Add support for stage-1 and nested stage translation
  xen/arm: smmuv3: Alloc io_domain for each device
  xen/arm: vIOMMU: add generic vIOMMU framework
  xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests
  xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param
  xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"
  xen/arm: vsmmuv3: Add support for registers emulation
  xen/arm: vsmmuv3: Add support for cmdqueue handling
  xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE
  xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware
  xen/arm: vsmmuv3: Add support for event queue and global error
  xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices
  xen/arm: vIOMMU: IOMMU device tree node for dom0
  xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less
  arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl
  xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3
  xen/arm: vsmmuv3: Add support to send stage-1 event to guest
  libxl/arm: vIOMMU: Modify the partial device tree for iommus
  xen/arm: vIOMMU: Modify the partial device tree for dom0less

 docs/designs/arm-viommu.rst             | 390 ++++++++++
 docs/man/xl.cfg.5.pod.in                |  13 +
 docs/misc/xen-command-line.pandoc       |   9 +
 tools/golang/xenlight/helpers.gen.go    |   2 +
 tools/golang/xenlight/types.gen.go      |   1 +
 tools/include/libxl.h                   |   5 +
 tools/include/xenctrl.h                 |  12 +
 tools/libs/ctrl/xc_domain.c             |  23 +
 tools/libs/light/libxl_arm.c            | 230 +++++-
 tools/libs/light/libxl_types.idl        |   6 +
 tools/xl/xl_parse.c                     |   9 +
 xen/arch/arm/dom0less-build.c           |  72 ++
 xen/arch/arm/domain.c                   |  26 +
 xen/arch/arm/domain_build.c             | 103 ++-
 xen/arch/arm/domctl.c                   |  31 +
 xen/arch/arm/include/asm/domain.h       |   4 +
 xen/arch/arm/include/asm/iommu.h        |   7 +
 xen/arch/arm/include/asm/viommu.h       | 113 +++
 xen/common/device-tree/dom0less-build.c |  55 +-
 xen/drivers/passthrough/Kconfig         |  13 +
 xen/drivers/passthrough/arm/Makefile    |   2 +
 xen/drivers/passthrough/arm/smmu-v3.c   | 369 ++++++++-
 xen/drivers/passthrough/arm/smmu-v3.h   |  49 +-
 xen/drivers/passthrough/arm/viommu.c    |  96 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c  | 958 ++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h  |  32 +
 xen/include/public/arch-arm.h           |  14 +-
 xen/include/public/device_tree_defs.h   |   1 +
 xen/include/public/domctl.h             |  18 +
 xen/include/xen/iommu.h                 |  13 +-
 xen/xsm/flask/hooks.c                   |   4 +
 xen/xsm/flask/policy/access_vectors     |   2 +
 32 files changed, 2627 insertions(+), 55 deletions(-)
 create mode 100644 docs/designs/arm-viommu.rst
 create mode 100644 xen/arch/arm/include/asm/viommu.h
 create mode 100644 xen/drivers/passthrough/arm/viommu.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h

-- 
2.43.0

[PATCH v2 01/23] xen/arm: smmuv3: Maintain a SID->device structure

[Milan Djokic]

From: Jean-Philippe Brucker <jean-philippe@linaro.org>

Backport Linux commit cdf315f907d4. This is the clean backport without
any changes.

When handling faults from the event or PRI queue, we need to find the
struct device associated with a SID. Add a rb_tree to keep track of
SIDs.

Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdf315f907d4
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 131 +++++++++++++++++++++-----
 xen/drivers/passthrough/arm/smmu-v3.h |  13 ++-
 2 files changed, 118 insertions(+), 26 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index bf153227db..73cc4ef08f 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -809,6 +809,27 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 	return 0;
 }
 
+__maybe_unused
+static struct arm_smmu_master *
+arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+{
+	struct rb_node *node;
+	struct arm_smmu_stream *stream;
+
+	node = smmu->streams.rb_node;
+	while (node) {
+		stream = rb_entry(node, struct arm_smmu_stream, node);
+		if (stream->id < sid)
+			node = node->rb_right;
+		else if (stream->id > sid)
+			node = node->rb_left;
+		else
+			return stream->master;
+	}
+
+	return NULL;
+}
+
 /* IRQ and event handlers */
 static void arm_smmu_evtq_tasklet(void *dev)
 {
@@ -1042,8 +1063,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
 	if (!master->ats_enabled)
 		return 0;
 
-	for (i = 0; i < master->num_sids; i++) {
-		cmd->atc.sid = master->sids[i];
+	for (i = 0; i < master->num_streams; i++) {
+		cmd->atc.sid = master->streams[i].id;
 		arm_smmu_cmdq_issue_cmd(master->smmu, cmd);
 	}
 
@@ -1271,13 +1292,13 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master)
 	int i, j;
 	struct arm_smmu_device *smmu = master->smmu;
 
-	for (i = 0; i < master->num_sids; ++i) {
-		u32 sid = master->sids[i];
+    for (i = 0; i < master->num_streams; ++i) {
+		u32 sid = master->streams[i].id;
 		__le64 *step = arm_smmu_get_step_for_sid(smmu, sid);
 
 		/* Bridged PCI devices may end up with duplicated IDs */
 		for (j = 0; j < i; j++)
-			if (master->sids[j] == sid)
+			if (master->streams[j].id == sid)
 				break;
 		if (j < i)
 			continue;
@@ -1486,6 +1507,80 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 
 	return sid < limit;
 }
+
+static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
+				  struct arm_smmu_master *master)
+{
+	int i;
+	int ret = 0;
+	struct arm_smmu_stream *new_stream, *cur_stream;
+	struct rb_node **new_node, *parent_node = NULL;
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
+
+	master->streams = _xzalloc_array(sizeof(*master->streams), sizeof(void *),
+					fwspec->num_ids);
+	if (!master->streams)
+		return -ENOMEM;
+	master->num_streams = fwspec->num_ids;
+
+	mutex_lock(&smmu->streams_mutex);
+	for (i = 0; i < fwspec->num_ids; i++) {
+		u32 sid = fwspec->ids[i];
+
+		new_stream = &master->streams[i];
+		new_stream->id = sid;
+		new_stream->master = master;
+
+		/*
+		 * Check the SIDs are in range of the SMMU and our stream table
+		 */
+		if (!arm_smmu_sid_in_range(smmu, sid)) {
+			ret = -ERANGE;
+			break;
+		}
+
+		/* Ensure l2 strtab is initialised */
+		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
+			ret = arm_smmu_init_l2_strtab(smmu, sid);
+			if (ret)
+				break;
+		}
+
+		/* Insert into SID tree */
+		new_node = &(smmu->streams.rb_node);
+		while (*new_node) {
+			cur_stream = rb_entry(*new_node, struct arm_smmu_stream,
+					      node);
+			parent_node = *new_node;
+			if (cur_stream->id > new_stream->id) {
+				new_node = &((*new_node)->rb_left);
+			} else if (cur_stream->id < new_stream->id) {
+				new_node = &((*new_node)->rb_right);
+			} else {
+				dev_warn(master->dev,
+					 "stream %u already in tree\n",
+					 cur_stream->id);
+				ret = -EINVAL;
+				break;
+			}
+		}
+		if (ret)
+			break;
+
+		rb_link_node(&new_stream->node, parent_node, new_node);
+		rb_insert_color(&new_stream->node, &smmu->streams);
+	}
+
+	if (ret) {
+		for (i--; i >= 0; i--)
+			rb_erase(&master->streams[i].node, &smmu->streams);
+		xfree(master->streams);
+	}
+	mutex_unlock(&smmu->streams_mutex);
+
+	return ret;
+}
+
 /* Forward declaration */
 static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev);
 static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct device *dev,
@@ -1495,7 +1590,7 @@ static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn,
 
 static int arm_smmu_add_device(u8 devfn, struct device *dev)
 {
-	int i, ret;
+	int ret;
 	struct arm_smmu_device *smmu;
 	struct arm_smmu_master *master;
 	struct iommu_fwspec *fwspec;
@@ -1532,26 +1627,11 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev)
 
 	master->dev = dev;
 	master->smmu = smmu;
-	master->sids = fwspec->ids;
-	master->num_sids = fwspec->num_ids;
 	dev_iommu_priv_set(dev, master);
 
-	/* Check the SIDs are in range of the SMMU and our stream table */
-	for (i = 0; i < master->num_sids; i++) {
-		u32 sid = master->sids[i];
-
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
-			goto err_free_master;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				goto err_free_master;
-		}
-	}
+	ret = arm_smmu_insert_master(smmu, master);
+	if (ret)
+		goto err_free_master;
 
 	/*
 	 * Note that PASID must be enabled before, and disabled after ATS:
@@ -1796,6 +1876,9 @@ static int __init arm_smmu_init_structures(struct arm_smmu_device *smmu)
 {
 	int ret;
 
+	mutex_init(&smmu->streams_mutex);
+	smmu->streams = RB_ROOT;
+
 	ret = arm_smmu_init_queues(smmu);
 	if (ret)
 		return ret;
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index ab07366294..ab1f29f6c7 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -639,6 +639,15 @@ struct arm_smmu_device {
 	struct tasklet		evtq_irq_tasklet;
 	struct tasklet		priq_irq_tasklet;
 	struct tasklet		combined_irq_tasklet;
+
+	struct rb_root		streams;
+	struct mutex		streams_mutex;
+};
+
+struct arm_smmu_stream {
+	u32							id;
+	struct arm_smmu_master		*master;
+	struct rb_node				node;
 };
 
 /* SMMU private data for each master */
@@ -647,8 +656,8 @@ struct arm_smmu_master {
 	struct device			*dev;
 	struct arm_smmu_domain		*domain;
 	struct list_head		domain_head;
-	u32				*sids;
-	unsigned int			num_sids;
+	struct arm_smmu_stream		*streams;
+	unsigned int				num_streams;
 	bool				ats_enabled;
 };
 
-- 
2.43.0

Re: [PATCH v2 01/23] xen/arm: smmuv3: Maintain a SID->device structure

[Mykola Kvach]

On 24/03/2026 00:51, Milan Djokic wrote:
> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
> 
> Backport Linux commit cdf315f907d4. This is the clean backport without
> any changes.
> 
> When handling faults from the event or PRI queue, we need to find the
> struct device associated with a SID. Add a rb_tree to keep track of
> SIDs.
> 
> Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Acked-by: Will Deacon <will@kernel.org>
> Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.org
> Signed-off-by: Joerg Roedel <jroedel@suse.de>
> Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdf315f907d4
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>

Your Signed-off-by is missing:
Signed-off-by: Milan Djokic <milan_djokic@epam.com>

> ---
>   xen/drivers/passthrough/arm/smmu-v3.c | 131 +++++++++++++++++++++-----
>   xen/drivers/passthrough/arm/smmu-v3.h |  13 ++-
>   2 files changed, 118 insertions(+), 26 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index bf153227db..73cc4ef08f 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -809,6 +809,27 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
>   	return 0;
>   }
>   
> +__maybe_unused
> +static struct arm_smmu_master *
> +arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
> +{
> +	struct rb_node *node;
> +	struct arm_smmu_stream *stream;
> +
> +	node = smmu->streams.rb_node;
> +	while (node) {
> +		stream = rb_entry(node, struct arm_smmu_stream, node);
> +		if (stream->id < sid)
> +			node = node->rb_right;
> +		else if (stream->id > sid)
> +			node = node->rb_left;
> +		else
> +			return stream->master;
> +	}
> +
> +	return NULL;
> +}
> +
>   /* IRQ and event handlers */
>   static void arm_smmu_evtq_tasklet(void *dev)
>   {
> @@ -1042,8 +1063,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
>   	if (!master->ats_enabled)
>   		return 0;
>   
> -	for (i = 0; i < master->num_sids; i++) {
> -		cmd->atc.sid = master->sids[i];
> +	for (i = 0; i < master->num_streams; i++) {
> +		cmd->atc.sid = master->streams[i].id;
>   		arm_smmu_cmdq_issue_cmd(master->smmu, cmd);
>   	}
>   
> @@ -1271,13 +1292,13 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master)
>   	int i, j;
>   	struct arm_smmu_device *smmu = master->smmu;
>   
> -	for (i = 0; i < master->num_sids; ++i) {
> -		u32 sid = master->sids[i];
> +    for (i = 0; i < master->num_streams; ++i) {
> +		u32 sid = master->streams[i].id;
>   		__le64 *step = arm_smmu_get_step_for_sid(smmu, sid);
>   
>   		/* Bridged PCI devices may end up with duplicated IDs */
>   		for (j = 0; j < i; j++)
> -			if (master->sids[j] == sid)
> +			if (master->streams[j].id == sid)
>   				break;
>   		if (j < i)
>   			continue;
> @@ -1486,6 +1507,80 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
>   
>   	return sid < limit;
>   }
> +
> +static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
> +				  struct arm_smmu_master *master)
> +{
> +	int i;
> +	int ret = 0;
> +	struct arm_smmu_stream *new_stream, *cur_stream;
> +	struct rb_node **new_node, *parent_node = NULL;
> +	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
> +
> +	master->streams = _xzalloc_array(sizeof(*master->streams), sizeof(void *),
> +					fwspec->num_ids);
> +	if (!master->streams)
> +		return -ENOMEM;
> +	master->num_streams = fwspec->num_ids;
> +
> +	mutex_lock(&smmu->streams_mutex);
> +	for (i = 0; i < fwspec->num_ids; i++) {
> +		u32 sid = fwspec->ids[i];
> +
> +		new_stream = &master->streams[i];
> +		new_stream->id = sid;
> +		new_stream->master = master;
> +
> +		/*
> +		 * Check the SIDs are in range of the SMMU and our stream table
> +		 */
> +		if (!arm_smmu_sid_in_range(smmu, sid)) {
> +			ret = -ERANGE;
> +			break;
> +		}
> +
> +		/* Ensure l2 strtab is initialised */
> +		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
> +			ret = arm_smmu_init_l2_strtab(smmu, sid);
> +			if (ret)
> +				break;
> +		}
> +
> +		/* Insert into SID tree */
> +		new_node = &(smmu->streams.rb_node);
> +		while (*new_node) {
> +			cur_stream = rb_entry(*new_node, struct arm_smmu_stream,
> +					      node);
> +			parent_node = *new_node;
> +			if (cur_stream->id > new_stream->id) {
> +				new_node = &((*new_node)->rb_left);
> +			} else if (cur_stream->id < new_stream->id) {
> +				new_node = &((*new_node)->rb_right);
> +			} else {
> +				dev_warn(master->dev,
> +					 "stream %u already in tree\n",
> +					 cur_stream->id);
> +				ret = -EINVAL;
> +				break;
> +			}
> +		}
> +		if (ret)
> +			break;
> +
> +		rb_link_node(&new_stream->node, parent_node, new_node);
> +		rb_insert_color(&new_stream->node, &smmu->streams);
> +	}
> +
> +	if (ret) {
> +		for (i--; i >= 0; i--)
> +			rb_erase(&master->streams[i].node, &smmu->streams);
> +		xfree(master->streams);
> +	}
> +	mutex_unlock(&smmu->streams_mutex);
> +
> +	return ret;
> +}
> +
>   /* Forward declaration */
>   static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev);
>   static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct device *dev,
> @@ -1495,7 +1590,7 @@ static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn,
>   
>   static int arm_smmu_add_device(u8 devfn, struct device *dev)
>   {
> -	int i, ret;
> +	int ret;
>   	struct arm_smmu_device *smmu;
>   	struct arm_smmu_master *master;
>   	struct iommu_fwspec *fwspec;
> @@ -1532,26 +1627,11 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev)
>   
>   	master->dev = dev;
>   	master->smmu = smmu;
> -	master->sids = fwspec->ids;
> -	master->num_sids = fwspec->num_ids;
>   	dev_iommu_priv_set(dev, master);
>   
> -	/* Check the SIDs are in range of the SMMU and our stream table */
> -	for (i = 0; i < master->num_sids; i++) {
> -		u32 sid = master->sids[i];
> -
> -		if (!arm_smmu_sid_in_range(smmu, sid)) {
> -			ret = -ERANGE;
> -			goto err_free_master;
> -		}
> -
> -		/* Ensure l2 strtab is initialised */
> -		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
> -			ret = arm_smmu_init_l2_strtab(smmu, sid);
> -			if (ret)
> -				goto err_free_master;
> -		}
> -	}
> +	ret = arm_smmu_insert_master(smmu, master);
> +	if (ret)
> +		goto err_free_master;
>   
>   	/*
>   	 * Note that PASID must be enabled before, and disabled after ATS:
> @@ -1796,6 +1876,9 @@ static int __init arm_smmu_init_structures(struct arm_smmu_device *smmu)
>   {
>   	int ret;
>   
> +	mutex_init(&smmu->streams_mutex);
> +	smmu->streams = RB_ROOT;
> +
>   	ret = arm_smmu_init_queues(smmu);
>   	if (ret)
>   		return ret;
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
> index ab07366294..ab1f29f6c7 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.h
> +++ b/xen/drivers/passthrough/arm/smmu-v3.h
> @@ -639,6 +639,15 @@ struct arm_smmu_device {
>   	struct tasklet		evtq_irq_tasklet;
>   	struct tasklet		priq_irq_tasklet;
>   	struct tasklet		combined_irq_tasklet;
> +
> +	struct rb_root		streams;
> +	struct mutex		streams_mutex;
> +};
> +
> +struct arm_smmu_stream {
> +	u32							id;
> +	struct arm_smmu_master		*master;
> +	struct rb_node				node;
>   };
>   
>   /* SMMU private data for each master */
> @@ -647,8 +656,8 @@ struct arm_smmu_master {
>   	struct device			*dev;
>   	struct arm_smmu_domain		*domain;
>   	struct list_head		domain_head;
> -	u32				*sids;
> -	unsigned int			num_sids;
> +	struct arm_smmu_stream		*streams;
> +	unsigned int				num_streams;
>   	bool				ats_enabled;
>   };
>   

[PATCH v2 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Xen SMMUv3 driver only supports stage-2 translation. Add support for
Stage-1 translation that is required to support nested stage
translation.

In true nested mode, both s1_cfg and s2_cfg will coexist.
Let's remove the union. When nested stage translation is setup, both
s1_cfg and s2_cfg are valid.

We introduce a new smmu_domain abort field that will be set
upon guest stage-1 configuration passing. If no guest stage-1
config has been attached, it is ignored when writing the STE.

arm_smmu_write_strtab_ent() is modified to write both stage
fields in the STE and deal with the abort field.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 93 +++++++++++++++++++++++----
 xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
 2 files changed, 91 insertions(+), 11 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 73cc4ef08f..f9c6837919 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 	 * 3. Update Config, sync
 	 */
 	u64 val = le64_to_cpu(dst[0]);
-	bool ste_live = false;
+	bool s1_live = false, s2_live = false, ste_live = false;
+	bool abort, translate = false;
 	struct arm_smmu_device *smmu = NULL;
+	struct arm_smmu_s1_cfg *s1_cfg = NULL;
 	struct arm_smmu_s2_cfg *s2_cfg = NULL;
 	struct arm_smmu_domain *smmu_domain = NULL;
 	struct arm_smmu_cmdq_ent prefetch_cmd = {
@@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		smmu = master->smmu;
 	}
 
-	if (smmu_domain)
-		s2_cfg = &smmu_domain->s2_cfg;
+	if (smmu_domain) {
+		switch (smmu_domain->stage) {
+		case ARM_SMMU_DOMAIN_NESTED:
+			s1_cfg = &smmu_domain->s1_cfg;
+			fallthrough;
+		case ARM_SMMU_DOMAIN_S2:
+			s2_cfg = &smmu_domain->s2_cfg;
+			break;
+		default:
+			break;
+		}
+		translate = !!s1_cfg || !!s2_cfg;
+	}
 
 	if (val & STRTAB_STE_0_V) {
 		switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
 		case STRTAB_STE_0_CFG_BYPASS:
 			break;
+		case STRTAB_STE_0_CFG_S1_TRANS:
+			s1_live = true;
+			break;
 		case STRTAB_STE_0_CFG_S2_TRANS:
-			ste_live = true;
+			s2_live = true;
+			break;
+		case STRTAB_STE_0_CFG_NESTED:
+			s1_live = true;
+			s2_live = true;
 			break;
 		case STRTAB_STE_0_CFG_ABORT:
-			BUG_ON(!disable_bypass);
 			break;
 		default:
 			BUG(); /* STE corruption */
 		}
 	}
 
+	ste_live = s1_live || s2_live;
+
 	/* Nuke the existing STE_0 value, as we're going to rewrite it */
 	val = STRTAB_STE_0_V;
 
 	/* Bypass/fault */
-	if (!smmu_domain || !(s2_cfg)) {
-		if (!smmu_domain && disable_bypass)
+	if (!smmu_domain)
+		abort = disable_bypass;
+	else
+		abort = smmu_domain->abort;
+
+	if (abort || !translate) {
+		if (abort)
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
 		else
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
@@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		return;
 	}
 
+	if (ste_live) {
+		/* First invalidate the live STE */
+		dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
+		arm_smmu_sync_ste_for_sid(smmu, sid);
+	}
+
+	if (s1_cfg) {
+		BUG_ON(s1_live);
+		dst[1] = cpu_to_le64(
+			 FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
+			 FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
+			 FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
+			 FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
+			 FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
+
+		if (smmu->features & ARM_SMMU_FEAT_STALLS &&
+		   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
+			dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
+
+		val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
+			FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
+			FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
+			FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
+	}
+
 	if (s2_cfg) {
+		u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
 		u64 strtab =
 			 FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
 			 FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
@@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 			 STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
 			 STRTAB_STE_2_S2R;
 
-		BUG_ON(ste_live);
+		if (s2_live) {
+			u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
+			BUG_ON(s2ttb != vttbr);
+		}
+
 		dst[2] = cpu_to_le64(strtab);
 
-		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
+		dst[3] = cpu_to_le64(vttbr);
 
 		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
+	} else {
+		dst[2] = 0;
+		dst[3] = 0;
 	}
 
 	if (master->ats_enabled)
@@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
 {
 	int ret;
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+	struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+	if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
+		(!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
+		 !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
+			dev_info(smmu_domain->smmu->dev,
+					"does not implement two stages\n");
+			return -EINVAL;
+	}
 
 	/* Restrict the stage to what we can actually support */
 	smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
@@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
 		break;
 	}
 
+	if (reg & IDR0_S1P)
+		smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
+
 	if (reg & IDR0_S2P)
 		smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
 
-	if (!(reg & IDR0_S2P)) {
-		dev_err(smmu->dev, "no stage-2 translation support!\n");
+	if (!(reg & (IDR0_S1P | IDR0_S2P))) {
+		dev_err(smmu->dev, "no translation support!\n");
 		return -ENXIO;
 	}
 
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index ab1f29f6c7..3fb13b7e21 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -197,6 +197,7 @@
 #define STRTAB_STE_0_CFG_BYPASS		4
 #define STRTAB_STE_0_CFG_S1_TRANS	5
 #define STRTAB_STE_0_CFG_S2_TRANS	6
+#define STRTAB_STE_0_CFG_NESTED		7
 
 #define STRTAB_STE_0_S1FMT		GENMASK_ULL(5, 4)
 #define STRTAB_STE_0_S1FMT_LINEAR	0
@@ -549,6 +550,12 @@ struct arm_smmu_strtab_l1_desc {
 	dma_addr_t			l2ptr_dma;
 };
 
+struct arm_smmu_s1_cfg {
+	u64				s1ctxptr;
+	u8				s1fmt;
+	u8				s1cdmax;
+};
+
 struct arm_smmu_s2_cfg {
 	u16				vmid;
 	u64				vttbr;
@@ -669,7 +676,9 @@ struct arm_smmu_domain {
 	atomic_t			nr_ats_masters;
 
 	enum arm_smmu_domain_stage	stage;
+	struct arm_smmu_s1_cfg	s1_cfg;
 	struct arm_smmu_s2_cfg	s2_cfg;
+	bool			abort;
 
 	/* Xen domain associated with this SMMU domain */
 	struct domain		*d;
-- 
2.43.0

Re: [PATCH v2 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Julien Grall]

Hi Milan,

On 24/03/2026 07:51, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Xen SMMUv3 driver only supports stage-2 translation. Add support for
> Stage-1 translation that is required to support nested stage
> translation.
> 
> In true nested mode, both s1_cfg and s2_cfg will coexist.
> Let's remove the union. When nested stage translation is setup, both
> s1_cfg and s2_cfg are valid.
> 
> We introduce a new smmu_domain abort field that will be set
> upon guest stage-1 configuration passing. If no guest stage-1
> config has been attached, it is ignored when writing the STE.
> 
> arm_smmu_write_strtab_ent() is modified to write both stage
> fields in the STE and deal with the abort field.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/drivers/passthrough/arm/smmu-v3.c | 93 +++++++++++++++++++++++----
>   xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
>   2 files changed, 91 insertions(+), 11 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index 73cc4ef08f..f9c6837919 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>   	 * 3. Update Config, sync
>   	 */
>   	u64 val = le64_to_cpu(dst[0]);
> -	bool ste_live = false;
> +	bool s1_live = false, s2_live = false, ste_live = false;
> +	bool abort, translate = false;
>   	struct arm_smmu_device *smmu = NULL;
> +	struct arm_smmu_s1_cfg *s1_cfg = NULL;
>   	struct arm_smmu_s2_cfg *s2_cfg = NULL;
>   	struct arm_smmu_domain *smmu_domain = NULL;
>   	struct arm_smmu_cmdq_ent prefetch_cmd = {
> @@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>   		smmu = master->smmu;
>   	}
>   
> -	if (smmu_domain)
> -		s2_cfg = &smmu_domain->s2_cfg;
> +	if (smmu_domain) {
> +		switch (smmu_domain->stage) {
> +		case ARM_SMMU_DOMAIN_NESTED:
> +			s1_cfg = &smmu_domain->s1_cfg;
> +			fallthrough;
> +		case ARM_SMMU_DOMAIN_S2:
> +			s2_cfg = &smmu_domain->s2_cfg;
> +			break;
> +		default:
> +			break;
> +		}
> +		translate = !!s1_cfg || !!s2_cfg;

NIT: translate is a bool. So do you actually need the !!?

> +	}
>   
>   	if (val & STRTAB_STE_0_V) {
>   		switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
>   		case STRTAB_STE_0_CFG_BYPASS:
>   			break;
> +		case STRTAB_STE_0_CFG_S1_TRANS:
> +			s1_live = true;
> +			break;
>   		case STRTAB_STE_0_CFG_S2_TRANS:
> -			ste_live = true;
> +			s2_live = true;
> +			break;
> +		case STRTAB_STE_0_CFG_NESTED:
> +			s1_live = true;
> +			s2_live = true;
>   			break;
>   		case STRTAB_STE_0_CFG_ABORT:
> -			BUG_ON(!disable_bypass);

I am not sure I understand why this was removed. Can you clarify?

>   			break;
>   		default:
>   			BUG(); /* STE corruption */
>   		}
>   	}
>   
> +	ste_live = s1_live || s2_live;
> +
>   	/* Nuke the existing STE_0 value, as we're going to rewrite it */
>   	val = STRTAB_STE_0_V;
>   
>   	/* Bypass/fault */
> -	if (!smmu_domain || !(s2_cfg)) {
> -		if (!smmu_domain && disable_bypass)
> +	if (!smmu_domain)
> +		abort = disable_bypass;
> +	else
> +		abort = smmu_domain->abort;
> +
> +	if (abort || !translate) {
> +		if (abort)
>   			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
>   		else
>   			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
> @@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>   		return;
>   	}
>   
> +	if (ste_live) {
> +		/* First invalidate the live STE */
> +		dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
> +		arm_smmu_sync_ste_for_sid(smmu, sid);
> +	}
> +
> +	if (s1_cfg) {
> +		BUG_ON(s1_live);
> +		dst[1] = cpu_to_le64(
> +			 FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
> +			 FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
> +			 FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
> +			 FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
> +			 FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
> +
> +		if (smmu->features & ARM_SMMU_FEAT_STALLS &&
> +		   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
> +			dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
> +
> +		val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
> +			FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
> +			FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
> +			FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
> +	}
> +
>   	if (s2_cfg) {
> +		u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
>   		u64 strtab =
>   			 FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
>   			 FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
> @@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>   			 STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
>   			 STRTAB_STE_2_S2R;
>   
> -		BUG_ON(ste_live);
> +		if (s2_live) {
> +			u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
> +			BUG_ON(s2ttb != vttbr);
> +		}
> +
>   		dst[2] = cpu_to_le64(strtab);
>   
> -		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
> +		dst[3] = cpu_to_le64(vttbr);
>   
>   		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
> +	} else {
> +		dst[2] = 0;
> +		dst[3] = 0;
>   	}
>   
>   	if (master->ats_enabled)
> @@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
>   {
>   	int ret;
>   	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> +	struct arm_smmu_device *smmu = smmu_domain->smmu;
> +
> +	if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
> +		(!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
> +		 !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
> +			dev_info(smmu_domain->smmu->dev,
> +					"does not implement two stages\n");
> +			return -EINVAL;
> +	}
>   
>   	/* Restrict the stage to what we can actually support */
>   	smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
> @@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
>   		break;
>   	}
>   
> +	if (reg & IDR0_S1P)
> +		smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
> +
>   	if (reg & IDR0_S2P)
>   		smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
>   
> -	if (!(reg & IDR0_S2P)) {
> -		dev_err(smmu->dev, "no stage-2 translation support!\n");
> +	if (!(reg & (IDR0_S1P | IDR0_S2P))) {

I believe the behavior is now slightly different. This would not print 
anything if the SMMU support stage-1 but not stage-2. However, Xen at 
least need the latter.

> +		dev_err(smmu->dev, "no translation support!\n");
>   		return -ENXIO;
>   	}
>   
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
> index ab1f29f6c7..3fb13b7e21 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.h
> +++ b/xen/drivers/passthrough/arm/smmu-v3.h
> @@ -197,6 +197,7 @@
>   #define STRTAB_STE_0_CFG_BYPASS		4
>   #define STRTAB_STE_0_CFG_S1_TRANS	5
>   #define STRTAB_STE_0_CFG_S2_TRANS	6
> +#define STRTAB_STE_0_CFG_NESTED		7
>   
>   #define STRTAB_STE_0_S1FMT		GENMASK_ULL(5, 4)
>   #define STRTAB_STE_0_S1FMT_LINEAR	0
> @@ -549,6 +550,12 @@ struct arm_smmu_strtab_l1_desc {
>   	dma_addr_t			l2ptr_dma;
>   };
>   
> +struct arm_smmu_s1_cfg {
> +	u64				s1ctxptr;
> +	u8				s1fmt;
> +	u8				s1cdmax;
> +};
> +
>   struct arm_smmu_s2_cfg {
>   	u16				vmid;
>   	u64				vttbr;
> @@ -669,7 +676,9 @@ struct arm_smmu_domain {
>   	atomic_t			nr_ats_masters;
>   
>   	enum arm_smmu_domain_stage	stage;
> +	struct arm_smmu_s1_cfg	s1_cfg;
>   	struct arm_smmu_s2_cfg	s2_cfg;
> +	bool			abort;

I think there might be some padding before s1_cfg. So I wonder whether 
this would be better be moved before to reduce the size of the structure.

>   
>   	/* Xen domain associated with this SMMU domain */
>   	struct domain		*d;

Cheers,

-- 
Julien Grall

Re: [PATCH v2 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Milan Djokic]

Hi Julien,

On 4/14/26 04:17, Julien Grall wrote:
> Hi Milan,
> 
> On 24/03/2026 07:51, Milan Djokic wrote:
>> From: Rahul Singh <rahul.singh@arm.com>
>>
>> Xen SMMUv3 driver only supports stage-2 translation. Add support for
>> Stage-1 translation that is required to support nested stage
>> translation.
>>
>> In true nested mode, both s1_cfg and s2_cfg will coexist.
>> Let's remove the union. When nested stage translation is setup, both
>> s1_cfg and s2_cfg are valid.
>>
>> We introduce a new smmu_domain abort field that will be set
>> upon guest stage-1 configuration passing. If no guest stage-1
>> config has been attached, it is ignored when writing the STE.
>>
>> arm_smmu_write_strtab_ent() is modified to write both stage
>> fields in the STE and deal with the abort field.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>>    xen/drivers/passthrough/arm/smmu-v3.c | 93 +++++++++++++++++++++++----
>>    xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
>>    2 files changed, 91 insertions(+), 11 deletions(-)
>>
>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
>> index 73cc4ef08f..f9c6837919 100644
>> --- a/xen/drivers/passthrough/arm/smmu-v3.c
>> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
>> @@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>>    	 * 3. Update Config, sync
>>    	 */
>>    	u64 val = le64_to_cpu(dst[0]);
>> -	bool ste_live = false;
>> +	bool s1_live = false, s2_live = false, ste_live = false;
>> +	bool abort, translate = false;
>>    	struct arm_smmu_device *smmu = NULL;
>> +	struct arm_smmu_s1_cfg *s1_cfg = NULL;
>>    	struct arm_smmu_s2_cfg *s2_cfg = NULL;
>>    	struct arm_smmu_domain *smmu_domain = NULL;
>>    	struct arm_smmu_cmdq_ent prefetch_cmd = {
>> @@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>>    		smmu = master->smmu;
>>    	}
>>    
>> -	if (smmu_domain)
>> -		s2_cfg = &smmu_domain->s2_cfg;
>> +	if (smmu_domain) {
>> +		switch (smmu_domain->stage) {
>> +		case ARM_SMMU_DOMAIN_NESTED:
>> +			s1_cfg = &smmu_domain->s1_cfg;
>> +			fallthrough;
>> +		case ARM_SMMU_DOMAIN_S2:
>> +			s2_cfg = &smmu_domain->s2_cfg;
>> +			break;
>> +		default:
>> +			break;
>> +		}
>> +		translate = !!s1_cfg || !!s2_cfg;
> 
> NIT: translate is a bool. So do you actually need the !!?
> 

No, !! is not necessary here, will fix this.

>> +	}
>>    
>>    	if (val & STRTAB_STE_0_V) {
>>    		switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
>>    		case STRTAB_STE_0_CFG_BYPASS:
>>    			break;
>> +		case STRTAB_STE_0_CFG_S1_TRANS:
>> +			s1_live = true;
>> +			break;
>>    		case STRTAB_STE_0_CFG_S2_TRANS:
>> -			ste_live = true;
>> +			s2_live = true;
>> +			break;
>> +		case STRTAB_STE_0_CFG_NESTED:
>> +			s1_live = true;
>> +			s2_live = true;
>>    			break;
>>    		case STRTAB_STE_0_CFG_ABORT:
>> -			BUG_ON(!disable_bypass);
> 
> I am not sure I understand why this was removed. Can you clarify?
> 

Yes. With the stage-1 support, abort is controlled per guest smmu 
configuration, so abort state is valid and not controlled by the global
disable_bypass, but with per-config smmu_domain->abort field instead.


>>    			break;
>>    		default:
>>    			BUG(); /* STE corruption */
>>    		}
>>    	}
>>    
>> +	ste_live = s1_live || s2_live;
>> +
>>    	/* Nuke the existing STE_0 value, as we're going to rewrite it */
>>    	val = STRTAB_STE_0_V;
>>    
>>    	/* Bypass/fault */
>> -	if (!smmu_domain || !(s2_cfg)) {
>> -		if (!smmu_domain && disable_bypass)
>> +	if (!smmu_domain)
>> +		abort = disable_bypass;
>> +	else
>> +		abort = smmu_domain->abort;
>> +
>> +	if (abort || !translate) {
>> +		if (abort)
>>    			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
>>    		else
>>    			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
>> @@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>>    		return;
>>    	}
>>    
>> +	if (ste_live) {
>> +		/* First invalidate the live STE */
>> +		dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
>> +		arm_smmu_sync_ste_for_sid(smmu, sid);
>> +	}
>> +
>> +	if (s1_cfg) {
>> +		BUG_ON(s1_live);
>> +		dst[1] = cpu_to_le64(
>> +			 FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
>> +			 FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
>> +			 FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
>> +			 FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
>> +			 FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
>> +
>> +		if (smmu->features & ARM_SMMU_FEAT_STALLS &&
>> +		   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
>> +			dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
>> +
>> +		val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
>> +			FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
>> +			FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
>> +			FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
>> +	}
>> +
>>    	if (s2_cfg) {
>> +		u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
>>    		u64 strtab =
>>    			 FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
>>    			 FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
>> @@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>>    			 STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
>>    			 STRTAB_STE_2_S2R;
>>    
>> -		BUG_ON(ste_live);
>> +		if (s2_live) {
>> +			u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
>> +			BUG_ON(s2ttb != vttbr);
>> +		}
>> +
>>    		dst[2] = cpu_to_le64(strtab);
>>    
>> -		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
>> +		dst[3] = cpu_to_le64(vttbr);
>>    
>>    		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
>> +	} else {
>> +		dst[2] = 0;
>> +		dst[3] = 0;
>>    	}
>>    
>>    	if (master->ats_enabled)
>> @@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
>>    {
>>    	int ret;
>>    	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
>> +	struct arm_smmu_device *smmu = smmu_domain->smmu;
>> +
>> +	if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
>> +		(!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
>> +		 !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
>> +			dev_info(smmu_domain->smmu->dev,
>> +					"does not implement two stages\n");
>> +			return -EINVAL;
>> +	}
>>    
>>    	/* Restrict the stage to what we can actually support */
>>    	smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
>> @@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
>>    		break;
>>    	}
>>    
>> +	if (reg & IDR0_S1P)
>> +		smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
>> +
>>    	if (reg & IDR0_S2P)
>>    		smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
>>    
>> -	if (!(reg & IDR0_S2P)) {
>> -		dev_err(smmu->dev, "no stage-2 translation support!\n");
>> +	if (!(reg & (IDR0_S1P | IDR0_S2P))) {
> 
> I believe the behavior is now slightly different. This would not print
> anything if the SMMU support stage-1 but not stage-2. However, Xen at
> least need the latter.
> 

The original idea was to also allow stage-1-only support. But I'm not 
sure if stage-1-only usecase is useful or even valid for Xen.. I will 
update the patch series with the missing parts for stage-1-only support, 
pointed out by Luca, but the question remains if this is needed at all. 
If not, I can revert to original state where stage-2 was always required.

>> +		dev_err(smmu->dev, "no translation support!\n");
>>    		return -ENXIO;
>>    	}
>>    
>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
>> index ab1f29f6c7..3fb13b7e21 100644
>> --- a/xen/drivers/passthrough/arm/smmu-v3.h
>> +++ b/xen/drivers/passthrough/arm/smmu-v3.h
>> @@ -197,6 +197,7 @@
>>    #define STRTAB_STE_0_CFG_BYPASS		4
>>    #define STRTAB_STE_0_CFG_S1_TRANS	5
>>    #define STRTAB_STE_0_CFG_S2_TRANS	6
>> +#define STRTAB_STE_0_CFG_NESTED		7
>>    
>>    #define STRTAB_STE_0_S1FMT		GENMASK_ULL(5, 4)
>>    #define STRTAB_STE_0_S1FMT_LINEAR	0
>> @@ -549,6 +550,12 @@ struct arm_smmu_strtab_l1_desc {
>>    	dma_addr_t			l2ptr_dma;
>>    };
>>    
>> +struct arm_smmu_s1_cfg {
>> +	u64				s1ctxptr;
>> +	u8				s1fmt;
>> +	u8				s1cdmax;
>> +};
>> +
>>    struct arm_smmu_s2_cfg {
>>    	u16				vmid;
>>    	u64				vttbr;
>> @@ -669,7 +676,9 @@ struct arm_smmu_domain {
>>    	atomic_t			nr_ats_masters;
>>    
>>    	enum arm_smmu_domain_stage	stage;
>> +	struct arm_smmu_s1_cfg	s1_cfg;
>>    	struct arm_smmu_s2_cfg	s2_cfg;
>> +	bool			abort;
> 
> I think there might be some padding before s1_cfg. So I wonder whether
> this would be better be moved before to reduce the size of the structure.
> 

Will do so.

>>    
>>    	/* Xen domain associated with this SMMU domain */
>>    	struct domain		*d;
> 
> Cheers,
> 

Best regards,
Milan

Re: [PATCH v2 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Julien Grall]

Hi Milan,

On 19/04/2026 18:34, Milan Djokic wrote:
> Hi Julien,
> 
> On 4/14/26 04:17, Julien Grall wrote:
>> Hi Milan,
>>
>> On 24/03/2026 07:51, Milan Djokic wrote:
>>> From: Rahul Singh <rahul.singh@arm.com>
>>>
>>> Xen SMMUv3 driver only supports stage-2 translation. Add support for
>>> Stage-1 translation that is required to support nested stage
>>> translation.
>>>
>>> In true nested mode, both s1_cfg and s2_cfg will coexist.
>>> Let's remove the union. When nested stage translation is setup, both
>>> s1_cfg and s2_cfg are valid.
>>>
>>> We introduce a new smmu_domain abort field that will be set
>>> upon guest stage-1 configuration passing. If no guest stage-1
>>> config has been attached, it is ignored when writing the STE.
>>>
>>> arm_smmu_write_strtab_ent() is modified to write both stage
>>> fields in the STE and deal with the abort field.
>>>
>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>>> ---
>>>    xen/drivers/passthrough/arm/smmu-v3.c | 93 ++++++++++++++++++++++ 
>>> +----
>>>    xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
>>>    2 files changed, 91 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/ 
>>> passthrough/arm/smmu-v3.c
>>> index 73cc4ef08f..f9c6837919 100644
>>> --- a/xen/drivers/passthrough/arm/smmu-v3.c
>>> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
>>> @@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct 
>>> arm_smmu_master *master, u32 sid,
>>>         * 3. Update Config, sync
>>>         */
>>>        u64 val = le64_to_cpu(dst[0]);
>>> -    bool ste_live = false;
>>> +    bool s1_live = false, s2_live = false, ste_live = false;
>>> +    bool abort, translate = false;
>>>        struct arm_smmu_device *smmu = NULL;
>>> +    struct arm_smmu_s1_cfg *s1_cfg = NULL;
>>>        struct arm_smmu_s2_cfg *s2_cfg = NULL;
>>>        struct arm_smmu_domain *smmu_domain = NULL;
>>>        struct arm_smmu_cmdq_ent prefetch_cmd = {
>>> @@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct 
>>> arm_smmu_master *master, u32 sid,
>>>            smmu = master->smmu;
>>>        }
>>> -    if (smmu_domain)
>>> -        s2_cfg = &smmu_domain->s2_cfg;
>>> +    if (smmu_domain) {
>>> +        switch (smmu_domain->stage) {
>>> +        case ARM_SMMU_DOMAIN_NESTED:
>>> +            s1_cfg = &smmu_domain->s1_cfg;
>>> +            fallthrough;
>>> +        case ARM_SMMU_DOMAIN_S2:
>>> +            s2_cfg = &smmu_domain->s2_cfg;
>>> +            break;
>>> +        default:
>>> +            break;
>>> +        }
>>> +        translate = !!s1_cfg || !!s2_cfg;
>>
>> NIT: translate is a bool. So do you actually need the !!?
>>
> 
> No, !! is not necessary here, will fix this.
> 
>>> +    }
>>>        if (val & STRTAB_STE_0_V) {
>>>            switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
>>>            case STRTAB_STE_0_CFG_BYPASS:
>>>                break;
>>> +        case STRTAB_STE_0_CFG_S1_TRANS:
>>> +            s1_live = true;
>>> +            break;
>>>            case STRTAB_STE_0_CFG_S2_TRANS:
>>> -            ste_live = true;
>>> +            s2_live = true;
>>> +            break;
>>> +        case STRTAB_STE_0_CFG_NESTED:
>>> +            s1_live = true;
>>> +            s2_live = true;
>>>                break;
>>>            case STRTAB_STE_0_CFG_ABORT:
>>> -            BUG_ON(!disable_bypass);
>>
>> I am not sure I understand why this was removed. Can you clarify?
>>
> 
> Yes. With the stage-1 support, abort is controlled per guest smmu 
> configuration, so abort state is valid and not controlled by the global
> disable_bypass, but with per-config smmu_domain->abort field instead.

Are we ok to allow the guest to control the bit? For instance, what does 
it mean if the guest decide to that no abort is necessary but the region 
is not mapped in stage-2?

[...]

> 
> The original idea was to also allow stage-1-only support. But I'm not 
> sure if stage-1-only usecase is useful or even valid for Xen.. I will 
> update the patch series with the missing parts for stage-1-only support, 
> pointed out by Luca, but the question remains if this is needed at all. 
> If not, I can revert to original state where stage-2 was always required.

By "stage-1 only" support, do you mean Xen would use the stage-1 in 
replacement of the stage-2? Or do you mean the guest will use the 
stage-1 page-table and there will be no isolation from Xen?

If the former, then I believe the page tables don't have the exact same 
format. Today, the page-tables are shared between the CPU and IOMMU, so 
this would need to be duplicated. For now, I am not sure this is worth 
to do.

If the latter, this would require the guest to be directly mapped (i.e. 
IPA == PA) but it would also open a big hole. So I would want to 
understand the exact use case first.

Cheers,

-- 
Julien Grall

Re: [PATCH v2 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Milan Djokic]

Hi Julien,

On 4/21/26 10:43, Julien Grall wrote:
> Hi Milan,
> 
> On 19/04/2026 18:34, Milan Djokic wrote:
>> Hi Julien,
>>
>> On 4/14/26 04:17, Julien Grall wrote:
>>> Hi Milan,
>>>
>>> On 24/03/2026 07:51, Milan Djokic wrote:
>>>> From: Rahul Singh <rahul.singh@arm.com>
>>>>
>>>> Xen SMMUv3 driver only supports stage-2 translation. Add support for
>>>> Stage-1 translation that is required to support nested stage
>>>> translation.
>>>>
>>>> In true nested mode, both s1_cfg and s2_cfg will coexist.
>>>> Let's remove the union. When nested stage translation is setup, both
>>>> s1_cfg and s2_cfg are valid.
>>>>
>>>> We introduce a new smmu_domain abort field that will be set
>>>> upon guest stage-1 configuration passing. If no guest stage-1
>>>> config has been attached, it is ignored when writing the STE.
>>>>
>>>> arm_smmu_write_strtab_ent() is modified to write both stage
>>>> fields in the STE and deal with the abort field.
>>>>
>>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>>>> ---
>>>>     xen/drivers/passthrough/arm/smmu-v3.c | 93 ++++++++++++++++++++++
>>>> +----
>>>>     xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
>>>>     2 files changed, 91 insertions(+), 11 deletions(-)
>>>>
>>>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/
>>>> passthrough/arm/smmu-v3.c
>>>> index 73cc4ef08f..f9c6837919 100644
>>>> --- a/xen/drivers/passthrough/arm/smmu-v3.c
>>>> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
>>>> @@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct
>>>> arm_smmu_master *master, u32 sid,
>>>>          * 3. Update Config, sync
>>>>          */
>>>>         u64 val = le64_to_cpu(dst[0]);
>>>> -    bool ste_live = false;
>>>> +    bool s1_live = false, s2_live = false, ste_live = false;
>>>> +    bool abort, translate = false;
>>>>         struct arm_smmu_device *smmu = NULL;
>>>> +    struct arm_smmu_s1_cfg *s1_cfg = NULL;
>>>>         struct arm_smmu_s2_cfg *s2_cfg = NULL;
>>>>         struct arm_smmu_domain *smmu_domain = NULL;
>>>>         struct arm_smmu_cmdq_ent prefetch_cmd = {
>>>> @@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct
>>>> arm_smmu_master *master, u32 sid,
>>>>             smmu = master->smmu;
>>>>         }
>>>> -    if (smmu_domain)
>>>> -        s2_cfg = &smmu_domain->s2_cfg;
>>>> +    if (smmu_domain) {
>>>> +        switch (smmu_domain->stage) {
>>>> +        case ARM_SMMU_DOMAIN_NESTED:
>>>> +            s1_cfg = &smmu_domain->s1_cfg;
>>>> +            fallthrough;
>>>> +        case ARM_SMMU_DOMAIN_S2:
>>>> +            s2_cfg = &smmu_domain->s2_cfg;
>>>> +            break;
>>>> +        default:
>>>> +            break;
>>>> +        }
>>>> +        translate = !!s1_cfg || !!s2_cfg;
>>>
>>> NIT: translate is a bool. So do you actually need the !!?
>>>
>>
>> No, !! is not necessary here, will fix this.
>>
>>>> +    }
>>>>         if (val & STRTAB_STE_0_V) {
>>>>             switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
>>>>             case STRTAB_STE_0_CFG_BYPASS:
>>>>                 break;
>>>> +        case STRTAB_STE_0_CFG_S1_TRANS:
>>>> +            s1_live = true;
>>>> +            break;
>>>>             case STRTAB_STE_0_CFG_S2_TRANS:
>>>> -            ste_live = true;
>>>> +            s2_live = true;
>>>> +            break;
>>>> +        case STRTAB_STE_0_CFG_NESTED:
>>>> +            s1_live = true;
>>>> +            s2_live = true;
>>>>                 break;
>>>>             case STRTAB_STE_0_CFG_ABORT:
>>>> -            BUG_ON(!disable_bypass);
>>>
>>> I am not sure I understand why this was removed. Can you clarify?
>>>
>>
>> Yes. With the stage-1 support, abort is controlled per guest smmu
>> configuration, so abort state is valid and not controlled by the global
>> disable_bypass, but with per-config smmu_domain->abort field instead.
> 
> Are we ok to allow the guest to control the bit? For instance, what does
> it mean if the guest decide to that no abort is necessary but the region
> is not mapped in stage-2?
> 

If the guest disables abort and translation is enabled, accesses to 
regions not mapped in stage-2 will still generate stage-2 translation 
faults. This won’t result in bypass.
In nested mode, both stage-1 and stage-2 are programmed, so stage-2 is 
always active. However, missing or invalid stage-2 mappings can still 
occur, and in such cases generating a stage-2 fault is the expected 
behavior, and would typically indicate either an invalid access from the 
guest or a missing stage-2 mapping.


> [...]
> 
>>
>> The original idea was to also allow stage-1-only support. But I'm not
>> sure if stage-1-only usecase is useful or even valid for Xen.. I will
>> update the patch series with the missing parts for stage-1-only support,
>> pointed out by Luca, but the question remains if this is needed at all.
>> If not, I can revert to original state where stage-2 was always required.
> 
> By "stage-1 only" support, do you mean Xen would use the stage-1 in
> replacement of the stage-2? Or do you mean the guest will use the
> stage-1 page-table and there will be no isolation from Xen?
> 
> If the former, then I believe the page tables don't have the exact same
> format. Today, the page-tables are shared between the CPU and IOMMU, so
> this would need to be duplicated. For now, I am not sure this is worth
> to do.
> 
> If the latter, this would require the guest to be directly mapped (i.e.
> IPA == PA) but it would also open a big hole. So I would want to
> understand the exact use case first.
> 

The latter. In this case, the guest would configure stage-1 while 
stage-2 translation is not used, so there is no additional isolation 
enforced by Xen. This would only be intended for specific usecases with 
trusted domains. But yes, this opens a significant hole if used with 
untrusted guests. If there is no strong usecase, we could restrict the 
implementation to always require stage-2.

> Cheers,
> 

Best regards,
Milan

[PATCH v2 03/23] xen/arm: smmuv3: Alloc io_domain for each device

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

In current implementation io_domain is allocated once for each xen
domain as Stage2 translation is common for all devices in same xen
domain.

Nested stage supports S1 and S2 configuration at the same time. Stage1
translation will be different for each device as linux kernel will
allocate page-table for each device.

Alloc io_domain for each device so that each device can have different
Stage-1 and Stage-2 configuration structure.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index f9c6837919..19e55b6c9b 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -2809,11 +2809,13 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
 static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 				struct device *dev)
 {
+	unsigned long flags;
 	struct iommu_domain *io_domain;
 	struct arm_smmu_domain *smmu_domain;
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
 	struct arm_smmu_device *smmu = arm_smmu_get_by_dev(fwspec->iommu_dev);
+	struct arm_smmu_master *master;
 
 	if (!smmu)
 		return NULL;
@@ -2824,8 +2826,15 @@ static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 	 */
 	list_for_each_entry(io_domain, &xen_domain->contexts, list) {
 		smmu_domain = to_smmu_domain(io_domain);
-		if (smmu_domain->smmu == smmu)
-			return io_domain;
+
+		spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+		list_for_each_entry(master, &smmu_domain->devices, domain_head) {
+			if (master->dev == dev) {
+				spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+				return io_domain;
+			}
+		}
+		spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
 	}
 	return NULL;
 }
-- 
2.43.0

[PATCH v2 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

This patch adds basic framework for vIOMMU.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/dom0less-build.c        |  2 +
 xen/arch/arm/domain.c                | 25 ++++++++++
 xen/arch/arm/domain_build.c          |  2 +
 xen/arch/arm/include/asm/viommu.h    | 70 ++++++++++++++++++++++++++++
 xen/drivers/passthrough/Kconfig      |  5 ++
 xen/drivers/passthrough/arm/Makefile |  1 +
 xen/drivers/passthrough/arm/viommu.c | 48 +++++++++++++++++++
 xen/include/public/arch-arm.h        |  4 ++
 8 files changed, 157 insertions(+)
 create mode 100644 xen/arch/arm/include/asm/viommu.h
 create mode 100644 xen/drivers/passthrough/arm/viommu.c

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 4181c10538..067835e5d0 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -23,6 +23,7 @@
 #include <asm/arm64/sve.h>
 #include <asm/domain_build.h>
 #include <asm/firmware/sci.h>
+#include <asm/viommu.h>
 #include <asm/grant_table.h>
 #include <asm/setup.h>
 
@@ -317,6 +318,7 @@ int __init arch_parse_dom0less_node(struct dt_device_node *node,
     uint32_t val;
 
     d_cfg->arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
+    d_cfg->arch.viommu_type = viommu_get_type();
     d_cfg->flags |= XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap;
 
     if ( domu_dt_sci_parse(node, d_cfg) )
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 94b9858ad2..37511c9323 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -28,6 +28,7 @@
 #include <asm/tee/tee.h>
 #include <asm/vfp.h>
 #include <asm/vgic.h>
+#include <asm/viommu.h>
 #include <asm/vtimer.h>
 
 #include "vpci.h"
@@ -626,6 +627,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
+         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    {
+        dprintk(XENLOG_INFO,
+                "vIOMMU requested while iommu not enabled for domain\n");
+        return -EINVAL;
+    }
+
+    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    {
+        dprintk(XENLOG_INFO,
+                "vIOMMU type requested not supported by the platform or Xen\n");
+        return -EINVAL;
+    }
+
     return sci_domain_sanitise_config(config);
 }
 
@@ -721,6 +737,9 @@ int arch_domain_create(struct domain *d,
     if ( (rc = sci_domain_init(d, config)) != 0 )
         goto fail;
 
+    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
+        goto fail;
+
     return 0;
 
 fail:
@@ -965,6 +984,7 @@ enum {
     PROG_pci = 1,
     PROG_sci,
     PROG_tee,
+    PROG_viommu,
     PROG_xen,
     PROG_page,
     PROG_mapping,
@@ -1021,6 +1041,11 @@ int domain_relinquish_resources(struct domain *d)
         if (ret )
             return ret;
 
+    PROGRESS(viommu):
+        ret = viommu_relinquish_resources(d);
+        if (ret )
+            return ret;
+
     PROGRESS(xen):
         ret = relinquish_memory(d, &d->xenpage_list);
         if ( ret )
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index e8795745dd..a51563ee3d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -35,6 +35,7 @@
 #include <asm/arm64/sve.h>
 #include <asm/cpufeature.h>
 #include <asm/domain_build.h>
+#include <asm/viommu.h>
 #include <xen/event.h>
 
 #include <xen/irq.h>
@@ -1946,6 +1947,7 @@ void __init create_dom0(void)
     dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
     dom0_cfg.arch.tee_type = tee_get_type();
     dom0_cfg.max_vcpus = dom0_max_vcpus();
+    dom0_cfg.arch.viommu_type = viommu_get_type();
 
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
new file mode 100644
index 0000000000..7cd3818a12
--- /dev/null
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+#ifndef __ARCH_ARM_VIOMMU_H__
+#define __ARCH_ARM_VIOMMU_H__
+
+#ifdef CONFIG_VIRTUAL_IOMMU
+
+#include <xen/lib.h>
+#include <xen/types.h>
+#include <public/xen.h>
+
+struct viommu_ops {
+    /*
+     * Called during domain construction if toolstack requests to enable
+     * vIOMMU support.
+     */
+    int (*domain_init)(struct domain *d);
+
+    /*
+     * Called during domain destruction to free resources used by vIOMMU.
+     */
+    int (*relinquish_resources)(struct domain *d);
+};
+
+struct viommu_desc {
+    /* vIOMMU domains init/free operations described above. */
+    const struct viommu_ops *ops;
+
+    /*
+     * ID of vIOMMU. Corresponds to xen_arch_domainconfig.viommu_type.
+     * Should be one of XEN_DOMCTL_CONFIG_VIOMMU_xxx
+     */
+    uint16_t viommu_type;
+};
+
+int domain_viommu_init(struct domain *d, uint16_t viommu_type);
+int viommu_relinquish_resources(struct domain *d);
+uint16_t viommu_get_type(void);
+
+#else
+
+static inline uint8_t viommu_get_type(void)
+{
+    return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+}
+
+static inline int domain_viommu_init(struct domain *d, uint16_t viommu_type)
+{
+    if ( likely(viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE) )
+        return 0;
+
+    return -ENODEV;
+}
+
+static inline int viommu_relinquish_resources(struct domain *d)
+{
+    return 0;
+}
+
+#endif /* CONFIG_VIRTUAL_IOMMU */
+
+#endif /* __ARCH_ARM_VIOMMU_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index b413c33a4c..487331607c 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -35,6 +35,11 @@ config IPMMU_VMSA
 	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
 	  translation table format and is able to use CPU's P2M table as is.
 
+config VIRTUAL_IOMMU
+	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
+	help
+	 Support virtual IOMMU infrastructure to implement vIOMMU.
+
 endif
 
 config AMD_IOMMU
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index c5fb3b58a5..4cc54f3f4d 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -2,3 +2,4 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
+obj-$(CONFIG_VIRTUAL_IOMMU) += viommu.o
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
new file mode 100644
index 0000000000..7ab6061e34
--- /dev/null
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/types.h>
+
+#include <asm/viommu.h>
+
+const struct viommu_desc __read_mostly *cur_viommu;
+
+int domain_viommu_init(struct domain *d, uint16_t viommu_type)
+{
+    if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+        return 0;
+
+    if ( !cur_viommu )
+        return -ENODEV;
+
+    if ( cur_viommu->viommu_type != viommu_type )
+        return -EINVAL;
+
+    return cur_viommu->ops->domain_init(d);
+}
+
+int viommu_relinquish_resources(struct domain *d)
+{
+    if ( !cur_viommu )
+        return 0;
+
+    return cur_viommu->ops->relinquish_resources(d);
+}
+
+uint16_t viommu_get_type(void)
+{
+    if ( !cur_viommu )
+        return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+
+    return cur_viommu->viommu_type;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index cd563cf706..b1db2bf19d 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -330,12 +330,16 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
 #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
 #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
 
+#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
+
 struct xen_arch_domainconfig {
     /* IN/OUT */
     uint8_t gic_version;
     /* IN - Contains SVE vector length divided by 128 */
     uint8_t sve_vl;
     /* IN */
+    uint8_t viommu_type;
+    /* IN */
     uint16_t tee_type;
     /* IN */
     uint32_t nr_spis;
-- 
2.43.0

Re: [PATCH v2 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Jan Beulich]

On 23.03.2026 23:51, Milan Djokic wrote:
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -35,6 +35,11 @@ config IPMMU_VMSA
>  	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
>  	  translation table format and is able to use CPU's P2M table as is.
>  
> +config VIRTUAL_IOMMU
> +	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
> +	help
> +	 Support virtual IOMMU infrastructure to implement vIOMMU.
> +
>  endif

This is being added to an Arm-only section, without it having an Arm-only
name. Judging from the diffstat in the cover letter, there's hardly any
abstraction in common code, so likely adding an ARM_ prefix to the name
would be the way to go (for now, i.e. if doing proper abstraction is
specifically not a goal).

Jan

Re: [PATCH v2 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Milan Djokic]

Hello Jan,

On 3/24/26 09:27, Jan Beulich wrote:
> On 23.03.2026 23:51, Milan Djokic wrote:
>> --- a/xen/drivers/passthrough/Kconfig
>> +++ b/xen/drivers/passthrough/Kconfig
>> @@ -35,6 +35,11 @@ config IPMMU_VMSA
>>   	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
>>   	  translation table format and is able to use CPU's P2M table as is.
>>   
>> +config VIRTUAL_IOMMU
>> +	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
>> +	help
>> +	 Support virtual IOMMU infrastructure to implement vIOMMU.
>> +
>>   endif
> 
> This is being added to an Arm-only section, without it having an Arm-only
> name. Judging from the diffstat in the cover letter, there's hardly any
> abstraction in common code, so likely adding an ARM_ prefix to the name
> would be the way to go (for now, i.e. if doing proper abstraction is
> specifically not a goal).
> 
> Jan

The idea is to reuse (if possible) the generic framework covered by this 
configuration option for RISC-V in the future. At the moment, most of 
the implementation resides under the ARM section, since only SMMUv3 
hardware is supported. I will add the ARM prefix for now and update the 
other patches according to the review comments.

BR,
Milan

Re: [PATCH v2 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Jan Beulich]

On 23.03.2026 23:51, Milan Djokic wrote:
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -330,12 +330,16 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>  #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>  #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>  
> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
> +
>  struct xen_arch_domainconfig {
>      /* IN/OUT */
>      uint8_t gic_version;
>      /* IN - Contains SVE vector length divided by 128 */
>      uint8_t sve_vl;
>      /* IN */
> +    uint8_t viommu_type;
> +    /* IN */
>      uint16_t tee_type;
>      /* IN */
>      uint32_t nr_spis;

You're altering structure layout, and hence an interface version change is
needed. Adding at the end would be less intrusive (no real change to layout),
but still would require an interface version bump due to the lack of
checking that the 3 padding bytes are actually 0. (IOW to help future
changes, you may want to add explicit padding bytes along with the new field
you need, and check them to be zero.)

Jan

[PATCH v2 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

domain_viommu_init() will be called during domain creation and will add
the dummy trap handler for virtual IOMMUs for guests.

A host IOMMU list will be created when host IOMMU devices are probed
and this list will be used to create the IOMMU device tree node for
dom0. For dom0, 1-1 mapping will be established between vIOMMU in dom0
and physical IOMMU.

For domUs, the 1-N mapping will be established between domU and physical
IOMMUs. A new area has been reserved in the arm guest physical map at
which the emulated vIOMMU node is created in the device tree.

Also set the vIOMMU type to vSMMUv3 to enable vIOMMU framework to call
vSMMUv3 domain creation/destroy functions.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain.c                  |   3 +-
 xen/arch/arm/include/asm/domain.h      |   4 +
 xen/arch/arm/include/asm/viommu.h      |  20 ++++
 xen/drivers/passthrough/Kconfig        |   8 ++
 xen/drivers/passthrough/arm/Makefile   |   1 +
 xen/drivers/passthrough/arm/smmu-v3.c  |   7 ++
 xen/drivers/passthrough/arm/viommu.c   |  30 ++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 124 +++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h |  20 ++++
 xen/include/public/arch-arm.h          |   7 +-
 10 files changed, 222 insertions(+), 2 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 37511c9323..badfdbfc63 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -635,7 +635,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
-    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE &&
+         config->arch.viommu_type != viommu_get_type() )
     {
         dprintk(XENLOG_INFO,
                 "vIOMMU type requested not supported by the platform or Xen\n");
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index 758ad807e4..35ff5c81bd 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -126,6 +126,10 @@ struct arch_domain
     void *sci_data;
 #endif
 
+#ifdef CONFIG_VIRTUAL_IOMMU
+    struct list_head viommu_list;     /* List of virtual IOMMUs */
+#endif
+
 }  __cacheline_aligned;
 
 struct arch_vcpu
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 7cd3818a12..4785877e2a 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -5,9 +5,21 @@
 #ifdef CONFIG_VIRTUAL_IOMMU
 
 #include <xen/lib.h>
+#include <xen/list.h>
 #include <xen/types.h>
 #include <public/xen.h>
 
+extern struct list_head host_iommu_list;
+
+/* data structure for each hardware IOMMU */
+struct host_iommu {
+    struct list_head entry;
+    const struct dt_device_node *dt_node;
+    paddr_t addr;
+    paddr_t size;
+    uint32_t irq;
+};
+
 struct viommu_ops {
     /*
      * Called during domain construction if toolstack requests to enable
@@ -35,6 +47,8 @@ struct viommu_desc {
 int domain_viommu_init(struct domain *d, uint16_t viommu_type);
 int viommu_relinquish_resources(struct domain *d);
 uint16_t viommu_get_type(void);
+void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                            const struct dt_device_node *node);
 
 #else
 
@@ -56,6 +70,12 @@ static inline int viommu_relinquish_resources(struct domain *d)
     return 0;
 }
 
+static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                                          const struct dt_device_node *node)
+{
+    return;
+}
+
 #endif /* CONFIG_VIRTUAL_IOMMU */
 
 #endif /* __ARCH_ARM_VIOMMU_H__ */
diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index 487331607c..54a72303d8 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -40,6 +40,14 @@ config VIRTUAL_IOMMU
 	help
 	 Support virtual IOMMU infrastructure to implement vIOMMU.
 
+config VIRTUAL_ARM_SMMU_V3
+	bool "ARM Ltd. Virtual SMMUv3 Support (UNSUPPORTED)" if UNSUPPORTED
+	depends on ARM_SMMU_V3 && VIRTUAL_IOMMU
+	help
+	 Support for implementations of the virtual ARM System MMU architecture
+	 version 3. Virtual SMMUv3 is unsupported feature and should not be used
+	 in production.
+
 endif
 
 config AMD_IOMMU
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index 4cc54f3f4d..e758a9d6aa 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -3,3 +3,4 @@ obj-$(CONFIG_ARM_SMMU) += smmu.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
 obj-$(CONFIG_VIRTUAL_IOMMU) += viommu.o
+obj-$(CONFIG_VIRTUAL_ARM_SMMU_V3) += vsmmu-v3.o
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 19e55b6c9b..87612df21d 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -93,6 +93,7 @@
 #include <asm/platform.h>
 
 #include "smmu-v3.h"
+#include "vsmmu-v3.h"
 
 #define ARM_SMMU_VTCR_SH_IS		3
 #define ARM_SMMU_VTCR_RGN_WBWA		1
@@ -2727,6 +2728,9 @@ static int __init arm_smmu_device_probe(struct platform_device *pdev)
 	list_add(&smmu->devices, &arm_smmu_devices);
 	spin_unlock(&arm_smmu_devices_lock);
 
+    /* Add to host IOMMU list to initialize vIOMMU for dom0 */
+	add_to_host_iommu_list(ioaddr, iosize, dev_to_dt(pdev));
+
 	return 0;
 
 
@@ -3058,6 +3062,9 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
 
 	platform_features &= smmu->features;
 
+	/* Set vIOMMU type to SMMUv3 */
+	vsmmuv3_set_type();
+
 	return 0;
 }
 
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 7ab6061e34..53ae46349a 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -2,12 +2,42 @@
 
 #include <xen/errno.h>
 #include <xen/init.h>
+#include <xen/irq.h>
 #include <xen/types.h>
 
 #include <asm/viommu.h>
 
+/* List of all host IOMMUs */
+LIST_HEAD(host_iommu_list);
+
 const struct viommu_desc __read_mostly *cur_viommu;
 
+/* Common function for adding to host_iommu_list */
+void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                            const struct dt_device_node *node)
+{
+    struct host_iommu *iommu_data;
+
+    iommu_data = xzalloc(struct host_iommu);
+    if ( !iommu_data )
+        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");
+
+    iommu_data->addr = addr;
+    iommu_data->size = size;
+    iommu_data->dt_node = node;
+    iommu_data->irq = platform_get_irq(node, 0);
+    if ( iommu_data->irq < 0 )
+    {
+        gdprintk(XENLOG_ERR,
+                 "vIOMMU: Cannot find a valid IOMMU irq\n");
+        return;
+    }
+
+    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
+
+    list_add_tail(&iommu_data->entry, &host_iommu_list);
+}
+
 int domain_viommu_init(struct domain *d, uint16_t viommu_type)
 {
     if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
new file mode 100644
index 0000000000..6b4009e5ef
--- /dev/null
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+
+#include <xen/param.h>
+#include <xen/sched.h>
+#include <asm/mmio.h>
+#include <asm/viommu.h>
+
+/* Struct to hold the vIOMMU ops and vIOMMU type */
+extern const struct viommu_desc __read_mostly *cur_viommu;
+
+struct virt_smmu {
+    struct      domain *d;
+    struct      list_head viommu_list;
+};
+
+static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
+                              register_t r, void *priv)
+{
+    return IO_HANDLED;
+}
+
+static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
+                             register_t *r, void *priv)
+{
+    return IO_HANDLED;
+}
+
+static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
+    .read  = vsmmuv3_mmio_read,
+    .write = vsmmuv3_mmio_write,
+};
+
+static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
+{
+    struct virt_smmu *smmu;
+
+    smmu = xzalloc(struct virt_smmu);
+    if ( !smmu )
+        return -ENOMEM;
+
+    smmu->d = d;
+
+    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
+
+    /* Register the vIOMMU to be able to clean it up later. */
+    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
+
+    return 0;
+}
+
+int domain_vsmmuv3_init(struct domain *d)
+{
+    int ret;
+    INIT_LIST_HEAD(&d->arch.viommu_list);
+
+    if ( is_hardware_domain(d) )
+    {
+        struct host_iommu *hw_iommu;
+
+        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
+        {
+            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
+            if ( ret )
+                return ret;
+        }
+    }
+    else
+    {
+        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+        if ( ret )
+            return ret;
+    }
+
+    return 0;
+}
+
+int vsmmuv3_relinquish_resources(struct domain *d)
+{
+    struct virt_smmu *pos, *temp;
+
+    /* Cope with unitialized vIOMMU */
+    if ( list_head_is_null(&d->arch.viommu_list) )
+        return 0;
+
+    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
+    {
+        list_del(&pos->viommu_list);
+        xfree(pos);
+    }
+
+    return 0;
+}
+
+static const struct viommu_ops vsmmuv3_ops = {
+    .domain_init = domain_vsmmuv3_init,
+    .relinquish_resources = vsmmuv3_relinquish_resources,
+};
+
+static const struct viommu_desc vsmmuv3_desc = {
+    .ops = &vsmmuv3_ops,
+    .viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3,
+};
+
+void __init vsmmuv3_set_type(void)
+{
+    const struct viommu_desc *desc = &vsmmuv3_desc;
+
+    if ( cur_viommu && (cur_viommu != desc) )
+    {
+        printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
+        return;
+    }
+
+    cur_viommu = desc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
new file mode 100644
index 0000000000..e11f85b431
--- /dev/null
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+#ifndef __ARCH_ARM_VSMMU_V3_H__
+#define __ARCH_ARM_VSMMU_V3_H__
+
+#include <asm/viommu.h>
+
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+
+void vsmmuv3_set_type(void);
+
+#else
+
+static inline void vsmmuv3_set_type(void)
+{
+    return;
+}
+
+#endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
+
+#endif /* __ARCH_ARM_VSMMU_V3_H__ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index b1db2bf19d..9371c163c4 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -330,7 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
 #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
 #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
 
-#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
+#define XEN_DOMCTL_CONFIG_VIOMMU_NONE       0
+#define XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3     1
 
 struct xen_arch_domainconfig {
     /* IN/OUT */
@@ -455,6 +456,10 @@ typedef uint64_t xen_callback_t;
 #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
 #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
 
+/* vsmmuv3 ITS mappings */
+#define GUEST_VSMMUV3_BASE     xen_mk_ullong(0x04040000)
+#define GUEST_VSMMUV3_SIZE     xen_mk_ullong(0x00040000)
+
 /*
  * 256 MB is reserved for VPCI configuration space based on calculation
  * 256 buses x 32 devices x 8 functions x 4 KB = 256 MB
-- 
2.43.0

Re: [PATCH v2 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Jan Beulich]

On 23.03.2026 23:51, Milan Djokic wrote:
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -40,6 +40,14 @@ config VIRTUAL_IOMMU
>  	help
>  	 Support virtual IOMMU infrastructure to implement vIOMMU.
>  
> +config VIRTUAL_ARM_SMMU_V3
> +	bool "ARM Ltd. Virtual SMMUv3 Support (UNSUPPORTED)" if UNSUPPORTED
> +	depends on ARM_SMMU_V3 && VIRTUAL_IOMMU

VIRTUAL_IOMMU already has its prompt conditional upon UNSUPPORTED. Does this
really need repeating here then?

Jan

[PATCH v2 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
enable/disable vIOMMU support for domains.

Also add viommu="N" parameter to xl domain configuration to enable the
vIOMMU for the domains. Currently, only the "smmuv3" type is supported
for ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/include/libxl.h                |  5 +++++
 tools/libs/light/libxl_arm.c         | 13 +++++++++++++
 tools/libs/light/libxl_types.idl     |  6 ++++++
 tools/xl/xl_parse.c                  |  9 +++++++++
 7 files changed, 49 insertions(+)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 27c455210b..f69cdee55c 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3162,6 +3162,19 @@ option.
 
 =back
 
+=over 4
+
+=item B<viommu="N">
+
+To enable viommu, user must specify the following option in the VM
+config file:
+
+viommu = "smmuv3"
+
+Currently, only the "smmuv3" type is supported for ARM.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 8909fe8a1b..4f0997f02f 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1195,6 +1195,7 @@ x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchArm.ArmSci.fromC(&xc.arch_arm.arm_sci);err != nil {
 return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
 }
+x.ArchArm.Viommu = ViommuType(xc.arch_arm.viommu)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1734,6 +1735,7 @@ xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchArm.ArmSci.toC(&xc.arch_arm.arm_sci); err != nil {
 return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
 }
+xc.arch_arm.viommu = C.libxl_viommu_type(x.ArchArm.Viommu)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index ab9d4ca7b4..8a37b52a82 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -610,6 +610,7 @@ Vuart VuartType
 SveVl SveType
 NrSpis uint32
 ArmSci ArmSci
+Viommu ViommuType
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index bc35e412da..f7d5c77e23 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -318,6 +318,11 @@
  */
 #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
 
+/*
+ * libxl_domain_build_info has the arch_arm.viommu_type field.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARM_VIOMMU 1
+
 /*
  * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
  * 'soft reset' for domains and there is 'soft_reset' shutdown reason
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 7e9f8a1bc3..a248793588 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -247,6 +247,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
     }
     LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
 
+    switch (d_config->b_info.arch_arm.viommu_type) {
+    case LIBXL_VIOMMU_TYPE_NONE:
+        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+        break;
+    case LIBXL_VIOMMU_TYPE_SMMUV3:
+        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3;
+        break;
+    default:
+        LOG(ERROR, "Unknown vIOMMU type %d",
+            d_config->b_info.arch_arm.viommu_type);
+        return ERROR_FAIL;
+    }
+
     return 0;
 }
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index d64a573ff3..c7ad0e77b2 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -561,6 +561,11 @@ libxl_arm_sci = Struct("arm_sci", [
     ("type", libxl_arm_sci_type),
     ])
 
+libxl_viommu_type = Enumeration("viommu_type", [
+    (0, "none"),
+    (1, "smmuv3")
+    ], init_val = "LIBXL_VIOMMU_TYPE_NONE")
+
 libxl_rdm_reserve = Struct("rdm_reserve", [
     ("strategy",    libxl_rdm_reserve_strategy),
     ("policy",      libxl_rdm_reserve_policy),
@@ -736,6 +741,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
                                ("sve_vl", libxl_sve_type),
                                ("nr_spis", uint32, {'init_val': 'LIBXL_NR_SPIS_DEFAULT'}),
                                ("arm_sci", libxl_arm_sci),
+                               ("viommu_type", libxl_viommu_type),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 1a2ea8b5d5..dcae8314fe 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -3033,6 +3033,15 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_string (config, "viommu", &buf, 1)) {
+        e = libxl_viommu_type_from_string(buf, &b_info->arch_arm.viommu_type);
+        if (e) {
+            fprintf(stderr,
+                    "Unknown vIOMMU type \"%s\" specified\n", buf);
+            exit(-ERROR_FAIL);
+        }
+    }
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.43.0

Re: [PATCH v2 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Nick Rosbrook]

On Mon, Mar 23, 2026 at 6:51 PM Milan Djokic <milan_djokic@epam.com> wrote:
>
> From: Rahul Singh <rahul.singh@arm.com>
>
> Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
> XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
> enable/disable vIOMMU support for domains.
>
> Also add viommu="N" parameter to xl domain configuration to enable the
> vIOMMU for the domains. Currently, only the "smmuv3" type is supported
> for ARM.
>
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>  docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
>  tools/golang/xenlight/helpers.gen.go |  2 ++
>  tools/golang/xenlight/types.gen.go   |  1 +
>  tools/include/libxl.h                |  5 +++++
>  tools/libs/light/libxl_arm.c         | 13 +++++++++++++
>  tools/libs/light/libxl_types.idl     |  6 ++++++
>  tools/xl/xl_parse.c                  |  9 +++++++++
>  7 files changed, 49 insertions(+)
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 27c455210b..f69cdee55c 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3162,6 +3162,19 @@ option.
>
>  =back
>
> +=over 4
> +
> +=item B<viommu="N">
> +
> +To enable viommu, user must specify the following option in the VM
> +config file:
> +
> +viommu = "smmuv3"
> +
> +Currently, only the "smmuv3" type is supported for ARM.
> +
> +=back
> +
>  =head3 x86
>
>  =over 4
> diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
> index 8909fe8a1b..4f0997f02f 100644
> --- a/tools/golang/xenlight/helpers.gen.go
> +++ b/tools/golang/xenlight/helpers.gen.go
> @@ -1195,6 +1195,7 @@ x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
>  if err := x.ArchArm.ArmSci.fromC(&xc.arch_arm.arm_sci);err != nil {
>  return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>  }
> +x.ArchArm.Viommu = ViommuType(xc.arch_arm.viommu)
>  if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
>  return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>  }
> @@ -1734,6 +1735,7 @@ xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
>  if err := x.ArchArm.ArmSci.toC(&xc.arch_arm.arm_sci); err != nil {
>  return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>  }
> +xc.arch_arm.viommu = C.libxl_viommu_type(x.ArchArm.Viommu)
>  if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
>  return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>  }
> diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
> index ab9d4ca7b4..8a37b52a82 100644
> --- a/tools/golang/xenlight/types.gen.go
> +++ b/tools/golang/xenlight/types.gen.go
> @@ -610,6 +610,7 @@ Vuart VuartType
>  SveVl SveType
>  NrSpis uint32
>  ArmSci ArmSci
> +Viommu ViommuType
>  }
>  ArchX86 struct {
>  MsrRelaxed Defbool

The generated go code doesn't look right - it appears to be missing a
definition for `ViommuType` and the associated constants for the
"viommu_type" Enumeration added to libxl_types.idl. Does the code need
re-generating?

> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index bc35e412da..f7d5c77e23 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -318,6 +318,11 @@
>   */
>  #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
>
> +/*
> + * libxl_domain_build_info has the arch_arm.viommu_type field.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARM_VIOMMU 1
> +
>  /*
>   * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
>   * 'soft reset' for domains and there is 'soft_reset' shutdown reason
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 7e9f8a1bc3..a248793588 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -247,6 +247,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>      }
>      LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
>
> +    switch (d_config->b_info.arch_arm.viommu_type) {
> +    case LIBXL_VIOMMU_TYPE_NONE:
> +        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_NONE;
> +        break;
> +    case LIBXL_VIOMMU_TYPE_SMMUV3:
> +        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3;
> +        break;
> +    default:
> +        LOG(ERROR, "Unknown vIOMMU type %d",
> +            d_config->b_info.arch_arm.viommu_type);
> +        return ERROR_FAIL;
> +    }
> +
>      return 0;
>  }
>
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index d64a573ff3..c7ad0e77b2 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -561,6 +561,11 @@ libxl_arm_sci = Struct("arm_sci", [
>      ("type", libxl_arm_sci_type),
>      ])
>
> +libxl_viommu_type = Enumeration("viommu_type", [
> +    (0, "none"),
> +    (1, "smmuv3")
> +    ], init_val = "LIBXL_VIOMMU_TYPE_NONE")
> +
>  libxl_rdm_reserve = Struct("rdm_reserve", [
>      ("strategy",    libxl_rdm_reserve_strategy),
>      ("policy",      libxl_rdm_reserve_policy),
> @@ -736,6 +741,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>                                 ("sve_vl", libxl_sve_type),
>                                 ("nr_spis", uint32, {'init_val': 'LIBXL_NR_SPIS_DEFAULT'}),
>                                 ("arm_sci", libxl_arm_sci),
> +                               ("viommu_type", libxl_viommu_type),
>                                ])),
>      ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>                                ])),
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 1a2ea8b5d5..dcae8314fe 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -3033,6 +3033,15 @@ skip_usbdev:
>          }
>      }
>
> +    if (!xlu_cfg_get_string (config, "viommu", &buf, 1)) {
> +        e = libxl_viommu_type_from_string(buf, &b_info->arch_arm.viommu_type);
> +        if (e) {
> +            fprintf(stderr,
> +                    "Unknown vIOMMU type \"%s\" specified\n", buf);
> +            exit(-ERROR_FAIL);
> +        }
> +    }
> +
>      parse_vkb_list(config, d_config);
>
>      d_config->virtios = NULL;
> --
> 2.43.0

-Nick

Re: [PATCH v2 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Milan Djokic]

On 3/25/26 19:52, Nick Rosbrook wrote:
> On Mon, Mar 23, 2026 at 6:51 PM Milan Djokic <milan_djokic@epam.com> wrote:
>>
>> From: Rahul Singh <rahul.singh@arm.com>
>>
>> Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
>> XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
>> enable/disable vIOMMU support for domains.
>>
>> Also add viommu="N" parameter to xl domain configuration to enable the
>> vIOMMU for the domains. Currently, only the "smmuv3" type is supported
>> for ARM.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>>   docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
>>   tools/golang/xenlight/helpers.gen.go |  2 ++
>>   tools/golang/xenlight/types.gen.go   |  1 +
>>   tools/include/libxl.h                |  5 +++++
>>   tools/libs/light/libxl_arm.c         | 13 +++++++++++++
>>   tools/libs/light/libxl_types.idl     |  6 ++++++
>>   tools/xl/xl_parse.c                  |  9 +++++++++
>>   7 files changed, 49 insertions(+)
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index 27c455210b..f69cdee55c 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -3162,6 +3162,19 @@ option.
>>
>>   =back
>>
>> +=over 4
>> +
>> +=item B<viommu="N">
>> +
>> +To enable viommu, user must specify the following option in the VM
>> +config file:
>> +
>> +viommu = "smmuv3"
>> +
>> +Currently, only the "smmuv3" type is supported for ARM.
>> +
>> +=back
>> +
>>   =head3 x86
>>
>>   =over 4
>> diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
>> index 8909fe8a1b..4f0997f02f 100644
>> --- a/tools/golang/xenlight/helpers.gen.go
>> +++ b/tools/golang/xenlight/helpers.gen.go
>> @@ -1195,6 +1195,7 @@ x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
>>   if err := x.ArchArm.ArmSci.fromC(&xc.arch_arm.arm_sci);err != nil {
>>   return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>>   }
>> +x.ArchArm.Viommu = ViommuType(xc.arch_arm.viommu)
>>   if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
>>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>>   }
>> @@ -1734,6 +1735,7 @@ xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
>>   if err := x.ArchArm.ArmSci.toC(&xc.arch_arm.arm_sci); err != nil {
>>   return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>>   }
>> +xc.arch_arm.viommu = C.libxl_viommu_type(x.ArchArm.Viommu)
>>   if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
>>   return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>>   }
>> diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
>> index ab9d4ca7b4..8a37b52a82 100644
>> --- a/tools/golang/xenlight/types.gen.go
>> +++ b/tools/golang/xenlight/types.gen.go
>> @@ -610,6 +610,7 @@ Vuart VuartType
>>   SveVl SveType
>>   NrSpis uint32
>>   ArmSci ArmSci
>> +Viommu ViommuType
>>   }
>>   ArchX86 struct {
>>   MsrRelaxed Defbool
> 
> The generated go code doesn't look right - it appears to be missing a
> definition for `ViommuType` and the associated constants for the
> "viommu_type" Enumeration added to libxl_types.idl. Does the code need
> re-generating?
> 

Yes, I will re-generate the code in the updated version. Thanks for 
noticing.

>> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
>> index bc35e412da..f7d5c77e23 100644
>> --- a/tools/include/libxl.h
>> +++ b/tools/include/libxl.h
>> @@ -318,6 +318,11 @@
>>    */
>>   #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
>>
>> +/*
>> + * libxl_domain_build_info has the arch_arm.viommu_type field.
>> + */
>> +#define LIBXL_HAVE_BUILDINFO_ARM_VIOMMU 1
>> +
>>   /*
>>    * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
>>    * 'soft reset' for domains and there is 'soft_reset' shutdown reason
>> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
>> index 7e9f8a1bc3..a248793588 100644
>> --- a/tools/libs/light/libxl_arm.c
>> +++ b/tools/libs/light/libxl_arm.c
>> @@ -247,6 +247,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>>       }
>>       LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
>>
>> +    switch (d_config->b_info.arch_arm.viommu_type) {
>> +    case LIBXL_VIOMMU_TYPE_NONE:
>> +        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_NONE;
>> +        break;
>> +    case LIBXL_VIOMMU_TYPE_SMMUV3:
>> +        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3;
>> +        break;
>> +    default:
>> +        LOG(ERROR, "Unknown vIOMMU type %d",
>> +            d_config->b_info.arch_arm.viommu_type);
>> +        return ERROR_FAIL;
>> +    }
>> +
>>       return 0;
>>   }
>>
>> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
>> index d64a573ff3..c7ad0e77b2 100644
>> --- a/tools/libs/light/libxl_types.idl
>> +++ b/tools/libs/light/libxl_types.idl
>> @@ -561,6 +561,11 @@ libxl_arm_sci = Struct("arm_sci", [
>>       ("type", libxl_arm_sci_type),
>>       ])
>>
>> +libxl_viommu_type = Enumeration("viommu_type", [
>> +    (0, "none"),
>> +    (1, "smmuv3")
>> +    ], init_val = "LIBXL_VIOMMU_TYPE_NONE")
>> +
>>   libxl_rdm_reserve = Struct("rdm_reserve", [
>>       ("strategy",    libxl_rdm_reserve_strategy),
>>       ("policy",      libxl_rdm_reserve_policy),
>> @@ -736,6 +741,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>>                                  ("sve_vl", libxl_sve_type),
>>                                  ("nr_spis", uint32, {'init_val': 'LIBXL_NR_SPIS_DEFAULT'}),
>>                                  ("arm_sci", libxl_arm_sci),
>> +                               ("viommu_type", libxl_viommu_type),
>>                                 ])),
>>       ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
>>                                 ])),
>> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
>> index 1a2ea8b5d5..dcae8314fe 100644
>> --- a/tools/xl/xl_parse.c
>> +++ b/tools/xl/xl_parse.c
>> @@ -3033,6 +3033,15 @@ skip_usbdev:
>>           }
>>       }
>>
>> +    if (!xlu_cfg_get_string (config, "viommu", &buf, 1)) {
>> +        e = libxl_viommu_type_from_string(buf, &b_info->arch_arm.viommu_type);
>> +        if (e) {
>> +            fprintf(stderr,
>> +                    "Unknown vIOMMU type \"%s\" specified\n", buf);
>> +            exit(-ERROR_FAIL);
>> +        }
>> +    }
>> +
>>       parse_vkb_list(config, d_config);
>>
>>       d_config->virtios = NULL;
>> --
>> 2.43.0
> 
> -Nick

BR,
Milan

[PATCH v2 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add cmdline boot option "viommu = <string>" to enable or disable the
virtual iommu support for guests on ARM (only viommu="smmuv3" supported
for now).

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 docs/misc/xen-command-line.pandoc      |  9 +++++++++
 xen/arch/arm/include/asm/viommu.h      | 12 ++++++++++++
 xen/drivers/passthrough/arm/viommu.c   | 11 +++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c |  3 +++
 4 files changed, 35 insertions(+)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index ebdca007d2..3a1cc562c4 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2855,6 +2855,15 @@ The optional `keep` parameter causes Xen to continue using the vga
 console even after dom0 has been started.  The default behaviour is to
 relinquish control to dom0.
 
+### viommu (arm)
+> `= <string>`
+
+> Default: ``
+
+Flag to enable or disable support for the virtual IOMMU for guests. Disabled by
+default. Enable by specifying target IOMMU type (if supported). Only "smmuv3"
+IOMMU emulation supported at this point.
+
 ### viridian-spinlock-retry-count (x86)
 > `= <integer>`
 
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 4785877e2a..811e9312d5 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -10,6 +10,7 @@
 #include <public/xen.h>
 
 extern struct list_head host_iommu_list;
+extern char viommu[];
 
 /* data structure for each hardware IOMMU */
 struct host_iommu {
@@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
 void add_to_host_iommu_list(paddr_t addr, paddr_t size,
                             const struct dt_device_node *node);
 
+static always_inline bool is_viommu_enabled(void)
+{
+    /* only smmuv3 emulation supported */
+    return !strcmp(viommu, "smmuv3");
+}
+
 #else
 
 static inline uint8_t viommu_get_type(void)
@@ -76,6 +83,11 @@ static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
     return;
 }
 
+static always_inline bool is_viommu_enabled(void)
+{
+    return false;
+}
+
 #endif /* CONFIG_VIRTUAL_IOMMU */
 
 #endif /* __ARCH_ARM_VIOMMU_H__ */
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 53ae46349a..5f5892fbb2 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -3,6 +3,7 @@
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/irq.h>
+#include <xen/param.h>
 #include <xen/types.h>
 
 #include <asm/viommu.h>
@@ -38,8 +39,18 @@ void add_to_host_iommu_list(paddr_t addr, paddr_t size,
     list_add_tail(&iommu_data->entry, &host_iommu_list);
 }
 
+/* By default viommu is disabled.
+ * If enabled, 'viommu' param indicates type (smmuv3 is only supported type atm)
+ */
+char __read_mostly viommu[10] = "";
+string_param("viommu", viommu);
+
 int domain_viommu_init(struct domain *d, uint16_t viommu_type)
 {
+    /* Enable viommu when it has been enabled explicitly (viommu="smmuv3"). */
+    if ( !is_viommu_enabled() )
+        return 0;
+
     if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
         return 0;
 
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 6b4009e5ef..e36f200ba5 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -105,6 +105,9 @@ void __init vsmmuv3_set_type(void)
 {
     const struct viommu_desc *desc = &vsmmuv3_desc;
 
+    if ( !is_viommu_enabled() )
+        return;
+
     if ( cur_viommu && (cur_viommu != desc) )
     {
         printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
-- 
2.43.0

[PATCH v2 08/23] xen/arm: vsmmuv3: Add support for registers emulation

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add initial support for various emulated registers for virtual SMMUv3
for guests and also add support for virtual cmdq and eventq.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.h  |   6 +
 xen/drivers/passthrough/arm/vsmmu-v3.c | 286 +++++++++++++++++++++++++
 2 files changed, 292 insertions(+)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index 3fb13b7e21..fab4fd5a26 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -60,6 +60,12 @@
 #define IDR5_VAX			GENMASK(11, 10)
 #define IDR5_VAX_52_BIT			1
 
+#define ARM_SMMU_IIDR			0x18
+#define IIDR_PRODUCTID			GENMASK(31, 20)
+#define IIDR_VARIANT			GENMASK(19, 16)
+#define IIDR_REVISION			GENMASK(15, 12)
+#define IIDR_IMPLEMENTER		GENMASK(11, 0)
+
 #define ARM_SMMU_CR0			0x20
 #define CR0_ATSCHK			(1 << 4)
 #define CR0_CMDQEN			(1 << 3)
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index e36f200ba5..3ae1e62a50 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -3,25 +3,307 @@
 #include <xen/param.h>
 #include <xen/sched.h>
 #include <asm/mmio.h>
+#include <asm/vgic-emul.h>
 #include <asm/viommu.h>
+#include <asm/vreg.h>
+
+#include "smmu-v3.h"
+
+/* Register Definition */
+#define ARM_SMMU_IDR2       0x8
+#define ARM_SMMU_IDR3       0xc
+#define ARM_SMMU_IDR4       0x10
+#define IDR0_TERM_MODEL     (1 << 26)
+#define IDR3_RIL            (1 << 10)
+#define CR0_RESERVED        0xFFFFFC20
+#define SMMU_IDR1_SIDSIZE   16
+#define SMMU_CMDQS          19
+#define SMMU_EVTQS          19
+#define DWORDS_BYTES        8
+#define ARM_SMMU_IIDR_VAL   0x12
 
 /* Struct to hold the vIOMMU ops and vIOMMU type */
 extern const struct viommu_desc __read_mostly *cur_viommu;
 
+/* virtual smmu queue */
+struct arm_vsmmu_queue {
+    uint64_t    q_base; /* base register */
+    uint32_t    prod;
+    uint32_t    cons;
+    uint8_t     ent_size;
+    uint8_t     max_n_shift;
+};
+
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
+    uint8_t     sid_split;
+    uint32_t    features;
+    uint32_t    cr[3];
+    uint32_t    cr0ack;
+    uint32_t    gerror;
+    uint32_t    gerrorn;
+    uint32_t    strtab_base_cfg;
+    uint64_t    strtab_base;
+    uint32_t    irq_ctrl;
+    uint64_t    gerror_irq_cfg0;
+    uint64_t    evtq_irq_cfg0;
+    struct      arm_vsmmu_queue evtq, cmdq;
 };
 
 static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
                               register_t r, void *priv)
 {
+    struct virt_smmu *smmu = priv;
+    uint64_t reg;
+    uint32_t reg32;
+
+    switch ( info->gpa & 0xffff )
+    {
+    case VREG32(ARM_SMMU_CR0):
+        reg32 = smmu->cr[0];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[0] = reg32;
+        smmu->cr0ack = reg32 & ~CR0_RESERVED;
+        break;
+
+    case VREG32(ARM_SMMU_CR1):
+        reg32 = smmu->cr[1];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[1] = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_CR2):
+        reg32 = smmu->cr[2];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[2] = reg32;
+        break;
+
+    case VREG64(ARM_SMMU_STRTAB_BASE):
+        reg = smmu->strtab_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->strtab_base = reg;
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
+        reg32 = smmu->strtab_base_cfg;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->strtab_base_cfg = reg32;
+
+        smmu->sid_split = FIELD_GET(STRTAB_BASE_CFG_SPLIT, reg32);
+        smmu->features |= STRTAB_BASE_CFG_FMT_2LVL;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_BASE):
+        reg = smmu->cmdq.q_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->cmdq.q_base = reg;
+        smmu->cmdq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->cmdq.q_base);
+        if ( smmu->cmdq.max_n_shift > SMMU_CMDQS )
+            smmu->cmdq.max_n_shift = SMMU_CMDQS;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_PROD):
+        reg32 = smmu->cmdq.prod;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cmdq.prod = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_CONS):
+        reg32 = smmu->cmdq.cons;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cmdq.cons = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_BASE):
+        reg = smmu->evtq.q_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->evtq.q_base = reg;
+        smmu->evtq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->evtq.q_base);
+        if ( smmu->cmdq.max_n_shift > SMMU_EVTQS )
+            smmu->cmdq.max_n_shift = SMMU_EVTQS;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_PROD):
+        reg32 = smmu->evtq.prod;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->evtq.prod = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_CONS):
+        reg32 = smmu->evtq.cons;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->evtq.cons = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_IRQ_CTRL):
+        reg32 = smmu->irq_ctrl;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->irq_ctrl = reg32;
+        break;
+
+    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
+        reg = smmu->gerror_irq_cfg0;
+        vreg_reg64_update(&reg, r, info);
+        smmu->gerror_irq_cfg0 = reg;
+        break;
+
+    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
+        reg = smmu->evtq_irq_cfg0;
+        vreg_reg64_update(&reg, r, info);
+        smmu->evtq_irq_cfg0 = reg;
+        break;
+
+    case VREG32(ARM_SMMU_GERRORN):
+        reg = smmu->gerrorn;
+        vreg_reg64_update(&reg, r, info);
+        smmu->gerrorn = reg;
+        break;
+
+    default:
+        printk(XENLOG_G_ERR
+               "%pv: vSMMUv3: unhandled write r%d offset %"PRIpaddr"\n",
+               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+        return IO_ABORT;
+    }
+
     return IO_HANDLED;
 }
 
 static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
                              register_t *r, void *priv)
 {
+    struct virt_smmu *smmu = priv;
+    uint64_t reg;
+
+    switch ( info->gpa & 0xffff )
+    {
+    case VREG32(ARM_SMMU_IDR0):
+        reg  = FIELD_PREP(IDR0_S1P, 1) | FIELD_PREP(IDR0_TTF, 2) |
+            FIELD_PREP(IDR0_COHACC, 0) | FIELD_PREP(IDR0_ASID16, 1) |
+            FIELD_PREP(IDR0_TTENDIAN, 0) | FIELD_PREP(IDR0_STALL_MODEL, 1) |
+            FIELD_PREP(IDR0_ST_LVL, 1) | FIELD_PREP(IDR0_TERM_MODEL, 1);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR1):
+        reg  = FIELD_PREP(IDR1_SIDSIZE, SMMU_IDR1_SIDSIZE) |
+            FIELD_PREP(IDR1_CMDQS, SMMU_CMDQS) |
+            FIELD_PREP(IDR1_EVTQS, SMMU_EVTQS);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR2):
+        goto read_reserved;
+
+    case VREG32(ARM_SMMU_IDR3):
+        reg  = FIELD_PREP(IDR3_RIL, 0);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR4):
+        goto read_impl_defined;
+
+    case VREG32(ARM_SMMU_IDR5):
+        reg  = FIELD_PREP(IDR5_GRAN4K, 1) | FIELD_PREP(IDR5_GRAN16K, 1) |
+            FIELD_PREP(IDR5_GRAN64K, 1) | FIELD_PREP(IDR5_OAS, IDR5_OAS_48_BIT);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IIDR):
+        *r = vreg_reg32_extract(ARM_SMMU_IIDR_VAL, info);
+        break;
+
+    case VREG32(ARM_SMMU_CR0):
+        *r = vreg_reg32_extract(smmu->cr[0], info);
+        break;
+
+    case VREG32(ARM_SMMU_CR0ACK):
+        *r = vreg_reg32_extract(smmu->cr0ack, info);
+        break;
+
+    case VREG32(ARM_SMMU_CR1):
+        *r = vreg_reg32_extract(smmu->cr[1], info);
+        break;
+
+    case VREG32(ARM_SMMU_CR2):
+        *r = vreg_reg32_extract(smmu->cr[2], info);
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE):
+        *r = vreg_reg64_extract(smmu->strtab_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
+        *r = vreg_reg32_extract(smmu->strtab_base_cfg, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_BASE):
+        *r = vreg_reg64_extract(smmu->cmdq.q_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_PROD):
+        *r = vreg_reg32_extract(smmu->cmdq.prod, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_CONS):
+        *r = vreg_reg32_extract(smmu->cmdq.cons, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_BASE):
+        *r = vreg_reg64_extract(smmu->evtq.q_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_PROD):
+        *r = vreg_reg32_extract(smmu->evtq.prod, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_CONS):
+        *r = vreg_reg32_extract(smmu->evtq.cons, info);
+        break;
+
+    case VREG32(ARM_SMMU_IRQ_CTRL):
+    case VREG32(ARM_SMMU_IRQ_CTRLACK):
+        *r = vreg_reg32_extract(smmu->irq_ctrl, info);
+        break;
+
+    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
+        *r = vreg_reg64_extract(smmu->gerror_irq_cfg0, info);
+        break;
+
+    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
+        *r = vreg_reg64_extract(smmu->evtq_irq_cfg0, info);
+        break;
+
+    case VREG32(ARM_SMMU_GERROR):
+        *r = vreg_reg64_extract(smmu->gerror, info);
+        break;
+
+    case VREG32(ARM_SMMU_GERRORN):
+        *r = vreg_reg64_extract(smmu->gerrorn, info);
+        break;
+
+    default:
+        printk(XENLOG_G_ERR
+               "%pv: vSMMUv3: unhandled read r%d offset %"PRIpaddr"\n",
+               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+        return IO_ABORT;
+    }
+
+    return IO_HANDLED;
+
+ read_impl_defined:
+    printk(XENLOG_G_DEBUG
+           "%pv: vSMMUv3: RAZ on implementation defined register offset %"PRIpaddr"\n",
+           v, info->gpa & 0xffff);
+    *r = 0;
+    return IO_HANDLED;
+
+ read_reserved:
+    printk(XENLOG_G_DEBUG
+           "%pv: vSMMUv3: RAZ on reserved register offset %"PRIpaddr"\n",
+           v, info->gpa & 0xffff);
+    *r = 0;
     return IO_HANDLED;
 }
 
@@ -39,6 +321,10 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
         return -ENOMEM;
 
     smmu->d = d;
+    smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
+    smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
+    smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
+    smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
 
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
-- 
2.43.0

[PATCH v2 09/23] xen/arm: vsmmuv3: Add support for cmdqueue handling

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add support for virtual cmdqueue handling for guests

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/vsmmu-v3.c | 101 +++++++++++++++++++++++++
 1 file changed, 101 insertions(+)

diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 3ae1e62a50..02fe6a4422 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
 
+#include <xen/guest_access.h>
 #include <xen/param.h>
 #include <xen/sched.h>
 #include <asm/mmio.h>
@@ -25,6 +26,26 @@
 /* Struct to hold the vIOMMU ops and vIOMMU type */
 extern const struct viommu_desc __read_mostly *cur_viommu;
 
+/* SMMUv3 command definitions */
+#define CMDQ_OP_PREFETCH_CFG    0x1
+#define CMDQ_OP_CFGI_STE        0x3
+#define CMDQ_OP_CFGI_ALL        0x4
+#define CMDQ_OP_CFGI_CD         0x5
+#define CMDQ_OP_CFGI_CD_ALL     0x6
+#define CMDQ_OP_TLBI_NH_ASID    0x11
+#define CMDQ_OP_TLBI_NH_VA      0x12
+#define CMDQ_OP_TLBI_NSNH_ALL   0x30
+#define CMDQ_OP_CMD_SYNC        0x46
+
+/* Queue Handling */
+#define Q_BASE(q)       ((q)->q_base & Q_BASE_ADDR_MASK)
+#define Q_CONS_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->cons) * (q)->ent_size)
+#define Q_PROD_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->prod) * (q)->ent_size)
+
+/* Helper Macros */
+#define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
+#define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
+
 /* virtual smmu queue */
 struct arm_vsmmu_queue {
     uint64_t    q_base; /* base register */
@@ -49,8 +70,80 @@ struct virt_smmu {
     uint64_t    gerror_irq_cfg0;
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
+    spinlock_t  cmd_queue_lock;
 };
 
+/* Queue manipulation functions */
+static bool queue_empty(struct arm_vsmmu_queue *q)
+{
+    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
+           Q_WRP(q, q->prod) == Q_WRP(q, q->cons);
+}
+
+static void queue_inc_cons(struct arm_vsmmu_queue *q)
+{
+    uint32_t cons = (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1;
+    q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
+}
+
+static void dump_smmu_command(uint64_t *command)
+{
+    gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
+             smmu_cmd_get_command(command[0]), command[0], command[1]);
+}
+static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
+{
+    struct arm_vsmmu_queue *q = &smmu->cmdq;
+    struct domain *d = smmu->d;
+    uint64_t command[CMDQ_ENT_DWORDS];
+    paddr_t addr;
+
+    if ( !smmu_get_cmdq_enabled(smmu->cr[0]) )
+        return 0;
+
+    while ( !queue_empty(q) )
+    {
+        int ret;
+
+        addr = Q_CONS_ENT(q);
+        ret = access_guest_memory_by_gpa(d, addr, command,
+                                         sizeof(command), false);
+        if ( ret )
+            return ret;
+
+        switch ( smmu_cmd_get_command(command[0]) )
+        {
+        case CMDQ_OP_CFGI_STE:
+            break;
+        case CMDQ_OP_PREFETCH_CFG:
+        case CMDQ_OP_CFGI_CD:
+        case CMDQ_OP_CFGI_CD_ALL:
+        case CMDQ_OP_CFGI_ALL:
+        case CMDQ_OP_CMD_SYNC:
+            break;
+        case CMDQ_OP_TLBI_NH_ASID:
+        case CMDQ_OP_TLBI_NSNH_ALL:
+        case CMDQ_OP_TLBI_NH_VA:
+            if ( !iommu_iotlb_flush_all(smmu->d, 1) )
+                break;
+        default:
+            gdprintk(XENLOG_ERR, "vSMMUv3: unhandled command\n");
+            dump_smmu_command(command);
+            break;
+        }
+
+        if ( ret )
+        {
+            gdprintk(XENLOG_ERR,
+                     "vSMMUv3: command error %d while handling command\n",
+                     ret);
+            dump_smmu_command(command);
+        }
+        queue_inc_cons(q);
+    }
+    return 0;
+}
+
 static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
                               register_t r, void *priv)
 {
@@ -104,9 +197,15 @@ static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
         break;
 
     case VREG32(ARM_SMMU_CMDQ_PROD):
+        spin_lock(&smmu->cmd_queue_lock);
         reg32 = smmu->cmdq.prod;
         vreg_reg32_update(&reg32, r, info);
         smmu->cmdq.prod = reg32;
+
+        if ( arm_vsmmu_handle_cmds(smmu) )
+            gdprintk(XENLOG_ERR, "error handling vSMMUv3 commands\n");
+
+        spin_unlock(&smmu->cmd_queue_lock);
         break;
 
     case VREG32(ARM_SMMU_CMDQ_CONS):
@@ -326,6 +425,8 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
 
+    spin_lock_init(&smmu->cmd_queue_lock);
+
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
     /* Register the vIOMMU to be able to clean it up later. */
-- 
2.43.0

[PATCH v2 10/23] xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

CMD_CFGI_STE is used to invalidate/validate the STE. Emulated vSMMUv3
driver in XEN will read the STE from the guest memory space and capture
the Stage-1 configuration required to support nested translation.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/vsmmu-v3.c | 148 +++++++++++++++++++++++++
 1 file changed, 148 insertions(+)

diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 02fe6a4422..39ed4dc577 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -45,6 +45,21 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 /* Helper Macros */
 #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
 #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
+#define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)
+#define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
+#define smmu_get_ste_s1fmt(x)       FIELD_GET(STRTAB_STE_0_S1FMT, x)
+#define smmu_get_ste_s1stalld(x)    FIELD_GET(STRTAB_STE_1_S1STALLD, x)
+#define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
+                                    FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
+
+/* stage-1 translation configuration */
+struct arm_vsmmu_s1_trans_cfg {
+    paddr_t s1ctxptr;
+    uint8_t s1fmt;
+    uint8_t s1cdmax;
+    bool    bypassed;             /* translation is bypassed */
+    bool    aborted;              /* translation is aborted */
+};
 
 /* virtual smmu queue */
 struct arm_vsmmu_queue {
@@ -91,6 +106,138 @@ static void dump_smmu_command(uint64_t *command)
     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
              smmu_cmd_get_command(command[0]), command[0], command[1]);
 }
+static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
+                              uint64_t *ste)
+{
+    paddr_t addr, strtab_base;
+    struct domain *d = smmu->d;
+    uint32_t log2size;
+    int strtab_size_shift;
+    int ret;
+
+    log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
+
+    if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
+        return -EINVAL;
+
+    if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
+    {
+        int idx, max_l2_ste, span;
+        paddr_t l1ptr, l2ptr;
+        uint64_t l1std;
+
+        strtab_size_shift = MAX(5, (int)log2size - smmu->sid_split - 1 + 3);
+        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
+                        ~GENMASK_ULL(strtab_size_shift, 0);
+        idx = (sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS;
+        l1ptr = (paddr_t)(strtab_base + idx * sizeof(l1std));
+
+        ret = access_guest_memory_by_gpa(d, l1ptr, &l1std,
+                                         sizeof(l1std), false);
+        if ( ret )
+        {
+            gdprintk(XENLOG_ERR,
+                     "Could not read L1PTR at 0X%"PRIx64"\n", l1ptr);
+            return ret;
+        }
+
+        span = FIELD_GET(STRTAB_L1_DESC_SPAN, l1std);
+        if ( !span )
+        {
+            gdprintk(XENLOG_ERR, "Bad StreamID span\n");
+            return -EINVAL;
+        }
+
+        max_l2_ste = (1 << span) - 1;
+        l2ptr = FIELD_PREP(STRTAB_L1_DESC_L2PTR_MASK,
+                    FIELD_GET(STRTAB_L1_DESC_L2PTR_MASK, l1std));
+        idx = sid & ((1 << smmu->sid_split) - 1);
+        if ( idx > max_l2_ste )
+        {
+            gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
+                     idx, max_l2_ste);
+            return -EINVAL;
+        }
+        addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
+    }
+    else
+    {
+        strtab_size_shift = log2size + 5;
+        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
+                      ~GENMASK_ULL(strtab_size_shift, 0);
+        addr = strtab_base + sid * sizeof(*ste) * STRTAB_STE_DWORDS;
+    }
+    ret = access_guest_memory_by_gpa(d, addr, ste, sizeof(*ste), false);
+    if ( ret )
+    {
+        gdprintk(XENLOG_ERR,
+                "Cannot fetch pte at address=0x%"PRIx64"\n", addr);
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
+static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
+                                struct arm_vsmmu_s1_trans_cfg *cfg,
+                                uint64_t *ste)
+{
+    uint64_t val = ste[0];
+
+    if ( !(val & STRTAB_STE_0_V) )
+        return -EAGAIN;
+
+    switch ( FIELD_GET(STRTAB_STE_0_CFG, val) )
+    {
+    case STRTAB_STE_0_CFG_BYPASS:
+        cfg->bypassed = true;
+        return 0;
+    case STRTAB_STE_0_CFG_ABORT:
+        cfg->aborted = true;
+        return 0;
+    case STRTAB_STE_0_CFG_S1_TRANS:
+        break;
+    case STRTAB_STE_0_CFG_S2_TRANS:
+        gdprintk(XENLOG_ERR, "vSMMUv3 does not support stage 2 yet\n");
+        goto bad_ste;
+    default:
+        BUG(); /* STE corruption */
+    }
+
+    cfg->s1ctxptr = smmu_get_ste_s1ctxptr(val);
+    cfg->s1fmt = smmu_get_ste_s1fmt(val);
+    cfg->s1cdmax = smmu_get_ste_s1cdmax(val);
+    if ( cfg->s1cdmax != 0 )
+    {
+        gdprintk(XENLOG_ERR,
+                 "vSMMUv3 does not support multiple context descriptors\n");
+        goto bad_ste;
+    }
+
+    return 0;
+
+bad_ste:
+    return -EINVAL;
+}
+
+static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
+{
+    int ret;
+    uint64_t ste[STRTAB_STE_DWORDS];
+    struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
+    uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
+
+    ret = arm_vsmmu_find_ste(smmu, sid, ste);
+    if ( ret )
+        return ret;
+
+    ret = arm_vsmmu_decode_ste(smmu, sid, &s1_cfg, ste);
+    if ( ret )
+        return (ret == -EAGAIN ) ? 0 : ret;
+
+    return 0;
+}
+
 static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
 {
     struct arm_vsmmu_queue *q = &smmu->cmdq;
@@ -114,6 +261,7 @@ static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
         switch ( smmu_cmd_get_command(command[0]) )
         {
         case CMDQ_OP_CFGI_STE:
+            ret = arm_vsmmu_handle_cfgi_ste(smmu, command);
             break;
         case CMDQ_OP_PREFETCH_CFG:
         case CMDQ_OP_CFGI_CD:
-- 
2.43.0

[PATCH v2 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Attach the Stage-1 configuration to device STE to support nested
translation for the guests.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/include/asm/iommu.h       |  7 +++
 xen/drivers/passthrough/arm/smmu-v3.c  | 79 ++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu-v3.h  |  1 +
 xen/drivers/passthrough/arm/vsmmu-v3.c | 18 ++++++
 xen/include/xen/iommu.h                | 13 +++--
 5 files changed, 114 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/include/asm/iommu.h b/xen/arch/arm/include/asm/iommu.h
index ad15477e24..56bc9314a7 100644
--- a/xen/arch/arm/include/asm/iommu.h
+++ b/xen/arch/arm/include/asm/iommu.h
@@ -20,6 +20,13 @@ struct arch_iommu
     void *priv;
 };
 
+struct iommu_guest_config {
+    paddr_t     s1ctxptr;
+    uint8_t     config;
+    uint8_t     s1fmt;
+    uint8_t     s1cdmax;
+};
+
 const struct iommu_ops *iommu_get_ops(void);
 void iommu_set_ops(const struct iommu_ops *ops);
 
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 87612df21d..cf8f638a49 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -2810,6 +2810,37 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
 	return NULL;
 }
 
+static struct iommu_domain *arm_smmu_get_domain_by_sid(struct domain *d,
+				u32 sid)
+{
+	int i;
+	unsigned long flags;
+	struct iommu_domain *io_domain;
+	struct arm_smmu_domain *smmu_domain;
+	struct arm_smmu_master *master;
+	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
+
+	/*
+	 * Loop through the &xen_domain->contexts to locate a context
+	 * assigned to this SMMU
+	 */
+	list_for_each_entry(io_domain, &xen_domain->contexts, list) {
+		smmu_domain = to_smmu_domain(io_domain);
+
+		spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+		list_for_each_entry(master, &smmu_domain->devices, domain_head) {
+			for (i = 0; i < master->num_streams; i++) {
+				if (sid != master->streams[i].id)
+					continue;
+				spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+				return io_domain;
+			}
+		}
+		spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+	}
+	return NULL;
+}
+
 static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 				struct device *dev)
 {
@@ -3022,6 +3053,53 @@ static void arm_smmu_iommu_xen_domain_teardown(struct domain *d)
 	xfree(xen_domain);
 }
 
+static int arm_smmu_attach_guest_config(struct domain *d, u32 sid,
+		struct iommu_guest_config *cfg)
+{
+	int ret = -EINVAL;
+	unsigned long flags;
+	struct arm_smmu_master *master;
+	struct arm_smmu_domain *smmu_domain;
+	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
+	struct iommu_domain *io_domain = arm_smmu_get_domain_by_sid(d, sid);
+
+	if (!io_domain)
+		return -ENODEV;
+
+	smmu_domain = to_smmu_domain(io_domain);
+
+	spin_lock(&xen_domain->lock);
+
+	switch (cfg->config) {
+	case ARM_SMMU_DOMAIN_ABORT:
+		smmu_domain->abort = true;
+		break;
+	case ARM_SMMU_DOMAIN_BYPASS:
+		smmu_domain->abort = false;
+		break;
+	case ARM_SMMU_DOMAIN_NESTED:
+		/* Enable Nested stage translation. */
+		smmu_domain->stage = ARM_SMMU_DOMAIN_NESTED;
+		smmu_domain->s1_cfg.s1ctxptr = cfg->s1ctxptr;
+		smmu_domain->s1_cfg.s1fmt = cfg->s1fmt;
+		smmu_domain->s1_cfg.s1cdmax = cfg->s1cdmax;
+		smmu_domain->abort = false;
+		break;
+	default:
+		goto out;
+	}
+
+	spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+	list_for_each_entry(master, &smmu_domain->devices, domain_head)
+		arm_smmu_install_ste_for_dev(master);
+	spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+
+	ret = 0;
+out:
+	spin_unlock(&xen_domain->lock);
+	return ret;
+}
+
 static const struct iommu_ops arm_smmu_iommu_ops = {
 	.page_sizes		= PAGE_SIZE_4K,
 	.init			= arm_smmu_iommu_xen_domain_init,
@@ -3034,6 +3112,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
 	.unmap_page		= arm_iommu_unmap_page,
 	.dt_xlate		= arm_smmu_dt_xlate,
 	.add_device		= arm_smmu_add_device,
+	.attach_guest_config = arm_smmu_attach_guest_config
 };
 
 static __init int arm_smmu_dt_init(struct dt_device_node *dev,
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index fab4fd5a26..df3b7ec1b5 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -398,6 +398,7 @@ enum arm_smmu_domain_stage {
 	ARM_SMMU_DOMAIN_S2,
 	ARM_SMMU_DOMAIN_NESTED,
 	ARM_SMMU_DOMAIN_BYPASS,
+	ARM_SMMU_DOMAIN_ABORT,
 };
 
 /* Xen specific code. */
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 39ed4dc577..6d3636b18b 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -224,8 +224,11 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
 {
     int ret;
     uint64_t ste[STRTAB_STE_DWORDS];
+    struct domain *d = smmu->d;
+    struct domain_iommu *hd = dom_iommu(d);
     struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
     uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
+    struct iommu_guest_config guest_cfg = {0};
 
     ret = arm_vsmmu_find_ste(smmu, sid, ste);
     if ( ret )
@@ -235,6 +238,21 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     if ( ret )
         return (ret == -EAGAIN ) ? 0 : ret;
 
+    guest_cfg.s1ctxptr = s1_cfg.s1ctxptr;
+    guest_cfg.s1fmt = s1_cfg.s1fmt;
+    guest_cfg.s1cdmax = s1_cfg.s1cdmax;
+
+    if ( s1_cfg.bypassed )
+        guest_cfg.config = ARM_SMMU_DOMAIN_BYPASS;
+    else if ( s1_cfg.aborted )
+        guest_cfg.config = ARM_SMMU_DOMAIN_ABORT;
+    else
+        guest_cfg.config = ARM_SMMU_DOMAIN_NESTED;
+
+    ret = hd->platform_ops->attach_guest_config(d, sid, &guest_cfg);
+    if ( ret )
+        return ret;
+
     return 0;
 }
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 37c4a1dc82..9fdcf03630 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -311,6 +311,10 @@ static inline int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev)
 
 #endif /* HAS_DEVICE_TREE_DISCOVERY */
 
+#if defined(CONFIG_HAS_PASSTHROUGH) || defined(CONFIG_ARM)
+#include <asm/iommu.h>
+#endif
+
 struct page_info;
 
 /*
@@ -387,6 +391,11 @@ struct iommu_ops {
 #endif
     /* Inhibit all interrupt generation, to be used at shutdown. */
     void (*quiesce)(void);
+
+#ifdef CONFIG_ARM
+    int (*attach_guest_config)(struct domain *d, u32 sid,
+                               struct iommu_guest_config *cfg);
+#endif
 };
 
 /*
@@ -407,10 +416,6 @@ extern int iommu_add_extra_reserved_device_memory(unsigned long start,
 extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
                                                   void *ctxt);
 
-#ifdef CONFIG_HAS_PASSTHROUGH
-#include <asm/iommu.h>
-#endif
-
 #ifndef iommu_call
 # define iommu_call(ops, fn, args...) ((ops)->fn(args))
 # define iommu_vcall iommu_call
-- 
2.43.0

Re: [PATCH v2 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Jan Beulich]

On 23.03.2026 23:51, Milan Djokic wrote:
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -311,6 +311,10 @@ static inline int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev)
>  
>  #endif /* HAS_DEVICE_TREE_DISCOVERY */
>  
> +#if defined(CONFIG_HAS_PASSTHROUGH) || defined(CONFIG_ARM)
> +#include <asm/iommu.h>
> +#endif
> +
>  struct page_info;

Why would this change be needed here?

> @@ -387,6 +391,11 @@ struct iommu_ops {
>  #endif
>      /* Inhibit all interrupt generation, to be used at shutdown. */
>      void (*quiesce)(void);
> +
> +#ifdef CONFIG_ARM
> +    int (*attach_guest_config)(struct domain *d, u32 sid,
> +                               struct iommu_guest_config *cfg);
> +#endif
>  };

Nothing here requires anything to be pulled in, so ...

> @@ -407,10 +416,6 @@ extern int iommu_add_extra_reserved_device_memory(unsigned long start,
>  extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
>                                                    void *ctxt);
>  
> -#ifdef CONFIG_HAS_PASSTHROUGH
> -#include <asm/iommu.h>
> -#endif
> -
>  #ifndef iommu_call
>  # define iommu_call(ops, fn, args...) ((ops)->fn(args))
>  # define iommu_vcall iommu_call

... keeping the #include untouched looks entirely possible.

Jan

[PATCH v2 13/23] xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

"iommus" property will be added for dom0 devices to virtual
IOMMU node to enable the dom0 linux kernel to configure the IOMMU

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain_build.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index a51563ee3d..30a4f1fd09 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -538,9 +538,12 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
             continue;
         }
 
-        if ( iommu_node )
+        /*
+         * Expose IOMMU specific properties to hwdom when vIOMMU is
+         * enabled.
+         */
+        if ( iommu_node && !is_viommu_enabled() )
         {
-            /* Don't expose IOMMU specific properties to hwdom */
             if ( dt_property_name_is_equal(prop, "iommus") )
                 continue;
 
-- 
2.43.0

[PATCH v2 12/23] xen/arm: vsmmuv3: Add support for event queue and global error

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Event queue is used to send the events to guest when there is an events/
faults. Add support for event queue to send events to guest.

Global error in SMMUv3 hw will be updated in smmu_gerror and
smmu_gerrorn register. Add support for global error registers to send
global error to guest.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.h  |  20 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 163 ++++++++++++++++++++++++-
 xen/include/public/arch-arm.h          |   5 +-
 3 files changed, 183 insertions(+), 5 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index df3b7ec1b5..8d3e1877aa 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -354,6 +354,26 @@
 
 #define EVTQ_0_ID			GENMASK_ULL(7, 0)
 
+#define EVT_ID_BAD_STREAMID		0x02
+#define EVT_ID_BAD_STE		    	0x04
+#define EVT_ID_TRANSLATION_FAULT	0x10
+#define EVT_ID_ADDR_SIZE_FAULT		0x11
+#define EVT_ID_ACCESS_FAULT		0x12
+#define EVT_ID_PERMISSION_FAULT		0x13
+
+#define EVTQ_0_SSV			(1UL << 11)
+#define EVTQ_0_SSID			GENMASK_ULL(31, 12)
+#define EVTQ_0_SID			GENMASK_ULL(63, 32)
+#define EVTQ_1_STAG			GENMASK_ULL(15, 0)
+#define EVTQ_1_STALL			(1UL << 31)
+#define EVTQ_1_PnU			(1UL << 33)
+#define EVTQ_1_InD			(1UL << 34)
+#define EVTQ_1_RnW			(1UL << 35)
+#define EVTQ_1_S2			(1UL << 39)
+#define EVTQ_1_CLASS			GENMASK_ULL(41, 40)
+#define EVTQ_1_TT_READ			(1UL << 44)
+#define EVTQ_2_ADDR			GENMASK_ULL(63, 0)
+#define EVTQ_3_IPA			GENMASK_ULL(51, 12)
 /* PRI queue */
 #define PRIQ_ENT_SZ_SHIFT		4
 #define PRIQ_ENT_DWORDS			((1 << PRIQ_ENT_SZ_SHIFT) >> 3)
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 6d3636b18b..7a6c18df53 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -44,6 +44,7 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 
 /* Helper Macros */
 #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
+#define smmu_get_evtq_enabled(x)    FIELD_GET(CR0_EVTQEN, x)
 #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
 #define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)
 #define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
@@ -52,6 +53,35 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
 
+/* event queue entry */
+struct arm_smmu_evtq_ent {
+    /* Common fields */
+    uint8_t     opcode;
+    uint32_t    sid;
+
+    /* Event-specific fields */
+    union {
+        struct {
+            uint32_t ssid;
+            bool ssv;
+        } c_bad_ste_streamid;
+
+        struct {
+            bool stall;
+            uint16_t stag;
+            uint32_t ssid;
+            bool ssv;
+            bool s2;
+            uint64_t addr;
+            bool rnw;
+            bool pnu;
+            bool ind;
+            uint8_t class;
+            uint64_t addr2;
+        } f_translation;
+    };
+};
+
 /* stage-1 translation configuration */
 struct arm_vsmmu_s1_trans_cfg {
     paddr_t s1ctxptr;
@@ -82,6 +112,7 @@ struct virt_smmu {
     uint32_t    strtab_base_cfg;
     uint64_t    strtab_base;
     uint32_t    irq_ctrl;
+    uint32_t    virq;
     uint64_t    gerror_irq_cfg0;
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
@@ -89,6 +120,12 @@ struct virt_smmu {
 };
 
 /* Queue manipulation functions */
+static bool queue_full(struct arm_vsmmu_queue *q)
+{
+    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
+           Q_WRP(q, q->prod) != Q_WRP(q, q->cons);
+}
+
 static bool queue_empty(struct arm_vsmmu_queue *q)
 {
     return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
@@ -101,11 +138,105 @@ static void queue_inc_cons(struct arm_vsmmu_queue *q)
     q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
 }
 
+static void queue_inc_prod(struct arm_vsmmu_queue *q)
+{
+    u32 prod = (Q_WRP(q, q->prod) | Q_IDX(q, q->prod)) + 1;
+    q->prod = Q_OVF(q->prod) | Q_WRP(q, prod) | Q_IDX(q, prod);
+}
+
 static void dump_smmu_command(uint64_t *command)
 {
     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
              smmu_cmd_get_command(command[0]), command[0], command[1]);
 }
+
+static void arm_vsmmu_inject_irq(struct virt_smmu *smmu, bool is_gerror,
+                                uint32_t gerror_err)
+{
+    uint32_t new_gerrors, pending;
+
+    if ( is_gerror )
+    {
+        /* trigger global error irq to guest */
+        pending = smmu->gerror ^ smmu->gerrorn;
+        new_gerrors = ~pending & gerror_err;
+
+        /* only toggle non pending errors */
+        if (!new_gerrors)
+            return;
+
+        smmu->gerror ^= new_gerrors;
+    }
+
+    vgic_inject_irq(smmu->d, NULL, smmu->virq, true);
+}
+
+static int arm_vsmmu_write_evtq(struct virt_smmu *smmu, uint64_t *evt)
+{
+    struct arm_vsmmu_queue *q = &smmu->evtq;
+    struct domain *d = smmu->d;
+    paddr_t addr;
+    int ret;
+
+    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
+        return -EINVAL;
+
+    if ( queue_full(q) )
+        return -EINVAL;
+
+    addr = Q_PROD_ENT(q);
+    ret = access_guest_memory_by_gpa(d, addr, evt,
+                                     sizeof(*evt) * EVTQ_ENT_DWORDS, true);
+    if ( ret )
+        return ret;
+
+    queue_inc_prod(q);
+
+    /* trigger eventq irq to guest */
+    if ( !queue_empty(q) )
+        arm_vsmmu_inject_irq(smmu, false, 0);
+
+    return 0;
+}
+
+void arm_vsmmu_send_event(struct virt_smmu *smmu,
+                          struct arm_smmu_evtq_ent *ent)
+{
+    uint64_t evt[EVTQ_ENT_DWORDS];
+    int ret;
+
+    memset(evt, 0, 1 << EVTQ_ENT_SZ_SHIFT);
+
+    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
+        return;
+
+    evt[0] |= FIELD_PREP(EVTQ_0_ID, ent->opcode);
+    evt[0] |= FIELD_PREP(EVTQ_0_SID, ent->sid);
+
+    switch (ent->opcode)
+    {
+    case EVT_ID_BAD_STREAMID:
+    case EVT_ID_BAD_STE:
+        evt[0] |= FIELD_PREP(EVTQ_0_SSID, ent->c_bad_ste_streamid.ssid);
+        evt[0] |= FIELD_PREP(EVTQ_0_SSV, ent->c_bad_ste_streamid.ssv);
+        break;
+    case EVT_ID_TRANSLATION_FAULT:
+    case EVT_ID_ADDR_SIZE_FAULT:
+    case EVT_ID_ACCESS_FAULT:
+    case EVT_ID_PERMISSION_FAULT:
+        break;
+    default:
+        gdprintk(XENLOG_WARNING, "vSMMUv3: event opcode is bad\n");
+        break;
+    }
+
+    ret = arm_vsmmu_write_evtq(smmu, evt);
+    if ( ret )
+        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
+
+    return;
+}
+
 static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
                               uint64_t *ste)
 {
@@ -114,11 +245,22 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
     uint32_t log2size;
     int strtab_size_shift;
     int ret;
+    struct arm_smmu_evtq_ent ent = {
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
 
     log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
 
     if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
+    {
+        ent.opcode = EVT_ID_BAD_STE;
+        arm_vsmmu_send_event(smmu, &ent);
         return -EINVAL;
+    }
 
     if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
     {
@@ -156,6 +298,8 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
         {
             gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
                      idx, max_l2_ste);
+            ent.opcode = EVT_ID_BAD_STREAMID;
+            arm_vsmmu_send_event(smmu, &ent);
             return -EINVAL;
         }
         addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
@@ -183,6 +327,14 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
                                 uint64_t *ste)
 {
     uint64_t val = ste[0];
+    struct arm_smmu_evtq_ent ent = {
+        .opcode = EVT_ID_BAD_STE,
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
 
     if ( !(val & STRTAB_STE_0_V) )
         return -EAGAIN;
@@ -217,6 +369,7 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
     return 0;
 
 bad_ste:
+    arm_vsmmu_send_event(smmu, &ent);
     return -EINVAL;
 }
 
@@ -577,7 +730,8 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
     .write = vsmmuv3_mmio_write,
 };
 
-static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
+static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
+                               paddr_t size, uint32_t virq)
 {
     struct virt_smmu *smmu;
 
@@ -586,6 +740,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
         return -ENOMEM;
 
     smmu->d = d;
+    smmu->virq = virq;
     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
@@ -612,14 +767,16 @@ int domain_vsmmuv3_init(struct domain *d)
 
         list_for_each_entry(hw_iommu, &host_iommu_list, entry)
         {
-            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
+            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size,
+                                      hw_iommu->irq);
             if ( ret )
                 return ret;
         }
     }
     else
     {
-        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE,
+                                  GUEST_VSMMU_SPI);
         if ( ret )
             return ret;
     }
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index 9371c163c4..c7fd2ee132 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -526,9 +526,10 @@ typedef uint64_t xen_callback_t;
 #define GUEST_EVTCHN_PPI        31
 
 #define GUEST_VPL011_SPI        32
+#define GUEST_VSMMU_SPI         33
 
-#define GUEST_VIRTIO_MMIO_SPI_FIRST   33
-#define GUEST_VIRTIO_MMIO_SPI_LAST    43
+#define GUEST_VIRTIO_MMIO_SPI_FIRST   34
+#define GUEST_VIRTIO_MMIO_SPI_LAST    44
 
 /*
  * SGI is the preferred delivery mechanism of FF-A pending notifications or
-- 
2.43.0

[PATCH v2 14/23] xen/arm: vIOMMU: IOMMU device tree node for dom0

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

XEN will create an IOMMU device tree node in the device tree
to enable the dom0 to discover the virtual SMMUv3 during dom0 boot.
IOMMU device tree node will only be created when cmdline option viommu
is enabled.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain_build.c       | 94 +++++++++++++++++++++++++++++++
 xen/arch/arm/include/asm/viommu.h |  1 +
 2 files changed, 95 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 30a4f1fd09..57072686f9 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1440,6 +1440,95 @@ int __init make_timer_node(const struct kernel_info *kinfo)
     return res;
 }
 
+#ifdef CONFIG_VIRTUAL_IOMMU
+static int make_hwdom_viommu_node(const struct kernel_info *kinfo)
+{
+    uint32_t len;
+    int res;
+    char buf[24];
+    void *fdt = kinfo->fdt;
+    const void *prop = NULL;
+    const struct dt_device_node *iommu = NULL;
+    struct host_iommu *iommu_data;
+    gic_interrupt_t intr;
+
+    if ( list_empty(&host_iommu_list) )
+        return 0;
+
+    list_for_each_entry( iommu_data, &host_iommu_list, entry )
+    {
+        if ( iommu_data->hwdom_node_created )
+            return 0;
+
+        iommu = iommu_data->dt_node;
+
+        snprintf(buf, sizeof(buf), "iommu@%"PRIx64, iommu_data->addr);
+
+        res = fdt_begin_node(fdt, buf);
+        if ( res )
+            return res;
+
+        prop = dt_get_property(iommu, "compatible", &len);
+        if ( !prop )
+        {
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "compatible", prop, len);
+        if ( res )
+            return res;
+
+        if ( iommu->phandle )
+        {
+            res = fdt_property_cell(fdt, "phandle", iommu->phandle);
+            if ( res )
+                return res;
+        }
+
+        /* Use the same reg regions as the IOMMU node in host DTB. */
+        prop = dt_get_property(iommu, "reg", &len);
+        if ( !prop )
+        {
+            printk(XENLOG_ERR "vIOMMU: Can't find IOMMU reg property.\n");
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "reg", prop, len);
+        if ( res )
+            return res;
+
+        prop = dt_get_property(iommu, "#iommu-cells", &len);
+        if ( !prop )
+        {
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "#iommu-cells", prop, len);
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "interrupt-names", "combined");
+        if ( res )
+            return res;
+
+        set_interrupt(intr, iommu_data->irq, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+        res = fdt_property_interrupts(kinfo, &intr, 1);
+        if ( res )
+            return res;
+
+        iommu_data->hwdom_node_created = true;
+
+        fdt_end_node(fdt);
+    }
+
+    return res;
+}
+#endif
+
 static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
                               struct dt_device_node *node,
                               p2m_type_t p2mt)
@@ -1508,6 +1597,11 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
     if ( dt_match_node(timer_matches, node) )
         return make_timer_node(kinfo);
 
+#ifdef CONFIG_VIRTUAL_IOMMU
+    if ( device_get_class(node) == DEVICE_IOMMU && is_viommu_enabled() )
+        return make_hwdom_viommu_node(kinfo);
+#endif
+
     /* Skip nodes used by Xen */
     if ( dt_device_used_by(node) == DOMID_XEN )
     {
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 811e9312d5..b519a73d4d 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -19,6 +19,7 @@ struct host_iommu {
     paddr_t addr;
     paddr_t size;
     uint32_t irq;
+    bool hwdom_node_created;
 };
 
 struct viommu_ops {
-- 
2.43.0

[PATCH v2 15/23] xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

XEN will create an Emulated SMMUv3 device tree node in the device tree
to enable the dom0less domains to discover the virtual SMMUv3 during
boot. Emulated SMMUv3 device tree node will only be created when cmdline
option vsmmuv3 is enabled.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/dom0less-build.c         | 53 +++++++++++++++++++++++++++
 xen/include/public/device_tree_defs.h |  1 +
 2 files changed, 54 insertions(+)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 067835e5d0..a48edb9568 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -218,10 +218,63 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
 }
 #endif
 
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
+{
+    int res;
+    char buf[24];
+    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
+    __be32 *cells;
+    void *fdt = kinfo->fdt;
+
+    snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);
+
+    res = fdt_begin_node(fdt, buf);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,smmu-v3");
+    if ( res )
+        return res;
+
+    /* Create reg property */
+    cells = &reg[0];
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+    res = fdt_property(fdt, "reg", reg,
+                       (GUEST_ROOT_ADDRESS_CELLS +
+                       GUEST_ROOT_SIZE_CELLS) * sizeof(*reg));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_VSMMUV3);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#iommu-cells", 1);
+    if ( res )
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+#endif
+
 int __init make_arch_nodes(struct kernel_info *kinfo)
 {
     int ret;
 
+
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+    if ( is_viommu_enabled() )
+    {
+        ret = make_vsmmuv3_node(kinfo);
+        if ( ret )
+            return -EINVAL;
+    }
+#endif
+
     ret = make_psci_node(kinfo->fdt);
     if ( ret )
         return -EINVAL;
diff --git a/xen/include/public/device_tree_defs.h b/xen/include/public/device_tree_defs.h
index 9e80d0499d..7846a0425c 100644
--- a/xen/include/public/device_tree_defs.h
+++ b/xen/include/public/device_tree_defs.h
@@ -14,6 +14,7 @@
  */
 #define GUEST_PHANDLE_GIC (65000)
 #define GUEST_PHANDLE_IOMMU (GUEST_PHANDLE_GIC + 1)
+#define GUEST_PHANDLE_VSMMUV3 (GUEST_PHANDLE_IOMMU + 1)
 
 #define GUEST_ROOT_ADDRESS_CELLS 2
 #define GUEST_ROOT_SIZE_CELLS 2
-- 
2.43.0

[PATCH v2 16/23] arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

libxl will create an Emulated SMMUv3 device tree node in the device
tree to enable the guest OS to discover the virtual SMMUv3 during guest
boot.

Emulated SMMUv3 device tree node will only be created when
"viommu=smmuv3" is set in xl domain configuration.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c | 37 ++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index a248793588..eb879473f5 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -901,6 +901,36 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
+                             const struct arch_info *ainfo,
+                             struct xc_dom_image *dom)
+{
+    int res;
+    const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, "arm,smmu-v3");
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+                            GUEST_ROOT_SIZE_CELLS, 1, GUEST_VSMMUV3_BASE,
+                            GUEST_VSMMUV3_SIZE);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_VSMMUV3);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#iommu-cells", 1);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
 static int make_vpci_node(libxl__gc *gc, void *fdt,
                           const struct arch_info *ainfo,
                           struct xc_dom_image *dom)
@@ -942,6 +972,10 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
         GUEST_VPCI_PREFETCH_MEM_SIZE);
     if (res) return res;
 
+    res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
+                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
+    if (res) return res;
+
     res = fdt_end_node(fdt);
     if (res) return res;
 
@@ -1408,6 +1442,9 @@ next_resize:
         if (d_config->num_pcidevs)
             FDT( make_vpci_node(gc, fdt, ainfo, dom) );
 
+        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3)
+            FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
+
         for (i = 0; i < d_config->num_disks; i++) {
             libxl_device_disk *disk = &d_config->disks[i];
 
-- 
2.43.0

[PATCH v2 17/23] xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Alloc and reserve virq for event queue and global error to send event to
guests. Also Modify the libxl to accomadate the new define virq.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c           | 28 ++++++++++++++++++++++++--
 xen/arch/arm/dom0less-build.c          | 17 ++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 13 ++++++++++++
 3 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index eb879473f5..803c3b39b7 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -86,8 +86,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 {
     uint32_t nr_spis = 0, cfg_nr_spis = d_config->b_info.arch_arm.nr_spis;
     unsigned int i;
-    uint32_t vuart_irq, virtio_irq = 0;
-    bool vuart_enabled = false, virtio_enabled = false;
+    uint32_t vuart_irq, virtio_irq = 0, vsmmu_irq = 0;
+    bool vuart_enabled = false, virtio_enabled = false, vsmmu_enabled = false;
     uint64_t virtio_mmio_base = GUEST_VIRTIO_MMIO_BASE;
     uint32_t virtio_mmio_irq = GUEST_VIRTIO_MMIO_SPI_FIRST;
     int rc;
@@ -102,6 +102,16 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
         vuart_enabled = true;
     }
 
+    /*
+     * If smmuv3 viommu is enabled then increment the nr_spis to allow allocation
+     * of SPI VIRQ for VSMMU.
+     */
+    if (d_config->b_info.arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
+        nr_spis += (GUEST_VSMMU_SPI - 32) + 1;
+        vsmmu_irq = GUEST_VSMMU_SPI;
+        vsmmu_enabled = true;
+    }
+
     for (i = 0; i < d_config->num_disks; i++) {
         libxl_device_disk *disk = &d_config->disks[i];
 
@@ -170,6 +180,11 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
             return ERROR_FAIL;
         }
 
+        if (vsmmu_enabled && irq == vsmmu_irq) {
+            LOG(ERROR, "Physical IRQ %u conflicting with vSMMUv3 SPI\n", irq);
+            return ERROR_FAIL;
+        }
+
         if (irq < 32)
             continue;
 
@@ -907,6 +922,7 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
 {
     int res;
     const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);
+    gic_interrupt intr;
 
     res = fdt_begin_node(fdt, name);
     if (res) return res;
@@ -925,6 +941,14 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
     res = fdt_property_cell(fdt, "#iommu-cells", 1);
     if (res) return res;
 
+    res = fdt_property_string(fdt, "interrupt-names", "combined");
+    if (res) return res;
+
+    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property_interrupts(gc, fdt, &intr, 1);
+    if (res) return res;
+
     res = fdt_end_node(fdt);
     if (res) return res;
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index a48edb9568..7380753fa2 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -225,6 +225,7 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
     char buf[24];
     __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
     __be32 *cells;
+    gic_interrupt_t intr;
     void *fdt = kinfo->fdt;
 
     snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);
@@ -255,6 +256,22 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
     if ( res )
         return res;
 
+    res = fdt_property_string(fdt, "interrupt-names", "combined");
+    if ( res )
+        return res;
+
+    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property(kinfo->fdt, "interrupts",
+                       intr, sizeof(intr));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(kinfo->fdt, "interrupt-parent",
+                            kinfo->phandle_intc);
+    if ( res )
+        return res;
+
     res = fdt_end_node(fdt);
 
     return res;
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 7a6c18df53..a5b9700369 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -733,6 +733,7 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
 static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
                                paddr_t size, uint32_t virq)
 {
+    int ret;
     struct virt_smmu *smmu;
 
     smmu = xzalloc(struct virt_smmu);
@@ -748,12 +749,24 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
 
     spin_lock_init(&smmu->cmd_queue_lock);
 
+    ret = vgic_reserve_virq(d, virq);
+    if ( !ret )
+    {
+        ret = -EINVAL;
+        goto out;
+    }
+
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
     /* Register the vIOMMU to be able to clean it up later. */
     list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
 
     return 0;
+
+out:
+    xfree(smmu);
+    vgic_free_virq(d, virq);
+    return ret;
 }
 
 int domain_vsmmuv3_init(struct domain *d)
-- 
2.43.0

[PATCH v2 18/23] xen/arm: vsmmuv3: Add support to send stage-1 event to guest

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Stage-1 translation is handled by guest, therefore stage-1 fault has to
be forwarded to guest.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c  | 48 ++++++++++++++++++++++++--
 xen/drivers/passthrough/arm/vsmmu-v3.c | 45 ++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h | 12 +++++++
 3 files changed, 103 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index cf8f638a49..4c1951d753 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -869,7 +869,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 	return 0;
 }
 
-__maybe_unused
 static struct arm_smmu_master *
 arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 {
@@ -890,10 +889,51 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 	return NULL;
 }
 
+static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
+{
+	int ret;
+	struct arm_smmu_master *master;
+	u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]);
+
+	switch (FIELD_GET(EVTQ_0_ID, evt[0])) {
+	case EVT_ID_TRANSLATION_FAULT:
+		break;
+	case EVT_ID_ADDR_SIZE_FAULT:
+		break;
+	case EVT_ID_ACCESS_FAULT:
+		break;
+	case EVT_ID_PERMISSION_FAULT:
+		break;
+	default:
+		return -EOPNOTSUPP;
+	}
+
+	/* Stage-2 event */
+	if (evt[1] & EVTQ_1_S2)
+		return -EFAULT;
+
+	mutex_lock(&smmu->streams_mutex);
+	master = arm_smmu_find_master(smmu, sid);
+	if (!master) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	ret = arm_vsmmu_handle_evt(master->domain->d, smmu->dev, evt);
+	if (ret) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+out_unlock:
+	mutex_unlock(&smmu->streams_mutex);
+	return ret;
+}
+
 /* IRQ and event handlers */
 static void arm_smmu_evtq_tasklet(void *dev)
 {
-	int i;
+	int i, ret;
 	struct arm_smmu_device *smmu = dev;
 	struct arm_smmu_queue *q = &smmu->evtq.q;
 	struct arm_smmu_ll_queue *llq = &q->llq;
@@ -903,6 +943,10 @@ static void arm_smmu_evtq_tasklet(void *dev)
 		while (!queue_remove_raw(q, evt)) {
 			u8 id = FIELD_GET(EVTQ_0_ID, evt[0]);
 
+			ret = arm_smmu_handle_evt(smmu, evt);
+			if (!ret)
+				continue;
+
 			dev_info(smmu->dev, "event 0x%02x received:\n", id);
 			for (i = 0; i < ARRAY_SIZE(evt); ++i)
 				dev_info(smmu->dev, "\t0x%016llx\n",
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index a5b9700369..5d0dabd2b2 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -103,6 +103,7 @@ struct arm_vsmmu_queue {
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
+    paddr_t     addr;
     uint8_t     sid_split;
     uint32_t    features;
     uint32_t    cr[3];
@@ -237,6 +238,49 @@ void arm_vsmmu_send_event(struct virt_smmu *smmu,
     return;
 }
 
+static struct virt_smmu *vsmmuv3_find_by_addr(struct domain *d, paddr_t paddr)
+{
+    struct virt_smmu *smmu;
+
+    list_for_each_entry( smmu, &d->arch.viommu_list, viommu_list )
+    {
+        if ( smmu->addr == paddr )
+            return smmu;
+    }
+
+    return NULL;
+}
+
+int arm_vsmmu_handle_evt(struct domain *d, struct device *dev, uint64_t *evt)
+{
+    int ret;
+    struct virt_smmu *smmu;
+
+    if ( is_hardware_domain(d) )
+    {
+        paddr_t paddr;
+        /* Base address */
+        ret = dt_device_get_address(dev_to_dt(dev), 0, &paddr, NULL);
+        if ( ret )
+            return -EINVAL;
+
+        smmu = vsmmuv3_find_by_addr(d, paddr);
+        if ( !smmu )
+            return -ENODEV;
+    }
+    else
+    {
+        smmu = list_entry(d->arch.viommu_list.next,
+                          struct virt_smmu, viommu_list);
+    }
+
+    ret = arm_vsmmu_write_evtq(smmu, evt);
+    if ( ret )
+        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
+
+    return 0;
+}
+
 static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
                               uint64_t *ste)
 {
@@ -742,6 +786,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
 
     smmu->d = d;
     smmu->virq = virq;
+    smmu->addr = addr;
     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
index e11f85b431..c7bfd3fb59 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.h
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
@@ -8,6 +8,12 @@
 
 void vsmmuv3_set_type(void);
 
+static inline int arm_vsmmu_handle_evt(struct domain *d,
+                                       struct device *dev, uint64_t *evt)
+{
+    return -EINVAL;
+}
+
 #else
 
 static inline void vsmmuv3_set_type(void)
@@ -15,6 +21,12 @@ static inline void vsmmuv3_set_type(void)
     return;
 }
 
+static inline int arm_vsmmu_handle_evt(struct domain *d,
+                                       struct device *dev, uint64_t *evt)
+{
+    return -EINVAL;
+}
+
 #endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
 
 #endif /* __ARCH_ARM_VSMMU_V3_H__ */
-- 
2.43.0

[PATCH v2 19/23] libxl/arm: vIOMMU: Modify the partial device tree for iommus

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

To configure IOMMU in guest for passthrough devices, user will need to
copy the unmodified "iommus" property from host device tree to partial
device tree. To enable the dom0 linux kernel to confiure the IOMMU
correctly replace the phandle in partial device tree with virtual
IOMMU phandle when "iommus" property is set.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Singed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c | 47 +++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 803c3b39b7..7b887898bb 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1326,6 +1326,41 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return 0;
 }
 
+static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+{
+    int nodeoff, proplen, i, r;
+    const fdt32_t *prop;
+    fdt32_t *prop_c;
+
+    nodeoff = fdt_path_offset(pfdt, "/passthrough");
+    if (nodeoff < 0)
+        return nodeoff;
+
+    for (nodeoff = fdt_first_subnode(pfdt, nodeoff);
+         nodeoff >= 0;
+         nodeoff = fdt_next_subnode(pfdt, nodeoff)) {
+
+        prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+        if (!prop)
+            continue;
+
+        prop_c = libxl__zalloc(gc, proplen);
+
+        for (i = 0; i < proplen / 8; ++i) {
+            prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+            prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        }
+
+        r = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
+        if (r) {
+            LOG(ERROR, "Can't set the iommus property in partial FDT");
+            return r;
+        }
+    }
+
+    return 0;
+}
+
 #else
 
 static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
@@ -1344,6 +1379,13 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return -FDT_ERR_INTERNAL;
 }
 
+static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+{
+    LOG(ERROR, "partial device tree not supported");
+
+    return ERROR_FAIL;
+}
+
 #endif /* ENABLE_PARTIAL_DEVICE_TREE */
 
 #define FDT_MAX_SIZE (1<<20)
@@ -1466,8 +1508,11 @@ next_resize:
         if (d_config->num_pcidevs)
             FDT( make_vpci_node(gc, fdt, ainfo, dom) );
 
-        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3)
+        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
             FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
+            if (pfdt)
+                FDT( modify_partial_fdt(gc, pfdt) );
+        }
 
         for (i = 0; i < d_config->num_disks; i++) {
             libxl_device_disk *disk = &d_config->disks[i];
-- 
2.43.0

[PATCH v2 20/23] xen/arm: vIOMMU: Modify the partial device tree for dom0less

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

To configure IOMMU in guest for passthrough devices, user will need to
copy the unmodified "iommus" property from host device tree to partial
device tree. To enable the dom0 linux kernel to confiure the IOMMU
correctly replace the phandle in partial device tree with virtual
IOMMU phandle when "iommus" property is set.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/common/device-tree/dom0less-build.c | 31 ++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 840d14419d..4b74d2f705 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -318,7 +318,35 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
 }
 
-static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
+static void modify_pfdt_node(void *pfdt, int nodeoff)
+{
+    int proplen, i, rc;
+    const fdt32_t *prop;
+    fdt32_t *prop_c;
+
+    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+    if ( !prop )
+        return;
+
+    prop_c = xzalloc_bytes(proplen);
+
+    for ( i = 0; i < proplen / 8; ++i )
+    {
+        prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+        prop_c[i * 2 + 1] = prop[i * 2 + 1];
+    }
+
+    rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
+    if ( rc )
+    {
+        dprintk(XENLOG_ERR, "Can't set the iommus property in partial FDT");
+        return;
+    }
+
+    return;
+}
+
+static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
                                  int nodeoff,
                                  uint32_t address_cells, uint32_t size_cells,
                                  bool scan_passthrough_prop)
@@ -344,6 +372,7 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
     node_next = fdt_first_subnode(pfdt, nodeoff);
     while ( node_next > 0 )
     {
+        modify_pfdt_node(pfdt, node_next);
         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
                             scan_passthrough_prop);
         if ( rc )
-- 
2.43.0

[PATCH v2 21/23] xen/arm: vIOMMU vSID->pSID mapping layer

[Milan Djokic]

Introduce vIOMMU mapping layer in order to support passthrough of IOMMU
devices attached to different physical IOMMUs (e.g. devices with the same streamID).
New generic vIOMMU API is added: viommu_allocate_free_vid().
This function will allocate a new guest vSID and map it to input pSID.
Once mapping is established, guest will use vSID for stage-1 commands
and xen will translate vSID->pSID and propagate it towards stage-2.
Introduced naming is generic (vID/pID), since this API could be used
for other IOMMU types in the future.
Implemented usage of the new API for dom0less guests. vSIDs are allocated
on guest device tree creation and the original pSID is
replaced with vSID which shall be used by the guest driver.

Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/include/asm/viommu.h       | 10 ++++
 xen/common/device-tree/dom0less-build.c | 32 +++++++++---
 xen/drivers/passthrough/arm/viommu.c    |  7 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c  | 67 ++++++++++++++++++++++++-
 4 files changed, 106 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index b519a73d4d..4cf8664ba3 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -33,6 +33,15 @@ struct viommu_ops {
      * Called during domain destruction to free resources used by vIOMMU.
      */
     int (*relinquish_resources)(struct domain *d);
+
+    /*
+     * Allocate free vSID/vRID for the guest device and establish vID->pID mapping
+     * Called during domain device assignment.
+     * Returns 0 on success and sets vid argument to newly allocated vSID/vRID
+     * mapped to physical ID (id argument).
+     * Negative error code returned if allocation fails.
+     */
+    int (*allocate_free_vid)(struct domain *d, uint32_t id, uint32_t *vid);
 };
 
 struct viommu_desc {
@@ -48,6 +57,7 @@ struct viommu_desc {
 
 int domain_viommu_init(struct domain *d, uint16_t viommu_type);
 int viommu_relinquish_resources(struct domain *d);
+int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid);
 uint16_t viommu_get_type(void);
 void add_to_host_iommu_list(paddr_t addr, paddr_t size,
                             const struct dt_device_node *node);
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 4b74d2f705..2bf09637b6 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -31,6 +31,8 @@
 #include <xen/static-memory.h>
 #include <xen/static-shmem.h>
 
+#include <asm/viommu.h>
+
 #define XENSTORE_PFN_LATE_ALLOC UINT64_MAX
 
 static domid_t __initdata xs_domid = DOMID_INVALID;
@@ -318,22 +320,33 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
 }
 
-static void modify_pfdt_node(void *pfdt, int nodeoff)
+#ifdef CONFIG_VIRTUAL_IOMMU
+static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d)
 {
     int proplen, i, rc;
     const fdt32_t *prop;
     fdt32_t *prop_c;
+    uint32_t vsid;
 
-    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen); 
     if ( !prop )
         return;
 
     prop_c = xzalloc_bytes(proplen);
 
+    /* 
+     * Assign <vIOMMU vSID> pairs to iommus property and establish
+     * vSID->pSID mappings
+    */
     for ( i = 0; i < proplen / 8; ++i )
     {
         prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
-        prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        rc = viommu_allocate_free_vid(d, fdt32_to_cpu(prop[i * 2 + 1]), &vsid);
+        if( rc ) {
+            dprintk(XENLOG_ERR, "Failed to allocate new vSID for iommu device");
+            return;
+        }
+        prop_c[i * 2 + 1] = cpu_to_fdt32(vsid);
     }
 
     rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
@@ -345,11 +358,14 @@ static void modify_pfdt_node(void *pfdt, int nodeoff)
 
     return;
 }
+#else
+    static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d) {}
+#endif
 
 static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
                                  int nodeoff,
                                  uint32_t address_cells, uint32_t size_cells,
-                                 bool scan_passthrough_prop)
+                                 bool scan_passthrough_prop, struct domain *d)
 {
     int rc = 0;
     void *fdt = kinfo->fdt;
@@ -372,9 +388,9 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
     node_next = fdt_first_subnode(pfdt, nodeoff);
     while ( node_next > 0 )
     {
-        modify_pfdt_node(pfdt, node_next);
+        modify_pfdt_node(pfdt, node_next, d);
         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
-                            scan_passthrough_prop);
+                            scan_passthrough_prop, d);
         if ( rc )
             return rc;
 
@@ -443,7 +459,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
             res = scan_pfdt_node(kinfo, pfdt, node_next,
                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
-                                 false);
+                                 false, d);
             if ( res )
                 goto out;
             continue;
@@ -453,7 +469,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
             res = scan_pfdt_node(kinfo, pfdt, node_next,
                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
-                                 true);
+                                 true, d);
             if ( res )
                 goto out;
             continue;
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 5f5892fbb2..4b7837a91f 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -71,6 +71,13 @@ int viommu_relinquish_resources(struct domain *d)
     return cur_viommu->ops->relinquish_resources(d);
 }
 
+int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
+    if ( !cur_viommu )
+        return -ENODEV;
+
+    return cur_viommu->ops->allocate_free_vid(d, id, vid);
+}
+
 uint16_t viommu_get_type(void)
 {
     if ( !cur_viommu )
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 5d0dabd2b2..604f09e980 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -53,6 +53,8 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
 
+#define MAX_VSID   (1 << SMMU_IDR1_SIDSIZE)
+
 /* event queue entry */
 struct arm_smmu_evtq_ent {
     /* Common fields */
@@ -100,6 +102,14 @@ struct arm_vsmmu_queue {
     uint8_t     max_n_shift;
 };
 
+/* vSID->pSID mapping entry */
+struct vsid_entry {
+    bool        valid;
+    uint32_t    vsid;
+    struct host_iommu *phys_smmu;
+    uint32_t    psid;
+};
+
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
@@ -118,6 +128,7 @@ struct virt_smmu {
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
     spinlock_t  cmd_queue_lock;
+    struct vsid_entry *vsids;
 };
 
 /* Queue manipulation functions */
@@ -426,6 +437,29 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
     uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
     struct iommu_guest_config guest_cfg = {0};
+    uint32_t psid;
+    struct arm_smmu_evtq_ent ent = {
+        .opcode = EVT_ID_BAD_STE,
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
+
+    /* SIDs identity mapped for HW domain */
+    if ( is_hardware_domain(d) )
+        psid = sid;
+    else {
+        /* vSID out of range or not mapped to pSID */
+        if ( sid >= MAX_VSID || !smmu->vsids[sid].valid )
+        {
+            arm_vsmmu_send_event(smmu, &ent);
+            return -EINVAL;
+        }
+
+        psid = smmu->vsids[sid].psid;
+    }
 
     ret = arm_vsmmu_find_ste(smmu, sid, ste);
     if ( ret )
@@ -446,7 +480,7 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     else
         guest_cfg.config = ARM_SMMU_DOMAIN_NESTED;
 
-    ret = hd->platform_ops->attach_guest_config(d, sid, &guest_cfg);
+    ret = hd->platform_ops->attach_guest_config(d, psid, &guest_cfg);
     if ( ret )
         return ret;
 
@@ -791,6 +825,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
+    smmu->vsids = xzalloc_array(struct vsid_entry, MAX_VSID);
 
     spin_lock_init(&smmu->cmd_queue_lock);
 
@@ -850,8 +885,9 @@ int vsmmuv3_relinquish_resources(struct domain *d)
     if ( list_head_is_null(&d->arch.viommu_list) )
         return 0;
 
-    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
+    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list)
     {
+        xfree(pos->vsids);
         list_del(&pos->viommu_list);
         xfree(pos);
     }
@@ -859,8 +895,35 @@ int vsmmuv3_relinquish_resources(struct domain *d)
     return 0;
 }
 
+int vsmmuv3_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
+    uint16_t i = 0;
+    struct virt_smmu *smmu;
+
+    if ( list_head_is_null(&d->arch.viommu_list) )
+        return -ENODEV;
+
+    smmu = list_first_entry(&d->arch.viommu_list, struct virt_smmu, viommu_list);
+
+    /* Get first free vSID index */
+    while ( smmu->vsids[i].valid && i++ < MAX_VSID );
+
+    /* Max number of vSIDs already allocated? */
+    if ( i == MAX_VSID) {
+        return -ENOMEM;
+    }
+
+    /* Establish vSID->pSID mapping */
+    smmu->vsids[i].valid = true;
+    smmu->vsids[i].vsid = i;
+    smmu->vsids[i].psid = id;
+    *vid = smmu->vsids[i].vsid;
+
+    return 0;
+}
+
 static const struct viommu_ops vsmmuv3_ops = {
     .domain_init = domain_vsmmuv3_init,
+    .allocate_free_vid = vsmmuv3_allocate_free_vid,
     .relinquish_resources = vsmmuv3_relinquish_resources,
 };
 
-- 
2.43.0

[PATCH v2 22/23] libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping

[Milan Djokic]

For guests created via control domain (xl, zephyr xenlib), partial device
tree is parsed and loaded on control domain side.
SIDs in guests device tree have to be replaced with
virtual SIDs which are mapped to physical SIDs. In order
to do that, control domain has to request from Xen to create
a new vSID and map it to original pSID for every guest device IOMMU
stream ID. For this purpose, new domctl command (XEN_DOMCTL_viommu_allocate_vid)
is introduced which control domain can use to request a new vSID mapping and
insert a new vSID into guest device tree once mapped.
Requested vSID allocation using this interface for vPCI/DT devices.

Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/include/xenctrl.h             |  12 +++
 tools/libs/ctrl/xc_domain.c         |  23 +++++
 tools/libs/light/libxl_arm.c        | 127 +++++++++++++++++++++++++---
 xen/arch/arm/domctl.c               |  31 +++++++
 xen/include/public/domctl.h         |  18 ++++
 xen/xsm/flask/hooks.c               |   4 +
 xen/xsm/flask/policy/access_vectors |   2 +
 7 files changed, 207 insertions(+), 10 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index d5dbf69c89..61be892cc8 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2659,6 +2659,18 @@ int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
                              const uint32_t *llc_colors,
                              uint32_t num_llc_colors);
 
+/*
+ * Allocate guest IOMMU vSID and establish its mapping to pSID.
+ * It can only be used on domain DT creation.
+ * Currently used for ARM only, possibly for RISC-V in the
+ * future. Function has no effect for x86.
+ */
+int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
+                                         uint32_t domid,
+                                         uint16_t nr_sids,
+                                         uint32_t first_psid,
+                                         uint32_t *first_vsid);
+
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index 01c0669c88..39ffe80e6d 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -2222,6 +2222,29 @@ out:
 
     return ret;
 }
+
+int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
+                                         uint32_t domid,
+                                         uint16_t nr_sids,
+                                         uint32_t first_psid,
+                                         uint32_t *first_vsid)
+{
+    int err;
+    struct xen_domctl domctl = {};
+
+    domctl.cmd = XEN_DOMCTL_viommu_alloc_vsid_range;
+    domctl.domain = domid;
+    domctl.u.viommu_alloc_vsid_range.first_psid = first_psid;
+    domctl.u.viommu_alloc_vsid_range.nr_sids = nr_sids;
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        return err;
+
+    *first_vsid = domctl.u.viommu_alloc_vsid_range.first_vsid;
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 7b887898bb..79904b746c 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -955,6 +955,13 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+/*
+ * Stores starting vSID of vPCI IOMMU SID range
+ * Used as a lookup value to avoid repeated
+ * vSID range allocation on every fdt resize.
+ */
+static int vpci_first_vsid = -1;
+
 static int make_vpci_node(libxl__gc *gc, void *fdt,
                           const struct arch_info *ainfo,
                           struct xc_dom_image *dom)
@@ -963,6 +970,9 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
     const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
     const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
     const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
+    uint16_t iommu_range_size = 0x1000;
+    uint32_t first_vsid;
+    uint32_t first_psid = 0;
 
     res = fdt_begin_node(fdt, name);
     if (res) return res;
@@ -996,8 +1006,20 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
         GUEST_VPCI_PREFETCH_MEM_SIZE);
     if (res) return res;
 
+    /* request vSID range allocation if not already allocated */
+    if (vpci_first_vsid < 0) {
+        res = xc_domain_viommu_allocate_vsid_range(CTX->xch, dom->guest_domid,
+            iommu_range_size, first_psid, &first_vsid);
+        if (res)
+            return res;
+        vpci_first_vsid = first_vsid;
+    }
+    else {
+        first_vsid = vpci_first_vsid;
+    }
+
     res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
-                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
+                             GUEST_PHANDLE_VSMMUV3, first_vsid, iommu_range_size);
     if (res) return res;
 
     res = fdt_end_node(fdt);
@@ -1326,11 +1348,92 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return 0;
 }
 
-static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+/*
+ * Store virtualized 'iommus' properties for every node attached to IOMMU
+ * and passthroughed to guest.
+ * Used as a lookup table for mapping <phandle pSID> -> <vhandle vSID>
+ */
+struct viommu_stream {
+    XEN_LIST_ENTRY(struct viommu_stream) entry;
+    char path[128];          /* DT path, stable across resizes */
+    fdt32_t *iommus;         /* fully virtualized iommus property */
+};
+
+static XEN_LIST_HEAD(, struct viommu_stream) viommu_stream_list;
+
+/*
+ * Helper function which creates mapping of dt node to
+ * to virtualized 'iommus' property
+ * Mappings stored in a global 'viommu_stream_list' to
+ * make it reusable for every fdt resize
+ */
+static int viommu_get_stream(libxl__gc *gc,
+                             uint32_t domid,
+                             const fdt32_t *prop,
+                             int proplen,
+                             const char* path, fdt32_t **iommus)
+{
+    int i, r;
+    uint32_t vsid, psid;
+    struct viommu_stream *viommu_stream;
+
+    /* Lookup if stream for target device is already allocated */
+    XEN_LIST_FOREACH(viommu_stream, &viommu_stream_list, entry)
+    {
+        if (!strcmp(viommu_stream->path, path)) {
+            *iommus = viommu_stream->iommus;
+            return 0;
+        }
+    }
+
+    /* Allocate new viommu stream */
+    viommu_stream = malloc(sizeof(struct viommu_stream));
+    if (!viommu_stream)
+        return ERROR_NOMEM;
+    memset(viommu_stream, 0, sizeof(struct viommu_stream));
+    viommu_stream->iommus = malloc(proplen);
+    if (!viommu_stream->iommus)
+        return ERROR_NOMEM;
+    memset(viommu_stream->iommus, 0, proplen);
+
+    LOG(DEBUG, "Creating vIOMMU stream for device %s",
+        path);
+
+    /*
+     * Virtualize device "iommus" property
+     * (replace pIOMMU with vIOMMU phandle and pSIDs with mapped vSIDs)
+     */
+    for (i = 0; i < proplen / 8; ++i) {
+        viommu_stream->iommus[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+        /* Allocate new vSID mapped to pSID */
+        psid = fdt32_to_cpu(prop[i * 2 + 1]);
+        r = xc_domain_viommu_allocate_vsid_range(CTX->xch, domid, 1, psid, &vsid);
+        if (r) {
+            LOG(ERROR, "Can't allocate new vSID/vRID for guest IOMMU device");
+            return r;
+        }
+        viommu_stream->iommus[i * 2 + 1] = cpu_to_fdt32(vsid);
+        LOG(DEBUG, "Mapped vSID: %u to pSID: %u", vsid, psid);
+    }
+
+    strcpy(viommu_stream->path, path);
+    *iommus =  viommu_stream->iommus;
+
+    XEN_LIST_INSERT_HEAD(&viommu_stream_list, viommu_stream, entry);
+
+    return 0;
+}
+
+/*
+ * Used to update partial fdt when vIOMMU is enabled
+ * Maps dt properties of IOMMU devices to virtual IOMMU
+ */
+static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
 {
-    int nodeoff, proplen, i, r;
+    int nodeoff, proplen, r;
     const fdt32_t *prop;
     fdt32_t *prop_c;
+    char path[128];
 
     nodeoff = fdt_path_offset(pfdt, "/passthrough");
     if (nodeoff < 0)
@@ -1344,11 +1447,16 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
         if (!prop)
             continue;
 
-        prop_c = libxl__zalloc(gc, proplen);
+        r = fdt_get_path(pfdt, nodeoff, path, sizeof(path));
+        if ( r < 0 ) {
+            LOG(ERROR, "Can't get passthrough node path");
+            return r;
+        }
 
-        for (i = 0; i < proplen / 8; ++i) {
-            prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
-            prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        r = viommu_get_stream(gc, domid, prop, proplen, path, &prop_c);
+        if (r) {
+            LOG(ERROR, "Can't get viommu stream");
+            return r;
         }
 
         r = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
@@ -1360,7 +1468,6 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
 
     return 0;
 }
-
 #else
 
 static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
@@ -1379,7 +1486,7 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return -FDT_ERR_INTERNAL;
 }
 
-static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
 {
     LOG(ERROR, "partial device tree not supported");
 
@@ -1511,7 +1618,7 @@ next_resize:
         if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
             FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
             if (pfdt)
-                FDT( modify_partial_fdt(gc, pfdt) );
+                FDT( viommu_modify_partial_fdt(gc, pfdt, dom->guest_domid) );
         }
 
         for (i = 0; i < d_config->num_disks; i++) {
diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad914c915f..9679c386e5 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -16,6 +16,7 @@
 #include <xen/types.h>
 #include <xsm/xsm.h>
 #include <public/domctl.h>
+#include <asm/viommu.h>
 
 void arch_get_domain_info(const struct domain *d,
                           struct xen_domctl_getdomaininfo *info)
@@ -179,6 +180,36 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
     }
     case XEN_DOMCTL_dt_overlay:
         return dt_overlay_domctl(d, &domctl->u.dt_overlay);
+
+#ifdef CONFIG_VIRTUAL_IOMMU
+    case XEN_DOMCTL_viommu_alloc_vsid_range:
+    {
+        int rc = 0;
+        uint16_t i;
+        uint32_t vsid;
+        struct xen_domctl_viommu_alloc_vsid_range *viommu_alloc_vsid_range =
+            &domctl->u.viommu_alloc_vsid_range;
+
+        for ( i = 0; i < viommu_alloc_vsid_range->nr_sids; i++ )
+        {
+            rc = viommu_allocate_free_vid(d, viommu_alloc_vsid_range->first_psid
+                                            + i, &vsid);
+            if( rc )
+                return rc;
+        }
+
+        if ( !rc )
+        {
+            /* Calculate first vSID from allocated range */
+            viommu_alloc_vsid_range->first_vsid = vsid -
+                viommu_alloc_vsid_range->nr_sids + 1;
+            rc = copy_to_guest(u_domctl, domctl, 1);
+        }
+
+        return rc;
+    }
+#endif
+
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 8f6708c0a7..bb6d867cb8 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1276,6 +1276,22 @@ struct xen_domctl_get_domain_state {
     uint64_t unique_id;      /* Unique domain identifier. */
 };
 
+/*
+ * XEN_DOMCTL_viommu_alloc_vsid_range
+ *
+ * Allocate guest vSID range and
+ * establish pSID->vSID mapping for target range.
+ * Allocated range is continous
+ */
+struct xen_domctl_viommu_alloc_vsid_range {
+    /* IN: Range first pSID  */
+    uint32_t first_psid;
+    /* IN: Number of vSIDs to allocate */
+    uint16_t nr_sids;
+    /* OUT: Mapped range first vSID */
+    uint32_t first_vsid;
+};
+
 struct xen_domctl {
 /* Stable domctl ops: interface_version is required to be 0.  */
     uint32_t cmd;
@@ -1368,6 +1384,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_gsi_permission                88
 #define XEN_DOMCTL_set_llc_colors                89
 #define XEN_DOMCTL_get_domain_state              90 /* stable interface */
+#define XEN_DOMCTL_viommu_alloc_vsid_range       91
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1436,6 +1453,7 @@ struct xen_domctl {
 #endif
         struct xen_domctl_set_llc_colors    set_llc_colors;
         struct xen_domctl_get_domain_state  get_domain_state;
+        struct xen_domctl_viommu_alloc_vsid_range viommu_alloc_vsid_range;
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index b250b27065..91e80ea80d 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -820,6 +820,10 @@ static int cf_check flask_domctl(struct domain *d, unsigned int cmd,
     case XEN_DOMCTL_set_llc_colors:
         return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SET_LLC_COLORS);
 
+    case XEN_DOMCTL_viommu_alloc_vsid_range:
+        return current_has_perm(d, SECCLASS_DOMAIN2,
+            DOMAIN2__VIOMMU_ALLOC_VSID_RANGE);
+
     default:
         return avc_unknown_permission("domctl", cmd);
     }
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index ce907d50a4..e4ffe2f5db 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -255,6 +255,8 @@ class domain2
     set_llc_colors
 # XEN_DOMCTL_get_domain_state
     get_domain_state
+# XEN_DOMCTL_viommu_alloc_vsid_range
+    viommu_alloc_vsid_range
 }
 
 # Similar to class domain, but primarily contains domctls related to HVM domains
-- 
2.43.0

Re: [PATCH v2 22/23] libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping

[Jan Beulich]

On 23.03.2026 23:51, Milan Djokic wrote:
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -1276,6 +1276,22 @@ struct xen_domctl_get_domain_state {
>      uint64_t unique_id;      /* Unique domain identifier. */
>  };
>  
> +/*
> + * XEN_DOMCTL_viommu_alloc_vsid_range
> + *
> + * Allocate guest vSID range and
> + * establish pSID->vSID mapping for target range.
> + * Allocated range is continous
> + */
> +struct xen_domctl_viommu_alloc_vsid_range {
> +    /* IN: Range first pSID  */
> +    uint32_t first_psid;
> +    /* IN: Number of vSIDs to allocate */
> +    uint16_t nr_sids;
> +    /* OUT: Mapped range first vSID */
> +    uint32_t first_vsid;
> +};

Padding fields want making explicit, and want checking to be zero on
input.

Jan

[PATCH v2 23/23] doc/arm: vIOMMU design document

[Milan Djokic]

---
 docs/designs/arm-viommu.rst | 390 ++++++++++++++++++++++++++++++++++++
 1 file changed, 390 insertions(+)
 create mode 100644 docs/designs/arm-viommu.rst

diff --git a/docs/designs/arm-viommu.rst b/docs/designs/arm-viommu.rst
new file mode 100644
index 0000000000..0cf55d7108
--- /dev/null
+++ b/docs/designs/arm-viommu.rst
@@ -0,0 +1,390 @@
+==========================================================
+Design Proposal: Add SMMUv3 Stage-1 Support for XEN Guests
+==========================================================
+
+:Author:     Milan Djokic <milan_djokic@epam.com>
+:Date:       2026-02-13
+:Status:     Draft
+
+Introduction
+============
+
+The SMMUv3 supports two stages of translation. Each stage of translation 
+can be
+independently enabled. An incoming address is logically translated from 
+VA to
+IPA in stage 1, then the IPA is input to stage 2 which translates the IPA to
+the output PA. Stage 1 translation support is required to provide 
+isolation between different
+devices within OS. XEN already supports Stage 2 translation but there is no
+support for Stage 1 translation.
+This design proposal outlines the introduction of Stage-1 SMMUv3 support 
+in Xen for ARM guests.
+
+Motivation
+==========
+
+ARM systems utilizing SMMUv3 require stage-1 address translation to 
+ensure secure DMA and
+guest managed I/O memory mappings.
+With stage-1 enabled, guest manages IOVA to IPA mappings through its own 
+IOMMU driver.
+
+This feature enables:
+
+- Stage-1 translation for the guest domain
+- Device passthrough with per-device I/O address space
+
+Design Overview
+===============
+
+These changes provide emulated SMMUv3 support:
+
+- **SMMUv3 Stage-1 Translation**: stage-1 and nested translation support
+  in SMMUv3 driver.
+- **vIOMMU Abstraction**: Virtual IOMMU framework for guest stage-1
+  handling.
+- **Register/Command Emulation**: SMMUv3 register emulation and command
+  queue handling.
+- **Device Tree Extensions**: Adds `iommus` and virtual SMMUv3 nodes to
+  device trees for dom0 and dom0less scenarios.
+- **Runtime Configuration**: Introduces a `viommu` boot parameter for
+  dynamic enablement.
+
+A single vIOMMU device is exposed to the guest and mapped to one or more
+physical IOMMUs through a Xen-managed translation layer.
+The vIOMMU feature provides a generic framework together with a backend
+implementation specific to the target IOMMU type. The backend is responsible
+for implementing the hardware-specific data structures and command handling
+logic (currently only SMMUv3 is supported).
+
+This modular design allows the stage-1 support to be reused
+for other IOMMU architectures in the future.
+
+vIOMMU architecture
+===================
+
+Responsibilities:
+
+Guest:
+ - Configures stage-1 via vIOMMU commands.
+ - Handles stage-1 faults received from Xen.
+
+Xen:
+ - Emulates the IOMMU interface (registers, commands, events).
+ - Provides vSID->pSID mappings.
+ - Programs stage-1/stage-2 configuration in the physical IOMMU.
+ - Propagate stage-1 faults to guest.
+
+vIOMMU commands and faults are transmitted between guest and Xen via
+command and event queues (one command/event queue created per guest).
+
+vIOMMU command Flow:
+
+::
+
+    Guest:
+        smmu_cmd(vSID, IOVA -> IPA)
+
+    Xen:
+        trap MMIO read/write
+        translate vSID->pSID
+        store stage-1 state
+        program pIOMMU for (pSID, IPA -> PA)
+
+All hardware programming of the physical IOMMU is performed exclusively by Xen.
+
+vIOMMU Stage-1 fault handling flow:
+
+::
+
+    Xen:
+        receives stage-1 fault
+        triggers vIOMMU callback
+        injects virtual fault
+
+    Guest:
+        receives and handles fault
+
+vSID Mapping Layer
+------------------
+
+Each guest-visible Stream ID (vSID) is mapped by Xen to a physical Stream ID
+(pSID). The mapping is maintained per-domain. The allocation policy guarantees
+vSID uniqueness within a domain while allowing reuse of pSIDs for different
+pIOMMUs.
+
+* Platform devices receive individually allocated vSIDs.
+* PCI devices receive a contiguous vSID range derived from RID space.
+
+
+Supported Device Model
+======================
+
+Currently, the vIOMMU framework supports only devices described via the
+Device Tree (DT) model. This includes platform devices and basic PCI
+devices support instantiated through the vPCI DT node. ACPI-described
+devices are not supported.
+
+Guest assigned platform devices are mapped via `iommus` property:
+
+::
+
+    <&pIOMMU pSID> -> <&vIOMMU vSID>
+
+PCI devices use RID-based mapping via the root complex `iommu-map`:
+
+::
+
+    <RID-base &viommu vSID-base length>
+
+PCI Topology Assumptions and Constraints:
+
+- RID space must be contiguous
+- Pre-defined continuous pSID space (0-0x1000)
+- No runtime PCI reconfiguration
+- Single root complex assumed
+- Mapping is fixed at guest DT construction
+
+Constraints for PCI devices will be addressed as part of the future work on
+this feature.
+
+Security Considerations
+=======================
+
+Stage-1 translation provides isolation between guest devices by
+enforcing a per-device I/O address space, preventing unauthorized DMA.
+With the introduction of emulated IOMMU, additional protection
+mechanisms are required to minimize security risks.
+
+1. Observation:
+---------------
+Support for Stage-1 translation in SMMUv3 introduces new data structures 
+(`s1_cfg` alongside `s2_cfg`)
+and logic to write both Stage-1 and Stage-2 entries in the Stream Table 
+Entry (STE), including an `abort`
+field to handle partial configuration states.
+
+**Risk:**
+Without proper handling, a partially applied configuration
+might leave guest DMA mappings in an inconsistent state, potentially
+enabling unauthorized access or causing cross-domain interference.
+
+**Mitigation:** *(Handled by design)*
+This feature introduces logic that writes both `s1_cfg` and `s2_cfg` to
+STE and manages the `abort` field - only considering
+configuration if fully attached. This ensures  incomplete or invalid
+device configurations are safely ignored by the hypervisor.
+
+2. Observation:
+---------------
+Guests can now invalidate Stage-1 caches; invalidation needs forwarding
+to SMMUv3 hardware to maintain coherence.
+
+**Risk:**
+Failing to propagate cache invalidation could allow stale mappings,
+enabling access to old mappings and possibly
+data leakage or misrouting between devices assigned to the same guest.
+
+**Mitigation:**
+The guest must issue appropriate invalidation commands whenever
+its stage-1 I/O mappings are modified to ensure that translation caches
+remain coherent.
+
+3. Observation:
+---------------
+Introducing optional per-guest enabled features (`viommu` argument in xl 
+guest config) means some guests
+may opt-out.
+
+**Risk:**
+Guests without vIOMMU enabled (stage-2 only) could potentially dominate
+access to the physical command and event queues, since they bypass the
+emulation layer and processing is faster comparing to vIOMMU-enabled guests.
+
+**Mitigation:**
+Audit the impact of emulation overhead effect on IOMMU processing fairness
+in a multi-guest environment.
+Consider enabling/disabling stage-1 on a system level, instead of per-domain.
+
+4. Observation:
+---------------
+Guests have the ability to issue Stage-1 IOMMU commands like cache 
+invalidation, stream table entries
+configuration, etc. An adversarial guest may issue a high volume of 
+commands in rapid succession.
+
+**Risk:**
+Excessive commands requests can cause high hypervisor CPU consumption 
+and disrupt scheduling,
+leading to degraded system responsiveness and potential 
+denial-of-service scenarios.
+
+**Mitigation:**
+
+- Implement vIOMMU commands execution restart and continuation support:
+
+  - Introduce processing budget with only a limited amount of commands
+    handled per invocation.
+  - If additional commands remain pending after the budget is exhausted,
+    defer further processing and resume it asynchronously, e.g. via a
+    per-domain tasklet.
+
+- Batch multiple commands of same type to reduce emulation overhead:
+
+  - Inspect the command queue and group commands that can be processed
+    together (e.g. multiple successive invalidation requests or STE
+    updates for the same SID).
+  - Execute the entire batch in one go, reducing repeated accesses to
+    guest memory and emulation overhead per command.
+  - This reduces CPU time spent in the vIOMMU command processing loop.
+    The optimization is applicable only when consecutive commands of the
+    same type operate on the same SID/context.
+
+5. Observation:
+---------------
+Some guest commands issued towards vIOMMU are propagated to pIOMMU 
+command queue (e.g. TLB invalidate).
+
+**Risk:**
+Excessive commands requests from abusive guest can cause flooding of 
+physical IOMMU command queue,
+leading to degraded pIOMMU responsiveness on commands issued from other 
+guests.
+
+**Mitigation:**
+
+- Batch commands that are propagated to the pIOMMU command queue and
+  implement batch execution pause/continuation.
+  Rely on the same mechanisms as in the previous observation
+  (command continuation and batching of pIOMMU-related commands of the same
+  type and context).
+- If possible, implement domain penalization by adding a per-domain budget
+  for vIOMMU/pIOMMU usage:
+
+  - Apply per-domain dynamic budgeting of allowed IOMMU commands to
+    execute per invocation, reducing the budget for guests with
+    excessive command requests over a longer period of time
+  - Combine with command continuation mechanism
+
+6. Observation:
+---------------
+The vIOMMU feature includes an event queue used to forward IOMMU events
+to the guest (e.g. translation faults, invalid Stream IDs, permission errors).
+A malicious guest may misconfigure its IOMMU state or intentionally trigger
+faults at a high rate.
+
+**Risk:**
+Occurrence of IOMMU events with high frequency can cause Xen to flood the
+event queue and disrupt scheduling with
+high hypervisor CPU load for events handling.
+
+**Mitigation:**
+
+- Implement fail-safe state by disabling events forwarding when faults 
+  are occurred with high frequency and
+  not processed by guest:
+
+  - Introduce a per-domain pending event counter.
+  - Stop forwarding events to the guest once the number of unprocessed
+    events reaches a predefined threshold.
+
+- Consider disabling the emulated event queue for untrusted guests.
+- Note that this risk is more general and may also apply to stage-2-only
+  guests. This section addresses mitigations in the emulated IOMMU layer
+  only. Mitigation of physical event queue flooding should also be considered
+  in the target pIOMMU driver.
+
+Performance Impact
+==================
+
+With iommu stage-1 and nested translation inclusion, performance 
+overhead is introduced comparing to existing,
+stage-2 only usage in Xen. Once mappings are established, translations 
+should not introduce significant overhead.
+Emulated paths may introduce moderate overhead, primarily affecting 
+device initialization and event/command handling.
+Testing is performed on Renesas R-Car platform.
+Performance is mostly impacted by emulated vIOMMU operations, results 
+shown in the following table.
+
++-------------------------------+---------------------------------+
+| vIOMMU Operation              | Execution time in guest         |
++===============================+=================================+
+| Reg read                      | median: 645ns, worst-case: 2us  |
++-------------------------------+---------------------------------+
+| Reg write                     | median: 630ns, worst-case: 1us  |
++-------------------------------+---------------------------------+
+| Invalidate TLB                | median: 2us, worst-case: 10us   |
++-------------------------------+---------------------------------+
+| Invalidate STE                | median: 5us worst_case: 100us   |
++-------------------------------+---------------------------------+
+
+With vIOMMU exposed to guest, guest OS has to initialize IOMMU device
+and configure stage-1 mappings for the devices
+attached to it.
+Following table shows initialization stages which impact stage-1 enabled 
+guest boot time and compares it with
+stage-1 disabled guest.
+
+NOTE: Device probe execution time varies depending on device complexity.
+A USB host controller was selected as the test device in this case.
+
++---------------------+-----------------------+------------------------+
+| Stage               | Stage-1 Enabled Guest | Stage-1 Disabled Guest |
++=====================+=======================+========================+
+| IOMMU Init          | ~10ms                 | /                      |
++---------------------+-----------------------+------------------------+
+| Dev Attach / Mapping| ~100ms                | ~90ms                  |
++---------------------+-----------------------+------------------------+
+
+For devices configured with dynamic DMA mappings, DMA allocate/map/unmap 
+operations performance is
+also impacted on stage-1 enabled guests.
+Dynamic DMA mapping operation trigger emulated IOMMU functions like mmio 
+write/read and TLB invalidations.
+
++---------------+---------------------------+--------------------------+
+| DMA Op        | Stage-1 Enabled Guest     | Stage-1 Disabled Guest   |
++===============+===========================+==========================+
+| dma_alloc     | median: 20us, worst: 5ms  | median: 8us, worst: 60us |
++---------------+---------------------------+--------------------------+
+| dma_free      | median: 500us, worst: 10ms| median: 6us, worst: 30us |
++---------------+---------------------------+--------------------------+
+| dma_map       | median: 12us, worst: 60us | median: 3us, worst: 20us |
++---------------+---------------------------+--------------------------+
+| dma_unmap     | median: 400us, worst: 5ms | median: 5us, worst: 20us |
++---------------+---------------------------+--------------------------+
+
+Testing
+=======
+
+- QEMU-based ARM system tests for Stage-1 translation.
+- Actual hardware validation to ensure compatibility with real SMMUv3 
+implementations.
+- Unit/Functional tests validating correct translations (not implemented).
+
+Migration and Compatibility
+===========================
+
+This optional feature defaults to disabled (`viommu=""`) for backward 
+compatibility.
+
+Future improvements
+===================
+
+- Implement the proposed mitigations to address security risks that are 
+  not covered by the current design
+  (events batching, commands execution continuation)
+- PCI support
+- Support for other IOMMU HW (Renesas, RISC-V, etc.)
+
+References
+==========
+
+- Original feature implemented by Rahul Singh:
+  
+https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/ 
+
+- SMMUv3 architecture documentation
+- Existing vIOMMU code patterns (KVM, QEMU)
\ No newline at end of file
-- 
2.43.0

Re: [PATCH v2 23/23] doc/arm: vIOMMU design document

[Mykola Kvach]

This patch seems to be missing both the Signed-off-by tag and the commit
message body.

On 24/03/2026 00:51, Milan Djokic wrote:
> ---
>   docs/designs/arm-viommu.rst | 390 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 390 insertions(+)
>   create mode 100644 docs/designs/arm-viommu.rst
> 
> diff --git a/docs/designs/arm-viommu.rst b/docs/designs/arm-viommu.rst
> new file mode 100644
> index 0000000000..0cf55d7108
> --- /dev/null
> +++ b/docs/designs/arm-viommu.rst
> @@ -0,0 +1,390 @@
> +==========================================================
> +Design Proposal: Add SMMUv3 Stage-1 Support for XEN Guests
> +==========================================================
> +
> +:Author:     Milan Djokic <milan_djokic@epam.com>
> +:Date:       2026-02-13
> +:Status:     Draft
> +
> +Introduction
> +============
> +
> +The SMMUv3 supports two stages of translation. Each stage of translation
> +can be
> +independently enabled. An incoming address is logically translated from
> +VA to
> +IPA in stage 1, then the IPA is input to stage 2 which translates the IPA to
> +the output PA. Stage 1 translation support is required to provide
> +isolation between different
> +devices within OS. XEN already supports Stage 2 translation but there is no
> +support for Stage 1 translation.
> +This design proposal outlines the introduction of Stage-1 SMMUv3 support
> +in Xen for ARM guests.
> +
> +Motivation
> +==========
> +
> +ARM systems utilizing SMMUv3 require stage-1 address translation to
> +ensure secure DMA and
> +guest managed I/O memory mappings.
> +With stage-1 enabled, guest manages IOVA to IPA mappings through its own
> +IOMMU driver.
> +
> +This feature enables:
> +
> +- Stage-1 translation for the guest domain
> +- Device passthrough with per-device I/O address space
> +
> +Design Overview
> +===============
> +
> +These changes provide emulated SMMUv3 support:
> +
> +- **SMMUv3 Stage-1 Translation**: stage-1 and nested translation support
> +  in SMMUv3 driver.
> +- **vIOMMU Abstraction**: Virtual IOMMU framework for guest stage-1
> +  handling.
> +- **Register/Command Emulation**: SMMUv3 register emulation and command
> +  queue handling.
> +- **Device Tree Extensions**: Adds `iommus` and virtual SMMUv3 nodes to
> +  device trees for dom0 and dom0less scenarios.
> +- **Runtime Configuration**: Introduces a `viommu` boot parameter for
> +  dynamic enablement.
> +
> +A single vIOMMU device is exposed to the guest and mapped to one or more
> +physical IOMMUs through a Xen-managed translation layer.
> +The vIOMMU feature provides a generic framework together with a backend
> +implementation specific to the target IOMMU type. The backend is responsible
> +for implementing the hardware-specific data structures and command handling
> +logic (currently only SMMUv3 is supported).
> +
> +This modular design allows the stage-1 support to be reused
> +for other IOMMU architectures in the future.
> +
> +vIOMMU architecture
> +===================
> +
> +Responsibilities:
> +
> +Guest:
> + - Configures stage-1 via vIOMMU commands.
> + - Handles stage-1 faults received from Xen.
> +
> +Xen:
> + - Emulates the IOMMU interface (registers, commands, events).
> + - Provides vSID->pSID mappings.
> + - Programs stage-1/stage-2 configuration in the physical IOMMU.
> + - Propagate stage-1 faults to guest.
> +
> +vIOMMU commands and faults are transmitted between guest and Xen via
> +command and event queues (one command/event queue created per guest).
> +
> +vIOMMU command Flow:
> +
> +::
> +
> +    Guest:
> +        smmu_cmd(vSID, IOVA -> IPA)
> +
> +    Xen:
> +        trap MMIO read/write
> +        translate vSID->pSID
> +        store stage-1 state
> +        program pIOMMU for (pSID, IPA -> PA)
> +
> +All hardware programming of the physical IOMMU is performed exclusively by Xen.
> +
> +vIOMMU Stage-1 fault handling flow:
> +
> +::
> +
> +    Xen:
> +        receives stage-1 fault
> +        triggers vIOMMU callback
> +        injects virtual fault
> +
> +    Guest:
> +        receives and handles fault
> +
> +vSID Mapping Layer
> +------------------
> +
> +Each guest-visible Stream ID (vSID) is mapped by Xen to a physical Stream ID
> +(pSID). The mapping is maintained per-domain. The allocation policy guarantees
> +vSID uniqueness within a domain while allowing reuse of pSIDs for different
> +pIOMMUs.
> +
> +* Platform devices receive individually allocated vSIDs.
> +* PCI devices receive a contiguous vSID range derived from RID space.
> +
> +
> +Supported Device Model
> +======================
> +
> +Currently, the vIOMMU framework supports only devices described via the
> +Device Tree (DT) model. This includes platform devices and basic PCI
> +devices support instantiated through the vPCI DT node. ACPI-described
> +devices are not supported.
> +
> +Guest assigned platform devices are mapped via `iommus` property:
> +
> +::
> +
> +    <&pIOMMU pSID> -> <&vIOMMU vSID>
> +
> +PCI devices use RID-based mapping via the root complex `iommu-map`:
> +
> +::
> +
> +    <RID-base &viommu vSID-base length>
> +
> +PCI Topology Assumptions and Constraints:
> +
> +- RID space must be contiguous
> +- Pre-defined continuous pSID space (0-0x1000)
> +- No runtime PCI reconfiguration
> +- Single root complex assumed
> +- Mapping is fixed at guest DT construction
> +
> +Constraints for PCI devices will be addressed as part of the future work on
> +this feature.
> +
> +Security Considerations
> +=======================
> +
> +Stage-1 translation provides isolation between guest devices by
> +enforcing a per-device I/O address space, preventing unauthorized DMA.
> +With the introduction of emulated IOMMU, additional protection
> +mechanisms are required to minimize security risks.
> +
> +1. Observation:
> +---------------
> +Support for Stage-1 translation in SMMUv3 introduces new data structures
> +(`s1_cfg` alongside `s2_cfg`)
> +and logic to write both Stage-1 and Stage-2 entries in the Stream Table
> +Entry (STE), including an `abort`
> +field to handle partial configuration states.
> +
> +**Risk:**
> +Without proper handling, a partially applied configuration
> +might leave guest DMA mappings in an inconsistent state, potentially
> +enabling unauthorized access or causing cross-domain interference.
> +
> +**Mitigation:** *(Handled by design)*
> +This feature introduces logic that writes both `s1_cfg` and `s2_cfg` to
> +STE and manages the `abort` field - only considering
> +configuration if fully attached. This ensures  incomplete or invalid
> +device configurations are safely ignored by the hypervisor.
> +
> +2. Observation:
> +---------------
> +Guests can now invalidate Stage-1 caches; invalidation needs forwarding
> +to SMMUv3 hardware to maintain coherence.
> +
> +**Risk:**
> +Failing to propagate cache invalidation could allow stale mappings,
> +enabling access to old mappings and possibly
> +data leakage or misrouting between devices assigned to the same guest.
> +
> +**Mitigation:**
> +The guest must issue appropriate invalidation commands whenever
> +its stage-1 I/O mappings are modified to ensure that translation caches
> +remain coherent.
> +
> +3. Observation:
> +---------------
> +Introducing optional per-guest enabled features (`viommu` argument in xl
> +guest config) means some guests
> +may opt-out.
> +
> +**Risk:**
> +Guests without vIOMMU enabled (stage-2 only) could potentially dominate
> +access to the physical command and event queues, since they bypass the
> +emulation layer and processing is faster comparing to vIOMMU-enabled guests.
> +
> +**Mitigation:**
> +Audit the impact of emulation overhead effect on IOMMU processing fairness
> +in a multi-guest environment.
> +Consider enabling/disabling stage-1 on a system level, instead of per-domain.
> +
> +4. Observation:
> +---------------
> +Guests have the ability to issue Stage-1 IOMMU commands like cache
> +invalidation, stream table entries
> +configuration, etc. An adversarial guest may issue a high volume of
> +commands in rapid succession.
> +
> +**Risk:**
> +Excessive commands requests can cause high hypervisor CPU consumption
> +and disrupt scheduling,
> +leading to degraded system responsiveness and potential
> +denial-of-service scenarios.
> +
> +**Mitigation:**
> +
> +- Implement vIOMMU commands execution restart and continuation support:
> +
> +  - Introduce processing budget with only a limited amount of commands
> +    handled per invocation.
> +  - If additional commands remain pending after the budget is exhausted,
> +    defer further processing and resume it asynchronously, e.g. via a
> +    per-domain tasklet.
> +
> +- Batch multiple commands of same type to reduce emulation overhead:
> +
> +  - Inspect the command queue and group commands that can be processed
> +    together (e.g. multiple successive invalidation requests or STE
> +    updates for the same SID).
> +  - Execute the entire batch in one go, reducing repeated accesses to
> +    guest memory and emulation overhead per command.
> +  - This reduces CPU time spent in the vIOMMU command processing loop.
> +    The optimization is applicable only when consecutive commands of the
> +    same type operate on the same SID/context.
> +
> +5. Observation:
> +---------------
> +Some guest commands issued towards vIOMMU are propagated to pIOMMU
> +command queue (e.g. TLB invalidate).
> +
> +**Risk:**
> +Excessive commands requests from abusive guest can cause flooding of
> +physical IOMMU command queue,
> +leading to degraded pIOMMU responsiveness on commands issued from other
> +guests.
> +
> +**Mitigation:**
> +
> +- Batch commands that are propagated to the pIOMMU command queue and
> +  implement batch execution pause/continuation.
> +  Rely on the same mechanisms as in the previous observation
> +  (command continuation and batching of pIOMMU-related commands of the same
> +  type and context).
> +- If possible, implement domain penalization by adding a per-domain budget
> +  for vIOMMU/pIOMMU usage:
> +
> +  - Apply per-domain dynamic budgeting of allowed IOMMU commands to
> +    execute per invocation, reducing the budget for guests with
> +    excessive command requests over a longer period of time
> +  - Combine with command continuation mechanism
> +
> +6. Observation:
> +---------------
> +The vIOMMU feature includes an event queue used to forward IOMMU events
> +to the guest (e.g. translation faults, invalid Stream IDs, permission errors).
> +A malicious guest may misconfigure its IOMMU state or intentionally trigger
> +faults at a high rate.
> +
> +**Risk:**
> +Occurrence of IOMMU events with high frequency can cause Xen to flood the
> +event queue and disrupt scheduling with
> +high hypervisor CPU load for events handling.
> +
> +**Mitigation:**
> +
> +- Implement fail-safe state by disabling events forwarding when faults
> +  are occurred with high frequency and
> +  not processed by guest:
> +
> +  - Introduce a per-domain pending event counter.
> +  - Stop forwarding events to the guest once the number of unprocessed
> +    events reaches a predefined threshold.
> +
> +- Consider disabling the emulated event queue for untrusted guests.
> +- Note that this risk is more general and may also apply to stage-2-only
> +  guests. This section addresses mitigations in the emulated IOMMU layer
> +  only. Mitigation of physical event queue flooding should also be considered
> +  in the target pIOMMU driver.
> +
> +Performance Impact
> +==================
> +
> +With iommu stage-1 and nested translation inclusion, performance
> +overhead is introduced comparing to existing,
> +stage-2 only usage in Xen. Once mappings are established, translations
> +should not introduce significant overhead.
> +Emulated paths may introduce moderate overhead, primarily affecting
> +device initialization and event/command handling.
> +Testing is performed on Renesas R-Car platform.
> +Performance is mostly impacted by emulated vIOMMU operations, results
> +shown in the following table.
> +
> ++-------------------------------+---------------------------------+
> +| vIOMMU Operation              | Execution time in guest         |
> ++===============================+=================================+
> +| Reg read                      | median: 645ns, worst-case: 2us  |
> ++-------------------------------+---------------------------------+
> +| Reg write                     | median: 630ns, worst-case: 1us  |
> ++-------------------------------+---------------------------------+
> +| Invalidate TLB                | median: 2us, worst-case: 10us   |
> ++-------------------------------+---------------------------------+
> +| Invalidate STE                | median: 5us worst_case: 100us   |
> ++-------------------------------+---------------------------------+
> +
> +With vIOMMU exposed to guest, guest OS has to initialize IOMMU device
> +and configure stage-1 mappings for the devices
> +attached to it.
> +Following table shows initialization stages which impact stage-1 enabled
> +guest boot time and compares it with
> +stage-1 disabled guest.
> +
> +NOTE: Device probe execution time varies depending on device complexity.
> +A USB host controller was selected as the test device in this case.
> +
> ++---------------------+-----------------------+------------------------+
> +| Stage               | Stage-1 Enabled Guest | Stage-1 Disabled Guest |
> ++=====================+=======================+========================+
> +| IOMMU Init          | ~10ms                 | /                      |
> ++---------------------+-----------------------+------------------------+
> +| Dev Attach / Mapping| ~100ms                | ~90ms                  |
> ++---------------------+-----------------------+------------------------+
> +
> +For devices configured with dynamic DMA mappings, DMA allocate/map/unmap
> +operations performance is
> +also impacted on stage-1 enabled guests.
> +Dynamic DMA mapping operation trigger emulated IOMMU functions like mmio
> +write/read and TLB invalidations.
> +
> ++---------------+---------------------------+--------------------------+
> +| DMA Op        | Stage-1 Enabled Guest     | Stage-1 Disabled Guest   |
> ++===============+===========================+==========================+
> +| dma_alloc     | median: 20us, worst: 5ms  | median: 8us, worst: 60us |
> ++---------------+---------------------------+--------------------------+
> +| dma_free      | median: 500us, worst: 10ms| median: 6us, worst: 30us |
> ++---------------+---------------------------+--------------------------+
> +| dma_map       | median: 12us, worst: 60us | median: 3us, worst: 20us |
> ++---------------+---------------------------+--------------------------+
> +| dma_unmap     | median: 400us, worst: 5ms | median: 5us, worst: 20us |
> ++---------------+---------------------------+--------------------------+
> +
> +Testing
> +=======
> +
> +- QEMU-based ARM system tests for Stage-1 translation.
> +- Actual hardware validation to ensure compatibility with real SMMUv3
> +implementations.
> +- Unit/Functional tests validating correct translations (not implemented).
> +
> +Migration and Compatibility
> +===========================
> +
> +This optional feature defaults to disabled (`viommu=""`) for backward
> +compatibility.
> +
> +Future improvements
> +===================
> +
> +- Implement the proposed mitigations to address security risks that are
> +  not covered by the current design
> +  (events batching, commands execution continuation)
> +- PCI support
> +- Support for other IOMMU HW (Renesas, RISC-V, etc.)
> +
> +References
> +==========
> +
> +- Original feature implemented by Rahul Singh:
> +
> +https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/
> +
> +- SMMUv3 architecture documentation
> +- Existing vIOMMU code patterns (KVM, QEMU)
> \ No newline at end of file

[PATCH v3 00/23] Add SMMUv3 Stage 1 Support for Xen guests

[Milan Djokic]

This patch series provides emulated SMMUv3 support in Xen, enabling stage-1
translation for the guest OS.

Stage 1 translation support is required to provide isolation between different
devices within OS. Xen already supports Stage 2 translation but there is no
support for Stage 1 translation. The goal of this work is to support Stage 1
translation for Xen guests.

This patch series represents a continuation of work from Rahul Singh:
https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/
Original patch series is aligned with the newest Xen structure, with the addition
of translation layer which provides 1:N vIOMMU->pIOMMU mapping, in order to
support passthrough of the devices attached to different physical IOMMUs.

We cannot trust the guest OS to control the SMMUv3 hardware directly as
compromised guest OS can corrupt the SMMUv3 configuration and make the system
vulnerable. The guest gets the ownership of the stage 1 page tables and also
owns stage 1 configuration structures. The Xen handles the root configuration
structure (for security reasons), including the stage 2 configuration.

XEN will emulate the SMMUv3 hardware and expose the virtual SMMUv3 to the
guest. Guest can use the native SMMUv3 driver to configure the stage 1
translation. When the guest configures the SMMUv3 for Stage 1, XEN will trap
the access and configure hardware.

SMMUv3 Driver(Guest OS) -> Configure the Stage-1 translation ->
XEN trap access -> XEN SMMUv3 driver configure the HW.

The final patch series commit provides a design document for the emulated
IOMMU (arm-viommu.rst), which was previously discussed with the maintainers.
Details regarding implementation, future work and security risks are outlined
in this document.

---
Changes in v2:
 - Updated design and implementation with vIOMMU->pIOMMU mapping layer
 - Addressed security risks in the design, provided initial performance
   measurements
 - Addressed comments from previous version
 - Tested on Renesas R-Car platform, initial performance measurements for
   stage-1 vs stage-1-less guests
---

---
Changes in v3:
 - Bump domctl version, added explicit padding for the new domctl structures
 - Remove unnecessary changes according to review comments
 - Add "ARM" prefix for vIOMMU Kconfig options, since only ARM architecture is
   supported at this point 
 - Re-generate go code
 - Add missing commit sign-off tags
---

Jean-Philippe Brucker (1):
  xen/arm: smmuv3: Maintain a SID->device structure

Milan Djokic (3):
  xen/arm: vIOMMU vSID->pSID mapping layer
  libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping
  doc/arm: vIOMMU design document

Rahul Singh (19):
  xen/arm: smmuv3: Add support for stage-1 and nested stage translation
  xen/arm: smmuv3: Alloc io_domain for each device
  xen/arm: vIOMMU: add generic vIOMMU framework
  xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests
  xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param
  xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"
  xen/arm: vsmmuv3: Add support for registers emulation
  xen/arm: vsmmuv3: Add support for cmdqueue handling
  xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE
  xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware
  xen/arm: vsmmuv3: Add support for event queue and global error
  xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices
  xen/arm: vIOMMU: IOMMU device tree node for dom0
  xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less
  arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl
  xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3
  xen/arm: vsmmuv3: Add support to send stage-1 event to guest
  libxl/arm: vIOMMU: Modify the partial device tree for iommus
  xen/arm: vIOMMU: Modify the partial device tree for dom0less

 docs/designs/arm-viommu.rst             | 390 ++++++++++
 docs/man/xl.cfg.5.pod.in                |  13 +
 docs/misc/xen-command-line.pandoc       |   9 +
 tools/golang/xenlight/helpers.gen.go    |   2 +
 tools/golang/xenlight/types.gen.go      |   7 +
 tools/include/libxl.h                   |   5 +
 tools/include/xenctrl.h                 |  12 +
 tools/libs/ctrl/xc_domain.c             |  23 +
 tools/libs/light/libxl_arm.c            | 230 +++++-
 tools/libs/light/libxl_types.idl        |   6 +
 tools/xl/xl_parse.c                     |   9 +
 xen/arch/arm/dom0less-build.c           |  72 ++
 xen/arch/arm/domain.c                   |  34 +
 xen/arch/arm/domain_build.c             | 103 ++-
 xen/arch/arm/domctl.c                   |  34 +
 xen/arch/arm/include/asm/domain.h       |   4 +
 xen/arch/arm/include/asm/iommu.h        |   7 +
 xen/arch/arm/include/asm/viommu.h       | 113 +++
 xen/common/device-tree/dom0less-build.c |  55 +-
 xen/drivers/passthrough/Kconfig         |  13 +
 xen/drivers/passthrough/arm/Makefile    |   2 +
 xen/drivers/passthrough/arm/smmu-v3.c   | 369 ++++++++-
 xen/drivers/passthrough/arm/smmu-v3.h   |  49 +-
 xen/drivers/passthrough/arm/viommu.c    |  96 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c  | 958 ++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h  |  32 +
 xen/include/public/arch-arm.h           |  15 +-
 xen/include/public/device_tree_defs.h   |   1 +
 xen/include/public/domctl.h             |  24 +-
 xen/include/xen/iommu.h                 |   6 +
 xen/xsm/flask/hooks.c                   |   4 +
 xen/xsm/flask/policy/access_vectors     |   2 +
 32 files changed, 2646 insertions(+), 53 deletions(-)
 create mode 100644 docs/designs/arm-viommu.rst
 create mode 100644 xen/arch/arm/include/asm/viommu.h
 create mode 100644 xen/drivers/passthrough/arm/viommu.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h

-- 
2.43.0

[PATCH v3 01/23] xen/arm: smmuv3: Maintain a SID->device structure

[Milan Djokic]

From: Jean-Philippe Brucker <jean-philippe@linaro.org>

Backport Linux commit cdf315f907d4. This is the clean backport without
any changes.

When handling faults from the event or PRI queue, we need to find the
struct device associated with a SID. Add a rb_tree to keep track of
SIDs.

Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdf315f907d4
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 131 +++++++++++++++++++++-----
 xen/drivers/passthrough/arm/smmu-v3.h |  13 ++-
 2 files changed, 118 insertions(+), 26 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index bf153227db..73cc4ef08f 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -809,6 +809,27 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 	return 0;
 }
 
+__maybe_unused
+static struct arm_smmu_master *
+arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+{
+	struct rb_node *node;
+	struct arm_smmu_stream *stream;
+
+	node = smmu->streams.rb_node;
+	while (node) {
+		stream = rb_entry(node, struct arm_smmu_stream, node);
+		if (stream->id < sid)
+			node = node->rb_right;
+		else if (stream->id > sid)
+			node = node->rb_left;
+		else
+			return stream->master;
+	}
+
+	return NULL;
+}
+
 /* IRQ and event handlers */
 static void arm_smmu_evtq_tasklet(void *dev)
 {
@@ -1042,8 +1063,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
 	if (!master->ats_enabled)
 		return 0;
 
-	for (i = 0; i < master->num_sids; i++) {
-		cmd->atc.sid = master->sids[i];
+	for (i = 0; i < master->num_streams; i++) {
+		cmd->atc.sid = master->streams[i].id;
 		arm_smmu_cmdq_issue_cmd(master->smmu, cmd);
 	}
 
@@ -1271,13 +1292,13 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master)
 	int i, j;
 	struct arm_smmu_device *smmu = master->smmu;
 
-	for (i = 0; i < master->num_sids; ++i) {
-		u32 sid = master->sids[i];
+    for (i = 0; i < master->num_streams; ++i) {
+		u32 sid = master->streams[i].id;
 		__le64 *step = arm_smmu_get_step_for_sid(smmu, sid);
 
 		/* Bridged PCI devices may end up with duplicated IDs */
 		for (j = 0; j < i; j++)
-			if (master->sids[j] == sid)
+			if (master->streams[j].id == sid)
 				break;
 		if (j < i)
 			continue;
@@ -1486,6 +1507,80 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 
 	return sid < limit;
 }
+
+static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
+				  struct arm_smmu_master *master)
+{
+	int i;
+	int ret = 0;
+	struct arm_smmu_stream *new_stream, *cur_stream;
+	struct rb_node **new_node, *parent_node = NULL;
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev);
+
+	master->streams = _xzalloc_array(sizeof(*master->streams), sizeof(void *),
+					fwspec->num_ids);
+	if (!master->streams)
+		return -ENOMEM;
+	master->num_streams = fwspec->num_ids;
+
+	mutex_lock(&smmu->streams_mutex);
+	for (i = 0; i < fwspec->num_ids; i++) {
+		u32 sid = fwspec->ids[i];
+
+		new_stream = &master->streams[i];
+		new_stream->id = sid;
+		new_stream->master = master;
+
+		/*
+		 * Check the SIDs are in range of the SMMU and our stream table
+		 */
+		if (!arm_smmu_sid_in_range(smmu, sid)) {
+			ret = -ERANGE;
+			break;
+		}
+
+		/* Ensure l2 strtab is initialised */
+		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
+			ret = arm_smmu_init_l2_strtab(smmu, sid);
+			if (ret)
+				break;
+		}
+
+		/* Insert into SID tree */
+		new_node = &(smmu->streams.rb_node);
+		while (*new_node) {
+			cur_stream = rb_entry(*new_node, struct arm_smmu_stream,
+					      node);
+			parent_node = *new_node;
+			if (cur_stream->id > new_stream->id) {
+				new_node = &((*new_node)->rb_left);
+			} else if (cur_stream->id < new_stream->id) {
+				new_node = &((*new_node)->rb_right);
+			} else {
+				dev_warn(master->dev,
+					 "stream %u already in tree\n",
+					 cur_stream->id);
+				ret = -EINVAL;
+				break;
+			}
+		}
+		if (ret)
+			break;
+
+		rb_link_node(&new_stream->node, parent_node, new_node);
+		rb_insert_color(&new_stream->node, &smmu->streams);
+	}
+
+	if (ret) {
+		for (i--; i >= 0; i--)
+			rb_erase(&master->streams[i].node, &smmu->streams);
+		xfree(master->streams);
+	}
+	mutex_unlock(&smmu->streams_mutex);
+
+	return ret;
+}
+
 /* Forward declaration */
 static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev);
 static int arm_smmu_assign_dev(struct domain *d, u8 devfn, struct device *dev,
@@ -1495,7 +1590,7 @@ static int arm_smmu_deassign_dev(struct domain *d, uint8_t devfn,
 
 static int arm_smmu_add_device(u8 devfn, struct device *dev)
 {
-	int i, ret;
+	int ret;
 	struct arm_smmu_device *smmu;
 	struct arm_smmu_master *master;
 	struct iommu_fwspec *fwspec;
@@ -1532,26 +1627,11 @@ static int arm_smmu_add_device(u8 devfn, struct device *dev)
 
 	master->dev = dev;
 	master->smmu = smmu;
-	master->sids = fwspec->ids;
-	master->num_sids = fwspec->num_ids;
 	dev_iommu_priv_set(dev, master);
 
-	/* Check the SIDs are in range of the SMMU and our stream table */
-	for (i = 0; i < master->num_sids; i++) {
-		u32 sid = master->sids[i];
-
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
-			goto err_free_master;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				goto err_free_master;
-		}
-	}
+	ret = arm_smmu_insert_master(smmu, master);
+	if (ret)
+		goto err_free_master;
 
 	/*
 	 * Note that PASID must be enabled before, and disabled after ATS:
@@ -1796,6 +1876,9 @@ static int __init arm_smmu_init_structures(struct arm_smmu_device *smmu)
 {
 	int ret;
 
+	mutex_init(&smmu->streams_mutex);
+	smmu->streams = RB_ROOT;
+
 	ret = arm_smmu_init_queues(smmu);
 	if (ret)
 		return ret;
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index ab07366294..ab1f29f6c7 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -639,6 +639,15 @@ struct arm_smmu_device {
 	struct tasklet		evtq_irq_tasklet;
 	struct tasklet		priq_irq_tasklet;
 	struct tasklet		combined_irq_tasklet;
+
+	struct rb_root		streams;
+	struct mutex		streams_mutex;
+};
+
+struct arm_smmu_stream {
+	u32							id;
+	struct arm_smmu_master		*master;
+	struct rb_node				node;
 };
 
 /* SMMU private data for each master */
@@ -647,8 +656,8 @@ struct arm_smmu_master {
 	struct device			*dev;
 	struct arm_smmu_domain		*domain;
 	struct list_head		domain_head;
-	u32				*sids;
-	unsigned int			num_sids;
+	struct arm_smmu_stream		*streams;
+	unsigned int				num_streams;
 	bool				ats_enabled;
 };
 
-- 
2.43.0

Re: [PATCH v3 01/23] xen/arm: smmuv3: Maintain a SID->device structure

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:51, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
> 
> Backport Linux commit cdf315f907d4. This is the clean backport without
> any changes.
> 
> When handling faults from the event or PRI queue, we need to find the
> struct device associated with a SID. Add a rb_tree to keep track of
> SIDs.
> 
> Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Acked-by: Will Deacon <will@kernel.org>
> Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.org
> Signed-off-by: Joerg Roedel <jroedel@suse.de>
> Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdf315f907d4
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---

I think that by only backporting this one we are introducing a regression for which
we don’t allow anymore duplicated stream id, have a look on this linux commit
that is fixing it: b00d24997a11c10d3e420614f0873b83ce358a34.

P.s. I was able to apply only until patch 3, I think this serie needs a rebase, do you have
an updated branch or can you point me to the staging SHA to build it? (hope it’s not too far
from now)

Cheers,
Luca

Re: [PATCH v3 01/23] xen/arm: smmuv3: Maintain a SID->device structure

[Milan Djokic]

Hi Luca,

On 4/9/26 16:59, Luca Fancellu wrote:
> Hi Milan,
> 
>> On 31 Mar 2026, at 02:51, Milan Djokic <milan_djokic@epam.com> wrote:
>>
>> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
>>
>> Backport Linux commit cdf315f907d4. This is the clean backport without
>> any changes.
>>
>> When handling faults from the event or PRI queue, we need to find the
>> struct device associated with a SID. Add a rb_tree to keep track of
>> SIDs.
>>
>> Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>> Reviewed-by: Eric Auger <eric.auger@redhat.com>
>> Reviewed-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
>> Acked-by: Will Deacon <will@kernel.org>
>> Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.org
>> Signed-off-by: Joerg Roedel <jroedel@suse.de>
>> Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdf315f907d4
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
> 
> I think that by only backporting this one we are introducing a regression for which
> we don’t allow anymore duplicated stream id, have a look on this linux commit
> that is fixing it: b00d24997a11c10d3e420614f0873b83ce358a34.
> 

I will backport the logic from this commit also

> P.s. I was able to apply only until patch 3, I think this serie needs a rebase, do you have
> an updated branch or can you point me to the staging SHA to build it? (hope it’s not too far
> from now)
> 

Yes, sorry for that, it's conflicting with the couple of recently merged 
commits, rebased for the new version.

> Cheers,
> Luca
> 
> 

Best regards,
Milan

[PATCH v3 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Xen SMMUv3 driver only supports stage-2 translation. Add support for
Stage-1 translation that is required to support nested stage
translation.

In true nested mode, both s1_cfg and s2_cfg will coexist.
Let's remove the union. When nested stage translation is setup, both
s1_cfg and s2_cfg are valid.

We introduce a new smmu_domain abort field that will be set
upon guest stage-1 configuration passing. If no guest stage-1
config has been attached, it is ignored when writing the STE.

arm_smmu_write_strtab_ent() is modified to write both stage
fields in the STE and deal with the abort field.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 93 +++++++++++++++++++++++----
 xen/drivers/passthrough/arm/smmu-v3.h |  9 +++
 2 files changed, 91 insertions(+), 11 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 73cc4ef08f..f9c6837919 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -683,8 +683,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 	 * 3. Update Config, sync
 	 */
 	u64 val = le64_to_cpu(dst[0]);
-	bool ste_live = false;
+	bool s1_live = false, s2_live = false, ste_live = false;
+	bool abort, translate = false;
 	struct arm_smmu_device *smmu = NULL;
+	struct arm_smmu_s1_cfg *s1_cfg = NULL;
 	struct arm_smmu_s2_cfg *s2_cfg = NULL;
 	struct arm_smmu_domain *smmu_domain = NULL;
 	struct arm_smmu_cmdq_ent prefetch_cmd = {
@@ -699,30 +701,54 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		smmu = master->smmu;
 	}
 
-	if (smmu_domain)
-		s2_cfg = &smmu_domain->s2_cfg;
+	if (smmu_domain) {
+		switch (smmu_domain->stage) {
+		case ARM_SMMU_DOMAIN_NESTED:
+			s1_cfg = &smmu_domain->s1_cfg;
+			fallthrough;
+		case ARM_SMMU_DOMAIN_S2:
+			s2_cfg = &smmu_domain->s2_cfg;
+			break;
+		default:
+			break;
+		}
+		translate = !!s1_cfg || !!s2_cfg;
+	}
 
 	if (val & STRTAB_STE_0_V) {
 		switch (FIELD_GET(STRTAB_STE_0_CFG, val)) {
 		case STRTAB_STE_0_CFG_BYPASS:
 			break;
+		case STRTAB_STE_0_CFG_S1_TRANS:
+			s1_live = true;
+			break;
 		case STRTAB_STE_0_CFG_S2_TRANS:
-			ste_live = true;
+			s2_live = true;
+			break;
+		case STRTAB_STE_0_CFG_NESTED:
+			s1_live = true;
+			s2_live = true;
 			break;
 		case STRTAB_STE_0_CFG_ABORT:
-			BUG_ON(!disable_bypass);
 			break;
 		default:
 			BUG(); /* STE corruption */
 		}
 	}
 
+	ste_live = s1_live || s2_live;
+
 	/* Nuke the existing STE_0 value, as we're going to rewrite it */
 	val = STRTAB_STE_0_V;
 
 	/* Bypass/fault */
-	if (!smmu_domain || !(s2_cfg)) {
-		if (!smmu_domain && disable_bypass)
+	if (!smmu_domain)
+		abort = disable_bypass;
+	else
+		abort = smmu_domain->abort;
+
+	if (abort || !translate) {
+		if (abort)
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
 		else
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
@@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		return;
 	}
 
+	if (ste_live) {
+		/* First invalidate the live STE */
+		dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
+		arm_smmu_sync_ste_for_sid(smmu, sid);
+	}
+
+	if (s1_cfg) {
+		BUG_ON(s1_live);
+		dst[1] = cpu_to_le64(
+			 FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
+			 FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
+			 FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
+			 FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
+			 FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
+
+		if (smmu->features & ARM_SMMU_FEAT_STALLS &&
+		   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
+			dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
+
+		val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
+			FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
+			FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
+			FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
+	}
+
 	if (s2_cfg) {
+		u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
 		u64 strtab =
 			 FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
 			 FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
@@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 			 STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
 			 STRTAB_STE_2_S2R;
 
-		BUG_ON(ste_live);
+		if (s2_live) {
+			u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
+			BUG_ON(s2ttb != vttbr);
+		}
+
 		dst[2] = cpu_to_le64(strtab);
 
-		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
+		dst[3] = cpu_to_le64(vttbr);
 
 		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
+	} else {
+		dst[2] = 0;
+		dst[3] = 0;
 	}
 
 	if (master->ats_enabled)
@@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
 {
 	int ret;
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+	struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+	if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
+		(!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
+		 !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
+			dev_info(smmu_domain->smmu->dev,
+					"does not implement two stages\n");
+			return -EINVAL;
+	}
 
 	/* Restrict the stage to what we can actually support */
 	smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
@@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
 		break;
 	}
 
+	if (reg & IDR0_S1P)
+		smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
+
 	if (reg & IDR0_S2P)
 		smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
 
-	if (!(reg & IDR0_S2P)) {
-		dev_err(smmu->dev, "no stage-2 translation support!\n");
+	if (!(reg & (IDR0_S1P | IDR0_S2P))) {
+		dev_err(smmu->dev, "no translation support!\n");
 		return -ENXIO;
 	}
 
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index ab1f29f6c7..3fb13b7e21 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -197,6 +197,7 @@
 #define STRTAB_STE_0_CFG_BYPASS		4
 #define STRTAB_STE_0_CFG_S1_TRANS	5
 #define STRTAB_STE_0_CFG_S2_TRANS	6
+#define STRTAB_STE_0_CFG_NESTED		7
 
 #define STRTAB_STE_0_S1FMT		GENMASK_ULL(5, 4)
 #define STRTAB_STE_0_S1FMT_LINEAR	0
@@ -549,6 +550,12 @@ struct arm_smmu_strtab_l1_desc {
 	dma_addr_t			l2ptr_dma;
 };
 
+struct arm_smmu_s1_cfg {
+	u64				s1ctxptr;
+	u8				s1fmt;
+	u8				s1cdmax;
+};
+
 struct arm_smmu_s2_cfg {
 	u16				vmid;
 	u64				vttbr;
@@ -669,7 +676,9 @@ struct arm_smmu_domain {
 	atomic_t			nr_ats_masters;
 
 	enum arm_smmu_domain_stage	stage;
+	struct arm_smmu_s1_cfg	s1_cfg;
 	struct arm_smmu_s2_cfg	s2_cfg;
+	bool			abort;
 
 	/* Xen domain associated with this SMMU domain */
 	struct domain		*d;
-- 
2.43.0

Re: [PATCH v3 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Luca Fancellu]

Hi Milan,

> @@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
> return;
> }
> 
> + if (ste_live) {
> + /* First invalidate the live STE */
> + dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
> + arm_smmu_sync_ste_for_sid(smmu, sid);
> + }
> +
> + if (s1_cfg) {
> + BUG_ON(s1_live);

I think this is wrong, if a guest issues another s1 update on the same sid, this will crash Xen, I think
that we’ve already invalidated the live status so this one should be removed

> + dst[1] = cpu_to_le64(
> + FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
> + FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
> + FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
> + FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
> + FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
> +
> + if (smmu->features & ARM_SMMU_FEAT_STALLS &&
> +   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
> + dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
> +
> + val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
> + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
> + FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
> + FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
> + }
> +
> if (s2_cfg) {
> + u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
> u64 strtab =
> FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
> FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
> @@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
> STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
> STRTAB_STE_2_S2R;
> 
> - BUG_ON(ste_live);
> + if (s2_live) {
> + u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
> + BUG_ON(s2ttb != vttbr);
> + }
> +
> dst[2] = cpu_to_le64(strtab);
> 
> - dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
> + dst[3] = cpu_to_le64(vttbr);
> 
> val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
> + } else {
> + dst[2] = 0;
> + dst[3] = 0;
> }
> 
> if (master->ats_enabled)
> @@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
> {
> int ret;
> struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> + struct arm_smmu_device *smmu = smmu_domain->smmu;
> +
> + if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
> + (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
> + !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
> + dev_info(smmu_domain->smmu->dev,
> + "does not implement two stages\n");
> + return -EINVAL;
> + }
> 
> /* Restrict the stage to what we can actually support */
> smmu_domain->stage = ARM_SMMU_DOMAIN_S2;

Here we set stage 2 as default, but in arm_smmu_device_hw_probe() we’ve
deleted the check for S2 required, so if we have an HW with only S1 the probe will
succeed but we will wrongly set here S2, so I would keep ...

> @@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
> break;
> }
> 
> + if (reg & IDR0_S1P)
> + smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
> +
> if (reg & IDR0_S2P)
> smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
> 
> - if (!(reg & IDR0_S2P)) {
> - dev_err(smmu->dev, "no stage-2 translation support!\n");

this change, rearranged in the way that is sensible with the new logic.

> + if (!(reg & (IDR0_S1P | IDR0_S2P))) {
> + dev_err(smmu->dev, "no translation support!\n");
> return -ENXIO;
> }
> 

Re: [PATCH v3 02/23] xen/arm: smmuv3: Add support for stage-1 and nested stage translation

[Milan Djokic]

Hi Luca,

On 4/10/26 11:49, Luca Fancellu wrote:
> Hi Milan,
> 
>> @@ -740,7 +766,33 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>> return;
>> }
>>
>> + if (ste_live) {
>> + /* First invalidate the live STE */
>> + dst[0] = cpu_to_le64(STRTAB_STE_0_CFG_ABORT);
>> + arm_smmu_sync_ste_for_sid(smmu, sid);
>> + }
>> +
>> + if (s1_cfg) {
>> + BUG_ON(s1_live);
> 
> I think this is wrong, if a guest issues another s1 update on the same sid, this will crash Xen, I think
> that we’ve already invalidated the live status so this one should be removed
> 

Yes, this is a leftover. I will remove this assertion.

>> + dst[1] = cpu_to_le64(
>> + FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) |
>> + FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
>> + FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
>> + FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
>> + FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
>> +
>> + if (smmu->features & ARM_SMMU_FEAT_STALLS &&
>> +   !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
>> + dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
>> +
>> + val |= (s1_cfg->s1ctxptr & STRTAB_STE_0_S1CTXPTR_MASK) |
>> + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
>> + FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
>> + FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
>> + }
>> +
>> if (s2_cfg) {
>> + u64 vttbr = s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK;
>> u64 strtab =
>> FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) |
>> FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) |
>> @@ -750,12 +802,19 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>> STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 |
>> STRTAB_STE_2_S2R;
>>
>> - BUG_ON(ste_live);
>> + if (s2_live) {
>> + u64 s2ttb = le64_to_cpu(dst[3]) & STRTAB_STE_3_S2TTB_MASK;
>> + BUG_ON(s2ttb != vttbr);
>> + }
>> +
>> dst[2] = cpu_to_le64(strtab);
>>
>> - dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
>> + dst[3] = cpu_to_le64(vttbr);
>>
>> val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
>> + } else {
>> + dst[2] = 0;
>> + dst[3] = 0;
>> }
>>
>> if (master->ats_enabled)
>> @@ -1254,6 +1313,15 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
>> {
>> int ret;
>> struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
>> + struct arm_smmu_device *smmu = smmu_domain->smmu;
>> +
>> + if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED &&
>> + (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1) ||
>> + !(smmu->features & ARM_SMMU_FEAT_TRANS_S2))) {
>> + dev_info(smmu_domain->smmu->dev,
>> + "does not implement two stages\n");
>> + return -EINVAL;
>> + }
>>
>> /* Restrict the stage to what we can actually support */
>> smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
> 
> Here we set stage 2 as default, but in arm_smmu_device_hw_probe() we’ve
> deleted the check for S2 required, so if we have an HW with only S1 the probe will
> succeed but we will wrongly set here S2, so I would keep ...
> 

Yes, handling of stage-1-only scenario is not correct, missing in ste 
and guest config handling also. I will update this in the new version.
Following comments from Julien on the same topic, I'm wondering if it's 
valid to provide stage-1-only support in Xen?

>> @@ -2353,11 +2421,14 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
>> break;
>> }
>>
>> + if (reg & IDR0_S1P)
>> + smmu->features |= ARM_SMMU_FEAT_TRANS_S1;
>> +
>> if (reg & IDR0_S2P)
>> smmu->features |= ARM_SMMU_FEAT_TRANS_S2;
>>
>> - if (!(reg & IDR0_S2P)) {
>> - dev_err(smmu->dev, "no stage-2 translation support!\n");
> 
> this change, rearranged in the way that is sensible with the new logic.
> 
>> + if (!(reg & (IDR0_S1P | IDR0_S2P))) {
>> + dev_err(smmu->dev, "no translation support!\n");
>> return -ENXIO;
>> }
>>
> 

Best regards,
Milan

[PATCH v3 03/23] xen/arm: smmuv3: Alloc io_domain for each device

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

In current implementation io_domain is allocated once for each xen
domain as Stage2 translation is common for all devices in same xen
domain.

Nested stage supports S1 and S2 configuration at the same time. Stage1
translation will be different for each device as linux kernel will
allocate page-table for each device.

Alloc io_domain for each device so that each device can have different
Stage-1 and Stage-2 configuration structure.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index f9c6837919..19e55b6c9b 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -2809,11 +2809,13 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
 static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 				struct device *dev)
 {
+	unsigned long flags;
 	struct iommu_domain *io_domain;
 	struct arm_smmu_domain *smmu_domain;
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
 	struct arm_smmu_device *smmu = arm_smmu_get_by_dev(fwspec->iommu_dev);
+	struct arm_smmu_master *master;
 
 	if (!smmu)
 		return NULL;
@@ -2824,8 +2826,15 @@ static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 	 */
 	list_for_each_entry(io_domain, &xen_domain->contexts, list) {
 		smmu_domain = to_smmu_domain(io_domain);
-		if (smmu_domain->smmu == smmu)
-			return io_domain;
+
+		spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+		list_for_each_entry(master, &smmu_domain->devices, domain_head) {
+			if (master->dev == dev) {
+				spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+				return io_domain;
+			}
+		}
+		spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
 	}
 	return NULL;
 }
-- 
2.43.0

Re: [PATCH v3 03/23] xen/arm: smmuv3: Alloc io_domain for each device

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> In current implementation io_domain is allocated once for each xen
> domain as Stage2 translation is common for all devices in same xen
> domain.
> 
> Nested stage supports S1 and S2 configuration at the same time. Stage1
> translation will be different for each device as linux kernel will
> allocate page-table for each device.
> 
> Alloc io_domain for each device so that each device can have different
> Stage-1 and Stage-2 configuration structure.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---

Looks ok to me

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

Cheers,
Luca

Re: [PATCH v3 03/23] xen/arm: smmuv3: Alloc io_domain for each device

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> In current implementation io_domain is allocated once for each xen
> domain as Stage2 translation is common for all devices in same xen
> domain.
> 
> Nested stage supports S1 and S2 configuration at the same time. Stage1
> translation will be different for each device as linux kernel will
> allocate page-table for each device.
> 
> Alloc io_domain for each device so that each device can have different
> Stage-1 and Stage-2 configuration structure.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/drivers/passthrough/arm/smmu-v3.c | 13 +++++++++++--
>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index f9c6837919..19e55b6c9b 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -2809,11 +2809,13 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
>   static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
>   				struct device *dev)
>   {
> +	unsigned long flags;

In Xen, we are trying to limit the scope of local variables. AFAICT, 
this is only used within the loop below. So this can be defined later. 
Same ...

>   	struct iommu_domain *io_domain;
>   	struct arm_smmu_domain *smmu_domain;
>   	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
>   	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
>   	struct arm_smmu_device *smmu = arm_smmu_get_by_dev(fwspec->iommu_dev);
> +	struct arm_smmu_master *master;

... here. Also, AFAICT, ``master`` is not meant ot be modified. So 
shouldn't this be 'const'?

Cheers,

-- 
Julien Grall

[PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

This patch adds basic framework for vIOMMU.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/dom0less-build.c        |  2 +
 xen/arch/arm/domain.c                | 33 +++++++++++++
 xen/arch/arm/domain_build.c          |  2 +
 xen/arch/arm/include/asm/viommu.h    | 70 ++++++++++++++++++++++++++++
 xen/drivers/passthrough/Kconfig      |  5 ++
 xen/drivers/passthrough/arm/Makefile |  1 +
 xen/drivers/passthrough/arm/viommu.c | 48 +++++++++++++++++++
 xen/include/public/arch-arm.h        |  5 ++
 xen/include/public/domctl.h          |  4 +-
 9 files changed, 168 insertions(+), 2 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/viommu.h
 create mode 100644 xen/drivers/passthrough/arm/viommu.c

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 4181c10538..067835e5d0 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -23,6 +23,7 @@
 #include <asm/arm64/sve.h>
 #include <asm/domain_build.h>
 #include <asm/firmware/sci.h>
+#include <asm/viommu.h>
 #include <asm/grant_table.h>
 #include <asm/setup.h>
 
@@ -317,6 +318,7 @@ int __init arch_parse_dom0less_node(struct dt_device_node *node,
     uint32_t val;
 
     d_cfg->arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
+    d_cfg->arch.viommu_type = viommu_get_type();
     d_cfg->flags |= XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap;
 
     if ( domu_dt_sci_parse(node, d_cfg) )
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 94b9858ad2..241f87386b 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -28,6 +28,7 @@
 #include <asm/tee/tee.h>
 #include <asm/vfp.h>
 #include <asm/vgic.h>
+#include <asm/viommu.h>
 #include <asm/vtimer.h>
 
 #include "vpci.h"
@@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    /* Check config structure padding */
+    if ( config->arch.pad )
+    {
+        dprintk(XENLOG_INFO,
+            "Invalid input config, padding must be zero\n");
+        return -EINVAL;
+    }
+
     /* Check feature flags */
     if ( sve_vl_bits > 0 )
     {
@@ -626,6 +635,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
+    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
+         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    {
+        dprintk(XENLOG_INFO,
+                "vIOMMU requested while iommu not enabled for domain\n");
+        return -EINVAL;
+    }
+
+    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    {
+        dprintk(XENLOG_INFO,
+                "vIOMMU type requested not supported by the platform or Xen\n");
+        return -EINVAL;
+    }
+
     return sci_domain_sanitise_config(config);
 }
 
@@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
     if ( (rc = sci_domain_init(d, config)) != 0 )
         goto fail;
 
+    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
+        goto fail;
+
     return 0;
 
 fail:
@@ -965,6 +992,7 @@ enum {
     PROG_pci = 1,
     PROG_sci,
     PROG_tee,
+    PROG_viommu,
     PROG_xen,
     PROG_page,
     PROG_mapping,
@@ -1021,6 +1049,11 @@ int domain_relinquish_resources(struct domain *d)
         if (ret )
             return ret;
 
+    PROGRESS(viommu):
+        ret = viommu_relinquish_resources(d);
+        if (ret )
+            return ret;
+
     PROGRESS(xen):
         ret = relinquish_memory(d, &d->xenpage_list);
         if ( ret )
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index e8795745dd..a51563ee3d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -35,6 +35,7 @@
 #include <asm/arm64/sve.h>
 #include <asm/cpufeature.h>
 #include <asm/domain_build.h>
+#include <asm/viommu.h>
 #include <xen/event.h>
 
 #include <xen/irq.h>
@@ -1946,6 +1947,7 @@ void __init create_dom0(void)
     dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
     dom0_cfg.arch.tee_type = tee_get_type();
     dom0_cfg.max_vcpus = dom0_max_vcpus();
+    dom0_cfg.arch.viommu_type = viommu_get_type();
 
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
new file mode 100644
index 0000000000..4598f543b8
--- /dev/null
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+#ifndef __ARCH_ARM_VIOMMU_H__
+#define __ARCH_ARM_VIOMMU_H__
+
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+
+#include <xen/lib.h>
+#include <xen/types.h>
+#include <public/xen.h>
+
+struct viommu_ops {
+    /*
+     * Called during domain construction if toolstack requests to enable
+     * vIOMMU support.
+     */
+    int (*domain_init)(struct domain *d);
+
+    /*
+     * Called during domain destruction to free resources used by vIOMMU.
+     */
+    int (*relinquish_resources)(struct domain *d);
+};
+
+struct viommu_desc {
+    /* vIOMMU domains init/free operations described above. */
+    const struct viommu_ops *ops;
+
+    /*
+     * ID of vIOMMU. Corresponds to xen_arch_domainconfig.viommu_type.
+     * Should be one of XEN_DOMCTL_CONFIG_VIOMMU_xxx
+     */
+    uint16_t viommu_type;
+};
+
+int domain_viommu_init(struct domain *d, uint16_t viommu_type);
+int viommu_relinquish_resources(struct domain *d);
+uint16_t viommu_get_type(void);
+
+#else
+
+static inline uint8_t viommu_get_type(void)
+{
+    return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+}
+
+static inline int domain_viommu_init(struct domain *d, uint16_t viommu_type)
+{
+    if ( likely(viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE) )
+        return 0;
+
+    return -ENODEV;
+}
+
+static inline int viommu_relinquish_resources(struct domain *d)
+{
+    return 0;
+}
+
+#endif /* CONFIG_ARM_VIRTUAL_IOMMU */
+
+#endif /* __ARCH_ARM_VIOMMU_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index b413c33a4c..3c174bc87b 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -35,6 +35,11 @@ config IPMMU_VMSA
 	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
 	  translation table format and is able to use CPU's P2M table as is.
 
+config ARM_VIRTUAL_IOMMU
+	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
+	help
+	 Support virtual IOMMU infrastructure to implement vIOMMU.
+
 endif
 
 config AMD_IOMMU
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index c5fb3b58a5..c3783188e3 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -2,3 +2,4 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
 obj-$(CONFIG_ARM_SMMU) += smmu.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
+obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
new file mode 100644
index 0000000000..7ab6061e34
--- /dev/null
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/types.h>
+
+#include <asm/viommu.h>
+
+const struct viommu_desc __read_mostly *cur_viommu;
+
+int domain_viommu_init(struct domain *d, uint16_t viommu_type)
+{
+    if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+        return 0;
+
+    if ( !cur_viommu )
+        return -ENODEV;
+
+    if ( cur_viommu->viommu_type != viommu_type )
+        return -EINVAL;
+
+    return cur_viommu->ops->domain_init(d);
+}
+
+int viommu_relinquish_resources(struct domain *d)
+{
+    if ( !cur_viommu )
+        return 0;
+
+    return cur_viommu->ops->relinquish_resources(d);
+}
+
+uint16_t viommu_get_type(void)
+{
+    if ( !cur_viommu )
+        return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+
+    return cur_viommu->viommu_type;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index cd563cf706..d4953d40fd 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -330,6 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
 #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
 #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
 
+#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
+
 struct xen_arch_domainconfig {
     /* IN/OUT */
     uint8_t gic_version;
@@ -355,6 +357,9 @@ struct xen_arch_domainconfig {
     uint32_t clock_frequency;
     /* IN */
     uint8_t arm_sci_type;
+    /* IN */
+    uint8_t viommu_type;
+    uint16_t pad;
 };
 #endif /* __XEN__ || __XEN_TOOLS__ */
 
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 8f6708c0a7..23124547f3 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -30,9 +30,9 @@
  * fields) don't require a change of the version.
  * Stable ops are NOT covered by XEN_DOMCTL_INTERFACE_VERSION!
  *
- * Last version bump: Xen 4.19
+ * Last version bump: Xen 4.22
  */
-#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
+#define XEN_DOMCTL_INTERFACE_VERSION 0x00000018
 
 /*
  * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
-- 
2.43.0

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Jan Beulich]

On 31.03.2026 03:52, Milan Djokic wrote:
> @@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>          return -EINVAL;
>      }
>  
> +    /* Check config structure padding */
> +    if ( config->arch.pad )
> +    {
> +        dprintk(XENLOG_INFO,
> +            "Invalid input config, padding must be zero\n");

Nit (since I had to look here): Bad indentation. In fact there's no reason
to wrap the statement in the first place.

> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -35,6 +35,11 @@ config IPMMU_VMSA
>  	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
>  	  translation table format and is able to use CPU's P2M table as is.
>  
> +config ARM_VIRTUAL_IOMMU
> +	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
> +	help
> +	 Support virtual IOMMU infrastructure to implement vIOMMU.

Nit: Bad indentation. Kconfig help text is to be indented by a hard tab and
two blanks. (Sorry, should have noticed already on the earlier version.)

Jan

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Luca Fancellu]

Hi Milan,

> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 94b9858ad2..241f87386b 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -28,6 +28,7 @@
> #include <asm/tee/tee.h>
> #include <asm/vfp.h>
> #include <asm/vgic.h>
> +#include <asm/viommu.h>
> #include <asm/vtimer.h>
> 
> #include "vpci.h"
> @@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>         return -EINVAL;
>     }
> 
> +    /* Check config structure padding */
> +    if ( config->arch.pad )
> +    {
> +        dprintk(XENLOG_INFO,
> +            "Invalid input config, padding must be zero\n");
> +        return -EINVAL;
> +    }
> +

This feels ok but unrelated to the patch, but also the text maybe should be something like “Invalid domain configuration during domain creation\n”.

>     /* Check feature flags */
>     if ( sve_vl_bits > 0 )
>     {
> @@ -626,6 +635,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>         return -EINVAL;
>     }
> 
> +    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
> +         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "vIOMMU requested while iommu not enabled for domain\n");
> +        return -EINVAL;
> +    }
> +
> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "vIOMMU type requested not supported by the platform or Xen\n");
> +        return -EINVAL;
> +    }
> +
>     return sci_domain_sanitise_config(config);
> }
> 
> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>     if ( (rc = sci_domain_init(d, config)) != 0 )
>         goto fail;
> 
> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
> +        goto fail;
> +
>     return 0;
> 
> fail:
> @@ -965,6 +992,7 @@ enum {
>     PROG_pci = 1,
>     PROG_sci,
>     PROG_tee,
> +    PROG_viommu,
>     PROG_xen,
>     PROG_page,
>     PROG_mapping,
> @@ -1021,6 +1049,11 @@ int domain_relinquish_resources(struct domain *d)
>         if (ret )
>             return ret;
> 
> +    PROGRESS(viommu):
> +        ret = viommu_relinquish_resources(d);
> +        if (ret )
> +            return ret;
> +
>     PROGRESS(xen):
>         ret = relinquish_memory(d, &d->xenpage_list);
>         if ( ret )
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index e8795745dd..a51563ee3d 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -35,6 +35,7 @@
> #include <asm/arm64/sve.h>
> #include <asm/cpufeature.h>
> #include <asm/domain_build.h>
> +#include <asm/viommu.h>

NIT: In my local branch I’ve rebased this on top of new staging

> 
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index cd563cf706..d4953d40fd 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -330,6 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
> 
> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
> +
> struct xen_arch_domainconfig {
>     /* IN/OUT */
>     uint8_t gic_version;
> @@ -355,6 +357,9 @@ struct xen_arch_domainconfig {
>     uint32_t clock_frequency;
>     /* IN */
>     uint8_t arm_sci_type;
> +    /* IN */
> +    uint8_t viommu_type;
> +    uint16_t pad;

Having the padding explicit feels ok to me, but I would rely on maintainer
choice.

> };
> #endif /* __XEN__ || __XEN_TOOLS__ */
> 
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 8f6708c0a7..23124547f3 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -30,9 +30,9 @@
>  * fields) don't require a change of the version.
>  * Stable ops are NOT covered by XEN_DOMCTL_INTERFACE_VERSION!
>  *
> - * Last version bump: Xen 4.19
> + * Last version bump: Xen 4.22
>  */
> -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
> +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000018

I don’t think the changes in this patch are breaking the ABI, so this should not be bumped;
said so, I would rely on @Andrew or another maintainer for this

Cheers,
Luca

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Julien Grall]

On 10/04/2026 19:41, Luca Fancellu wrote:
> Hi Milan,
> 
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 94b9858ad2..241f87386b 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -28,6 +28,7 @@
>> #include <asm/tee/tee.h>
>> #include <asm/vfp.h>
>> #include <asm/vgic.h>
>> +#include <asm/viommu.h>
>> #include <asm/vtimer.h>
>>
>> #include "vpci.h"
>> @@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>          return -EINVAL;
>>      }
>>
>> +    /* Check config structure padding */
>> +    if ( config->arch.pad )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +            "Invalid input config, padding must be zero\n");
>> +        return -EINVAL;
>> +    }
>> +
> 
> This feels ok but unrelated to the patch

In general, we want to have padding explictly define and zeroed. I 
wouldn't mind if this is kept in this patch but this would need to be 
mentioned in the commit message.

> , but also the text maybe should be something like “Invalid domain configuration during domain creation\n”.
> 
>>      /* Check feature flags */
>>      if ( sve_vl_bits > 0 )
>>      {
>> @@ -626,6 +635,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>          return -EINVAL;
>>      }
>>
>> +    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
>> +         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +                "vIOMMU requested while iommu not enabled for domain\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +                "vIOMMU type requested not supported by the platform or Xen\n");
>> +        return -EINVAL;
>> +    }
>> +
>>      return sci_domain_sanitise_config(config);
>> }
>>
>> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>>      if ( (rc = sci_domain_init(d, config)) != 0 )
>>          goto fail;
>>
>> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
>> +        goto fail;
>> +
>>      return 0;
>>
>> fail:
>> @@ -965,6 +992,7 @@ enum {
>>      PROG_pci = 1,
>>      PROG_sci,
>>      PROG_tee,
>> +    PROG_viommu,
>>      PROG_xen,
>>      PROG_page,
>>      PROG_mapping,
>> @@ -1021,6 +1049,11 @@ int domain_relinquish_resources(struct domain *d)
>>          if (ret )
>>              return ret;
>>
>> +    PROGRESS(viommu):
>> +        ret = viommu_relinquish_resources(d);
>> +        if (ret )
>> +            return ret;
>> +
>>      PROGRESS(xen):
>>          ret = relinquish_memory(d, &d->xenpage_list);
>>          if ( ret )
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index e8795745dd..a51563ee3d 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -35,6 +35,7 @@
>> #include <asm/arm64/sve.h>
>> #include <asm/cpufeature.h>
>> #include <asm/domain_build.h>
>> +#include <asm/viommu.h>
> 
> NIT: In my local branch I’ve rebased this on top of new staging
> 
>>
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>> index cd563cf706..d4953d40fd 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -330,6 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>> #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>>
>> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
>> +
>> struct xen_arch_domainconfig {
>>      /* IN/OUT */
>>      uint8_t gic_version;
>> @@ -355,6 +357,9 @@ struct xen_arch_domainconfig {
>>      uint32_t clock_frequency;
>>      /* IN */
>>      uint8_t arm_sci_type;
>> +    /* IN */
>> +    uint8_t viommu_type;
>> +    uint16_t pad;
> 
> Having the padding explicit feels ok to me, but I would rely on maintainer
> choice.

See above. To expand what I wrote, if we don't define the padding 
explicitly and then check against zero, then we can't re-use them 
without breaking the ABI. So...

> 
>> };
>> #endif /* __XEN__ || __XEN_TOOLS__ */
>>
>> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
>> index 8f6708c0a7..23124547f3 100644
>> --- a/xen/include/public/domctl.h
>> +++ b/xen/include/public/domctl.h
>> @@ -30,9 +30,9 @@
>>   * fields) don't require a change of the version.
>>   * Stable ops are NOT covered by XEN_DOMCTL_INTERFACE_VERSION!
>>   *
>> - * Last version bump: Xen 4.19
>> + * Last version bump: Xen 4.22
>>   */
>> -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
>> +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000018
> 
> I don’t think the changes in this patch are breaking the ABI, so this should not be bumped;
> said so, I would rely on @Andrew or another maintainer for this

... this is needed because older toolstack may not zero the field.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Luca Fancellu]

HI Milan,

apologies I missed one bit in this patch

> 
> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>     if ( (rc = sci_domain_init(d, config)) != 0 )
>         goto fail;
> 
> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
> +        goto fail;
> +

Here if this fails, we call arch_domain_destroy(), however in that function we don’t call
viommu_relinquish_resources()

Cheers,
Luca

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Milan Djokic]

Hi Luca,

On 4/10/26 13:39, Luca Fancellu wrote:
> HI Milan,
> 
> apologies I missed one bit in this patch
> 
>>
>> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>>      if ( (rc = sci_domain_init(d, config)) != 0 )
>>          goto fail;
>>
>> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
>> +        goto fail;
>> +
> 
> Here if this fails, we call arch_domain_destroy(), however in that function we don’t call
> viommu_relinquish_resources()
> 

Yes, this is missing. I will add it in the updated version.

> Cheers,
> Luca
> 

BR,
Milan

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> This patch adds basic framework for vIOMMU.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/arch/arm/dom0less-build.c        |  2 +
>   xen/arch/arm/domain.c                | 33 +++++++++++++
>   xen/arch/arm/domain_build.c          |  2 +
>   xen/arch/arm/include/asm/viommu.h    | 70 ++++++++++++++++++++++++++++
>   xen/drivers/passthrough/Kconfig      |  5 ++
>   xen/drivers/passthrough/arm/Makefile |  1 +
>   xen/drivers/passthrough/arm/viommu.c | 48 +++++++++++++++++++
>   xen/include/public/arch-arm.h        |  5 ++
>   xen/include/public/domctl.h          |  4 +-
>   9 files changed, 168 insertions(+), 2 deletions(-)
>   create mode 100644 xen/arch/arm/include/asm/viommu.h
>   create mode 100644 xen/drivers/passthrough/arm/viommu.c
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 4181c10538..067835e5d0 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -23,6 +23,7 @@
>   #include <asm/arm64/sve.h>
>   #include <asm/domain_build.h>
>   #include <asm/firmware/sci.h>
> +#include <asm/viommu.h>
>   #include <asm/grant_table.h>
>   #include <asm/setup.h>
>   
> @@ -317,6 +318,7 @@ int __init arch_parse_dom0less_node(struct dt_device_node *node,
>       uint32_t val;
>   
>       d_cfg->arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> +    d_cfg->arch.viommu_type = viommu_get_type();
>       d_cfg->flags |= XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap;
>   
>       if ( domu_dt_sci_parse(node, d_cfg) )
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 94b9858ad2..241f87386b 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -28,6 +28,7 @@
>   #include <asm/tee/tee.h>
>   #include <asm/vfp.h>
>   #include <asm/vgic.h>
> +#include <asm/viommu.h>
>   #include <asm/vtimer.h>
>   
>   #include "vpci.h"
> @@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>           return -EINVAL;
>       }
>   
> +    /* Check config structure padding */
> +    if ( config->arch.pad )
> +    {
> +        dprintk(XENLOG_INFO,
> +            "Invalid input config, padding must be zero\n");
> +        return -EINVAL;
> +    }
> +
>       /* Check feature flags */
>       if ( sve_vl_bits > 0 )
>       {
> @@ -626,6 +635,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>           return -EINVAL;
>       }
>   
> +    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
> +         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "vIOMMU requested while iommu not enabled for domain\n");
> +        return -EINVAL;
> +    }
> +
> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +    {
> +        dprintk(XENLOG_INFO,
> +                "vIOMMU type requested not supported by the platform or Xen\n");
> +        return -EINVAL;
> +    }
> +
>       return sci_domain_sanitise_config(config);
>   }
>   
> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>       if ( (rc = sci_domain_init(d, config)) != 0 )
>           goto fail;
>   
> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
> +        goto fail;
> +
>       return 0;
>   
>   fail:
> @@ -965,6 +992,7 @@ enum {
>       PROG_pci = 1,
>       PROG_sci,
>       PROG_tee,
> +    PROG_viommu,

I am not entirely sure about the position. Is the intention to 
relinquish the viommu state *after* the devices are detached? If so, it 
would be better to move this call just after 'PROG_pci' and add a 
comment indicating the dependency.

>       PROG_xen,
>       PROG_page,
>       PROG_mapping,
> @@ -1021,6 +1049,11 @@ int domain_relinquish_resources(struct domain *d)
>           if (ret )
>               return ret;
>   
> +    PROGRESS(viommu):
> +        ret = viommu_relinquish_resources(d);
> +        if (ret )
> +            return ret;
> +
>       PROGRESS(xen):
>           ret = relinquish_memory(d, &d->xenpage_list);
>           if ( ret )
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index e8795745dd..a51563ee3d 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -35,6 +35,7 @@
>   #include <asm/arm64/sve.h>
>   #include <asm/cpufeature.h>
>   #include <asm/domain_build.h>
> +#include <asm/viommu.h>
>   #include <xen/event.h>
>   
>   #include <xen/irq.h>
> @@ -1946,6 +1947,7 @@ void __init create_dom0(void)
>       dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
>       dom0_cfg.arch.tee_type = tee_get_type();
>       dom0_cfg.max_vcpus = dom0_max_vcpus();
> +    dom0_cfg.arch.viommu_type = viommu_get_type();
>   
>       if ( iommu_enabled )
>           dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
> new file mode 100644
> index 0000000000..4598f543b8
> --- /dev/null
> +++ b/xen/arch/arm/include/asm/viommu.h
> @@ -0,0 +1,70 @@
> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +#ifndef __ARCH_ARM_VIOMMU_H__
> +#define __ARCH_ARM_VIOMMU_H__
> +
> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
> +
> +#include <xen/lib.h>
> +#include <xen/types.h>
> +#include <public/xen.h>
> +
> +struct viommu_ops {
> +    /*
> +     * Called during domain construction if toolstack requests to enable
> +     * vIOMMU support.
> +     */
> +    int (*domain_init)(struct domain *d);
> +
> +    /*
> +     * Called during domain destruction to free resources used by vIOMMU.
> +     */
> +    int (*relinquish_resources)(struct domain *d);
> +};
> +
> +struct viommu_desc {
> +    /* vIOMMU domains init/free operations described above. */
> +    const struct viommu_ops *ops;
> +
> +    /*
> +     * ID of vIOMMU. Corresponds to xen_arch_domainconfig.viommu_type.
> +     * Should be one of XEN_DOMCTL_CONFIG_VIOMMU_xxx
> +     */
> +    uint16_t viommu_type;

Below, you define viommu_type as 'uint8_t'. So shouldn't this also be 
'uint8_t'?

> +};
> +
> +int domain_viommu_init(struct domain *d, uint16_t viommu_type);
> +int viommu_relinquish_resources(struct domain *d);
> +uint16_t viommu_get_type(void);
> +
> +#else
> +
> +static inline uint8_t viommu_get_type(void)
> +{
> +    return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
> +}
> +
> +static inline int domain_viommu_init(struct domain *d, uint16_t viommu_type)
> +{
> +    if ( likely(viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE) )
> +        return 0;
> +
> +    return -ENODEV;
> +}
> +
> +static inline int viommu_relinquish_resources(struct domain *d)
> +{
> +    return 0;
> +}
> +
> +#endif /* CONFIG_ARM_VIRTUAL_IOMMU */
> +
> +#endif /* __ARCH_ARM_VIOMMU_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
> index b413c33a4c..3c174bc87b 100644
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -35,6 +35,11 @@ config IPMMU_VMSA
>   	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
>   	  translation table format and is able to use CPU's P2M table as is.
>   
> +config ARM_VIRTUAL_IOMMU
> +	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
> +	help
> +	 Support virtual IOMMU infrastructure to implement vIOMMU.
> +
>   endif
>   
>   config AMD_IOMMU
> diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
> index c5fb3b58a5..c3783188e3 100644
> --- a/xen/drivers/passthrough/arm/Makefile
> +++ b/xen/drivers/passthrough/arm/Makefile
> @@ -2,3 +2,4 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
>   obj-$(CONFIG_ARM_SMMU) += smmu.o
>   obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
>   obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
> +obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
> new file mode 100644
> index 0000000000..7ab6061e34
> --- /dev/null
> +++ b/xen/drivers/passthrough/arm/viommu.c
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +
> +#include <xen/errno.h>
> +#include <xen/init.h>
> +#include <xen/types.h>
> +
> +#include <asm/viommu.h>
> +
> +const struct viommu_desc __read_mostly *cur_viommu;

You don't seem to define 'cur_viommmu' in the header. So shouldn't this 
be 'static'? Also, AFAICT, 'cur_viommu' would be set only once at boot. 
So rather than using __read_mostly, you probably want to use 
'__ro_after_init'.

> +
> +int domain_viommu_init(struct domain *d, uint16_t viommu_type)
> +{
> +    if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +        return 0;
> +
> +    if ( !cur_viommu )
> +        return -ENODEV;
> +
> +    if ( cur_viommu->viommu_type != viommu_type )
> +        return -EINVAL;
> +
> +    return cur_viommu->ops->domain_init(d);
> +}
> +
> +int viommu_relinquish_resources(struct domain *d)
> +{
> +    if ( !cur_viommu )
> +        return 0;
> +
> +    return cur_viommu->ops->relinquish_resources(d);
> +}
> +
> +uint16_t viommu_get_type(void)
> +{
> +    if ( !cur_viommu )
> +        return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
> +
> +    return cur_viommu->viommu_type;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index cd563cf706..d4953d40fd 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -330,6 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>   #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>   #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>   
> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
> +
>   struct xen_arch_domainconfig {
>       /* IN/OUT */
>       uint8_t gic_version;
> @@ -355,6 +357,9 @@ struct xen_arch_domainconfig {
>       uint32_t clock_frequency;
>       /* IN */
>       uint8_t arm_sci_type;
> +    /* IN */
> +    uint8_t viommu_type;
> +    uint16_t pad;
>   };
>   #endif /* __XEN__ || __XEN_TOOLS__ */
>   
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 8f6708c0a7..23124547f3 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -30,9 +30,9 @@
>    * fields) don't require a change of the version.
>    * Stable ops are NOT covered by XEN_DOMCTL_INTERFACE_VERSION!
>    *
> - * Last version bump: Xen 4.19
> + * Last version bump: Xen 4.22
>    */
> -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
> +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000018
>   
>   /*
>    * NB. xen_domctl.domain is an IN/OUT parameter for this operation.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 04/23] xen/arm: vIOMMU: add generic vIOMMU framework

[Milan Djokic]

Hi Julien,

On 4/14/26 08:15, Julien Grall wrote:
> Hi Milan,
> 
> On 31/03/2026 10:52, Milan Djokic wrote:
>> From: Rahul Singh <rahul.singh@arm.com>
>>
>> This patch adds basic framework for vIOMMU.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>>    xen/arch/arm/dom0less-build.c        |  2 +
>>    xen/arch/arm/domain.c                | 33 +++++++++++++
>>    xen/arch/arm/domain_build.c          |  2 +
>>    xen/arch/arm/include/asm/viommu.h    | 70 ++++++++++++++++++++++++++++
>>    xen/drivers/passthrough/Kconfig      |  5 ++
>>    xen/drivers/passthrough/arm/Makefile |  1 +
>>    xen/drivers/passthrough/arm/viommu.c | 48 +++++++++++++++++++
>>    xen/include/public/arch-arm.h        |  5 ++
>>    xen/include/public/domctl.h          |  4 +-
>>    9 files changed, 168 insertions(+), 2 deletions(-)
>>    create mode 100644 xen/arch/arm/include/asm/viommu.h
>>    create mode 100644 xen/drivers/passthrough/arm/viommu.c
>>
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
>> index 4181c10538..067835e5d0 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -23,6 +23,7 @@
>>    #include <asm/arm64/sve.h>
>>    #include <asm/domain_build.h>
>>    #include <asm/firmware/sci.h>
>> +#include <asm/viommu.h>
>>    #include <asm/grant_table.h>
>>    #include <asm/setup.h>
>>    
>> @@ -317,6 +318,7 @@ int __init arch_parse_dom0less_node(struct dt_device_node *node,
>>        uint32_t val;
>>    
>>        d_cfg->arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>> +    d_cfg->arch.viommu_type = viommu_get_type();
>>        d_cfg->flags |= XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap;
>>    
>>        if ( domu_dt_sci_parse(node, d_cfg) )
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 94b9858ad2..241f87386b 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -28,6 +28,7 @@
>>    #include <asm/tee/tee.h>
>>    #include <asm/vfp.h>
>>    #include <asm/vgic.h>
>> +#include <asm/viommu.h>
>>    #include <asm/vtimer.h>
>>    
>>    #include "vpci.h"
>> @@ -550,6 +551,14 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>            return -EINVAL;
>>        }
>>    
>> +    /* Check config structure padding */
>> +    if ( config->arch.pad )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +            "Invalid input config, padding must be zero\n");
>> +        return -EINVAL;
>> +    }
>> +
>>        /* Check feature flags */
>>        if ( sve_vl_bits > 0 )
>>        {
>> @@ -626,6 +635,21 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>            return -EINVAL;
>>        }
>>    
>> +    if ( !(config->flags & XEN_DOMCTL_CDF_iommu) &&
>> +         config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +                "vIOMMU requested while iommu not enabled for domain\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +    {
>> +        dprintk(XENLOG_INFO,
>> +                "vIOMMU type requested not supported by the platform or Xen\n");
>> +        return -EINVAL;
>> +    }
>> +
>>        return sci_domain_sanitise_config(config);
>>    }
>>    
>> @@ -721,6 +745,9 @@ int arch_domain_create(struct domain *d,
>>        if ( (rc = sci_domain_init(d, config)) != 0 )
>>            goto fail;
>>    
>> +    if ( (rc = domain_viommu_init(d, config->arch.viommu_type)) != 0 )
>> +        goto fail;
>> +
>>        return 0;
>>    
>>    fail:
>> @@ -965,6 +992,7 @@ enum {
>>        PROG_pci = 1,
>>        PROG_sci,
>>        PROG_tee,
>> +    PROG_viommu,
> 
> I am not entirely sure about the position. Is the intention to
> relinquish the viommu state *after* the devices are detached? If so, it
> would be better to move this call just after 'PROG_pci' and add a
> comment indicating the dependency.
> 

Yes, viommu cleanup should be performed after the devices are released.
After the rebase, it ended up in the wrong place because new components 
were added in the meantime.
I will update this and add a note for dependecy.

>>        PROG_xen,
>>        PROG_page,
>>        PROG_mapping,
>> @@ -1021,6 +1049,11 @@ int domain_relinquish_resources(struct domain *d)
>>            if (ret )
>>                return ret;
>>    
>> +    PROGRESS(viommu):
>> +        ret = viommu_relinquish_resources(d);
>> +        if (ret )
>> +            return ret;
>> +
>>        PROGRESS(xen):
>>            ret = relinquish_memory(d, &d->xenpage_list);
>>            if ( ret )
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index e8795745dd..a51563ee3d 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -35,6 +35,7 @@
>>    #include <asm/arm64/sve.h>
>>    #include <asm/cpufeature.h>
>>    #include <asm/domain_build.h>
>> +#include <asm/viommu.h>
>>    #include <xen/event.h>
>>    
>>    #include <xen/irq.h>
>> @@ -1946,6 +1947,7 @@ void __init create_dom0(void)
>>        dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
>>        dom0_cfg.arch.tee_type = tee_get_type();
>>        dom0_cfg.max_vcpus = dom0_max_vcpus();
>> +    dom0_cfg.arch.viommu_type = viommu_get_type();
>>    
>>        if ( iommu_enabled )
>>            dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
>> new file mode 100644
>> index 0000000000..4598f543b8
>> --- /dev/null
>> +++ b/xen/arch/arm/include/asm/viommu.h
>> @@ -0,0 +1,70 @@
>> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>> +#ifndef __ARCH_ARM_VIOMMU_H__
>> +#define __ARCH_ARM_VIOMMU_H__
>> +
>> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
>> +
>> +#include <xen/lib.h>
>> +#include <xen/types.h>
>> +#include <public/xen.h>
>> +
>> +struct viommu_ops {
>> +    /*
>> +     * Called during domain construction if toolstack requests to enable
>> +     * vIOMMU support.
>> +     */
>> +    int (*domain_init)(struct domain *d);
>> +
>> +    /*
>> +     * Called during domain destruction to free resources used by vIOMMU.
>> +     */
>> +    int (*relinquish_resources)(struct domain *d);
>> +};
>> +
>> +struct viommu_desc {
>> +    /* vIOMMU domains init/free operations described above. */
>> +    const struct viommu_ops *ops;
>> +
>> +    /*
>> +     * ID of vIOMMU. Corresponds to xen_arch_domainconfig.viommu_type.
>> +     * Should be one of XEN_DOMCTL_CONFIG_VIOMMU_xxx
>> +     */
>> +    uint16_t viommu_type;
> 
> Below, you define viommu_type as 'uint8_t'. So shouldn't this also be
> 'uint8_t'?
> 

Should be uint8_t, will fix it.

>> +};
>> +
>> +int domain_viommu_init(struct domain *d, uint16_t viommu_type);
>> +int viommu_relinquish_resources(struct domain *d);
>> +uint16_t viommu_get_type(void);
>> +
>> +#else
>> +
>> +static inline uint8_t viommu_get_type(void)
>> +{
>> +    return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
>> +}
>> +
>> +static inline int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>> +{
>> +    if ( likely(viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE) )
>> +        return 0;
>> +
>> +    return -ENODEV;
>> +}
>> +
>> +static inline int viommu_relinquish_resources(struct domain *d)
>> +{
>> +    return 0;
>> +}
>> +
>> +#endif /* CONFIG_ARM_VIRTUAL_IOMMU */
>> +
>> +#endif /* __ARCH_ARM_VIOMMU_H__ */
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
>> index b413c33a4c..3c174bc87b 100644
>> --- a/xen/drivers/passthrough/Kconfig
>> +++ b/xen/drivers/passthrough/Kconfig
>> @@ -35,6 +35,11 @@ config IPMMU_VMSA
>>    	  (H3 ES3.0, M3-W+, etc) or Gen4 SoCs which IPMMU hardware supports stage 2
>>    	  translation table format and is able to use CPU's P2M table as is.
>>    
>> +config ARM_VIRTUAL_IOMMU
>> +	bool "Virtual IOMMU Support (UNSUPPORTED)" if UNSUPPORTED
>> +	help
>> +	 Support virtual IOMMU infrastructure to implement vIOMMU.
>> +
>>    endif
>>    
>>    config AMD_IOMMU
>> diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
>> index c5fb3b58a5..c3783188e3 100644
>> --- a/xen/drivers/passthrough/arm/Makefile
>> +++ b/xen/drivers/passthrough/arm/Makefile
>> @@ -2,3 +2,4 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o
>>    obj-$(CONFIG_ARM_SMMU) += smmu.o
>>    obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
>>    obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
>> +obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
>> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
>> new file mode 100644
>> index 0000000000..7ab6061e34
>> --- /dev/null
>> +++ b/xen/drivers/passthrough/arm/viommu.c
>> @@ -0,0 +1,48 @@
>> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>> +
>> +#include <xen/errno.h>
>> +#include <xen/init.h>
>> +#include <xen/types.h>
>> +
>> +#include <asm/viommu.h>
>> +
>> +const struct viommu_desc __read_mostly *cur_viommu;
> 
> You don't seem to define 'cur_viommmu' in the header. So shouldn't this
> be 'static'? Also, AFAICT, 'cur_viommu' would be set only once at boot.
> So rather than using __read_mostly, you probably want to use
> '__ro_after_init'.
> 

I will update the 'cur_viommu' properties and add a set function for it.


>> +
>> +int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>> +{
>> +    if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +        return 0;
>> +
>> +    if ( !cur_viommu )
>> +        return -ENODEV;
>> +
>> +    if ( cur_viommu->viommu_type != viommu_type )
>> +        return -EINVAL;
>> +
>> +    return cur_viommu->ops->domain_init(d);
>> +}
>> +
>> +int viommu_relinquish_resources(struct domain *d)
>> +{
>> +    if ( !cur_viommu )
>> +        return 0;
>> +
>> +    return cur_viommu->ops->relinquish_resources(d);
>> +}
>> +
>> +uint16_t viommu_get_type(void)
>> +{
>> +    if ( !cur_viommu )
>> +        return XEN_DOMCTL_CONFIG_VIOMMU_NONE;
>> +
>> +    return cur_viommu->viommu_type;
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>> index cd563cf706..d4953d40fd 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -330,6 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>>    #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>>    #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>>    
>> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
>> +
>>    struct xen_arch_domainconfig {
>>        /* IN/OUT */
>>        uint8_t gic_version;
>> @@ -355,6 +357,9 @@ struct xen_arch_domainconfig {
>>        uint32_t clock_frequency;
>>        /* IN */
>>        uint8_t arm_sci_type;
>> +    /* IN */
>> +    uint8_t viommu_type;
>> +    uint16_t pad;
>>    };
>>    #endif /* __XEN__ || __XEN_TOOLS__ */
>>    
>> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
>> index 8f6708c0a7..23124547f3 100644
>> --- a/xen/include/public/domctl.h
>> +++ b/xen/include/public/domctl.h
>> @@ -30,9 +30,9 @@
>>     * fields) don't require a change of the version.
>>     * Stable ops are NOT covered by XEN_DOMCTL_INTERFACE_VERSION!
>>     *
>> - * Last version bump: Xen 4.19
>> + * Last version bump: Xen 4.22
>>     */
>> -#define XEN_DOMCTL_INTERFACE_VERSION 0x00000017
>> +#define XEN_DOMCTL_INTERFACE_VERSION 0x00000018
>>    
>>    /*
>>     * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
> 
> Cheers,
> 

BR,
Milan

[PATCH v3 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

domain_viommu_init() will be called during domain creation and will add
the dummy trap handler for virtual IOMMUs for guests.

A host IOMMU list will be created when host IOMMU devices are probed
and this list will be used to create the IOMMU device tree node for
dom0. For dom0, 1-1 mapping will be established between vIOMMU in dom0
and physical IOMMU.

For domUs, the 1-N mapping will be established between domU and physical
IOMMUs. A new area has been reserved in the arm guest physical map at
which the emulated vIOMMU node is created in the device tree.

Also set the vIOMMU type to vSMMUv3 to enable vIOMMU framework to call
vSMMUv3 domain creation/destroy functions.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain.c                  |   3 +-
 xen/arch/arm/include/asm/domain.h      |   4 +
 xen/arch/arm/include/asm/viommu.h      |  20 ++++
 xen/drivers/passthrough/Kconfig        |   8 ++
 xen/drivers/passthrough/arm/Makefile   |   1 +
 xen/drivers/passthrough/arm/smmu-v3.c  |   7 ++
 xen/drivers/passthrough/arm/viommu.c   |  30 ++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 124 +++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h |  20 ++++
 xen/include/public/arch-arm.h          |   7 +-
 10 files changed, 222 insertions(+), 2 deletions(-)
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
 create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 241f87386b..b982d79b3b 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -643,7 +643,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
-    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
+    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE &&
+         config->arch.viommu_type != viommu_get_type() )
     {
         dprintk(XENLOG_INFO,
                 "vIOMMU type requested not supported by the platform or Xen\n");
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index 758ad807e4..61108d0068 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -126,6 +126,10 @@ struct arch_domain
     void *sci_data;
 #endif
 
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+    struct list_head viommu_list;     /* List of virtual IOMMUs */
+#endif
+
 }  __cacheline_aligned;
 
 struct arch_vcpu
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 4598f543b8..2a6742de73 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -5,9 +5,21 @@
 #ifdef CONFIG_ARM_VIRTUAL_IOMMU
 
 #include <xen/lib.h>
+#include <xen/list.h>
 #include <xen/types.h>
 #include <public/xen.h>
 
+extern struct list_head host_iommu_list;
+
+/* data structure for each hardware IOMMU */
+struct host_iommu {
+    struct list_head entry;
+    const struct dt_device_node *dt_node;
+    paddr_t addr;
+    paddr_t size;
+    uint32_t irq;
+};
+
 struct viommu_ops {
     /*
      * Called during domain construction if toolstack requests to enable
@@ -35,6 +47,8 @@ struct viommu_desc {
 int domain_viommu_init(struct domain *d, uint16_t viommu_type);
 int viommu_relinquish_resources(struct domain *d);
 uint16_t viommu_get_type(void);
+void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                            const struct dt_device_node *node);
 
 #else
 
@@ -56,6 +70,12 @@ static inline int viommu_relinquish_resources(struct domain *d)
     return 0;
 }
 
+static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                                          const struct dt_device_node *node)
+{
+    return;
+}
+
 #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
 
 #endif /* __ARCH_ARM_VIOMMU_H__ */
diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
index 3c174bc87b..9c48e7415e 100644
--- a/xen/drivers/passthrough/Kconfig
+++ b/xen/drivers/passthrough/Kconfig
@@ -40,6 +40,14 @@ config ARM_VIRTUAL_IOMMU
 	help
 	 Support virtual IOMMU infrastructure to implement vIOMMU.
 
+config VIRTUAL_ARM_SMMU_V3
+	bool "ARM Ltd. Virtual SMMUv3 Support (UNSUPPORTED)"
+	depends on ARM_SMMU_V3 && ARM_VIRTUAL_IOMMU
+	help
+	 Support for implementations of the virtual ARM System MMU architecture
+	 version 3. Virtual SMMUv3 is unsupported feature and should not be used
+	 in production.
+
 endif
 
 config AMD_IOMMU
diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
index c3783188e3..c8f0a5f802 100644
--- a/xen/drivers/passthrough/arm/Makefile
+++ b/xen/drivers/passthrough/arm/Makefile
@@ -3,3 +3,4 @@ obj-$(CONFIG_ARM_SMMU) += smmu.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
 obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
+obj-$(CONFIG_VIRTUAL_ARM_SMMU_V3) += vsmmu-v3.o
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 19e55b6c9b..87612df21d 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -93,6 +93,7 @@
 #include <asm/platform.h>
 
 #include "smmu-v3.h"
+#include "vsmmu-v3.h"
 
 #define ARM_SMMU_VTCR_SH_IS		3
 #define ARM_SMMU_VTCR_RGN_WBWA		1
@@ -2727,6 +2728,9 @@ static int __init arm_smmu_device_probe(struct platform_device *pdev)
 	list_add(&smmu->devices, &arm_smmu_devices);
 	spin_unlock(&arm_smmu_devices_lock);
 
+    /* Add to host IOMMU list to initialize vIOMMU for dom0 */
+	add_to_host_iommu_list(ioaddr, iosize, dev_to_dt(pdev));
+
 	return 0;
 
 
@@ -3058,6 +3062,9 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
 
 	platform_features &= smmu->features;
 
+	/* Set vIOMMU type to SMMUv3 */
+	vsmmuv3_set_type();
+
 	return 0;
 }
 
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 7ab6061e34..53ae46349a 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -2,12 +2,42 @@
 
 #include <xen/errno.h>
 #include <xen/init.h>
+#include <xen/irq.h>
 #include <xen/types.h>
 
 #include <asm/viommu.h>
 
+/* List of all host IOMMUs */
+LIST_HEAD(host_iommu_list);
+
 const struct viommu_desc __read_mostly *cur_viommu;
 
+/* Common function for adding to host_iommu_list */
+void add_to_host_iommu_list(paddr_t addr, paddr_t size,
+                            const struct dt_device_node *node)
+{
+    struct host_iommu *iommu_data;
+
+    iommu_data = xzalloc(struct host_iommu);
+    if ( !iommu_data )
+        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");
+
+    iommu_data->addr = addr;
+    iommu_data->size = size;
+    iommu_data->dt_node = node;
+    iommu_data->irq = platform_get_irq(node, 0);
+    if ( iommu_data->irq < 0 )
+    {
+        gdprintk(XENLOG_ERR,
+                 "vIOMMU: Cannot find a valid IOMMU irq\n");
+        return;
+    }
+
+    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
+
+    list_add_tail(&iommu_data->entry, &host_iommu_list);
+}
+
 int domain_viommu_init(struct domain *d, uint16_t viommu_type)
 {
     if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
new file mode 100644
index 0000000000..6b4009e5ef
--- /dev/null
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+
+#include <xen/param.h>
+#include <xen/sched.h>
+#include <asm/mmio.h>
+#include <asm/viommu.h>
+
+/* Struct to hold the vIOMMU ops and vIOMMU type */
+extern const struct viommu_desc __read_mostly *cur_viommu;
+
+struct virt_smmu {
+    struct      domain *d;
+    struct      list_head viommu_list;
+};
+
+static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
+                              register_t r, void *priv)
+{
+    return IO_HANDLED;
+}
+
+static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
+                             register_t *r, void *priv)
+{
+    return IO_HANDLED;
+}
+
+static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
+    .read  = vsmmuv3_mmio_read,
+    .write = vsmmuv3_mmio_write,
+};
+
+static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
+{
+    struct virt_smmu *smmu;
+
+    smmu = xzalloc(struct virt_smmu);
+    if ( !smmu )
+        return -ENOMEM;
+
+    smmu->d = d;
+
+    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
+
+    /* Register the vIOMMU to be able to clean it up later. */
+    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
+
+    return 0;
+}
+
+int domain_vsmmuv3_init(struct domain *d)
+{
+    int ret;
+    INIT_LIST_HEAD(&d->arch.viommu_list);
+
+    if ( is_hardware_domain(d) )
+    {
+        struct host_iommu *hw_iommu;
+
+        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
+        {
+            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
+            if ( ret )
+                return ret;
+        }
+    }
+    else
+    {
+        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+        if ( ret )
+            return ret;
+    }
+
+    return 0;
+}
+
+int vsmmuv3_relinquish_resources(struct domain *d)
+{
+    struct virt_smmu *pos, *temp;
+
+    /* Cope with unitialized vIOMMU */
+    if ( list_head_is_null(&d->arch.viommu_list) )
+        return 0;
+
+    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
+    {
+        list_del(&pos->viommu_list);
+        xfree(pos);
+    }
+
+    return 0;
+}
+
+static const struct viommu_ops vsmmuv3_ops = {
+    .domain_init = domain_vsmmuv3_init,
+    .relinquish_resources = vsmmuv3_relinquish_resources,
+};
+
+static const struct viommu_desc vsmmuv3_desc = {
+    .ops = &vsmmuv3_ops,
+    .viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3,
+};
+
+void __init vsmmuv3_set_type(void)
+{
+    const struct viommu_desc *desc = &vsmmuv3_desc;
+
+    if ( cur_viommu && (cur_viommu != desc) )
+    {
+        printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
+        return;
+    }
+
+    cur_viommu = desc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
new file mode 100644
index 0000000000..e11f85b431
--- /dev/null
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+#ifndef __ARCH_ARM_VSMMU_V3_H__
+#define __ARCH_ARM_VSMMU_V3_H__
+
+#include <asm/viommu.h>
+
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+
+void vsmmuv3_set_type(void);
+
+#else
+
+static inline void vsmmuv3_set_type(void)
+{
+    return;
+}
+
+#endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
+
+#endif /* __ARCH_ARM_VSMMU_V3_H__ */
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index d4953d40fd..ebac02ed63 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -330,7 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
 #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
 #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
 
-#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
+#define XEN_DOMCTL_CONFIG_VIOMMU_NONE       0
+#define XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3     1
 
 struct xen_arch_domainconfig {
     /* IN/OUT */
@@ -456,6 +457,10 @@ typedef uint64_t xen_callback_t;
 #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
 #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
 
+/* vsmmuv3 ITS mappings */
+#define GUEST_VSMMUV3_BASE     xen_mk_ullong(0x04040000)
+#define GUEST_VSMMUV3_SIZE     xen_mk_ullong(0x00040000)
+
 /*
  * 256 MB is reserved for VPCI configuration space based on calculation
  * 256 buses x 32 devices x 8 functions x 4 KB = 256 MB
-- 
2.43.0

Re: [PATCH v3 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Luca Fancellu]

Hi Milan,


> 
> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
> index 7ab6061e34..53ae46349a 100644
> --- a/xen/drivers/passthrough/arm/viommu.c
> +++ b/xen/drivers/passthrough/arm/viommu.c
> @@ -2,12 +2,42 @@
> 
> #include <xen/errno.h>
> #include <xen/init.h>
> +#include <xen/irq.h>
> #include <xen/types.h>
> 
> #include <asm/viommu.h>
> 
> +/* List of all host IOMMUs */
> +LIST_HEAD(host_iommu_list);
> +
> const struct viommu_desc __read_mostly *cur_viommu;
> 
> +/* Common function for adding to host_iommu_list */
> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
> +                            const struct dt_device_node *node)
> +{
> +    struct host_iommu *iommu_data;
> +
> +    iommu_data = xzalloc(struct host_iommu);
> +    if ( !iommu_data )
> +        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");
> +
> +    iommu_data->addr = addr;
> +    iommu_data->size = size;
> +    iommu_data->dt_node = node;
> +    iommu_data->irq = platform_get_irq(node, 0);
> +    if ( iommu_data->irq < 0 )
> +    {
> +        gdprintk(XENLOG_ERR,
> +                 "vIOMMU: Cannot find a valid IOMMU irq\n");

We need to free iommu_data here

> +        return;
> +    }
> +
> +    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
> +
> +    list_add_tail(&iommu_data->entry, &host_iommu_list);
> +}
> +
> int domain_viommu_init(struct domain *d, uint16_t viommu_type)
> {
>     if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> new file mode 100644
> index 0000000000..6b4009e5ef
> --- /dev/null
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -0,0 +1,124 @@
> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +
> +#include <xen/param.h>
> +#include <xen/sched.h>
> +#include <asm/mmio.h>
> +#include <asm/viommu.h>
> +
> +/* Struct to hold the vIOMMU ops and vIOMMU type */
> +extern const struct viommu_desc __read_mostly *cur_viommu;
> +
> +struct virt_smmu {
> +    struct      domain *d;
> +    struct      list_head viommu_list;
> +};
> +
> +static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                              register_t r, void *priv)
> +{
> +    return IO_HANDLED;
> +}
> +
> +static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                             register_t *r, void *priv)
> +{
> +    return IO_HANDLED;

If this has to be treated for now as RAZ, being a dummy implementation,
I would add *r = 0;

> +}
> +
> +static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
> +    .read  = vsmmuv3_mmio_read,
> +    .write = vsmmuv3_mmio_write,
> +};
> +
> +static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
> +{
> +    struct virt_smmu *smmu;
> +
> +    smmu = xzalloc(struct virt_smmu);
> +    if ( !smmu )
> +        return -ENOMEM;
> +
> +    smmu->d = d;
> +
> +    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
> +
> +    /* Register the vIOMMU to be able to clean it up later. */
> +    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
> +
> +    return 0;
> +}
> +
> +int domain_vsmmuv3_init(struct domain *d)
> +{
> +    int ret;
> +    INIT_LIST_HEAD(&d->arch.viommu_list);
> +
> +    if ( is_hardware_domain(d) )
> +    {
> +        struct host_iommu *hw_iommu;
> +
> +        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
> +        {
> +            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
> +            if ( ret )
> +                return ret;
> +        }
> +    }
> +    else
> +    {
> +        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
> +int vsmmuv3_relinquish_resources(struct domain *d)
> +{
> +    struct virt_smmu *pos, *temp;
> +
> +    /* Cope with unitialized vIOMMU */

Typo s/unitialized/uninitialized/


Cheers,
Luca

Re: [PATCH v3 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Milan Djokic]

Hi Luca,

On 4/10/26 13:59, Luca Fancellu wrote:
> Hi Milan,
> 
> 
>>
>> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
>> index 7ab6061e34..53ae46349a 100644
>> --- a/xen/drivers/passthrough/arm/viommu.c
>> +++ b/xen/drivers/passthrough/arm/viommu.c
>> @@ -2,12 +2,42 @@
>>
>> #include <xen/errno.h>
>> #include <xen/init.h>
>> +#include <xen/irq.h>
>> #include <xen/types.h>
>>
>> #include <asm/viommu.h>
>>
>> +/* List of all host IOMMUs */
>> +LIST_HEAD(host_iommu_list);
>> +
>> const struct viommu_desc __read_mostly *cur_viommu;
>>
>> +/* Common function for adding to host_iommu_list */
>> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>> +                            const struct dt_device_node *node)
>> +{
>> +    struct host_iommu *iommu_data;
>> +
>> +    iommu_data = xzalloc(struct host_iommu);
>> +    if ( !iommu_data )
>> +        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");
>> +
>> +    iommu_data->addr = addr;
>> +    iommu_data->size = size;
>> +    iommu_data->dt_node = node;
>> +    iommu_data->irq = platform_get_irq(node, 0);
>> +    if ( iommu_data->irq < 0 )
>> +    {
>> +        gdprintk(XENLOG_ERR,
>> +                 "vIOMMU: Cannot find a valid IOMMU irq\n");
> 
> We need to free iommu_data here
> 

Yes, will fix this.

>> +        return;
>> +    }
>> +
>> +    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
>> +
>> +    list_add_tail(&iommu_data->entry, &host_iommu_list);
>> +}
>> +
>> int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>> {
>>      if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> new file mode 100644
>> index 0000000000..6b4009e5ef
>> --- /dev/null
>> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> @@ -0,0 +1,124 @@
>> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>> +
>> +#include <xen/param.h>
>> +#include <xen/sched.h>
>> +#include <asm/mmio.h>
>> +#include <asm/viommu.h>
>> +
>> +/* Struct to hold the vIOMMU ops and vIOMMU type */
>> +extern const struct viommu_desc __read_mostly *cur_viommu;
>> +
>> +struct virt_smmu {
>> +    struct      domain *d;
>> +    struct      list_head viommu_list;
>> +};
>> +
>> +static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>> +                              register_t r, void *priv)
>> +{
>> +    return IO_HANDLED;
>> +}
>> +
>> +static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
>> +                             register_t *r, void *priv)
>> +{
>> +    return IO_HANDLED;
> 
> If this has to be treated for now as RAZ, being a dummy implementation,
> I would add *r = 0;
> 

As suggested in later comments, I will insert BUG_ON("unimplemented") 
for functions which are not complete at this point.

>> +}
>> +
>> +static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
>> +    .read  = vsmmuv3_mmio_read,
>> +    .write = vsmmuv3_mmio_write,
>> +};
>> +
>> +static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
>> +{
>> +    struct virt_smmu *smmu;
>> +
>> +    smmu = xzalloc(struct virt_smmu);
>> +    if ( !smmu )
>> +        return -ENOMEM;
>> +
>> +    smmu->d = d;
>> +
>> +    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
>> +
>> +    /* Register the vIOMMU to be able to clean it up later. */
>> +    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
>> +
>> +    return 0;
>> +}
>> +
>> +int domain_vsmmuv3_init(struct domain *d)
>> +{
>> +    int ret;
>> +    INIT_LIST_HEAD(&d->arch.viommu_list);
>> +
>> +    if ( is_hardware_domain(d) )
>> +    {
>> +        struct host_iommu *hw_iommu;
>> +
>> +        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
>> +        {
>> +            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
>> +            if ( ret )
>> +                return ret;
>> +        }
>> +    }
>> +    else
>> +    {
>> +        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
>> +        if ( ret )
>> +            return ret;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +int vsmmuv3_relinquish_resources(struct domain *d)
>> +{
>> +    struct virt_smmu *pos, *temp;
>> +
>> +    /* Cope with unitialized vIOMMU */
> 
> Typo s/unitialized/uninitialized/
> 
> 
> Cheers,
> Luca
> 
> 

BR,
Milan

Re: [PATCH v3 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> domain_viommu_init() will be called during domain creation and will add
> the dummy trap handler for virtual IOMMUs for guests.
> 
> A host IOMMU list will be created when host IOMMU devices are probed
> and this list will be used to create the IOMMU device tree node for
> dom0. For dom0, 1-1 mapping will be established between vIOMMU in dom0
> and physical IOMMU.
> 
> For domUs, the 1-N mapping will be established between domU and physical
> IOMMUs. A new area has been reserved in the arm guest physical map at
> which the emulated vIOMMU node is created in the device tree.
> 
> Also set the vIOMMU type to vSMMUv3 to enable vIOMMU framework to call
> vSMMUv3 domain creation/destroy functions.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/arch/arm/domain.c                  |   3 +-
>   xen/arch/arm/include/asm/domain.h      |   4 +
>   xen/arch/arm/include/asm/viommu.h      |  20 ++++
>   xen/drivers/passthrough/Kconfig        |   8 ++
>   xen/drivers/passthrough/arm/Makefile   |   1 +
>   xen/drivers/passthrough/arm/smmu-v3.c  |   7 ++
>   xen/drivers/passthrough/arm/viommu.c   |  30 ++++++
>   xen/drivers/passthrough/arm/vsmmu-v3.c | 124 +++++++++++++++++++++++++
>   xen/drivers/passthrough/arm/vsmmu-v3.h |  20 ++++
>   xen/include/public/arch-arm.h          |   7 +-
>   10 files changed, 222 insertions(+), 2 deletions(-)
>   create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
>   create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 241f87386b..b982d79b3b 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -643,7 +643,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>           return -EINVAL;
>       }
>   
> -    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE &&
> +         config->arch.viommu_type != viommu_get_type() )
>       {
>           dprintk(XENLOG_INFO,
>                   "vIOMMU type requested not supported by the platform or Xen\n");
> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
> index 758ad807e4..61108d0068 100644
> --- a/xen/arch/arm/include/asm/domain.h
> +++ b/xen/arch/arm/include/asm/domain.h
> @@ -126,6 +126,10 @@ struct arch_domain
>       void *sci_data;
>   #endif
>   
> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
> +    struct list_head viommu_list;     /* List of virtual IOMMUs */
> +#endif
> +
>   }  __cacheline_aligned;
>   
>   struct arch_vcpu
> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
> index 4598f543b8..2a6742de73 100644
> --- a/xen/arch/arm/include/asm/viommu.h
> +++ b/xen/arch/arm/include/asm/viommu.h
> @@ -5,9 +5,21 @@
>   #ifdef CONFIG_ARM_VIRTUAL_IOMMU
>   
>   #include <xen/lib.h>
> +#include <xen/list.h>
>   #include <xen/types.h>
>   #include <public/xen.h>
>   
> +extern struct list_head host_iommu_list;
> +
> +/* data structure for each hardware IOMMU */
> +struct host_iommu {
> +    struct list_head entry;
> +    const struct dt_device_node *dt_node;
> +    paddr_t addr;
> +    paddr_t size;
> +    uint32_t irq;

You don't seem to use ``irq`` in this patch. What is this meant to be 
used for?

> +};
> +
>   struct viommu_ops {
>       /*
>        * Called during domain construction if toolstack requests to enable
> @@ -35,6 +47,8 @@ struct viommu_desc {
>   int domain_viommu_init(struct domain *d, uint16_t viommu_type);
>   int viommu_relinquish_resources(struct domain *d);
>   uint16_t viommu_get_type(void);
> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
> +                            const struct dt_device_node *node);
>   
>   #else
>   
> @@ -56,6 +70,12 @@ static inline int viommu_relinquish_resources(struct domain *d)
>       return 0;
>   }
>   
> +static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
> +                                          const struct dt_device_node *node)
> +{
> +    return;
> +}
> +
>   #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
>   
>   #endif /* __ARCH_ARM_VIOMMU_H__ */
> diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
> index 3c174bc87b..9c48e7415e 100644
> --- a/xen/drivers/passthrough/Kconfig
> +++ b/xen/drivers/passthrough/Kconfig
> @@ -40,6 +40,14 @@ config ARM_VIRTUAL_IOMMU
>   	help
>   	 Support virtual IOMMU infrastructure to implement vIOMMU.
>   
> +config VIRTUAL_ARM_SMMU_V3
> +	bool "ARM Ltd. Virtual SMMUv3 Support (UNSUPPORTED)"
> +	depends on ARM_SMMU_V3 && ARM_VIRTUAL_IOMMU
> +	help
> +	 Support for implementations of the virtual ARM System MMU architecture
> +	 version 3. Virtual SMMUv3 is unsupported feature and should not be used
> +	 in production.
> +
>   endif
>   
>   config AMD_IOMMU
> diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
> index c3783188e3..c8f0a5f802 100644
> --- a/xen/drivers/passthrough/arm/Makefile
> +++ b/xen/drivers/passthrough/arm/Makefile
> @@ -3,3 +3,4 @@ obj-$(CONFIG_ARM_SMMU) += smmu.o
>   obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
>   obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
>   obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
> +obj-$(CONFIG_VIRTUAL_ARM_SMMU_V3) += vsmmu-v3.o
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index 19e55b6c9b..87612df21d 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -93,6 +93,7 @@
>   #include <asm/platform.h>
>   
>   #include "smmu-v3.h"
> +#include "vsmmu-v3.h"
>   
>   #define ARM_SMMU_VTCR_SH_IS		3
>   #define ARM_SMMU_VTCR_RGN_WBWA		1
> @@ -2727,6 +2728,9 @@ static int __init arm_smmu_device_probe(struct platform_device *pdev)
>   	list_add(&smmu->devices, &arm_smmu_devices);
>   	spin_unlock(&arm_smmu_devices_lock);
>   
> +    /* Add to host IOMMU list to initialize vIOMMU for dom0 */

The indentation looks odd. Is this correctly aligned?

> +	add_to_host_iommu_list(ioaddr, iosize, dev_to_dt(pdev));
> +
>   	return 0;
>   
>   
> @@ -3058,6 +3062,9 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>   
>   	platform_features &= smmu->features;
>   
> +	/* Set vIOMMU type to SMMUv3 */
> +	vsmmuv3_set_type();
> +
>   	return 0;
>   }
>   
> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
> index 7ab6061e34..53ae46349a 100644
> --- a/xen/drivers/passthrough/arm/viommu.c
> +++ b/xen/drivers/passthrough/arm/viommu.c
> @@ -2,12 +2,42 @@
>   
>   #include <xen/errno.h>
>   #include <xen/init.h>
> +#include <xen/irq.h>
>   #include <xen/types.h>
>   
>   #include <asm/viommu.h>
>   
> +/* List of all host IOMMUs */
> +LIST_HEAD(host_iommu_list);

I don't quite follow why this is part of the common code. That said, why 
do we need to register the host IOMMU? Wouldn't it be simpler to go 
through the list of pIOMMU in the vSMMU v3 implementation?

> +
>   const struct viommu_desc __read_mostly *cur_viommu;
>   
> +/* Common function for adding to host_iommu_list */
> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
> +                            const struct dt_device_node *node)

Is this supposed to only be called during __init? If so, this will help 
to justify the ...

> +{
> +    struct host_iommu *iommu_data;
> +
> +    iommu_data = xzalloc(struct host_iommu);
> +    if ( !iommu_data )
> +        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");

... panic(). If not, then this function needs to return an error.

> +
> +    iommu_data->addr = addr;
> +    iommu_data->size = size;
> +    iommu_data->dt_node = node;
> +    iommu_data->irq = platform_get_irq(node, 0);
> +    if ( iommu_data->irq < 0 )
> +    {
> +        gdprintk(XENLOG_ERR,
> +                 "vIOMMU: Cannot find a valid IOMMU irq\n");

Shouldn't you free the allocated memory? That said, why is it ok to 
ignore the vIOMMU in this case?

> +        return;
> +    }
> +
> +    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
> +
> +    list_add_tail(&iommu_data->entry, &host_iommu_list);
> +}
> +
>   int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>   {
>       if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> new file mode 100644
> index 0000000000..6b4009e5ef
> --- /dev/null
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -0,0 +1,124 @@
> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +
> +#include <xen/param.h>
> +#include <xen/sched.h>
> +#include <asm/mmio.h>
> +#include <asm/viommu.h>
> +
> +/* Struct to hold the vIOMMU ops and vIOMMU type */
> +extern const struct viommu_desc __read_mostly *cur_viommu;

Why is this defined in the C file? Shouldn't this be defined in 
``viomum.h``? That said, looking at the use, I think we want to 
introduce a helper to set the cur_viommu and not export ``cur_viommu``.

> +
> +struct virt_smmu {
> +    struct      domain *d;
> +    struct      list_head viommu_list;
> +};
> +
> +static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                              register_t r, void *priv)
> +{
> +    return IO_HANDLED;
> +}
> +
> +static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                             register_t *r, void *priv)
> +{
> +    return IO_HANDLED;
> +}

I undertand that both helpers are meant to be updated in a later patch. 
However, I think it makes quite difficult to know whether the code is 
complete by the end of the series. I would strongly recommend adding 
'BUG_ON("unimplemented");' so we can easily grep whether there is 
anything missing by the end.

> +
> +static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
> +    .read  = vsmmuv3_mmio_read,
> +    .write = vsmmuv3_mmio_write,
> +};
> +
> +static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
> +{
> +    struct virt_smmu *smmu;
> +
> +    smmu = xzalloc(struct virt_smmu);
> +    if ( !smmu )
> +        return -ENOMEM;
> +
> +    smmu->d = d;
> +
> +    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);

I can't seem to find a place where the number of handler supported is 
incremented (see domain_io_init()) for the vIOMMU. Are you sure we 
always have enough space today?

> +
> +    /* Register the vIOMMU to be able to clean it up later. */
> +    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
> +
> +    return 0;
> +}
> +
> +int domain_vsmmuv3_init(struct domain *d)
> +{
> +    int ret;

Coding style: newline missing.

> +    INIT_LIST_HEAD(&d->arch.viommu_list);
> +
> +    if ( is_hardware_domain(d) )
> +    {
> +        struct host_iommu *hw_iommu;
> +
> +        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
> +        {
> +            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
> +            if ( ret )
> +                return ret;
> +        }
> +    }
> +    else
> +    {
> +        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
> +        if ( ret )
> +            return ret;
> +    }
> +
> +    return 0;
> +}
> +
> +int vsmmuv3_relinquish_resources(struct domain *d)
> +{
> +    struct virt_smmu *pos, *temp;
> +
> +    /* Cope with unitialized vIOMMU */
> +    if ( list_head_is_null(&d->arch.viommu_list) )
> +        return 0;
> +
> +    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
> +    {
> +        list_del(&pos->viommu_list);
> +        xfree(pos);
> +    }
> +
> +    return 0;
> +}
> +
> +static const struct viommu_ops vsmmuv3_ops = {
> +    .domain_init = domain_vsmmuv3_init,
> +    .relinquish_resources = vsmmuv3_relinquish_resources,
> +};
> +
> +static const struct viommu_desc vsmmuv3_desc = {
> +    .ops = &vsmmuv3_ops,
> +    .viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3,
> +};
> +
> +void __init vsmmuv3_set_type(void)
> +{
> +    const struct viommu_desc *desc = &vsmmuv3_desc;
> +
> +    if ( cur_viommu && (cur_viommu != desc) )
> +    {
> +        printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
> +        return;
> +    }
> +
> +    cur_viommu = desc;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
> new file mode 100644
> index 0000000000..e11f85b431
> --- /dev/null
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +#ifndef __ARCH_ARM_VSMMU_V3_H__
> +#define __ARCH_ARM_VSMMU_V3_H__
> +
> +#include <asm/viommu.h>
> +
> +#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
> +
> +void vsmmuv3_set_type(void);
> +
> +#else
> +
> +static inline void vsmmuv3_set_type(void)
> +{
> +    return;
> +}
> +
> +#endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
> +
> +#endif /* __ARCH_ARM_VSMMU_V3_H__ */
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index d4953d40fd..ebac02ed63 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -330,7 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>   #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>   #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>   
> -#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE       0

I am generally against trying to align values because this is just 
introducing unnecessary churn in the code.

> +#define XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3     1
>   
>   struct xen_arch_domainconfig {
>       /* IN/OUT */
> @@ -456,6 +457,10 @@ typedef uint64_t xen_callback_t;
>   #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
>   #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
>   
> +/* vsmmuv3 ITS mappings */

What is ITS stand for in this context?

> +#define GUEST_VSMMUV3_BASE     xen_mk_ullong(0x04040000)
> +#define GUEST_VSMMUV3_SIZE     xen_mk_ullong(0x00040000)

If I am not mistaken, you are reserving 256KiB. However, looking at the 
SMMU spec (section 6.1, ARM IHI 0070 H.a), the minimum is 128 KiB. Are 
you intending to expose additional features?

> +
>   /*
>    * 256 MB is reserved for VPCI configuration space based on calculation
>    * 256 buses x 32 devices x 8 functions x 4 KB = 256 MB

Cheers,

-- 
Julien Grall

Re: [PATCH v3 05/23] xen/arm: vsmmuv3: Add dummy support for virtual SMMUv3 for guests

[Milan Djokic]

Hi Julien,

On 4/14/26 09:09, Julien Grall wrote:
> Hi Milan,
> 
> On 31/03/2026 10:52, Milan Djokic wrote:
>> From: Rahul Singh <rahul.singh@arm.com>
>>
>> domain_viommu_init() will be called during domain creation and will add
>> the dummy trap handler for virtual IOMMUs for guests.
>>
>> A host IOMMU list will be created when host IOMMU devices are probed
>> and this list will be used to create the IOMMU device tree node for
>> dom0. For dom0, 1-1 mapping will be established between vIOMMU in dom0
>> and physical IOMMU.
>>
>> For domUs, the 1-N mapping will be established between domU and physical
>> IOMMUs. A new area has been reserved in the arm guest physical map at
>> which the emulated vIOMMU node is created in the device tree.
>>
>> Also set the vIOMMU type to vSMMUv3 to enable vIOMMU framework to call
>> vSMMUv3 domain creation/destroy functions.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>>    xen/arch/arm/domain.c                  |   3 +-
>>    xen/arch/arm/include/asm/domain.h      |   4 +
>>    xen/arch/arm/include/asm/viommu.h      |  20 ++++
>>    xen/drivers/passthrough/Kconfig        |   8 ++
>>    xen/drivers/passthrough/arm/Makefile   |   1 +
>>    xen/drivers/passthrough/arm/smmu-v3.c  |   7 ++
>>    xen/drivers/passthrough/arm/viommu.c   |  30 ++++++
>>    xen/drivers/passthrough/arm/vsmmu-v3.c | 124 +++++++++++++++++++++++++
>>    xen/drivers/passthrough/arm/vsmmu-v3.h |  20 ++++
>>    xen/include/public/arch-arm.h          |   7 +-
>>    10 files changed, 222 insertions(+), 2 deletions(-)
>>    create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.c
>>    create mode 100644 xen/drivers/passthrough/arm/vsmmu-v3.h
>>
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 241f87386b..b982d79b3b 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -643,7 +643,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>            return -EINVAL;
>>        }
>>    
>> -    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> +    if ( config->arch.viommu_type != XEN_DOMCTL_CONFIG_VIOMMU_NONE &&
>> +         config->arch.viommu_type != viommu_get_type() )
>>        {
>>            dprintk(XENLOG_INFO,
>>                    "vIOMMU type requested not supported by the platform or Xen\n");
>> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
>> index 758ad807e4..61108d0068 100644
>> --- a/xen/arch/arm/include/asm/domain.h
>> +++ b/xen/arch/arm/include/asm/domain.h
>> @@ -126,6 +126,10 @@ struct arch_domain
>>        void *sci_data;
>>    #endif
>>    
>> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
>> +    struct list_head viommu_list;     /* List of virtual IOMMUs */
>> +#endif
>> +
>>    }  __cacheline_aligned;
>>    
>>    struct arch_vcpu
>> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
>> index 4598f543b8..2a6742de73 100644
>> --- a/xen/arch/arm/include/asm/viommu.h
>> +++ b/xen/arch/arm/include/asm/viommu.h
>> @@ -5,9 +5,21 @@
>>    #ifdef CONFIG_ARM_VIRTUAL_IOMMU
>>    
>>    #include <xen/lib.h>
>> +#include <xen/list.h>
>>    #include <xen/types.h>
>>    #include <public/xen.h>
>>    
>> +extern struct list_head host_iommu_list;
>> +
>> +/* data structure for each hardware IOMMU */
>> +struct host_iommu {
>> +    struct list_head entry;
>> +    const struct dt_device_node *dt_node;
>> +    paddr_t addr;
>> +    paddr_t size;
>> +    uint32_t irq;
> 
> You don't seem to use ``irq`` in this patch. What is this meant to be
> used for?
> 

This field will be used for vIOMMU event queue creation for the hardware 
domain in a later patch (xen/arm: vsmmuv3: Add support for event queue 
and global error). The emulated IRQ and MMIO region for the hardware 
domain vIOMMU will match those of the host IOMMU.

>> +};
>> +
>>    struct viommu_ops {
>>        /*
>>         * Called during domain construction if toolstack requests to enable
>> @@ -35,6 +47,8 @@ struct viommu_desc {
>>    int domain_viommu_init(struct domain *d, uint16_t viommu_type);
>>    int viommu_relinquish_resources(struct domain *d);
>>    uint16_t viommu_get_type(void);
>> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>> +                            const struct dt_device_node *node);
>>    
>>    #else
>>    
>> @@ -56,6 +70,12 @@ static inline int viommu_relinquish_resources(struct domain *d)
>>        return 0;
>>    }
>>    
>> +static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>> +                                          const struct dt_device_node *node)
>> +{
>> +    return;
>> +}
>> +
>>    #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
>>    
>>    #endif /* __ARCH_ARM_VIOMMU_H__ */
>> diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig
>> index 3c174bc87b..9c48e7415e 100644
>> --- a/xen/drivers/passthrough/Kconfig
>> +++ b/xen/drivers/passthrough/Kconfig
>> @@ -40,6 +40,14 @@ config ARM_VIRTUAL_IOMMU
>>    	help
>>    	 Support virtual IOMMU infrastructure to implement vIOMMU.
>>    
>> +config VIRTUAL_ARM_SMMU_V3
>> +	bool "ARM Ltd. Virtual SMMUv3 Support (UNSUPPORTED)"
>> +	depends on ARM_SMMU_V3 && ARM_VIRTUAL_IOMMU
>> +	help
>> +	 Support for implementations of the virtual ARM System MMU architecture
>> +	 version 3. Virtual SMMUv3 is unsupported feature and should not be used
>> +	 in production.
>> +
>>    endif
>>    
>>    config AMD_IOMMU
>> diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile
>> index c3783188e3..c8f0a5f802 100644
>> --- a/xen/drivers/passthrough/arm/Makefile
>> +++ b/xen/drivers/passthrough/arm/Makefile
>> @@ -3,3 +3,4 @@ obj-$(CONFIG_ARM_SMMU) += smmu.o
>>    obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
>>    obj-$(CONFIG_ARM_SMMU_V3) += smmu-v3.o
>>    obj-$(CONFIG_ARM_VIRTUAL_IOMMU) += viommu.o
>> +obj-$(CONFIG_VIRTUAL_ARM_SMMU_V3) += vsmmu-v3.o
>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
>> index 19e55b6c9b..87612df21d 100644
>> --- a/xen/drivers/passthrough/arm/smmu-v3.c
>> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
>> @@ -93,6 +93,7 @@
>>    #include <asm/platform.h>
>>    
>>    #include "smmu-v3.h"
>> +#include "vsmmu-v3.h"
>>    
>>    #define ARM_SMMU_VTCR_SH_IS		3
>>    #define ARM_SMMU_VTCR_RGN_WBWA		1
>> @@ -2727,6 +2728,9 @@ static int __init arm_smmu_device_probe(struct platform_device *pdev)
>>    	list_add(&smmu->devices, &arm_smmu_devices);
>>    	spin_unlock(&arm_smmu_devices_lock);
>>    
>> +    /* Add to host IOMMU list to initialize vIOMMU for dom0 */
> 
> The indentation looks odd. Is this correctly aligned?
> 

Yes, it's wrong, will fix it.

>> +	add_to_host_iommu_list(ioaddr, iosize, dev_to_dt(pdev));
>> +
>>    	return 0;
>>    
>>    
>> @@ -3058,6 +3062,9 @@ static __init int arm_smmu_dt_init(struct dt_device_node *dev,
>>    
>>    	platform_features &= smmu->features;
>>    
>> +	/* Set vIOMMU type to SMMUv3 */
>> +	vsmmuv3_set_type();
>> +
>>    	return 0;
>>    }
>>    
>> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
>> index 7ab6061e34..53ae46349a 100644
>> --- a/xen/drivers/passthrough/arm/viommu.c
>> +++ b/xen/drivers/passthrough/arm/viommu.c
>> @@ -2,12 +2,42 @@
>>    
>>    #include <xen/errno.h>
>>    #include <xen/init.h>
>> +#include <xen/irq.h>
>>    #include <xen/types.h>
>>    
>>    #include <asm/viommu.h>
>>    
>> +/* List of all host IOMMUs */
>> +LIST_HEAD(host_iommu_list);
> 
> I don't quite follow why this is part of the common code. That said, why
> do we need to register the host IOMMU? Wouldn't it be simpler to go
> through the list of pIOMMU in the vSMMU v3 implementation?
> 

``host_iommu_list`` is part of the generic code to allow reuse for other 
IOMMU types in the future. For example, it can be reused for Renesas 
IPMMU, rather than duplicating it.
As for why we need to register IOMMUs, it seems more suitable to create 
this list at initialization and add IOMMUs with the necessary properties 
when the pIOMMU is probed.
We can't reuse the same list from the SMMU driver because vIOMMU needs 
raw DT properties (address, size), which I don't think we can extract 
from the host driver list.


>> +
>>    const struct viommu_desc __read_mostly *cur_viommu;
>>    
>> +/* Common function for adding to host_iommu_list */
>> +void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>> +                            const struct dt_device_node *node)
> 
> Is this supposed to only be called during __init? If so, this will help
> to justify the ...
> 
>> +{
>> +    struct host_iommu *iommu_data;
>> +
>> +    iommu_data = xzalloc(struct host_iommu);
>> +    if ( !iommu_data )
>> +        panic("vIOMMU: Cannot allocate memory for host IOMMU data\n");
> 
> ... panic(). If not, then this function needs to return an error.
> 

Yes, this is called during init, on pIOMMU driver probe 
(arm_smmu_device_probe())

>> +
>> +    iommu_data->addr = addr;
>> +    iommu_data->size = size;
>> +    iommu_data->dt_node = node;
>> +    iommu_data->irq = platform_get_irq(node, 0);
>> +    if ( iommu_data->irq < 0 )
>> +    {
>> +        gdprintk(XENLOG_ERR,
>> +                 "vIOMMU: Cannot find a valid IOMMU irq\n");
> 
> Shouldn't you free the allocated memory? That said, why is it ok to
> ignore the vIOMMU in this case?
> 

Yes, this is missing. We ignore the host IOMMU with an invalid IRQ 
because event queue emulation won't work in this case.

>> +        return;
>> +    }
>> +
>> +    printk("vIOMMU: Found IOMMU @0x%"PRIx64"\n", addr);
>> +
>> +    list_add_tail(&iommu_data->entry, &host_iommu_list);
>> +}
>> +
>>    int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>>    {
>>        if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> new file mode 100644
>> index 0000000000..6b4009e5ef
>> --- /dev/null
>> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> @@ -0,0 +1,124 @@
>> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>> +
>> +#include <xen/param.h>
>> +#include <xen/sched.h>
>> +#include <asm/mmio.h>
>> +#include <asm/viommu.h>
>> +
>> +/* Struct to hold the vIOMMU ops and vIOMMU type */
>> +extern const struct viommu_desc __read_mostly *cur_viommu;
> 
> Why is this defined in the C file? Shouldn't this be defined in
> ``viomum.h``? That said, looking at the use, I think we want to
> introduce a helper to set the cur_viommu and not export ``cur_viommu``.
> 

I will introduce a helper, set function for ``cur_viommu``. After this 
is done, there's no need to move its definition to ``viommu.h``, since 
it should remain private to the vIOMMU code.

>> +
>> +struct virt_smmu {
>> +    struct      domain *d;
>> +    struct      list_head viommu_list;
>> +};
>> +
>> +static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>> +                              register_t r, void *priv)
>> +{
>> +    return IO_HANDLED;
>> +}
>> +
>> +static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
>> +                             register_t *r, void *priv)
>> +{
>> +    return IO_HANDLED;
>> +}
> 
> I undertand that both helpers are meant to be updated in a later patch.
> However, I think it makes quite difficult to know whether the code is
> complete by the end of the series. I would strongly recommend adding
> 'BUG_ON("unimplemented");' so we can easily grep whether there is
> anything missing by the end.
> 

Sure, I'll add BUG_ON() for the unimplemented functions.

>> +
>> +static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
>> +    .read  = vsmmuv3_mmio_read,
>> +    .write = vsmmuv3_mmio_write,
>> +};
>> +
>> +static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
>> +{
>> +    struct virt_smmu *smmu;
>> +
>> +    smmu = xzalloc(struct virt_smmu);
>> +    if ( !smmu )
>> +        return -ENOMEM;
>> +
>> +    smmu->d = d;
>> +
>> +    register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
> 
> I can't seem to find a place where the number of handler supported is
> incremented (see domain_io_init()) for the vIOMMU. Are you sure we
> always have enough space today?
> 

I'll fix this. The handler count should be incremented during domain 
creation if vIOMMU is enabled.

>> +
>> +    /* Register the vIOMMU to be able to clean it up later. */
>> +    list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
>> +
>> +    return 0;
>> +}
>> +
>> +int domain_vsmmuv3_init(struct domain *d)
>> +{
>> +    int ret;
> 
> Coding style: newline missing.
> 
>> +    INIT_LIST_HEAD(&d->arch.viommu_list);
>> +
>> +    if ( is_hardware_domain(d) )
>> +    {
>> +        struct host_iommu *hw_iommu;
>> +
>> +        list_for_each_entry(hw_iommu, &host_iommu_list, entry)
>> +        {
>> +            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
>> +            if ( ret )
>> +                return ret;
>> +        }
>> +    }
>> +    else
>> +    {
>> +        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
>> +        if ( ret )
>> +            return ret;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +int vsmmuv3_relinquish_resources(struct domain *d)
>> +{
>> +    struct virt_smmu *pos, *temp;
>> +
>> +    /* Cope with unitialized vIOMMU */
>> +    if ( list_head_is_null(&d->arch.viommu_list) )
>> +        return 0;
>> +
>> +    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
>> +    {
>> +        list_del(&pos->viommu_list);
>> +        xfree(pos);
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static const struct viommu_ops vsmmuv3_ops = {
>> +    .domain_init = domain_vsmmuv3_init,
>> +    .relinquish_resources = vsmmuv3_relinquish_resources,
>> +};
>> +
>> +static const struct viommu_desc vsmmuv3_desc = {
>> +    .ops = &vsmmuv3_ops,
>> +    .viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3,
>> +};
>> +
>> +void __init vsmmuv3_set_type(void)
>> +{
>> +    const struct viommu_desc *desc = &vsmmuv3_desc;
>> +
>> +    if ( cur_viommu && (cur_viommu != desc) )
>> +    {
>> +        printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
>> +        return;
>> +    }
>> +
>> +    cur_viommu = desc;
>> +}
>> +
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-file-style: "BSD"
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
>> new file mode 100644
>> index 0000000000..e11f85b431
>> --- /dev/null
>> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
>> @@ -0,0 +1,20 @@
>> +/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>> +#ifndef __ARCH_ARM_VSMMU_V3_H__
>> +#define __ARCH_ARM_VSMMU_V3_H__
>> +
>> +#include <asm/viommu.h>
>> +
>> +#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
>> +
>> +void vsmmuv3_set_type(void);
>> +
>> +#else
>> +
>> +static inline void vsmmuv3_set_type(void)
>> +{
>> +    return;
>> +}
>> +
>> +#endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
>> +
>> +#endif /* __ARCH_ARM_VSMMU_V3_H__ */
>> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
>> index d4953d40fd..ebac02ed63 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -330,7 +330,8 @@ DEFINE_XEN_GUEST_HANDLE(vcpu_guest_context_t);
>>    #define XEN_DOMCTL_CONFIG_ARM_SCI_NONE      0
>>    #define XEN_DOMCTL_CONFIG_ARM_SCI_SCMI_SMC  1
>>    
>> -#define XEN_DOMCTL_CONFIG_VIOMMU_NONE   0
>> +#define XEN_DOMCTL_CONFIG_VIOMMU_NONE       0
> 
> I am generally against trying to align values because this is just
> introducing unnecessary churn in the code.
> 
>> +#define XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3     1
>>    
>>    struct xen_arch_domainconfig {
>>        /* IN/OUT */
>> @@ -456,6 +457,10 @@ typedef uint64_t xen_callback_t;
>>    #define GUEST_GICV3_GICR0_BASE     xen_mk_ullong(0x03020000) /* vCPU0..127 */
>>    #define GUEST_GICV3_GICR0_SIZE     xen_mk_ullong(0x01000000)
>>    
>> +/* vsmmuv3 ITS mappings */
> 
> What is ITS stand for in this context?
> 

This is wrong, should be MMIO mappings instead of ITS. I'll fix this.

>> +#define GUEST_VSMMUV3_BASE     xen_mk_ullong(0x04040000)
>> +#define GUEST_VSMMUV3_SIZE     xen_mk_ullong(0x00040000)
> 
> If I am not mistaken, you are reserving 256KiB. However, looking at the
> SMMU spec (section 6.1, ARM IHI 0070 H.a), the minimum is 128 KiB. Are
> you intending to expose additional features?
> 

You're correct, there are no additional features exposed. The size was 
chosen during the early implementation, but I don't think we need to 
reserve more than the architectural minimum at this point.

>> +
>>    /*
>>     * 256 MB is reserved for VPCI configuration space based on calculation
>>     * 256 buses x 32 devices x 8 functions x 4 KB = 256 MB
> 
> Cheers,
> 

Best regards,
Milan

[PATCH v3 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
enable/disable vIOMMU support for domains.

Also add viommu="N" parameter to xl domain configuration to enable the
vIOMMU for the domains. Currently, only the "smmuv3" type is supported
for ARM.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  7 +++++++
 tools/include/libxl.h                |  5 +++++
 tools/libs/light/libxl_arm.c         | 13 +++++++++++++
 tools/libs/light/libxl_types.idl     |  6 ++++++
 tools/xl/xl_parse.c                  |  9 +++++++++
 7 files changed, 55 insertions(+)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 3aac0bc4fb..4de8db42ac 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3175,6 +3175,19 @@ option.
 
 =back
 
+=over 4
+
+=item B<viommu="N">
+
+To enable viommu, user must specify the following option in the VM
+config file:
+
+viommu = "smmuv3"
+
+Currently, only the "smmuv3" type is supported for ARM.
+
+=back
+
 =head3 x86
 
 =over 4
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index b0c09da910..b24cfd0533 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1273,6 +1273,7 @@ x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
 if err := x.ArchArm.ArmSci.fromC(&xc.arch_arm.arm_sci);err != nil {
 return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
 }
+x.ArchArm.ViommuType = ViommuType(xc.arch_arm.viommu_type)
 if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
@@ -1815,6 +1816,7 @@ xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
 if err := x.ArchArm.ArmSci.toC(&xc.arch_arm.arm_sci); err != nil {
 return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
 }
+xc.arch_arm.viommu_type = C.libxl_viommu_type(x.ArchArm.ViommuType)
 if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
 return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
 }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index e0fd78ec03..e306f9c1ac 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -530,6 +530,12 @@ type ArmSci struct {
 Type ArmSciType
 }
 
+type ViommuType int
+const(
+ViommuTypeNone ViommuType = 0
+ViommuTypeSmmuv3 ViommuType = 1
+)
+
 type RdmReserve struct {
 Strategy RdmReserveStrategy
 Policy RdmReservePolicy
@@ -619,6 +625,7 @@ Vuart VuartType
 SveVl SveType
 NrSpis uint32
 ArmSci ArmSci
+ViommuType ViommuType
 }
 ArchX86 struct {
 MsrRelaxed Defbool
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 80e3ec8de9..231dbff5d9 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -318,6 +318,11 @@
  */
 #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
 
+/*
+ * libxl_domain_build_info has the arch_arm.viommu_type field.
+ */
+#define LIBXL_HAVE_BUILDINFO_ARM_VIOMMU 1
+
 /*
  * LIBXL_HAVE_SOFT_RESET indicates that libxl supports performing
  * 'soft reset' for domains and there is 'soft_reset' shutdown reason
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 7e9f8a1bc3..a248793588 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -247,6 +247,19 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
     }
     LOG(DEBUG, " - SCI type=%u", config->arch.arm_sci_type);
 
+    switch (d_config->b_info.arch_arm.viommu_type) {
+    case LIBXL_VIOMMU_TYPE_NONE:
+        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_NONE;
+        break;
+    case LIBXL_VIOMMU_TYPE_SMMUV3:
+        config->arch.viommu_type = XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3;
+        break;
+    default:
+        LOG(ERROR, "Unknown vIOMMU type %d",
+            d_config->b_info.arch_arm.viommu_type);
+        return ERROR_FAIL;
+    }
+
     return 0;
 }
 
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index a7893460f0..f0a9a21ba4 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -561,6 +561,11 @@ libxl_arm_sci = Struct("arm_sci", [
     ("type", libxl_arm_sci_type),
     ])
 
+libxl_viommu_type = Enumeration("viommu_type", [
+    (0, "none"),
+    (1, "smmuv3")
+    ], init_val = "LIBXL_VIOMMU_TYPE_NONE")
+
 libxl_rdm_reserve = Struct("rdm_reserve", [
     ("strategy",    libxl_rdm_reserve_strategy),
     ("policy",      libxl_rdm_reserve_policy),
@@ -745,6 +750,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
                                ("sve_vl", libxl_sve_type),
                                ("nr_spis", uint32, {'init_val': 'LIBXL_NR_SPIS_DEFAULT'}),
                                ("arm_sci", libxl_arm_sci),
+                               ("viommu_type", libxl_viommu_type),
                               ])),
     ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
                               ])),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 48c72dce9c..f2a2bf4b23 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -3078,6 +3078,15 @@ skip_usbdev:
         }
     }
 
+    if (!xlu_cfg_get_string (config, "viommu", &buf, 1)) {
+        e = libxl_viommu_type_from_string(buf, &b_info->arch_arm.viommu_type);
+        if (e) {
+            fprintf(stderr,
+                    "Unknown vIOMMU type \"%s\" specified\n", buf);
+            exit(-ERROR_FAIL);
+        }
+    }
+
     parse_vkb_list(config, d_config);
 
     d_config->virtios = NULL;
-- 
2.43.0

Re: [PATCH v3 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Jan Beulich]

On 31.03.2026 03:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
> XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
> enable/disable vIOMMU support for domains.
> 
> Also add viommu="N" parameter to xl domain configuration to enable the
> vIOMMU for the domains. Currently, only the "smmuv3" type is supported
> for ARM.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>  docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
>  tools/golang/xenlight/helpers.gen.go |  2 ++
>  tools/golang/xenlight/types.gen.go   |  7 +++++++
>  tools/include/libxl.h                |  5 +++++
>  tools/libs/light/libxl_arm.c         | 13 +++++++++++++
>  tools/libs/light/libxl_types.idl     |  6 ++++++
>  tools/xl/xl_parse.c                  |  9 +++++++++
>  7 files changed, 55 insertions(+)

Nit: The subject prefix suggests a hypervisor change, when this is all docs
and tools.

Jan

Re: [PATCH v3 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Nick Rosbrook]

On Mon, Mar 30, 2026 at 9:52 PM Milan Djokic <milan_djokic@epam.com> wrote:
>
> From: Rahul Singh <rahul.singh@arm.com>
>
> Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
> XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
> enable/disable vIOMMU support for domains.
>
> Also add viommu="N" parameter to xl domain configuration to enable the
> vIOMMU for the domains. Currently, only the "smmuv3" type is supported
> for ARM.
>
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>  docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
>  tools/golang/xenlight/helpers.gen.go |  2 ++
>  tools/golang/xenlight/types.gen.go   |  7 +++++++
>  tools/include/libxl.h                |  5 +++++
>  tools/libs/light/libxl_arm.c         | 13 +++++++++++++
>  tools/libs/light/libxl_types.idl     |  6 ++++++
>  tools/xl/xl_parse.c                  |  9 +++++++++
>  7 files changed, 55 insertions(+)
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 3aac0bc4fb..4de8db42ac 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3175,6 +3175,19 @@ option.
>
>  =back
>
> +=over 4
> +
> +=item B<viommu="N">
> +
> +To enable viommu, user must specify the following option in the VM
> +config file:
> +
> +viommu = "smmuv3"
> +
> +Currently, only the "smmuv3" type is supported for ARM.
> +
> +=back
> +
>  =head3 x86
>
>  =over 4
> diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
> index b0c09da910..b24cfd0533 100644
> --- a/tools/golang/xenlight/helpers.gen.go
> +++ b/tools/golang/xenlight/helpers.gen.go
> @@ -1273,6 +1273,7 @@ x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis)
>  if err := x.ArchArm.ArmSci.fromC(&xc.arch_arm.arm_sci);err != nil {
>  return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>  }
> +x.ArchArm.ViommuType = ViommuType(xc.arch_arm.viommu_type)
>  if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil {
>  return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>  }
> @@ -1815,6 +1816,7 @@ xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis)
>  if err := x.ArchArm.ArmSci.toC(&xc.arch_arm.arm_sci); err != nil {
>  return fmt.Errorf("converting field ArchArm.ArmSci: %v", err)
>  }
> +xc.arch_arm.viommu_type = C.libxl_viommu_type(x.ArchArm.ViommuType)
>  if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil {
>  return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err)
>  }
> diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
> index e0fd78ec03..e306f9c1ac 100644
> --- a/tools/golang/xenlight/types.gen.go
> +++ b/tools/golang/xenlight/types.gen.go
> @@ -530,6 +530,12 @@ type ArmSci struct {
>  Type ArmSciType
>  }
>
> +type ViommuType int
> +const(
> +ViommuTypeNone ViommuType = 0
> +ViommuTypeSmmuv3 ViommuType = 1
> +)
> +
>  type RdmReserve struct {
>  Strategy RdmReserveStrategy
>  Policy RdmReservePolicy
> @@ -619,6 +625,7 @@ Vuart VuartType
>  SveVl SveType
>  NrSpis uint32
>  ArmSci ArmSci
> +ViommuType ViommuType
>  }
>  ArchX86 struct {
>  MsrRelaxed Defbool

This looks right now. For the golang bits:

Acked-by: Nick Rosbrook <enr0n@ubuntu.com>

-Nick

Re: [PATCH v3 06/23] xen/domctl: Add XEN_DOMCTL_CONFIG_VIOMMU_* and viommu config param

[Luca Fancellu]

HI Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add new viommu_type field and field values XEN_DOMCTL_CONFIG_VIOMMU_NONE
> XEN_DOMCTL_CONFIG_VIOMMU_SMMUV3 in xen_arch_domainconfig to
> enable/disable vIOMMU support for domains.
> 
> Also add viommu="N" parameter to xl domain configuration to enable the
> vIOMMU for the domains. Currently, only the "smmuv3" type is supported
> for ARM.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> Acked-by: Nick Rosbrook <enr0n@ubuntu.com>
> ---
> docs/man/xl.cfg.5.pod.in             | 13 +++++++++++++
> tools/golang/xenlight/helpers.gen.go |  2 ++
> tools/golang/xenlight/types.gen.go   |  7 +++++++
> tools/include/libxl.h                |  5 +++++
> tools/libs/light/libxl_arm.c         | 13 +++++++++++++
> tools/libs/light/libxl_types.idl     |  6 ++++++
> tools/xl/xl_parse.c                  |  9 +++++++++
> 7 files changed, 55 insertions(+)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 3aac0bc4fb..4de8db42ac 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3175,6 +3175,19 @@ option.
> 
> =back
> 
> +=over 4
> +
> +=item B<viommu="N">

“N” is misleading, I can think about a number. I think we can have “STRING” here

> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index 80e3ec8de9..231dbff5d9 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -318,6 +318,11 @@
>  */
> #define LIBXL_HAVE_BUILDINFO_ARCH_ARM_SCI 1
> 
> +/*
> + * libxl_domain_build_info has the arch_arm.viommu_type field.
> + */
> +#define LIBXL_HAVE_BUILDINFO_ARM_VIOMMU 1

I think this should be LIBXL_HAVE_BUILDINFO_ARCH_ARM_VIOMMU

Cheers,
Luca

[PATCH v3 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add cmdline boot option "viommu = <string>" to enable or disable the
virtual iommu support for guests on ARM (only viommu="smmuv3" supported
for now).

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 docs/misc/xen-command-line.pandoc      |  9 +++++++++
 xen/arch/arm/include/asm/viommu.h      | 12 ++++++++++++
 xen/drivers/passthrough/arm/viommu.c   | 11 +++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c |  3 +++
 4 files changed, 35 insertions(+)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 6c77129732..6531c2355c 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2850,6 +2850,15 @@ The optional `keep` parameter causes Xen to continue using the vga
 console even after dom0 has been started.  The default behaviour is to
 relinquish control to dom0.
 
+### viommu (arm)
+> `= <string>`
+
+> Default: ``
+
+Flag to enable or disable support for the virtual IOMMU for guests. Disabled by
+default. Enable by specifying target IOMMU type (if supported). Only "smmuv3"
+IOMMU emulation supported at this point.
+
 ### viridian-spinlock-retry-count (x86)
 > `= <integer>`
 
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 2a6742de73..ed338fe0ec 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -10,6 +10,7 @@
 #include <public/xen.h>
 
 extern struct list_head host_iommu_list;
+extern char viommu[];
 
 /* data structure for each hardware IOMMU */
 struct host_iommu {
@@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
 void add_to_host_iommu_list(paddr_t addr, paddr_t size,
                             const struct dt_device_node *node);
 
+static always_inline bool is_viommu_enabled(void)
+{
+    /* only smmuv3 emulation supported */
+    return !strcmp(viommu, "smmuv3");
+}
+
 #else
 
 static inline uint8_t viommu_get_type(void)
@@ -76,6 +83,11 @@ static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
     return;
 }
 
+static always_inline bool is_viommu_enabled(void)
+{
+    return false;
+}
+
 #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
 
 #endif /* __ARCH_ARM_VIOMMU_H__ */
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 53ae46349a..5f5892fbb2 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -3,6 +3,7 @@
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/irq.h>
+#include <xen/param.h>
 #include <xen/types.h>
 
 #include <asm/viommu.h>
@@ -38,8 +39,18 @@ void add_to_host_iommu_list(paddr_t addr, paddr_t size,
     list_add_tail(&iommu_data->entry, &host_iommu_list);
 }
 
+/* By default viommu is disabled.
+ * If enabled, 'viommu' param indicates type (smmuv3 is only supported type atm)
+ */
+char __read_mostly viommu[10] = "";
+string_param("viommu", viommu);
+
 int domain_viommu_init(struct domain *d, uint16_t viommu_type)
 {
+    /* Enable viommu when it has been enabled explicitly (viommu="smmuv3"). */
+    if ( !is_viommu_enabled() )
+        return 0;
+
     if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
         return 0;
 
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 6b4009e5ef..e36f200ba5 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -105,6 +105,9 @@ void __init vsmmuv3_set_type(void)
 {
     const struct viommu_desc *desc = &vsmmuv3_desc;
 
+    if ( !is_viommu_enabled() )
+        return;
+
     if ( cur_viommu && (cur_viommu != desc) )
     {
         printk("WARNING: Cannot set vIOMMU, already set to a different value\n");
-- 
2.43.0

Re: [PATCH v3 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Luca Fancellu]

HI Milan,

> 
> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
> index 2a6742de73..ed338fe0ec 100644
> --- a/xen/arch/arm/include/asm/viommu.h
> +++ b/xen/arch/arm/include/asm/viommu.h
> @@ -10,6 +10,7 @@
> #include <public/xen.h>
> 
> extern struct list_head host_iommu_list;
> +extern char viommu[];

As far as I can see this one is used only in this header by ...

> 
> /* data structure for each hardware IOMMU */
> struct host_iommu {
> @@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
> void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>                             const struct dt_device_node *node);
> 
> +static always_inline bool is_viommu_enabled(void)
> +{
> +    /* only smmuv3 emulation supported */
> +    return !strcmp(viommu, "smmuv3");
> +}

this function, it seems cleaner to me if viommu is static inside xen/xen/drivers/passthrough/arm/viommu.c
and this one can be defined there as well, so here we will have only the declaration.

Cheers,
Luca

Re: [PATCH v3 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Milan Djokic]

Hi Luca,

On 4/10/26 16:28, Luca Fancellu wrote:
> HI Milan,
> 
>>
>> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
>> index 2a6742de73..ed338fe0ec 100644
>> --- a/xen/arch/arm/include/asm/viommu.h
>> +++ b/xen/arch/arm/include/asm/viommu.h
>> @@ -10,6 +10,7 @@
>> #include <public/xen.h>
>>
>> extern struct list_head host_iommu_list;
>> +extern char viommu[];
> 
> As far as I can see this one is used only in this header by ...
> 
>>
>> /* data structure for each hardware IOMMU */
>> struct host_iommu {
>> @@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
>> void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>>                              const struct dt_device_node *node);
>>
>> +static always_inline bool is_viommu_enabled(void)
>> +{
>> +    /* only smmuv3 emulation supported */
>> +    return !strcmp(viommu, "smmuv3");
>> +}
> 
> this function, it seems cleaner to me if viommu is static inside xen/xen/drivers/passthrough/arm/viommu.c
> and this one can be defined there as well, so here we will have only the declaration.
> 

I will remove this patch, including the additional boot option and 
is_viommu_enabled(). Relying on the existing vIOMMU config options 
should be sufficient for vIOMMU enable/disable control.

> Cheers,
> Luca
> 

Best regards,
Milan

Re: [PATCH v3 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add cmdline boot option "viommu = <string>" to enable or disable the
> virtual iommu support for guests on ARM (only viommu="smmuv3" supported
> for now).

In Xen terminology, 'guests' refers to domUs. IOW, this doesn't include 
dom0. Is this what you meant? If so, how would you enable it for dom0?

That said, is there any particular reason why this can't be 
automatically enabled based on the SMMUv3 discovered?

> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   docs/misc/xen-command-line.pandoc      |  9 +++++++++
>   xen/arch/arm/include/asm/viommu.h      | 12 ++++++++++++
>   xen/drivers/passthrough/arm/viommu.c   | 11 +++++++++++
>   xen/drivers/passthrough/arm/vsmmu-v3.c |  3 +++
>   4 files changed, 35 insertions(+)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 6c77129732..6531c2355c 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -2850,6 +2850,15 @@ The optional `keep` parameter causes Xen to continue using the vga
>   console even after dom0 has been started.  The default behaviour is to
>   relinquish control to dom0.
>   
> +### viommu (arm)
> +> `= <string>`
> +
> +> Default: ``
> +
> +Flag to enable or disable support for the virtual IOMMU for guests. Disabled by
> +default. Enable by specifying target IOMMU type (if supported). Only "smmuv3"
> +IOMMU emulation supported at this point.
> +
>   ### viridian-spinlock-retry-count (x86)
>   > `= <integer>`
>   
> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
> index 2a6742de73..ed338fe0ec 100644
> --- a/xen/arch/arm/include/asm/viommu.h
> +++ b/xen/arch/arm/include/asm/viommu.h
> @@ -10,6 +10,7 @@
>   #include <public/xen.h>
>   
>   extern struct list_head host_iommu_list;
> +extern char viommu[];
>   
>   /* data structure for each hardware IOMMU */
>   struct host_iommu {
> @@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
>   void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>                               const struct dt_device_node *node);
>   
> +static always_inline bool is_viommu_enabled(void)

Regardless what Luca wrote, why do we need to force "always_inline"?

> +{
> +    /* only smmuv3 emulation supported */
> +    return !strcmp(viommu, "smmuv3");
> +}
> +
>   #else
>   
>   static inline uint8_t viommu_get_type(void)
> @@ -76,6 +83,11 @@ static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>       return;
>   }
>   
> +static always_inline bool is_viommu_enabled(void)
> +{
> +    return false;
> +}
> +
>   #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
>   
>   #endif /* __ARCH_ARM_VIOMMU_H__ */
> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
> index 53ae46349a..5f5892fbb2 100644
> --- a/xen/drivers/passthrough/arm/viommu.c
> +++ b/xen/drivers/passthrough/arm/viommu.c
> @@ -3,6 +3,7 @@
>   #include <xen/errno.h>
>   #include <xen/init.h>
>   #include <xen/irq.h>
> +#include <xen/param.h>
>   #include <xen/types.h>
>   
>   #include <asm/viommu.h>
> @@ -38,8 +39,18 @@ void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>       list_add_tail(&iommu_data->entry, &host_iommu_list);
>   }
>   
> +/* By default viommu is disabled.
> + * If enabled, 'viommu' param indicates type (smmuv3 is only supported type atm)
> + */
> +char __read_mostly viommu[10] = "";
> +string_param("viommu", viommu);
> +
>   int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>   {
> +    /* Enable viommu when it has been enabled explicitly (viommu="smmuv3"). */
> +    if ( !is_viommu_enabled() )
 > +        return 0;> +
>       if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>           return 0;
>   
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 6b4009e5ef..e36f200ba5 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -105,6 +105,9 @@ void __init vsmmuv3_set_type(void)
>   {
>       const struct viommu_desc *desc = &vsmmuv3_desc;
>   
> +    if ( !is_viommu_enabled() )

This is likely going to go wrong in the future if we add support for 
other vIOMMU in the future. If we decide to continue using the command 
line option (see above), you would want want an helper to return the 
selected emulation and check against SMMUv3 here.

Cheers,

-- 
Julien Grall

Re: [PATCH v3 07/23] xen/arm: vIOMMU: Add cmdline boot option "viommu = <string>"

[Milan Djokic]

Hi Julien,

On 4/14/26 09:18, Julien Grall wrote:
> Hi Milan,
> 
> On 31/03/2026 10:52, Milan Djokic wrote:
>> From: Rahul Singh <rahul.singh@arm.com>
>>
>> Add cmdline boot option "viommu = <string>" to enable or disable the
>> virtual iommu support for guests on ARM (only viommu="smmuv3" supported
>> for now).
> 
> In Xen terminology, 'guests' refers to domUs. IOW, this doesn't include
> dom0. Is this what you meant? If so, how would you enable it for dom0?
> 

This boot option enables vIOMMU initialization in Xen itself and also 
enables vIOMMU for dom0. For domUs, vIOMMU is enabled via the guest 
configuration file.
The exception is dom0less mode, where setting this boot option enables 
vIOMMU for all guests.

> That said, is there any particular reason why this can't be
> automatically enabled based on the SMMUv3 discovered?
> 

This boot option was intended to provide a runtime enable/disable 
mechanism for vIOMMU even when the vIOMMU build options 
(CONFIG_VIRTUAL_IOMMU / CONFIG_VIRTUAL_ARM_SMMU_V3) are enabled.

However, this may be unnecessary, and relying on the build options for 
enabling vIOMMU seems sufficient. I will remove this commit from the 
patch series.


>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>>    docs/misc/xen-command-line.pandoc      |  9 +++++++++
>>    xen/arch/arm/include/asm/viommu.h      | 12 ++++++++++++
>>    xen/drivers/passthrough/arm/viommu.c   | 11 +++++++++++
>>    xen/drivers/passthrough/arm/vsmmu-v3.c |  3 +++
>>    4 files changed, 35 insertions(+)
>>
>> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
>> index 6c77129732..6531c2355c 100644
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -2850,6 +2850,15 @@ The optional `keep` parameter causes Xen to continue using the vga
>>    console even after dom0 has been started.  The default behaviour is to
>>    relinquish control to dom0.
>>    
>> +### viommu (arm)
>> +> `= <string>`
>> +
>> +> Default: ``
>> +
>> +Flag to enable or disable support for the virtual IOMMU for guests. Disabled by
>> +default. Enable by specifying target IOMMU type (if supported). Only "smmuv3"
>> +IOMMU emulation supported at this point.
>> +
>>    ### viridian-spinlock-retry-count (x86)
>>    > `= <integer>`
>>    
>> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
>> index 2a6742de73..ed338fe0ec 100644
>> --- a/xen/arch/arm/include/asm/viommu.h
>> +++ b/xen/arch/arm/include/asm/viommu.h
>> @@ -10,6 +10,7 @@
>>    #include <public/xen.h>
>>    
>>    extern struct list_head host_iommu_list;
>> +extern char viommu[];
>>    
>>    /* data structure for each hardware IOMMU */
>>    struct host_iommu {
>> @@ -50,6 +51,12 @@ uint16_t viommu_get_type(void);
>>    void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>>                                const struct dt_device_node *node);
>>    
>> +static always_inline bool is_viommu_enabled(void)
> 
> Regardless what Luca wrote, why do we need to force "always_inline"?
> 

I don't think it's necessary to use always_inline here, anyway this 
function will be removed.

>> +{
>> +    /* only smmuv3 emulation supported */
>> +    return !strcmp(viommu, "smmuv3");
>> +}
>> +
>>    #else
>>    
>>    static inline uint8_t viommu_get_type(void)
>> @@ -76,6 +83,11 @@ static inline void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>>        return;
>>    }
>>    
>> +static always_inline bool is_viommu_enabled(void)
>> +{
>> +    return false;
>> +}
>> +
>>    #endif /* CONFIG_ARM_VIRTUAL_IOMMU */
>>    
>>    #endif /* __ARCH_ARM_VIOMMU_H__ */
>> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
>> index 53ae46349a..5f5892fbb2 100644
>> --- a/xen/drivers/passthrough/arm/viommu.c
>> +++ b/xen/drivers/passthrough/arm/viommu.c
>> @@ -3,6 +3,7 @@
>>    #include <xen/errno.h>
>>    #include <xen/init.h>
>>    #include <xen/irq.h>
>> +#include <xen/param.h>
>>    #include <xen/types.h>
>>    
>>    #include <asm/viommu.h>
>> @@ -38,8 +39,18 @@ void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>>        list_add_tail(&iommu_data->entry, &host_iommu_list);
>>    }
>>    
>> +/* By default viommu is disabled.
>> + * If enabled, 'viommu' param indicates type (smmuv3 is only supported type atm)
>> + */
>> +char __read_mostly viommu[10] = "";
>> +string_param("viommu", viommu);
>> +
>>    int domain_viommu_init(struct domain *d, uint16_t viommu_type)
>>    {
>> +    /* Enable viommu when it has been enabled explicitly (viommu="smmuv3"). */
>> +    if ( !is_viommu_enabled() )
>   > +        return 0;> +
>>        if ( viommu_type == XEN_DOMCTL_CONFIG_VIOMMU_NONE )
>>            return 0;
>>    
>> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> index 6b4009e5ef..e36f200ba5 100644
>> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
>> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
>> @@ -105,6 +105,9 @@ void __init vsmmuv3_set_type(void)
>>    {
>>        const struct viommu_desc *desc = &vsmmuv3_desc;
>>    
>> +    if ( !is_viommu_enabled() )
> 
> This is likely going to go wrong in the future if we add support for
> other vIOMMU in the future. If we decide to continue using the command
> line option (see above), you would want want an helper to return the
> selected emulation and check against SMMUv3 here.
> 

Yes, I will remove the command-line option and use the viommu_get_type() 
helper where needed to check the selected vIOMMU emulation type.

> Cheers,
> 

Best regards,
Milan

[PATCH v3 09/23] xen/arm: vsmmuv3: Add support for cmdqueue handling

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add support for virtual cmdqueue handling for guests

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/vsmmu-v3.c | 101 +++++++++++++++++++++++++
 1 file changed, 101 insertions(+)

diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 3ae1e62a50..02fe6a4422 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
 
+#include <xen/guest_access.h>
 #include <xen/param.h>
 #include <xen/sched.h>
 #include <asm/mmio.h>
@@ -25,6 +26,26 @@
 /* Struct to hold the vIOMMU ops and vIOMMU type */
 extern const struct viommu_desc __read_mostly *cur_viommu;
 
+/* SMMUv3 command definitions */
+#define CMDQ_OP_PREFETCH_CFG    0x1
+#define CMDQ_OP_CFGI_STE        0x3
+#define CMDQ_OP_CFGI_ALL        0x4
+#define CMDQ_OP_CFGI_CD         0x5
+#define CMDQ_OP_CFGI_CD_ALL     0x6
+#define CMDQ_OP_TLBI_NH_ASID    0x11
+#define CMDQ_OP_TLBI_NH_VA      0x12
+#define CMDQ_OP_TLBI_NSNH_ALL   0x30
+#define CMDQ_OP_CMD_SYNC        0x46
+
+/* Queue Handling */
+#define Q_BASE(q)       ((q)->q_base & Q_BASE_ADDR_MASK)
+#define Q_CONS_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->cons) * (q)->ent_size)
+#define Q_PROD_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->prod) * (q)->ent_size)
+
+/* Helper Macros */
+#define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
+#define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
+
 /* virtual smmu queue */
 struct arm_vsmmu_queue {
     uint64_t    q_base; /* base register */
@@ -49,8 +70,80 @@ struct virt_smmu {
     uint64_t    gerror_irq_cfg0;
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
+    spinlock_t  cmd_queue_lock;
 };
 
+/* Queue manipulation functions */
+static bool queue_empty(struct arm_vsmmu_queue *q)
+{
+    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
+           Q_WRP(q, q->prod) == Q_WRP(q, q->cons);
+}
+
+static void queue_inc_cons(struct arm_vsmmu_queue *q)
+{
+    uint32_t cons = (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1;
+    q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
+}
+
+static void dump_smmu_command(uint64_t *command)
+{
+    gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
+             smmu_cmd_get_command(command[0]), command[0], command[1]);
+}
+static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
+{
+    struct arm_vsmmu_queue *q = &smmu->cmdq;
+    struct domain *d = smmu->d;
+    uint64_t command[CMDQ_ENT_DWORDS];
+    paddr_t addr;
+
+    if ( !smmu_get_cmdq_enabled(smmu->cr[0]) )
+        return 0;
+
+    while ( !queue_empty(q) )
+    {
+        int ret;
+
+        addr = Q_CONS_ENT(q);
+        ret = access_guest_memory_by_gpa(d, addr, command,
+                                         sizeof(command), false);
+        if ( ret )
+            return ret;
+
+        switch ( smmu_cmd_get_command(command[0]) )
+        {
+        case CMDQ_OP_CFGI_STE:
+            break;
+        case CMDQ_OP_PREFETCH_CFG:
+        case CMDQ_OP_CFGI_CD:
+        case CMDQ_OP_CFGI_CD_ALL:
+        case CMDQ_OP_CFGI_ALL:
+        case CMDQ_OP_CMD_SYNC:
+            break;
+        case CMDQ_OP_TLBI_NH_ASID:
+        case CMDQ_OP_TLBI_NSNH_ALL:
+        case CMDQ_OP_TLBI_NH_VA:
+            if ( !iommu_iotlb_flush_all(smmu->d, 1) )
+                break;
+        default:
+            gdprintk(XENLOG_ERR, "vSMMUv3: unhandled command\n");
+            dump_smmu_command(command);
+            break;
+        }
+
+        if ( ret )
+        {
+            gdprintk(XENLOG_ERR,
+                     "vSMMUv3: command error %d while handling command\n",
+                     ret);
+            dump_smmu_command(command);
+        }
+        queue_inc_cons(q);
+    }
+    return 0;
+}
+
 static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
                               register_t r, void *priv)
 {
@@ -104,9 +197,15 @@ static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
         break;
 
     case VREG32(ARM_SMMU_CMDQ_PROD):
+        spin_lock(&smmu->cmd_queue_lock);
         reg32 = smmu->cmdq.prod;
         vreg_reg32_update(&reg32, r, info);
         smmu->cmdq.prod = reg32;
+
+        if ( arm_vsmmu_handle_cmds(smmu) )
+            gdprintk(XENLOG_ERR, "error handling vSMMUv3 commands\n");
+
+        spin_unlock(&smmu->cmd_queue_lock);
         break;
 
     case VREG32(ARM_SMMU_CMDQ_CONS):
@@ -326,6 +425,8 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
 
+    spin_lock_init(&smmu->cmd_queue_lock);
+
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
     /* Register the vIOMMU to be able to clean it up later. */
-- 
2.43.0

Re: [PATCH v3 09/23] xen/arm: vsmmuv3: Add support for cmdqueue handling

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add support for virtual cmdqueue handling for guests
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/drivers/passthrough/arm/vsmmu-v3.c | 101 +++++++++++++++++++++++++
> 1 file changed, 101 insertions(+)
> 
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 3ae1e62a50..02fe6a4422 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -1,5 +1,6 @@
> /* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> 
> +#include <xen/guest_access.h>
> #include <xen/param.h>
> #include <xen/sched.h>
> #include <asm/mmio.h>
> @@ -25,6 +26,26 @@
> /* Struct to hold the vIOMMU ops and vIOMMU type */
> extern const struct viommu_desc __read_mostly *cur_viommu;
> 
> +/* SMMUv3 command definitions */
> +#define CMDQ_OP_PREFETCH_CFG    0x1
> +#define CMDQ_OP_CFGI_STE        0x3
> +#define CMDQ_OP_CFGI_ALL        0x4
> +#define CMDQ_OP_CFGI_CD         0x5
> +#define CMDQ_OP_CFGI_CD_ALL     0x6
> +#define CMDQ_OP_TLBI_NH_ASID    0x11
> +#define CMDQ_OP_TLBI_NH_VA      0x12
> +#define CMDQ_OP_TLBI_NSNH_ALL   0x30
> +#define CMDQ_OP_CMD_SYNC        0x46
> +
> +/* Queue Handling */
> +#define Q_BASE(q)       ((q)->q_base & Q_BASE_ADDR_MASK)
> +#define Q_CONS_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->cons) * (q)->ent_size)
> +#define Q_PROD_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->prod) * (q)->ent_size)
> +
> +/* Helper Macros */
> +#define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
> +#define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
> +
> /* virtual smmu queue */
> struct arm_vsmmu_queue {
>     uint64_t    q_base; /* base register */
> @@ -49,8 +70,80 @@ struct virt_smmu {
>     uint64_t    gerror_irq_cfg0;
>     uint64_t    evtq_irq_cfg0;
>     struct      arm_vsmmu_queue evtq, cmdq;
> +    spinlock_t  cmd_queue_lock;
> };
> 
> +/* Queue manipulation functions */
> +static bool queue_empty(struct arm_vsmmu_queue *q)
> +{
> +    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
> +           Q_WRP(q, q->prod) == Q_WRP(q, q->cons);
> +}
> +
> +static void queue_inc_cons(struct arm_vsmmu_queue *q)
> +{
> +    uint32_t cons = (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1;
> +    q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
> +}
> +
> +static void dump_smmu_command(uint64_t *command)
> +{
> +    gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",

I think using PRIx64 is preferred

> +             smmu_cmd_get_command(command[0]), command[0], command[1]);
> +}
> +static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
> +{
> +    struct arm_vsmmu_queue *q = &smmu->cmdq;
> +    struct domain *d = smmu->d;
> +    uint64_t command[CMDQ_ENT_DWORDS];
> +    paddr_t addr;

can we declare ‘int ret = 0;’ here and ...

> +
> +    if ( !smmu_get_cmdq_enabled(smmu->cr[0]) )
> +        return 0;
> +
> +    while ( !queue_empty(q) )
> +    {
> +        int ret;

remove this

> +
> +        addr = Q_CONS_ENT(q);
> +        ret = access_guest_memory_by_gpa(d, addr, command,
> +                                         sizeof(command), false);
> +        if ( ret )
> +            return ret;

Here we should at least have queue_inc_cons(q); before returning, since
otherwise we will indefinitely execute on the same element.

> +
> +        switch ( smmu_cmd_get_command(command[0]) )
> +        {
> +        case CMDQ_OP_CFGI_STE:
> +            break;
> +        case CMDQ_OP_PREFETCH_CFG:
> +        case CMDQ_OP_CFGI_CD:
> +        case CMDQ_OP_CFGI_CD_ALL:
> +        case CMDQ_OP_CFGI_ALL:
> +        case CMDQ_OP_CMD_SYNC:
> +            break;
> +        case CMDQ_OP_TLBI_NH_ASID:
> +        case CMDQ_OP_TLBI_NSNH_ALL:
> +        case CMDQ_OP_TLBI_NH_VA:
> +            if ( !iommu_iotlb_flush_all(smmu->d, 1) )
> +                break;

ret = iommu_iotlb_flush_all(smmu->d, 1);
if ( !ret )
    break;

> +        default:
> +            gdprintk(XENLOG_ERR, "vSMMUv3: unhandled command\n");
> +            dump_smmu_command(command);
> +            break;
> +        }
> +
> +        if ( ret )
> +        {
> +            gdprintk(XENLOG_ERR,
> +                     "vSMMUv3: command error %d while handling command\n",
> +                     ret);
> +            dump_smmu_command(command);
> +        }
> +        queue_inc_cons(q);
> +    }
> +    return 0;

return ret;

In this way we don’t suppress the iotbl error but we propagate to the caller.

> +}
> +
> static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>                               register_t r, void *priv)
> {
> @@ -104,9 +197,15 @@ static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>         break;
> 
>     case VREG32(ARM_SMMU_CMDQ_PROD):
> +        spin_lock(&smmu->cmd_queue_lock);
>         reg32 = smmu->cmdq.prod;
>         vreg_reg32_update(&reg32, r, info);
>         smmu->cmdq.prod = reg32;
> +
> +        if ( arm_vsmmu_handle_cmds(smmu) )
> +            gdprintk(XENLOG_ERR, "error handling vSMMUv3 commands\n");
> +
> +        spin_unlock(&smmu->cmd_queue_lock);
>         break;
> 
>     case VREG32(ARM_SMMU_CMDQ_CONS):
> @@ -326,6 +425,8 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
>     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
>     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
> 
> +    spin_lock_init(&smmu->cmd_queue_lock);
> +
>     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
> 
>     /* Register the vIOMMU to be able to clean it up later. */
> 

Cheers,
Luca

Re: [PATCH v3 09/23] xen/arm: vsmmuv3: Add support for cmdqueue handling

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add support for virtual cmdqueue handling for guests

This commit message is quite light. There are quite a few pitfalss with 
the command queue because it can be long running which require some 
explaining on the plan to handle it.

If this is delayed for later, then it would be useful to document in the 
code what's missing so it is easier to know whether the vSMMUv3 
implementation can be security supported (I assume this will be the goal).

> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/drivers/passthrough/arm/vsmmu-v3.c | 101 +++++++++++++++++++++++++
>   1 file changed, 101 insertions(+)
> 
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 3ae1e62a50..02fe6a4422 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -1,5 +1,6 @@
>   /* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
>   
> +#include <xen/guest_access.h>
>   #include <xen/param.h>
>   #include <xen/sched.h>
>   #include <asm/mmio.h>
> @@ -25,6 +26,26 @@
>   /* Struct to hold the vIOMMU ops and vIOMMU type */
>   extern const struct viommu_desc __read_mostly *cur_viommu;
>   
> +/* SMMUv3 command definitions */
> +#define CMDQ_OP_PREFETCH_CFG    0x1
> +#define CMDQ_OP_CFGI_STE        0x3
> +#define CMDQ_OP_CFGI_ALL        0x4
> +#define CMDQ_OP_CFGI_CD         0x5
> +#define CMDQ_OP_CFGI_CD_ALL     0x6
> +#define CMDQ_OP_TLBI_NH_ASID    0x11
> +#define CMDQ_OP_TLBI_NH_VA      0x12
> +#define CMDQ_OP_TLBI_NSNH_ALL   0x30
> +#define CMDQ_OP_CMD_SYNC        0x46
> +
> +/* Queue Handling */
> +#define Q_BASE(q)       ((q)->q_base & Q_BASE_ADDR_MASK)
> +#define Q_CONS_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->cons) * (q)->ent_size)
> +#define Q_PROD_ENT(q)   (Q_BASE(q) + Q_IDX(q, (q)->prod) * (q)->ent_size)
> +
> +/* Helper Macros */
> +#define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
> +#define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
> +
>   /* virtual smmu queue */
>   struct arm_vsmmu_queue {
>       uint64_t    q_base; /* base register */
> @@ -49,8 +70,80 @@ struct virt_smmu {
>       uint64_t    gerror_irq_cfg0;
>       uint64_t    evtq_irq_cfg0;
>       struct      arm_vsmmu_queue evtq, cmdq;
> +    spinlock_t  cmd_queue_lock;
>   };
>   
> +/* Queue manipulation functions */
> +static bool queue_empty(struct arm_vsmmu_queue *q)
> +{
> +    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
> +           Q_WRP(q, q->prod) == Q_WRP(q, q->cons);
> +}
> +
> +static void queue_inc_cons(struct arm_vsmmu_queue *q)
> +{
> +    uint32_t cons = (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1;
> +    q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
> +}
> +
> +static void dump_smmu_command(uint64_t *command)
> +{
> +    gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
> +             smmu_cmd_get_command(command[0]), command[0], command[1]);

I would consider using gprintk() because this could be useful even in 
non-production build.

> +}
> +static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
> +{
> +    struct arm_vsmmu_queue *q = &smmu->cmdq;
> +    struct domain *d = smmu->d;
> +    uint64_t command[CMDQ_ENT_DWORDS];
> +    paddr_t addr;
> +
> +    if ( !smmu_get_cmdq_enabled(smmu->cr[0]) )
> +        return 0;
> +
> +    while ( !queue_empty(q) )
 > +    {> +        int ret;
> +
> +        addr = Q_CONS_ENT(q);
> +        ret = access_guest_memory_by_gpa(d, addr, command,
> +                                         sizeof(command), false);
> +        if ( ret )
> +            return ret;
> +
> +        switch ( smmu_cmd_get_command(command[0]) )
> +        {
> +        case CMDQ_OP_CFGI_STE:
> +            break;
> +        case CMDQ_OP_PREFETCH_CFG:
> +        case CMDQ_OP_CFGI_CD:
> +        case CMDQ_OP_CFGI_CD_ALL:
> +        case CMDQ_OP_CFGI_ALL:
> +        case CMDQ_OP_CMD_SYNC:

Is this empty because there is nothing to do? Or is this empty because 
they are not yet implemented?

> +            break;
> +        case CMDQ_OP_TLBI_NH_ASID:
> +        case CMDQ_OP_TLBI_NSNH_ALL:
> +        case CMDQ_OP_TLBI_NH_VA:
> +            if ( !iommu_iotlb_flush_all(smmu->d, 1) )

This is quite a bigger hammer when the guest may only want to flush the 
S1 TLB for a single device. I am ok for now, but it would be good to add 
a TODO for optimizing it.

> +                break;
> +        default:
> +            gdprintk(XENLOG_ERR, "vSMMUv3: unhandled command\n");
> +            dump_smmu_command(command);
> +            break;
> +        }
> +
> +        if ( ret )
> +        {
> +            gdprintk(XENLOG_ERR,
> +                     "vSMMUv3: command error %d while handling command\n",
> +                     ret);
> +            dump_smmu_command(command);
> +        }
> +        queue_inc_cons(q);
> +    }
> +    return 0;
> +}
> +
>   static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>                                 register_t r, void *priv)
>   {
> @@ -104,9 +197,15 @@ static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>           break;
>   
>       case VREG32(ARM_SMMU_CMDQ_PROD):
> +        spin_lock(&smmu->cmd_queue_lock);
>           reg32 = smmu->cmdq.prod;
>           vreg_reg32_update(&reg32, r, info);
>           smmu->cmdq.prod = reg32;
> +
> +        if ( arm_vsmmu_handle_cmds(smmu) )
> +            gdprintk(XENLOG_ERR, "error handling vSMMUv3 commands\n");
> +
> +        spin_unlock(&smmu->cmd_queue_lock);
>           break;
>   
>       case VREG32(ARM_SMMU_CMDQ_CONS):
> @@ -326,6 +425,8 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
>       smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
>       smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
>   
> +    spin_lock_init(&smmu->cmd_queue_lock);
> +
>       register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
>   
>       /* Register the vIOMMU to be able to clean it up later. */

Cheers,

-- 
Julien Grall

[PATCH v3 08/23] xen/arm: vsmmuv3: Add support for registers emulation

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Add initial support for various emulated registers for virtual SMMUv3
for guests and also add support for virtual cmdq and eventq.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.h  |   6 +
 xen/drivers/passthrough/arm/vsmmu-v3.c | 286 +++++++++++++++++++++++++
 2 files changed, 292 insertions(+)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index 3fb13b7e21..fab4fd5a26 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -60,6 +60,12 @@
 #define IDR5_VAX			GENMASK(11, 10)
 #define IDR5_VAX_52_BIT			1
 
+#define ARM_SMMU_IIDR			0x18
+#define IIDR_PRODUCTID			GENMASK(31, 20)
+#define IIDR_VARIANT			GENMASK(19, 16)
+#define IIDR_REVISION			GENMASK(15, 12)
+#define IIDR_IMPLEMENTER		GENMASK(11, 0)
+
 #define ARM_SMMU_CR0			0x20
 #define CR0_ATSCHK			(1 << 4)
 #define CR0_CMDQEN			(1 << 3)
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index e36f200ba5..3ae1e62a50 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -3,25 +3,307 @@
 #include <xen/param.h>
 #include <xen/sched.h>
 #include <asm/mmio.h>
+#include <asm/vgic-emul.h>
 #include <asm/viommu.h>
+#include <asm/vreg.h>
+
+#include "smmu-v3.h"
+
+/* Register Definition */
+#define ARM_SMMU_IDR2       0x8
+#define ARM_SMMU_IDR3       0xc
+#define ARM_SMMU_IDR4       0x10
+#define IDR0_TERM_MODEL     (1 << 26)
+#define IDR3_RIL            (1 << 10)
+#define CR0_RESERVED        0xFFFFFC20
+#define SMMU_IDR1_SIDSIZE   16
+#define SMMU_CMDQS          19
+#define SMMU_EVTQS          19
+#define DWORDS_BYTES        8
+#define ARM_SMMU_IIDR_VAL   0x12
 
 /* Struct to hold the vIOMMU ops and vIOMMU type */
 extern const struct viommu_desc __read_mostly *cur_viommu;
 
+/* virtual smmu queue */
+struct arm_vsmmu_queue {
+    uint64_t    q_base; /* base register */
+    uint32_t    prod;
+    uint32_t    cons;
+    uint8_t     ent_size;
+    uint8_t     max_n_shift;
+};
+
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
+    uint8_t     sid_split;
+    uint32_t    features;
+    uint32_t    cr[3];
+    uint32_t    cr0ack;
+    uint32_t    gerror;
+    uint32_t    gerrorn;
+    uint32_t    strtab_base_cfg;
+    uint64_t    strtab_base;
+    uint32_t    irq_ctrl;
+    uint64_t    gerror_irq_cfg0;
+    uint64_t    evtq_irq_cfg0;
+    struct      arm_vsmmu_queue evtq, cmdq;
 };
 
 static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
                               register_t r, void *priv)
 {
+    struct virt_smmu *smmu = priv;
+    uint64_t reg;
+    uint32_t reg32;
+
+    switch ( info->gpa & 0xffff )
+    {
+    case VREG32(ARM_SMMU_CR0):
+        reg32 = smmu->cr[0];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[0] = reg32;
+        smmu->cr0ack = reg32 & ~CR0_RESERVED;
+        break;
+
+    case VREG32(ARM_SMMU_CR1):
+        reg32 = smmu->cr[1];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[1] = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_CR2):
+        reg32 = smmu->cr[2];
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cr[2] = reg32;
+        break;
+
+    case VREG64(ARM_SMMU_STRTAB_BASE):
+        reg = smmu->strtab_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->strtab_base = reg;
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
+        reg32 = smmu->strtab_base_cfg;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->strtab_base_cfg = reg32;
+
+        smmu->sid_split = FIELD_GET(STRTAB_BASE_CFG_SPLIT, reg32);
+        smmu->features |= STRTAB_BASE_CFG_FMT_2LVL;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_BASE):
+        reg = smmu->cmdq.q_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->cmdq.q_base = reg;
+        smmu->cmdq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->cmdq.q_base);
+        if ( smmu->cmdq.max_n_shift > SMMU_CMDQS )
+            smmu->cmdq.max_n_shift = SMMU_CMDQS;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_PROD):
+        reg32 = smmu->cmdq.prod;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cmdq.prod = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_CONS):
+        reg32 = smmu->cmdq.cons;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->cmdq.cons = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_BASE):
+        reg = smmu->evtq.q_base;
+        vreg_reg64_update(&reg, r, info);
+        smmu->evtq.q_base = reg;
+        smmu->evtq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->evtq.q_base);
+        if ( smmu->cmdq.max_n_shift > SMMU_EVTQS )
+            smmu->cmdq.max_n_shift = SMMU_EVTQS;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_PROD):
+        reg32 = smmu->evtq.prod;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->evtq.prod = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_CONS):
+        reg32 = smmu->evtq.cons;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->evtq.cons = reg32;
+        break;
+
+    case VREG32(ARM_SMMU_IRQ_CTRL):
+        reg32 = smmu->irq_ctrl;
+        vreg_reg32_update(&reg32, r, info);
+        smmu->irq_ctrl = reg32;
+        break;
+
+    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
+        reg = smmu->gerror_irq_cfg0;
+        vreg_reg64_update(&reg, r, info);
+        smmu->gerror_irq_cfg0 = reg;
+        break;
+
+    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
+        reg = smmu->evtq_irq_cfg0;
+        vreg_reg64_update(&reg, r, info);
+        smmu->evtq_irq_cfg0 = reg;
+        break;
+
+    case VREG32(ARM_SMMU_GERRORN):
+        reg = smmu->gerrorn;
+        vreg_reg64_update(&reg, r, info);
+        smmu->gerrorn = reg;
+        break;
+
+    default:
+        printk(XENLOG_G_ERR
+               "%pv: vSMMUv3: unhandled write r%d offset %"PRIpaddr"\n",
+               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+        return IO_ABORT;
+    }
+
     return IO_HANDLED;
 }
 
 static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
                              register_t *r, void *priv)
 {
+    struct virt_smmu *smmu = priv;
+    uint64_t reg;
+
+    switch ( info->gpa & 0xffff )
+    {
+    case VREG32(ARM_SMMU_IDR0):
+        reg  = FIELD_PREP(IDR0_S1P, 1) | FIELD_PREP(IDR0_TTF, 2) |
+            FIELD_PREP(IDR0_COHACC, 0) | FIELD_PREP(IDR0_ASID16, 1) |
+            FIELD_PREP(IDR0_TTENDIAN, 0) | FIELD_PREP(IDR0_STALL_MODEL, 1) |
+            FIELD_PREP(IDR0_ST_LVL, 1) | FIELD_PREP(IDR0_TERM_MODEL, 1);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR1):
+        reg  = FIELD_PREP(IDR1_SIDSIZE, SMMU_IDR1_SIDSIZE) |
+            FIELD_PREP(IDR1_CMDQS, SMMU_CMDQS) |
+            FIELD_PREP(IDR1_EVTQS, SMMU_EVTQS);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR2):
+        goto read_reserved;
+
+    case VREG32(ARM_SMMU_IDR3):
+        reg  = FIELD_PREP(IDR3_RIL, 0);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IDR4):
+        goto read_impl_defined;
+
+    case VREG32(ARM_SMMU_IDR5):
+        reg  = FIELD_PREP(IDR5_GRAN4K, 1) | FIELD_PREP(IDR5_GRAN16K, 1) |
+            FIELD_PREP(IDR5_GRAN64K, 1) | FIELD_PREP(IDR5_OAS, IDR5_OAS_48_BIT);
+        *r = vreg_reg32_extract(reg, info);
+        break;
+
+    case VREG32(ARM_SMMU_IIDR):
+        *r = vreg_reg32_extract(ARM_SMMU_IIDR_VAL, info);
+        break;
+
+    case VREG32(ARM_SMMU_CR0):
+        *r = vreg_reg32_extract(smmu->cr[0], info);
+        break;
+
+    case VREG32(ARM_SMMU_CR0ACK):
+        *r = vreg_reg32_extract(smmu->cr0ack, info);
+        break;
+
+    case VREG32(ARM_SMMU_CR1):
+        *r = vreg_reg32_extract(smmu->cr[1], info);
+        break;
+
+    case VREG32(ARM_SMMU_CR2):
+        *r = vreg_reg32_extract(smmu->cr[2], info);
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE):
+        *r = vreg_reg64_extract(smmu->strtab_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
+        *r = vreg_reg32_extract(smmu->strtab_base_cfg, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_BASE):
+        *r = vreg_reg64_extract(smmu->cmdq.q_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_PROD):
+        *r = vreg_reg32_extract(smmu->cmdq.prod, info);
+        break;
+
+    case VREG32(ARM_SMMU_CMDQ_CONS):
+        *r = vreg_reg32_extract(smmu->cmdq.cons, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_BASE):
+        *r = vreg_reg64_extract(smmu->evtq.q_base, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_PROD):
+        *r = vreg_reg32_extract(smmu->evtq.prod, info);
+        break;
+
+    case VREG32(ARM_SMMU_EVTQ_CONS):
+        *r = vreg_reg32_extract(smmu->evtq.cons, info);
+        break;
+
+    case VREG32(ARM_SMMU_IRQ_CTRL):
+    case VREG32(ARM_SMMU_IRQ_CTRLACK):
+        *r = vreg_reg32_extract(smmu->irq_ctrl, info);
+        break;
+
+    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
+        *r = vreg_reg64_extract(smmu->gerror_irq_cfg0, info);
+        break;
+
+    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
+        *r = vreg_reg64_extract(smmu->evtq_irq_cfg0, info);
+        break;
+
+    case VREG32(ARM_SMMU_GERROR):
+        *r = vreg_reg64_extract(smmu->gerror, info);
+        break;
+
+    case VREG32(ARM_SMMU_GERRORN):
+        *r = vreg_reg64_extract(smmu->gerrorn, info);
+        break;
+
+    default:
+        printk(XENLOG_G_ERR
+               "%pv: vSMMUv3: unhandled read r%d offset %"PRIpaddr"\n",
+               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
+        return IO_ABORT;
+    }
+
+    return IO_HANDLED;
+
+ read_impl_defined:
+    printk(XENLOG_G_DEBUG
+           "%pv: vSMMUv3: RAZ on implementation defined register offset %"PRIpaddr"\n",
+           v, info->gpa & 0xffff);
+    *r = 0;
+    return IO_HANDLED;
+
+ read_reserved:
+    printk(XENLOG_G_DEBUG
+           "%pv: vSMMUv3: RAZ on reserved register offset %"PRIpaddr"\n",
+           v, info->gpa & 0xffff);
+    *r = 0;
     return IO_HANDLED;
 }
 
@@ -39,6 +321,10 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
         return -ENOMEM;
 
     smmu->d = d;
+    smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
+    smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
+    smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
+    smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
 
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
-- 
2.43.0

Re: [PATCH v3 08/23] xen/arm: vsmmuv3: Add support for registers emulation

[Luca Fancellu]

Hi Milan,

> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index e36f200ba5..3ae1e62a50 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> 
[...]
> static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>                               register_t r, void *priv)
> {
> +    struct virt_smmu *smmu = priv;
> +    uint64_t reg;
> +    uint32_t reg32;
> +
> +    switch ( info->gpa & 0xffff )
> +    {
> +    case VREG32(ARM_SMMU_CR0):
> +        reg32 = smmu->cr[0];
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cr[0] = reg32;
> +        smmu->cr0ack = reg32 & ~CR0_RESERVED;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR1):
> +        reg32 = smmu->cr[1];
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cr[1] = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR2):
> +        reg32 = smmu->cr[2];
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cr[2] = reg32;
> +        break;
> +
> +    case VREG64(ARM_SMMU_STRTAB_BASE):
> +        reg = smmu->strtab_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->strtab_base = reg;
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
> +        reg32 = smmu->strtab_base_cfg;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->strtab_base_cfg = reg32;
> +
> +        smmu->sid_split = FIELD_GET(STRTAB_BASE_CFG_SPLIT, reg32);
> +        smmu->features |= STRTAB_BASE_CFG_FMT_2LVL;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_BASE):

I think this needs to be VREG64

> +        reg = smmu->cmdq.q_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->cmdq.q_base = reg;
> +        smmu->cmdq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->cmdq.q_base);
> +        if ( smmu->cmdq.max_n_shift > SMMU_CMDQS )
> +            smmu->cmdq.max_n_shift = SMMU_CMDQS;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_PROD):
> +        reg32 = smmu->cmdq.prod;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cmdq.prod = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_CONS):
> +        reg32 = smmu->cmdq.cons;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cmdq.cons = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_BASE):

And also this one VREG64

> +        reg = smmu->evtq.q_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->evtq.q_base = reg;
> +        smmu->evtq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->evtq.q_base);
> +        if ( smmu->cmdq.max_n_shift > SMMU_EVTQS )
> +            smmu->cmdq.max_n_shift = SMMU_EVTQS;

This seems a typo, I think here we want to modify the evtq, not the cmdq?
So in case both condition and assignment needs to change.

> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_PROD):
> +        reg32 = smmu->evtq.prod;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->evtq.prod = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_CONS):
> +        reg32 = smmu->evtq.cons;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->evtq.cons = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_IRQ_CTRL):
> +        reg32 = smmu->irq_ctrl;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->irq_ctrl = reg32;
> +        break;
> +
> +    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
> +        reg = smmu->gerror_irq_cfg0;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->gerror_irq_cfg0 = reg;
> +        break;
> +
> +    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
> +        reg = smmu->evtq_irq_cfg0;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->evtq_irq_cfg0 = reg;
> +        break;
> +
> +    case VREG32(ARM_SMMU_GERRORN):
> +        reg = smmu->gerrorn;
> +        vreg_reg64_update(&reg, r, info);

this should be vreg_reg32_update

> +        smmu->gerrorn = reg;
> +        break;
> +
> +    default:
> +        printk(XENLOG_G_ERR
> +               "%pv: vSMMUv3: unhandled write r%d offset %"PRIpaddr"\n",
> +               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> +        return IO_ABORT;
> +    }
> +
>     return IO_HANDLED;
> }
> 
> static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
>                              register_t *r, void *priv)
> {
> +    struct virt_smmu *smmu = priv;
> +    uint64_t reg;
> +
> +    switch ( info->gpa & 0xffff )
> +    {
> +    case VREG32(ARM_SMMU_IDR0):
> +        reg  = FIELD_PREP(IDR0_S1P, 1) | FIELD_PREP(IDR0_TTF, 2) |
> +            FIELD_PREP(IDR0_COHACC, 0) | FIELD_PREP(IDR0_ASID16, 1) |
> +            FIELD_PREP(IDR0_TTENDIAN, 0) | FIELD_PREP(IDR0_STALL_MODEL, 1) |
> +            FIELD_PREP(IDR0_ST_LVL, 1) | FIELD_PREP(IDR0_TERM_MODEL, 1);
> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR1):
> +        reg  = FIELD_PREP(IDR1_SIDSIZE, SMMU_IDR1_SIDSIZE) |
> +            FIELD_PREP(IDR1_CMDQS, SMMU_CMDQS) |
> +            FIELD_PREP(IDR1_EVTQS, SMMU_EVTQS);
> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR2):
> +        goto read_reserved;
> +
> +    case VREG32(ARM_SMMU_IDR3):
> +        reg  = FIELD_PREP(IDR3_RIL, 0);
> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR4):
> +        goto read_impl_defined;
> +
> +    case VREG32(ARM_SMMU_IDR5):
> +        reg  = FIELD_PREP(IDR5_GRAN4K, 1) | FIELD_PREP(IDR5_GRAN16K, 1) |
> +            FIELD_PREP(IDR5_GRAN64K, 1) | FIELD_PREP(IDR5_OAS, IDR5_OAS_48_BIT);
> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IIDR):
> +        *r = vreg_reg32_extract(ARM_SMMU_IIDR_VAL, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR0):
> +        *r = vreg_reg32_extract(smmu->cr[0], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR0ACK):
> +        *r = vreg_reg32_extract(smmu->cr0ack, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR1):
> +        *r = vreg_reg32_extract(smmu->cr[1], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR2):
> +        *r = vreg_reg32_extract(smmu->cr[2], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE):

this needs to be VREG64 I think

> +        *r = vreg_reg64_extract(smmu->strtab_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
> +        *r = vreg_reg32_extract(smmu->strtab_base_cfg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_BASE):

VREG64

> +        *r = vreg_reg64_extract(smmu->cmdq.q_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_PROD):
> +        *r = vreg_reg32_extract(smmu->cmdq.prod, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_CONS):
> +        *r = vreg_reg32_extract(smmu->cmdq.cons, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_BASE):

VREG64

> +        *r = vreg_reg64_extract(smmu->evtq.q_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_PROD):
> +        *r = vreg_reg32_extract(smmu->evtq.prod, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_CONS):
> +        *r = vreg_reg32_extract(smmu->evtq.cons, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IRQ_CTRL):
> +    case VREG32(ARM_SMMU_IRQ_CTRLACK):
> +        *r = vreg_reg32_extract(smmu->irq_ctrl, info);
> +        break;
> +
> +    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
> +        *r = vreg_reg64_extract(smmu->gerror_irq_cfg0, info);
> +        break;
> +
> +    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
> +        *r = vreg_reg64_extract(smmu->evtq_irq_cfg0, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_GERROR):
> +        *r = vreg_reg64_extract(smmu->gerror, info);

vreg_reg32_extract

> +        break;
> +
> +    case VREG32(ARM_SMMU_GERRORN):
> +        *r = vreg_reg64_extract(smmu->gerrorn, info);

vreg_reg32_extract

> +        break;
> +
> +    default:
> +        printk(XENLOG_G_ERR
> +               "%pv: vSMMUv3: unhandled read r%d offset %"PRIpaddr"\n",
> +               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> +        return IO_ABORT;
> +    }
> +
> +    return IO_HANDLED;
> +
> + read_impl_defined:
> +    printk(XENLOG_G_DEBUG
> +           "%pv: vSMMUv3: RAZ on implementation defined register offset %"PRIpaddr"\n",
> +           v, info->gpa & 0xffff);
> +    *r = 0;
> +    return IO_HANDLED;
> +
> + read_reserved:
> +    printk(XENLOG_G_DEBUG
> +           "%pv: vSMMUv3: RAZ on reserved register offset %"PRIpaddr"\n",
> +           v, info->gpa & 0xffff);
> +    *r = 0;
>     return IO_HANDLED;
> }

Cheers,
Luca

Re: [PATCH v3 08/23] xen/arm: vsmmuv3: Add support for registers emulation

[Julien Grall]

Hi Milan,

On 31/03/2026 10:52, Milan Djokic wrote:
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Add initial support for various emulated registers for virtual SMMUv3
> for guests and also add support for virtual cmdq and eventq.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
>   xen/drivers/passthrough/arm/smmu-v3.h  |   6 +
>   xen/drivers/passthrough/arm/vsmmu-v3.c | 286 +++++++++++++++++++++++++
>   2 files changed, 292 insertions(+)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
> index 3fb13b7e21..fab4fd5a26 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.h
> +++ b/xen/drivers/passthrough/arm/smmu-v3.h
> @@ -60,6 +60,12 @@
>   #define IDR5_VAX			GENMASK(11, 10)
>   #define IDR5_VAX_52_BIT			1
>   
> +#define ARM_SMMU_IIDR			0x18
> +#define IIDR_PRODUCTID			GENMASK(31, 20)
> +#define IIDR_VARIANT			GENMASK(19, 16)
> +#define IIDR_REVISION			GENMASK(15, 12)
> +#define IIDR_IMPLEMENTER		GENMASK(11, 0)
> +
>   #define ARM_SMMU_CR0			0x20
>   #define CR0_ATSCHK			(1 << 4)
>   #define CR0_CMDQEN			(1 << 3)
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index e36f200ba5..3ae1e62a50 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -3,25 +3,307 @@
>   #include <xen/param.h>
>   #include <xen/sched.h>
>   #include <asm/mmio.h>
> +#include <asm/vgic-emul.h>

vgic-emul.h is intended to only be used in the vGIC code. I am fine if 
you want to use it in vsmmu-v3.c but it needs to be renamed. Maybe to 
vdev-emul.h.

>   #include <asm/viommu.h>
> +#include <asm/vreg.h>
> +
> +#include "smmu-v3.h"
> +
> +/* Register Definition */
> +#define ARM_SMMU_IDR2       0x8
> +#define ARM_SMMU_IDR3       0xc
> +#define ARM_SMMU_IDR4       0x10
> +#define IDR0_TERM_MODEL     (1 << 26)
> +#define IDR3_RIL            (1 << 10)
> +#define CR0_RESERVED        0xFFFFFC20

AFAIU, this is covering all the bits defined by the SMMU spec. But some 
of them are optional. Does this mean we will expose those optional features?

> +#define SMMU_IDR1_SIDSIZE   16
> +#define SMMU_CMDQS          19

Can you add some details how you decided the size of the command and ...

> +#define SMMU_EVTQS          19

... even queues?

> +#define DWORDS_BYTES        8
> +#define ARM_SMMU_IIDR_VAL   0x12

I am not sure which implementer this is referring to. But how do you 
plan to handle errata? Are we sure they can always be handled by Xen?

>   
>   /* Struct to hold the vIOMMU ops and vIOMMU type */
>   extern const struct viommu_desc __read_mostly *cur_viommu;
>   
> +/* virtual smmu queue */
> +struct arm_vsmmu_queue {
> +    uint64_t    q_base; /* base register */
> +    uint32_t    prod;
> +    uint32_t    cons;
> +    uint8_t     ent_size;
> +    uint8_t     max_n_shift;
> +};
> +
>   struct virt_smmu {
>       struct      domain *d;
>       struct      list_head viommu_list;
> +    uint8_t     sid_split;
> +    uint32_t    features;
> +    uint32_t    cr[3];
> +    uint32_t    cr0ack;
> +    uint32_t    gerror;
> +    uint32_t    gerrorn;
> +    uint32_t    strtab_base_cfg;
> +    uint64_t    strtab_base;
> +    uint32_t    irq_ctrl;
> +    uint64_t    gerror_irq_cfg0;
> +    uint64_t    evtq_irq_cfg0;
> +    struct      arm_vsmmu_queue evtq, cmdq;
>   };
>   
>   static int vsmmuv3_mmio_write(struct vcpu *v, mmio_info_t *info,
>                                 register_t r, void *priv)
>   {
> +    struct virt_smmu *smmu = priv;
> +    uint64_t reg;
> +    uint32_t reg32;

Looking at this helper and the read one, I am bit surprised there is no 
lock taken nor we check the access size.  Can you explain why?

For instance, we should not allow 64-bit access on 32-bit register. The 
rest of the size (8-bit and 16-bit) is IMP DEFINED so it may be easier 
just not allow them.

> +
> +    switch ( info->gpa & 0xffff )
> +    {
> +    case VREG32(ARM_SMMU_CR0):
 > +        reg32 = smmu->cr[0];> +        vreg_reg32_update(&reg32, r, 
info);
> +        smmu->cr[0] = reg32;
> +        smmu->cr0ack = reg32 & ~CR0_RESERVED;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR1):
> +        reg32 = smmu->cr[1];
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cr[1] = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR2):
> +        reg32 = smmu->cr[2];
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cr[2] = reg32;
> +        break;
> +
> +    case VREG64(ARM_SMMU_STRTAB_BASE):

Looking at the SMMU spec (6.3.24 in ARM IHI 0070 H.a), the behavior of 
writing to the register is constrained unpredictable before SMMUv3.2, 
but after it should be ignored if SMMU_CR0.SMMUEN == 1.

So this implementation would not be valid for SMMUv3.2 and later. For 
convenience it would be best to just ignore the write (which is also 
valid for SMMUv3.1 and ealier).

> +        reg = smmu->strtab_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->strtab_base = reg;
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):

Similar to above, there are some conditions when this field can be 
written (see 6.3.25).

> +        reg32 = smmu->strtab_base_cfg;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->strtab_base_cfg = reg32;
> +
> +        smmu->sid_split = FIELD_GET(STRTAB_BASE_CFG_SPLIT, reg32);

The information for sid_split is already stored in 
``smmu->strtab_base_cfg``. So why do we need to store it differently?

> +        smmu->features |= STRTAB_BASE_CFG_FMT_2LVL;

I haven't checked the rest of the code yet. But from the name, I would 
assume it indicates whether 2-level stream table is supported. From my 
understanding of the specification, this is selectable by the guest OS. 
So why is this unconditionally set?

> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_BASE):

Similar to above, there are some condition when this field is RO.

> +        reg = smmu->cmdq.q_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->cmdq.q_base = reg;
> +        smmu->cmdq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->cmdq.q_base);
> +        if ( smmu->cmdq.max_n_shift > SMMU_CMDQS )
> +            smmu->cmdq.max_n_shift = SMMU_CMDQS;
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_PROD):
> +        reg32 = smmu->cmdq.prod;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cmdq.prod = reg32;

AFAIU, this implementation is not yet complete. If so, it would be good 
to mark it as such with a comment of BUG_ON("Not yet implemented"). Same 
for everywhere in this file and the rest of the series.

> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_CONS):
> +        reg32 = smmu->cmdq.cons;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->cmdq.cons = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_BASE):
> +        reg = smmu->evtq.q_base;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->evtq.q_base = reg;
> +        smmu->evtq.max_n_shift = FIELD_GET(Q_BASE_LOG2SIZE, smmu->evtq.q_base);
> +        if ( smmu->cmdq.max_n_shift > SMMU_EVTQS )
> +            smmu->cmdq.max_n_shift = SMMU_EVTQS;
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_PROD):
> +        reg32 = smmu->evtq.prod;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->evtq.prod = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_CONS):
> +        reg32 = smmu->evtq.cons;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->evtq.cons = reg32;
> +        break;
> +
> +    case VREG32(ARM_SMMU_IRQ_CTRL):
> +        reg32 = smmu->irq_ctrl;
> +        vreg_reg32_update(&reg32, r, info);
> +        smmu->irq_ctrl = reg32;
> +        break;
> +
> +    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
> +        reg = smmu->gerror_irq_cfg0;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->gerror_irq_cfg0 = reg;
> +        break;
> +
> +    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
> +        reg = smmu->evtq_irq_cfg0;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->evtq_irq_cfg0 = reg;
> +        break;
> +
> +    case VREG32(ARM_SMMU_GERRORN):
> +        reg = smmu->gerrorn;
> +        vreg_reg64_update(&reg, r, info);
> +        smmu->gerrorn = reg;
> +        break;
> +
> +    default:
> +        printk(XENLOG_G_ERR
> +               "%pv: vSMMUv3: unhandled write r%d offset %"PRIpaddr"\n",

NIT: The vIOMMU is per-domain so it is sufficient to print "%pd".

> +               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> +        return IO_ABORT;

Per section 6 of the SMMU:

"
For all pages except Page 1, undefined register locations are RES0. For 
Page 1, access to undefined/Reserved
register locations is CONSTRAINED UNPREDICTABLE and an implementation 
has one of the following behaviors:
[...]
"

Here you seem to implement page0 so the default case should be write 
ignore and therefore IO_HANDLED should be returned. BTW, you don't seem 
to handle page1. Is this going to be handled later on?

> +    }
> +
>       return IO_HANDLED;
>   }
>   
>   static int vsmmuv3_mmio_read(struct vcpu *v, mmio_info_t *info,
>                                register_t *r, void *priv)
>   {
> +    struct virt_smmu *smmu = priv;
> +    uint64_t reg;
> +
> +    switch ( info->gpa & 0xffff )
> +    {
> +    case VREG32(ARM_SMMU_IDR0):
> +        reg  = FIELD_PREP(IDR0_S1P, 1) | FIELD_PREP(IDR0_TTF, 2) |

As the page-table will be used by the HW, shouldn't TTF reflect what the 
HW supports? This would allow the vIOMMU to work for 32-bit domains.

> +            FIELD_PREP(IDR0_COHACC, 0) | FIELD_PREP(IDR0_ASID16, 1) |

Here you set COHACC to 0 which means the guest OS will have to clean the 
cache every time. This is safe everywhere, but it will have an impact on 
performance. I am not asking to allow COHACC when the HW supports it, 
but I think a TODO would be worth.

For ASID16, shouldn't the value be based on the HW?

As an aside, I guess we don't allow BTM because we only expose a single 
vSMMU?

> +            FIELD_PREP(IDR0_TTENDIAN, 0) | FIELD_PREP(IDR0_STALL_MODEL, 1) |

For TTENDIAN, it is the same as above.

For STALL_MODEL, I think 1 is ok.

> +            FIELD_PREP(IDR0_ST_LVL, 1) | FIELD_PREP(IDR0_TERM_MODEL, 1);

Overall, it feels the value set in IDR0 and IDR1 (below) needs some comment.

> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR1):
> +        reg  = FIELD_PREP(IDR1_SIDSIZE, SMMU_IDR1_SIDSIZE) |
> +            FIELD_PREP(IDR1_CMDQS, SMMU_CMDQS) |
> +            FIELD_PREP(IDR1_EVTQS, SMMU_EVTQS);
> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR2):
> +        goto read_reserved;
> +
> +    case VREG32(ARM_SMMU_IDR3):
> +        reg  = FIELD_PREP(IDR3_RIL, 0);

I am not sure why we explicitely need to set RIL but not the other fields?

 > +        *r = vreg_reg32_extract(reg, info);> +        break;
> +
> +    case VREG32(ARM_SMMU_IDR4):
> +        goto read_impl_defined;
> +
> +    case VREG32(ARM_SMMU_IDR5):
> +        reg  = FIELD_PREP(IDR5_GRAN4K, 1) | FIELD_PREP(IDR5_GRAN16K, 1) |
> +            FIELD_PREP(IDR5_GRAN64K, 1) | FIELD_PREP(IDR5_OAS, IDR5_OAS_48_BIT);

Similar to the other fields in IDR0, isn't this based on what the HW 
supports?	

> +        *r = vreg_reg32_extract(reg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IIDR):
> +        *r = vreg_reg32_extract(ARM_SMMU_IIDR_VAL, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR0):
> +        *r = vreg_reg32_extract(smmu->cr[0], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR0ACK):
> +        *r = vreg_reg32_extract(smmu->cr0ack, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR1):
> +        *r = vreg_reg32_extract(smmu->cr[1], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CR2):
> +        *r = vreg_reg32_extract(smmu->cr[2], info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE):
> +        *r = vreg_reg64_extract(smmu->strtab_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_STRTAB_BASE_CFG):
> +        *r = vreg_reg32_extract(smmu->strtab_base_cfg, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_BASE):
> +        *r = vreg_reg64_extract(smmu->cmdq.q_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_PROD):
> +        *r = vreg_reg32_extract(smmu->cmdq.prod, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_CMDQ_CONS):
> +        *r = vreg_reg32_extract(smmu->cmdq.cons, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_BASE):
> +        *r = vreg_reg64_extract(smmu->evtq.q_base, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_PROD):
> +        *r = vreg_reg32_extract(smmu->evtq.prod, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_EVTQ_CONS):
> +        *r = vreg_reg32_extract(smmu->evtq.cons, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_IRQ_CTRL):
> +    case VREG32(ARM_SMMU_IRQ_CTRLACK):
> +        *r = vreg_reg32_extract(smmu->irq_ctrl, info);
> +        break;
> +
> +    case VREG64(ARM_SMMU_GERROR_IRQ_CFG0):
> +        *r = vreg_reg64_extract(smmu->gerror_irq_cfg0, info);
> +        break;
> +
> +    case VREG64(ARM_SMMU_EVTQ_IRQ_CFG0):
> +        *r = vreg_reg64_extract(smmu->evtq_irq_cfg0, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_GERROR):
> +        *r = vreg_reg64_extract(smmu->gerror, info);
> +        break;
> +
> +    case VREG32(ARM_SMMU_GERRORN):
> +        *r = vreg_reg64_extract(smmu->gerrorn, info);
> +        break;
> +
> +    default:
> +        printk(XENLOG_G_ERR
> +               "%pv: vSMMUv3: unhandled read r%d offset %"PRIpaddr"\n",
> +               v, info->dabt.reg, (unsigned long)info->gpa & 0xffff);
> +        return IO_ABORT;
> +    }
> +
> +    return IO_HANDLED;
> +
> + read_impl_defined:
> +    printk(XENLOG_G_DEBUG
> +           "%pv: vSMMUv3: RAZ on implementation defined register offset %"PRIpaddr"\n",
> +           v, info->gpa & 0xffff);
> +    *r = 0;
> +    return IO_HANDLED;
> +
> + read_reserved:
> +    printk(XENLOG_G_DEBUG
> +           "%pv: vSMMUv3: RAZ on reserved register offset %"PRIpaddr"\n",
> +           v, info->gpa & 0xffff);
> +    *r = 0;
>       return IO_HANDLED;
>   }
>   
> @@ -39,6 +321,10 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
>           return -ENOMEM;
>   
>       smmu->d = d;
> +    smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
> +    smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
> +    smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
> +    smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;

I understand why we initialize ent_size. But I am not sure to understand 
why we need to initialize q_base. Can you clarify?

>   
>       register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
>   

Cheers,

-- 
Julien Grall

[PATCH v3 10/23] xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

CMD_CFGI_STE is used to invalidate/validate the STE. Emulated vSMMUv3
driver in XEN will read the STE from the guest memory space and capture
the Stage-1 configuration required to support nested translation.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/vsmmu-v3.c | 148 +++++++++++++++++++++++++
 1 file changed, 148 insertions(+)

diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 02fe6a4422..39ed4dc577 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -45,6 +45,21 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 /* Helper Macros */
 #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
 #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
+#define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)
+#define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
+#define smmu_get_ste_s1fmt(x)       FIELD_GET(STRTAB_STE_0_S1FMT, x)
+#define smmu_get_ste_s1stalld(x)    FIELD_GET(STRTAB_STE_1_S1STALLD, x)
+#define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
+                                    FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
+
+/* stage-1 translation configuration */
+struct arm_vsmmu_s1_trans_cfg {
+    paddr_t s1ctxptr;
+    uint8_t s1fmt;
+    uint8_t s1cdmax;
+    bool    bypassed;             /* translation is bypassed */
+    bool    aborted;              /* translation is aborted */
+};
 
 /* virtual smmu queue */
 struct arm_vsmmu_queue {
@@ -91,6 +106,138 @@ static void dump_smmu_command(uint64_t *command)
     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
              smmu_cmd_get_command(command[0]), command[0], command[1]);
 }
+static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
+                              uint64_t *ste)
+{
+    paddr_t addr, strtab_base;
+    struct domain *d = smmu->d;
+    uint32_t log2size;
+    int strtab_size_shift;
+    int ret;
+
+    log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
+
+    if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
+        return -EINVAL;
+
+    if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
+    {
+        int idx, max_l2_ste, span;
+        paddr_t l1ptr, l2ptr;
+        uint64_t l1std;
+
+        strtab_size_shift = MAX(5, (int)log2size - smmu->sid_split - 1 + 3);
+        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
+                        ~GENMASK_ULL(strtab_size_shift, 0);
+        idx = (sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS;
+        l1ptr = (paddr_t)(strtab_base + idx * sizeof(l1std));
+
+        ret = access_guest_memory_by_gpa(d, l1ptr, &l1std,
+                                         sizeof(l1std), false);
+        if ( ret )
+        {
+            gdprintk(XENLOG_ERR,
+                     "Could not read L1PTR at 0X%"PRIx64"\n", l1ptr);
+            return ret;
+        }
+
+        span = FIELD_GET(STRTAB_L1_DESC_SPAN, l1std);
+        if ( !span )
+        {
+            gdprintk(XENLOG_ERR, "Bad StreamID span\n");
+            return -EINVAL;
+        }
+
+        max_l2_ste = (1 << span) - 1;
+        l2ptr = FIELD_PREP(STRTAB_L1_DESC_L2PTR_MASK,
+                    FIELD_GET(STRTAB_L1_DESC_L2PTR_MASK, l1std));
+        idx = sid & ((1 << smmu->sid_split) - 1);
+        if ( idx > max_l2_ste )
+        {
+            gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
+                     idx, max_l2_ste);
+            return -EINVAL;
+        }
+        addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
+    }
+    else
+    {
+        strtab_size_shift = log2size + 5;
+        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
+                      ~GENMASK_ULL(strtab_size_shift, 0);
+        addr = strtab_base + sid * sizeof(*ste) * STRTAB_STE_DWORDS;
+    }
+    ret = access_guest_memory_by_gpa(d, addr, ste, sizeof(*ste), false);
+    if ( ret )
+    {
+        gdprintk(XENLOG_ERR,
+                "Cannot fetch pte at address=0x%"PRIx64"\n", addr);
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
+static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
+                                struct arm_vsmmu_s1_trans_cfg *cfg,
+                                uint64_t *ste)
+{
+    uint64_t val = ste[0];
+
+    if ( !(val & STRTAB_STE_0_V) )
+        return -EAGAIN;
+
+    switch ( FIELD_GET(STRTAB_STE_0_CFG, val) )
+    {
+    case STRTAB_STE_0_CFG_BYPASS:
+        cfg->bypassed = true;
+        return 0;
+    case STRTAB_STE_0_CFG_ABORT:
+        cfg->aborted = true;
+        return 0;
+    case STRTAB_STE_0_CFG_S1_TRANS:
+        break;
+    case STRTAB_STE_0_CFG_S2_TRANS:
+        gdprintk(XENLOG_ERR, "vSMMUv3 does not support stage 2 yet\n");
+        goto bad_ste;
+    default:
+        BUG(); /* STE corruption */
+    }
+
+    cfg->s1ctxptr = smmu_get_ste_s1ctxptr(val);
+    cfg->s1fmt = smmu_get_ste_s1fmt(val);
+    cfg->s1cdmax = smmu_get_ste_s1cdmax(val);
+    if ( cfg->s1cdmax != 0 )
+    {
+        gdprintk(XENLOG_ERR,
+                 "vSMMUv3 does not support multiple context descriptors\n");
+        goto bad_ste;
+    }
+
+    return 0;
+
+bad_ste:
+    return -EINVAL;
+}
+
+static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
+{
+    int ret;
+    uint64_t ste[STRTAB_STE_DWORDS];
+    struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
+    uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
+
+    ret = arm_vsmmu_find_ste(smmu, sid, ste);
+    if ( ret )
+        return ret;
+
+    ret = arm_vsmmu_decode_ste(smmu, sid, &s1_cfg, ste);
+    if ( ret )
+        return (ret == -EAGAIN ) ? 0 : ret;
+
+    return 0;
+}
+
 static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
 {
     struct arm_vsmmu_queue *q = &smmu->cmdq;
@@ -114,6 +261,7 @@ static int arm_vsmmu_handle_cmds(struct virt_smmu *smmu)
         switch ( smmu_cmd_get_command(command[0]) )
         {
         case CMDQ_OP_CFGI_STE:
+            ret = arm_vsmmu_handle_cfgi_ste(smmu, command);
             break;
         case CMDQ_OP_PREFETCH_CFG:
         case CMDQ_OP_CFGI_CD:
-- 
2.43.0

Re: [PATCH v3 10/23] xen/arm: vsmmuv3: Add support for command CMD_CFGI_STE

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> CMD_CFGI_STE is used to invalidate/validate the STE. Emulated vSMMUv3
> driver in XEN will read the STE from the guest memory space and capture
> the Stage-1 configuration required to support nested translation.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/drivers/passthrough/arm/vsmmu-v3.c | 148 +++++++++++++++++++++++++
> 1 file changed, 148 insertions(+)
> 
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 02fe6a4422..39ed4dc577 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -45,6 +45,21 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
> /* Helper Macros */
> #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
> #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
> +#define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)

Was CMDQ_CFGI_0_SID intended here? we use it in arm_vsmmu_handle_cfgi_ste()

> +#define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
> +#define smmu_get_ste_s1fmt(x)       FIELD_GET(STRTAB_STE_0_S1FMT, x)
> +#define smmu_get_ste_s1stalld(x)    FIELD_GET(STRTAB_STE_1_S1STALLD, x)
> +#define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
> +                                    FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
> +
> +/* stage-1 translation configuration */
> +struct arm_vsmmu_s1_trans_cfg {
> +    paddr_t s1ctxptr;
> +    uint8_t s1fmt;
> +    uint8_t s1cdmax;
> +    bool    bypassed;             /* translation is bypassed */
> +    bool    aborted;              /* translation is aborted */
> +};
> 
> /* virtual smmu queue */
> struct arm_vsmmu_queue {
> @@ -91,6 +106,138 @@ static void dump_smmu_command(uint64_t *command)
>     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
>              smmu_cmd_get_command(command[0]), command[0], command[1]);
> }
> +static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
> +                              uint64_t *ste)
> +{
> +    paddr_t addr, strtab_base;
> +    struct domain *d = smmu->d;
> +    uint32_t log2size;
> +    int strtab_size_shift;
> +    int ret;
> +
> +    log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
> +
> +    if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
> +        return -EINVAL;
> +
> +    if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
> +    {
> +        int idx, max_l2_ste, span;
> +        paddr_t l1ptr, l2ptr;
> +        uint64_t l1std;
> +
> +        strtab_size_shift = MAX(5, (int)log2size - smmu->sid_split - 1 + 3);
> +        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
> +                        ~GENMASK_ULL(strtab_size_shift, 0);
> +        idx = (sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS;

I think here we should shift by smmu->sid_split instead of STRTAB_SPLIT?

> +        l1ptr = (paddr_t)(strtab_base + idx * sizeof(l1std));
> +
> +        ret = access_guest_memory_by_gpa(d, l1ptr, &l1std,
> +                                         sizeof(l1std), false);
> +        if ( ret )
> +        {
> +            gdprintk(XENLOG_ERR,
> +                     "Could not read L1PTR at 0X%"PRIx64"\n", l1ptr);
> +            return ret;
> +        }
> +
> +        span = FIELD_GET(STRTAB_L1_DESC_SPAN, l1std);
> +        if ( !span )
> +        {
> +            gdprintk(XENLOG_ERR, "Bad StreamID span\n");
> +            return -EINVAL;
> +        }
> +
> +        max_l2_ste = (1 << span) - 1;
> +        l2ptr = FIELD_PREP(STRTAB_L1_DESC_L2PTR_MASK,
> +                    FIELD_GET(STRTAB_L1_DESC_L2PTR_MASK, l1std));
> +        idx = sid & ((1 << smmu->sid_split) - 1);
> +        if ( idx > max_l2_ste )
> +        {
> +            gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
> +                     idx, max_l2_ste);
> +            return -EINVAL;
> +        }
> +        addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
> +    }
> +    else
> +    {
> +        strtab_size_shift = log2size + 5;
> +        strtab_base = smmu->strtab_base & STRTAB_BASE_ADDR_MASK &
> +                      ~GENMASK_ULL(strtab_size_shift, 0);
> +        addr = strtab_base + sid * sizeof(*ste) * STRTAB_STE_DWORDS;
> +    }
> +    ret = access_guest_memory_by_gpa(d, addr, ste, sizeof(*ste), false);
> +    if ( ret )
> +    {
> +        gdprintk(XENLOG_ERR,
> +                "Cannot fetch pte at address=0x%"PRIx64"\n", addr);
> +        return -EINVAL;
> +    }
> +
> +    return 0;
> +}
> +
> +static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
> +                                struct arm_vsmmu_s1_trans_cfg *cfg,
> +                                uint64_t *ste)
> +{
> +    uint64_t val = ste[0];
> +
> +    if ( !(val & STRTAB_STE_0_V) )
> +        return -EAGAIN;
> +
> +    switch ( FIELD_GET(STRTAB_STE_0_CFG, val) )
> +    {
> +    case STRTAB_STE_0_CFG_BYPASS:
> +        cfg->bypassed = true;
> +        return 0;
> +    case STRTAB_STE_0_CFG_ABORT:
> +        cfg->aborted = true;
> +        return 0;
> +    case STRTAB_STE_0_CFG_S1_TRANS:
> +        break;
> +    case STRTAB_STE_0_CFG_S2_TRANS:
> +        gdprintk(XENLOG_ERR, "vSMMUv3 does not support stage 2 yet\n");
> +        goto bad_ste;
> +    default:
> +        BUG(); /* STE corruption */

This will crash Xen based on the input of the guest, we should print something and
jump to bad_ste instead, in my opinion.

> +    }
> +
> +    cfg->s1ctxptr = smmu_get_ste_s1ctxptr(val);
> +    cfg->s1fmt = smmu_get_ste_s1fmt(val);
> +    cfg->s1cdmax = smmu_get_ste_s1cdmax(val);
> +    if ( cfg->s1cdmax != 0 )
> +    {
> +        gdprintk(XENLOG_ERR,
> +                 "vSMMUv3 does not support multiple context descriptors\n");
> +        goto bad_ste;
> +    }
> +
> +    return 0;
> +
> +bad_ste:

NIT: code style, one space before labels

Cheers,
Luca

[PATCH v3 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Attach the Stage-1 configuration to device STE to support nested
translation for the guests.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/include/asm/iommu.h       |  7 +++
 xen/drivers/passthrough/arm/smmu-v3.c  | 79 ++++++++++++++++++++++++++
 xen/drivers/passthrough/arm/smmu-v3.h  |  1 +
 xen/drivers/passthrough/arm/vsmmu-v3.c | 18 ++++++
 xen/include/xen/iommu.h                |  6 ++
 5 files changed, 111 insertions(+)

diff --git a/xen/arch/arm/include/asm/iommu.h b/xen/arch/arm/include/asm/iommu.h
index ad15477e24..56bc9314a7 100644
--- a/xen/arch/arm/include/asm/iommu.h
+++ b/xen/arch/arm/include/asm/iommu.h
@@ -20,6 +20,13 @@ struct arch_iommu
     void *priv;
 };
 
+struct iommu_guest_config {
+    paddr_t     s1ctxptr;
+    uint8_t     config;
+    uint8_t     s1fmt;
+    uint8_t     s1cdmax;
+};
+
 const struct iommu_ops *iommu_get_ops(void);
 void iommu_set_ops(const struct iommu_ops *ops);
 
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index 87612df21d..cf8f638a49 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -2810,6 +2810,37 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
 	return NULL;
 }
 
+static struct iommu_domain *arm_smmu_get_domain_by_sid(struct domain *d,
+				u32 sid)
+{
+	int i;
+	unsigned long flags;
+	struct iommu_domain *io_domain;
+	struct arm_smmu_domain *smmu_domain;
+	struct arm_smmu_master *master;
+	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
+
+	/*
+	 * Loop through the &xen_domain->contexts to locate a context
+	 * assigned to this SMMU
+	 */
+	list_for_each_entry(io_domain, &xen_domain->contexts, list) {
+		smmu_domain = to_smmu_domain(io_domain);
+
+		spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+		list_for_each_entry(master, &smmu_domain->devices, domain_head) {
+			for (i = 0; i < master->num_streams; i++) {
+				if (sid != master->streams[i].id)
+					continue;
+				spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+				return io_domain;
+			}
+		}
+		spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+	}
+	return NULL;
+}
+
 static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
 				struct device *dev)
 {
@@ -3022,6 +3053,53 @@ static void arm_smmu_iommu_xen_domain_teardown(struct domain *d)
 	xfree(xen_domain);
 }
 
+static int arm_smmu_attach_guest_config(struct domain *d, u32 sid,
+		struct iommu_guest_config *cfg)
+{
+	int ret = -EINVAL;
+	unsigned long flags;
+	struct arm_smmu_master *master;
+	struct arm_smmu_domain *smmu_domain;
+	struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
+	struct iommu_domain *io_domain = arm_smmu_get_domain_by_sid(d, sid);
+
+	if (!io_domain)
+		return -ENODEV;
+
+	smmu_domain = to_smmu_domain(io_domain);
+
+	spin_lock(&xen_domain->lock);
+
+	switch (cfg->config) {
+	case ARM_SMMU_DOMAIN_ABORT:
+		smmu_domain->abort = true;
+		break;
+	case ARM_SMMU_DOMAIN_BYPASS:
+		smmu_domain->abort = false;
+		break;
+	case ARM_SMMU_DOMAIN_NESTED:
+		/* Enable Nested stage translation. */
+		smmu_domain->stage = ARM_SMMU_DOMAIN_NESTED;
+		smmu_domain->s1_cfg.s1ctxptr = cfg->s1ctxptr;
+		smmu_domain->s1_cfg.s1fmt = cfg->s1fmt;
+		smmu_domain->s1_cfg.s1cdmax = cfg->s1cdmax;
+		smmu_domain->abort = false;
+		break;
+	default:
+		goto out;
+	}
+
+	spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+	list_for_each_entry(master, &smmu_domain->devices, domain_head)
+		arm_smmu_install_ste_for_dev(master);
+	spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+
+	ret = 0;
+out:
+	spin_unlock(&xen_domain->lock);
+	return ret;
+}
+
 static const struct iommu_ops arm_smmu_iommu_ops = {
 	.page_sizes		= PAGE_SIZE_4K,
 	.init			= arm_smmu_iommu_xen_domain_init,
@@ -3034,6 +3112,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
 	.unmap_page		= arm_iommu_unmap_page,
 	.dt_xlate		= arm_smmu_dt_xlate,
 	.add_device		= arm_smmu_add_device,
+	.attach_guest_config = arm_smmu_attach_guest_config
 };
 
 static __init int arm_smmu_dt_init(struct dt_device_node *dev,
diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index fab4fd5a26..df3b7ec1b5 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -398,6 +398,7 @@ enum arm_smmu_domain_stage {
 	ARM_SMMU_DOMAIN_S2,
 	ARM_SMMU_DOMAIN_NESTED,
 	ARM_SMMU_DOMAIN_BYPASS,
+	ARM_SMMU_DOMAIN_ABORT,
 };
 
 /* Xen specific code. */
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 39ed4dc577..6d3636b18b 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -224,8 +224,11 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
 {
     int ret;
     uint64_t ste[STRTAB_STE_DWORDS];
+    struct domain *d = smmu->d;
+    struct domain_iommu *hd = dom_iommu(d);
     struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
     uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
+    struct iommu_guest_config guest_cfg = {0};
 
     ret = arm_vsmmu_find_ste(smmu, sid, ste);
     if ( ret )
@@ -235,6 +238,21 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     if ( ret )
         return (ret == -EAGAIN ) ? 0 : ret;
 
+    guest_cfg.s1ctxptr = s1_cfg.s1ctxptr;
+    guest_cfg.s1fmt = s1_cfg.s1fmt;
+    guest_cfg.s1cdmax = s1_cfg.s1cdmax;
+
+    if ( s1_cfg.bypassed )
+        guest_cfg.config = ARM_SMMU_DOMAIN_BYPASS;
+    else if ( s1_cfg.aborted )
+        guest_cfg.config = ARM_SMMU_DOMAIN_ABORT;
+    else
+        guest_cfg.config = ARM_SMMU_DOMAIN_NESTED;
+
+    ret = hd->platform_ops->attach_guest_config(d, sid, &guest_cfg);
+    if ( ret )
+        return ret;
+
     return 0;
 }
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 37c4a1dc82..19e59095c3 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -312,6 +312,7 @@ static inline int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev)
 #endif /* HAS_DEVICE_TREE_DISCOVERY */
 
 struct page_info;
+struct iommu_guest_config;
 
 /*
  * Any non-zero value returned from callbacks of this type will cause the
@@ -387,6 +388,11 @@ struct iommu_ops {
 #endif
     /* Inhibit all interrupt generation, to be used at shutdown. */
     void (*quiesce)(void);
+
+#ifdef CONFIG_ARM
+    int (*attach_guest_config)(struct domain *d, u32 sid,
+                               struct iommu_guest_config *cfg);
+#endif
 };
 
 /*
-- 
2.43.0

Re: [PATCH v3 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Jan Beulich]

On 31.03.2026 03:52, Milan Djokic wrote:
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -312,6 +312,7 @@ static inline int iommu_add_dt_pci_sideband_ids(struct pci_dev *pdev)
>  #endif /* HAS_DEVICE_TREE_DISCOVERY */
>  
>  struct page_info;
> +struct iommu_guest_config;
>  
>  /*
>   * Any non-zero value returned from callbacks of this type will cause the
> @@ -387,6 +388,11 @@ struct iommu_ops {
>  #endif
>      /* Inhibit all interrupt generation, to be used at shutdown. */
>      void (*quiesce)(void);
> +
> +#ifdef CONFIG_ARM
> +    int (*attach_guest_config)(struct domain *d, u32 sid,

Nit: uint32_t please, provided a fixed-width type is actually warranted here
(see ./CODING_STYLE).

> +                               struct iommu_guest_config *cfg);

Pointer-to-const?

Jan

Re: [PATCH v3 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Luca Fancellu]

Hi Milan,


> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Attach the Stage-1 configuration to device STE to support nested
> translation for the guests.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/arch/arm/include/asm/iommu.h       |  7 +++
> xen/drivers/passthrough/arm/smmu-v3.c  | 79 ++++++++++++++++++++++++++
> xen/drivers/passthrough/arm/smmu-v3.h  |  1 +
> xen/drivers/passthrough/arm/vsmmu-v3.c | 18 ++++++
> xen/include/xen/iommu.h                |  6 ++
> 5 files changed, 111 insertions(+)
> 
> diff --git a/xen/arch/arm/include/asm/iommu.h b/xen/arch/arm/include/asm/iommu.h
> index ad15477e24..56bc9314a7 100644
> --- a/xen/arch/arm/include/asm/iommu.h
> +++ b/xen/arch/arm/include/asm/iommu.h
> @@ -20,6 +20,13 @@ struct arch_iommu
>     void *priv;
> };
> 
> +struct iommu_guest_config {
> +    paddr_t     s1ctxptr;
> +    uint8_t     config;
> +    uint8_t     s1fmt;
> +    uint8_t     s1cdmax;
> +};
> +
> const struct iommu_ops *iommu_get_ops(void);
> void iommu_set_ops(const struct iommu_ops *ops);
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index 87612df21d..cf8f638a49 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -2810,6 +2810,37 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
> return NULL;
> }
> 
> +static struct iommu_domain *arm_smmu_get_domain_by_sid(struct domain *d,
> + u32 sid)

I think this might be wrong, a system can have multiple SMMU and the SID is unique only
on each SMMU, not on the overall platform, I think arm_smmu_attach_guest_config should
pass also the smmu for the selected sid.

> +{
> + int i;
> + unsigned long flags;
> + struct iommu_domain *io_domain;
> + struct arm_smmu_domain *smmu_domain;
> + struct arm_smmu_master *master;
> + struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
> +
> + /*
> + * Loop through the &xen_domain->contexts to locate a context
> + * assigned to this SMMU
> + */
> + list_for_each_entry(io_domain, &xen_domain->contexts, list) {
> + smmu_domain = to_smmu_domain(io_domain);
> +
> + spin_lock_irqsave(&smmu_domain->devices_lock, flags);
> + list_for_each_entry(master, &smmu_domain->devices, domain_head) {
> + for (i = 0; i < master->num_streams; i++) {
> + if (sid != master->streams[i].id)
> + continue;
> + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
> + return io_domain;
> + }
> + }
> + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
> + }
> + return NULL;
> +}
> +
> static struct iommu_domain *arm_smmu_get_domain(struct domain *d,
> struct device *dev)
> {
> @@ -3022,6 +3053,53 @@ static void arm_smmu_iommu_xen_domain_teardown(struct domain *d)
> xfree(xen_domain);
> }
> 
> +static int arm_smmu_attach_guest_config(struct domain *d, u32 sid,
> + struct iommu_guest_config *cfg)
> +{
> + int ret = -EINVAL;
> + unsigned long flags;
> + struct arm_smmu_master *master;
> + struct arm_smmu_domain *smmu_domain;
> + struct arm_smmu_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
> + struct iommu_domain *io_domain = arm_smmu_get_domain_by_sid(d, sid);
> +
> + if (!io_domain)
> + return -ENODEV;
> +
> + smmu_domain = to_smmu_domain(io_domain);
> +
> + spin_lock(&xen_domain->lock);
> +
> + switch (cfg->config) {
> + case ARM_SMMU_DOMAIN_ABORT:
> + smmu_domain->abort = true;

Shold we change also smmu_domain->stage here and ...

> + break;
> + case ARM_SMMU_DOMAIN_BYPASS:
> + smmu_domain->abort = false;

Here? I see arm_smmu_write_strtab_ent() uses it to select
s1_cfg or s2_cfg configuration

> + break;
> + case ARM_SMMU_DOMAIN_NESTED:
> + /* Enable Nested stage translation. */
> + smmu_domain->stage = ARM_SMMU_DOMAIN_NESTED;
> + smmu_domain->s1_cfg.s1ctxptr = cfg->s1ctxptr;
> + smmu_domain->s1_cfg.s1fmt = cfg->s1fmt;
> + smmu_domain->s1_cfg.s1cdmax = cfg->s1cdmax;
> + smmu_domain->abort = false;
> + break;
> + default:
> + goto out;
> + }
> +
> + spin_lock_irqsave(&smmu_domain->devices_lock, flags);
> + list_for_each_entry(master, &smmu_domain->devices, domain_head)
> + arm_smmu_install_ste_for_dev(master);
> + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
> +
> + ret = 0;
> +out:
> + spin_unlock(&xen_domain->lock);
> + return ret;
> +}
> +
> static const struct iommu_ops arm_smmu_iommu_ops = {
> .page_sizes = PAGE_SIZE_4K,
> .init = arm_smmu_iommu_xen_domain_init,
> @@ -3034,6 +3112,7 @@ static const struct iommu_ops arm_smmu_iommu_ops = {
> .unmap_page = arm_iommu_unmap_page,
> .dt_xlate = arm_smmu_dt_xlate,
> .add_device = arm_smmu_add_device,
> + .attach_guest_config = arm_smmu_attach_guest_config
> };
> 
> static __init int arm_smmu_dt_init(struct dt_device_node *dev,

Cheers,
Luca

Re: [PATCH v3 11/23] xen/arm: vsmmuv3: Attach Stage-1 configuration to SMMUv3 hardware

[Luca Fancellu]

HI Milan,

> On 13 Apr 2026, at 11:26, Luca Fancellu <Luca.Fancellu@arm.com> wrote:
> 
> Hi Milan,
> 
> 
>> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
>> 
>> From: Rahul Singh <rahul.singh@arm.com>
>> 
>> Attach the Stage-1 configuration to device STE to support nested
>> translation for the guests.
>> 
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
>> ---
>> xen/arch/arm/include/asm/iommu.h       |  7 +++
>> xen/drivers/passthrough/arm/smmu-v3.c  | 79 ++++++++++++++++++++++++++
>> xen/drivers/passthrough/arm/smmu-v3.h  |  1 +
>> xen/drivers/passthrough/arm/vsmmu-v3.c | 18 ++++++
>> xen/include/xen/iommu.h                |  6 ++
>> 5 files changed, 111 insertions(+)
>> 
>> diff --git a/xen/arch/arm/include/asm/iommu.h b/xen/arch/arm/include/asm/iommu.h
>> index ad15477e24..56bc9314a7 100644
>> --- a/xen/arch/arm/include/asm/iommu.h
>> +++ b/xen/arch/arm/include/asm/iommu.h
>> @@ -20,6 +20,13 @@ struct arch_iommu
>>    void *priv;
>> };
>> 
>> +struct iommu_guest_config {
>> +    paddr_t     s1ctxptr;
>> +    uint8_t     config;
>> +    uint8_t     s1fmt;
>> +    uint8_t     s1cdmax;
>> +};
>> +
>> const struct iommu_ops *iommu_get_ops(void);
>> void iommu_set_ops(const struct iommu_ops *ops);
>> 
>> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
>> index 87612df21d..cf8f638a49 100644
>> --- a/xen/drivers/passthrough/arm/smmu-v3.c
>> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
>> @@ -2810,6 +2810,37 @@ static struct arm_smmu_device *arm_smmu_get_by_dev(const struct device *dev)
>> return NULL;
>> }
>> 
>> +static struct iommu_domain *arm_smmu_get_domain_by_sid(struct domain *d,
>> + u32 sid)
> 
> I think this might be wrong, a system can have multiple SMMU and the SID is unique only
> on each SMMU, not on the overall platform, I think arm_smmu_attach_guest_config should
> pass also the smmu for the selected sid.

Apologies, I’ve read the design, I think this is intentional. I withdraw this comment.

Cheers,
Luca

[PATCH v3 12/23] xen/arm: vsmmuv3: Add support for event queue and global error

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Event queue is used to send the events to guest when there is an events/
faults. Add support for event queue to send events to guest.

Global error in SMMUv3 hw will be updated in smmu_gerror and
smmu_gerrorn register. Add support for global error registers to send
global error to guest.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.h  |  20 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 163 ++++++++++++++++++++++++-
 xen/include/public/arch-arm.h          |   5 +-
 3 files changed, 183 insertions(+), 5 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.h b/xen/drivers/passthrough/arm/smmu-v3.h
index df3b7ec1b5..8d3e1877aa 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.h
+++ b/xen/drivers/passthrough/arm/smmu-v3.h
@@ -354,6 +354,26 @@
 
 #define EVTQ_0_ID			GENMASK_ULL(7, 0)
 
+#define EVT_ID_BAD_STREAMID		0x02
+#define EVT_ID_BAD_STE		    	0x04
+#define EVT_ID_TRANSLATION_FAULT	0x10
+#define EVT_ID_ADDR_SIZE_FAULT		0x11
+#define EVT_ID_ACCESS_FAULT		0x12
+#define EVT_ID_PERMISSION_FAULT		0x13
+
+#define EVTQ_0_SSV			(1UL << 11)
+#define EVTQ_0_SSID			GENMASK_ULL(31, 12)
+#define EVTQ_0_SID			GENMASK_ULL(63, 32)
+#define EVTQ_1_STAG			GENMASK_ULL(15, 0)
+#define EVTQ_1_STALL			(1UL << 31)
+#define EVTQ_1_PnU			(1UL << 33)
+#define EVTQ_1_InD			(1UL << 34)
+#define EVTQ_1_RnW			(1UL << 35)
+#define EVTQ_1_S2			(1UL << 39)
+#define EVTQ_1_CLASS			GENMASK_ULL(41, 40)
+#define EVTQ_1_TT_READ			(1UL << 44)
+#define EVTQ_2_ADDR			GENMASK_ULL(63, 0)
+#define EVTQ_3_IPA			GENMASK_ULL(51, 12)
 /* PRI queue */
 #define PRIQ_ENT_SZ_SHIFT		4
 #define PRIQ_ENT_DWORDS			((1 << PRIQ_ENT_SZ_SHIFT) >> 3)
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 6d3636b18b..7a6c18df53 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -44,6 +44,7 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 
 /* Helper Macros */
 #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
+#define smmu_get_evtq_enabled(x)    FIELD_GET(CR0_EVTQEN, x)
 #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
 #define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)
 #define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
@@ -52,6 +53,35 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
 
+/* event queue entry */
+struct arm_smmu_evtq_ent {
+    /* Common fields */
+    uint8_t     opcode;
+    uint32_t    sid;
+
+    /* Event-specific fields */
+    union {
+        struct {
+            uint32_t ssid;
+            bool ssv;
+        } c_bad_ste_streamid;
+
+        struct {
+            bool stall;
+            uint16_t stag;
+            uint32_t ssid;
+            bool ssv;
+            bool s2;
+            uint64_t addr;
+            bool rnw;
+            bool pnu;
+            bool ind;
+            uint8_t class;
+            uint64_t addr2;
+        } f_translation;
+    };
+};
+
 /* stage-1 translation configuration */
 struct arm_vsmmu_s1_trans_cfg {
     paddr_t s1ctxptr;
@@ -82,6 +112,7 @@ struct virt_smmu {
     uint32_t    strtab_base_cfg;
     uint64_t    strtab_base;
     uint32_t    irq_ctrl;
+    uint32_t    virq;
     uint64_t    gerror_irq_cfg0;
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
@@ -89,6 +120,12 @@ struct virt_smmu {
 };
 
 /* Queue manipulation functions */
+static bool queue_full(struct arm_vsmmu_queue *q)
+{
+    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
+           Q_WRP(q, q->prod) != Q_WRP(q, q->cons);
+}
+
 static bool queue_empty(struct arm_vsmmu_queue *q)
 {
     return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
@@ -101,11 +138,105 @@ static void queue_inc_cons(struct arm_vsmmu_queue *q)
     q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
 }
 
+static void queue_inc_prod(struct arm_vsmmu_queue *q)
+{
+    u32 prod = (Q_WRP(q, q->prod) | Q_IDX(q, q->prod)) + 1;
+    q->prod = Q_OVF(q->prod) | Q_WRP(q, prod) | Q_IDX(q, prod);
+}
+
 static void dump_smmu_command(uint64_t *command)
 {
     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
              smmu_cmd_get_command(command[0]), command[0], command[1]);
 }
+
+static void arm_vsmmu_inject_irq(struct virt_smmu *smmu, bool is_gerror,
+                                uint32_t gerror_err)
+{
+    uint32_t new_gerrors, pending;
+
+    if ( is_gerror )
+    {
+        /* trigger global error irq to guest */
+        pending = smmu->gerror ^ smmu->gerrorn;
+        new_gerrors = ~pending & gerror_err;
+
+        /* only toggle non pending errors */
+        if (!new_gerrors)
+            return;
+
+        smmu->gerror ^= new_gerrors;
+    }
+
+    vgic_inject_irq(smmu->d, NULL, smmu->virq, true);
+}
+
+static int arm_vsmmu_write_evtq(struct virt_smmu *smmu, uint64_t *evt)
+{
+    struct arm_vsmmu_queue *q = &smmu->evtq;
+    struct domain *d = smmu->d;
+    paddr_t addr;
+    int ret;
+
+    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
+        return -EINVAL;
+
+    if ( queue_full(q) )
+        return -EINVAL;
+
+    addr = Q_PROD_ENT(q);
+    ret = access_guest_memory_by_gpa(d, addr, evt,
+                                     sizeof(*evt) * EVTQ_ENT_DWORDS, true);
+    if ( ret )
+        return ret;
+
+    queue_inc_prod(q);
+
+    /* trigger eventq irq to guest */
+    if ( !queue_empty(q) )
+        arm_vsmmu_inject_irq(smmu, false, 0);
+
+    return 0;
+}
+
+void arm_vsmmu_send_event(struct virt_smmu *smmu,
+                          struct arm_smmu_evtq_ent *ent)
+{
+    uint64_t evt[EVTQ_ENT_DWORDS];
+    int ret;
+
+    memset(evt, 0, 1 << EVTQ_ENT_SZ_SHIFT);
+
+    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
+        return;
+
+    evt[0] |= FIELD_PREP(EVTQ_0_ID, ent->opcode);
+    evt[0] |= FIELD_PREP(EVTQ_0_SID, ent->sid);
+
+    switch (ent->opcode)
+    {
+    case EVT_ID_BAD_STREAMID:
+    case EVT_ID_BAD_STE:
+        evt[0] |= FIELD_PREP(EVTQ_0_SSID, ent->c_bad_ste_streamid.ssid);
+        evt[0] |= FIELD_PREP(EVTQ_0_SSV, ent->c_bad_ste_streamid.ssv);
+        break;
+    case EVT_ID_TRANSLATION_FAULT:
+    case EVT_ID_ADDR_SIZE_FAULT:
+    case EVT_ID_ACCESS_FAULT:
+    case EVT_ID_PERMISSION_FAULT:
+        break;
+    default:
+        gdprintk(XENLOG_WARNING, "vSMMUv3: event opcode is bad\n");
+        break;
+    }
+
+    ret = arm_vsmmu_write_evtq(smmu, evt);
+    if ( ret )
+        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
+
+    return;
+}
+
 static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
                               uint64_t *ste)
 {
@@ -114,11 +245,22 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
     uint32_t log2size;
     int strtab_size_shift;
     int ret;
+    struct arm_smmu_evtq_ent ent = {
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
 
     log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
 
     if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
+    {
+        ent.opcode = EVT_ID_BAD_STE;
+        arm_vsmmu_send_event(smmu, &ent);
         return -EINVAL;
+    }
 
     if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
     {
@@ -156,6 +298,8 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
         {
             gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
                      idx, max_l2_ste);
+            ent.opcode = EVT_ID_BAD_STREAMID;
+            arm_vsmmu_send_event(smmu, &ent);
             return -EINVAL;
         }
         addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
@@ -183,6 +327,14 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
                                 uint64_t *ste)
 {
     uint64_t val = ste[0];
+    struct arm_smmu_evtq_ent ent = {
+        .opcode = EVT_ID_BAD_STE,
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
 
     if ( !(val & STRTAB_STE_0_V) )
         return -EAGAIN;
@@ -217,6 +369,7 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
     return 0;
 
 bad_ste:
+    arm_vsmmu_send_event(smmu, &ent);
     return -EINVAL;
 }
 
@@ -577,7 +730,8 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
     .write = vsmmuv3_mmio_write,
 };
 
-static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
+static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
+                               paddr_t size, uint32_t virq)
 {
     struct virt_smmu *smmu;
 
@@ -586,6 +740,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
         return -ENOMEM;
 
     smmu->d = d;
+    smmu->virq = virq;
     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
@@ -612,14 +767,16 @@ int domain_vsmmuv3_init(struct domain *d)
 
         list_for_each_entry(hw_iommu, &host_iommu_list, entry)
         {
-            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
+            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size,
+                                      hw_iommu->irq);
             if ( ret )
                 return ret;
         }
     }
     else
     {
-        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE,
+                                  GUEST_VSMMU_SPI);
         if ( ret )
             return ret;
     }
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index ebac02ed63..1b606e20fd 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -527,9 +527,10 @@ typedef uint64_t xen_callback_t;
 #define GUEST_EVTCHN_PPI        31
 
 #define GUEST_VPL011_SPI        32
+#define GUEST_VSMMU_SPI         33
 
-#define GUEST_VIRTIO_MMIO_SPI_FIRST   33
-#define GUEST_VIRTIO_MMIO_SPI_LAST    43
+#define GUEST_VIRTIO_MMIO_SPI_FIRST   34
+#define GUEST_VIRTIO_MMIO_SPI_LAST    44
 
 /*
  * SGI is the preferred delivery mechanism of FF-A pending notifications or
-- 
2.43.0

Re: [PATCH v3 12/23] xen/arm: vsmmuv3: Add support for event queue and global error

[Luca Fancellu]

Hi Milan,

> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 6d3636b18b..7a6c18df53 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -44,6 +44,7 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
> 
> /* Helper Macros */
> #define smmu_get_cmdq_enabled(x)    FIELD_GET(CR0_CMDQEN, x)
> +#define smmu_get_evtq_enabled(x)    FIELD_GET(CR0_EVTQEN, x)
> #define smmu_cmd_get_command(x)     FIELD_GET(CMDQ_0_OP, x)
> #define smmu_cmd_get_sid(x)         FIELD_GET(CMDQ_PREFETCH_0_SID, x)
> #define smmu_get_ste_s1cdmax(x)     FIELD_GET(STRTAB_STE_0_S1CDMAX, x)
> @@ -52,6 +53,35 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
> #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
>                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
> 
> +/* event queue entry */
> +struct arm_smmu_evtq_ent {
> +    /* Common fields */
> +    uint8_t     opcode;
> +    uint32_t    sid;
> +
> +    /* Event-specific fields */
> +    union {
> +        struct {
> +            uint32_t ssid;
> +            bool ssv;
> +        } c_bad_ste_streamid;
> +
> +        struct {
> +            bool stall;
> +            uint16_t stag;
> +            uint32_t ssid;
> +            bool ssv;
> +            bool s2;
> +            uint64_t addr;
> +            bool rnw;
> +            bool pnu;
> +            bool ind;
> +            uint8_t class;
> +            uint64_t addr2;
> +        } f_translation;
> +    };
> +};
> +
> /* stage-1 translation configuration */
> struct arm_vsmmu_s1_trans_cfg {
>     paddr_t s1ctxptr;
> @@ -82,6 +112,7 @@ struct virt_smmu {
>     uint32_t    strtab_base_cfg;
>     uint64_t    strtab_base;
>     uint32_t    irq_ctrl;
> +    uint32_t    virq;
>     uint64_t    gerror_irq_cfg0;
>     uint64_t    evtq_irq_cfg0;
>     struct      arm_vsmmu_queue evtq, cmdq;
> @@ -89,6 +120,12 @@ struct virt_smmu {
> };
> 
> /* Queue manipulation functions */
> +static bool queue_full(struct arm_vsmmu_queue *q)
> +{
> +    return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
> +           Q_WRP(q, q->prod) != Q_WRP(q, q->cons);
> +}
> +
> static bool queue_empty(struct arm_vsmmu_queue *q)
> {
>     return Q_IDX(q, q->prod) == Q_IDX(q, q->cons) &&
> @@ -101,11 +138,105 @@ static void queue_inc_cons(struct arm_vsmmu_queue *q)
>     q->cons = Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons);
> }
> 
> +static void queue_inc_prod(struct arm_vsmmu_queue *q)
> +{
> +    u32 prod = (Q_WRP(q, q->prod) | Q_IDX(q, q->prod)) + 1;
> +    q->prod = Q_OVF(q->prod) | Q_WRP(q, prod) | Q_IDX(q, prod);
> +}
> +
> static void dump_smmu_command(uint64_t *command)
> {
>     gdprintk(XENLOG_ERR, "cmd 0x%02llx: %016lx %016lx\n",
>              smmu_cmd_get_command(command[0]), command[0], command[1]);
> }
> +
> +static void arm_vsmmu_inject_irq(struct virt_smmu *smmu, bool is_gerror,
> +                                uint32_t gerror_err)
> +{
> +    uint32_t new_gerrors, pending;
> +
> +    if ( is_gerror )
> +    {
> +        /* trigger global error irq to guest */
> +        pending = smmu->gerror ^ smmu->gerrorn;
> +        new_gerrors = ~pending & gerror_err;
> +
> +        /* only toggle non pending errors */
> +        if (!new_gerrors)

NIT: codestyle, spaces inside parentheses 

> +            return;
> +
> +        smmu->gerror ^= new_gerrors;
> +    }
> +
> +    vgic_inject_irq(smmu->d, NULL, smmu->virq, true);

I don’t understand this, shouldn’t we have the irq level high or low depending on
event queue or global error pending? Also this doesn’t take into consideration
irq enabled (IRQ_CTRL_EVTQ_IRQEN / IRQ_CTRL_GERROR_IRQEN)

> +}
> +
> +static int arm_vsmmu_write_evtq(struct virt_smmu *smmu, uint64_t *evt)
> +{
> +    struct arm_vsmmu_queue *q = &smmu->evtq;
> +    struct domain *d = smmu->d;
> +    paddr_t addr;
> +    int ret;
> +
> +    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
> +        return -EINVAL;
> +
> +    if ( queue_full(q) )
> +        return -EINVAL;
> +
> +    addr = Q_PROD_ENT(q);
> +    ret = access_guest_memory_by_gpa(d, addr, evt,
> +                                     sizeof(*evt) * EVTQ_ENT_DWORDS, true);
> +    if ( ret )
> +        return ret;
> +
> +    queue_inc_prod(q);
> +
> +    /* trigger eventq irq to guest */
> +    if ( !queue_empty(q) )
> +        arm_vsmmu_inject_irq(smmu, false, 0);
> +
> +    return 0;
> +}
> +
> +void arm_vsmmu_send_event(struct virt_smmu *smmu,
> +                          struct arm_smmu_evtq_ent *ent)
> +{
> +    uint64_t evt[EVTQ_ENT_DWORDS];
> +    int ret;
> +
> +    memset(evt, 0, 1 << EVTQ_ENT_SZ_SHIFT);
> +
> +    if ( !smmu_get_evtq_enabled(smmu->cr[0]) )
> +        return;
> +
> +    evt[0] |= FIELD_PREP(EVTQ_0_ID, ent->opcode);
> +    evt[0] |= FIELD_PREP(EVTQ_0_SID, ent->sid);
> +
> +    switch (ent->opcode)

NIT: codestyle, spaces inside parentheses 

> +    {
> +    case EVT_ID_BAD_STREAMID:
> +    case EVT_ID_BAD_STE:
> +        evt[0] |= FIELD_PREP(EVTQ_0_SSID, ent->c_bad_ste_streamid.ssid);
> +        evt[0] |= FIELD_PREP(EVTQ_0_SSV, ent->c_bad_ste_streamid.ssv);
> +        break;
> +    case EVT_ID_TRANSLATION_FAULT:
> +    case EVT_ID_ADDR_SIZE_FAULT:
> +    case EVT_ID_ACCESS_FAULT:
> +    case EVT_ID_PERMISSION_FAULT:
> +        break;
> +    default:
> +        gdprintk(XENLOG_WARNING, "vSMMUv3: event opcode is bad\n");
> +        break;
> +    }
> +
> +    ret = arm_vsmmu_write_evtq(smmu, evt);
> +    if ( ret )
> +        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
> +
> +    return;
> +}
> +
> static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
>                               uint64_t *ste)
> {
> @@ -114,11 +245,22 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
>     uint32_t log2size;
>     int strtab_size_shift;
>     int ret;
> +    struct arm_smmu_evtq_ent ent = {
> +        .sid = sid,
> +        .c_bad_ste_streamid = {
> +            .ssid = 0,
> +            .ssv = false,
> +        },
> +    };
> 
>     log2size = FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, smmu->strtab_base_cfg);
> 
>     if ( sid >= (1 << MIN(log2size, SMMU_IDR1_SIDSIZE)) )
> +    {
> +        ent.opcode = EVT_ID_BAD_STE;
> +        arm_vsmmu_send_event(smmu, &ent);
>         return -EINVAL;
> +    }
> 
>     if ( smmu->features & STRTAB_BASE_CFG_FMT_2LVL )
>     {
> @@ -156,6 +298,8 @@ static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
>         {
>             gdprintk(XENLOG_ERR, "idx=%d > max_l2_ste=%d\n",
>                      idx, max_l2_ste);
> +            ent.opcode = EVT_ID_BAD_STREAMID;
> +            arm_vsmmu_send_event(smmu, &ent);
>             return -EINVAL;
>         }
>         addr = l2ptr + idx * sizeof(*ste) * STRTAB_STE_DWORDS;
> @@ -183,6 +327,14 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
>                                 uint64_t *ste)
> {
>     uint64_t val = ste[0];
> +    struct arm_smmu_evtq_ent ent = {
> +        .opcode = EVT_ID_BAD_STE,
> +        .sid = sid,
> +        .c_bad_ste_streamid = {
> +            .ssid = 0,
> +            .ssv = false,
> +        },
> +    };
> 
>     if ( !(val & STRTAB_STE_0_V) )
>         return -EAGAIN;
> @@ -217,6 +369,7 @@ static int arm_vsmmu_decode_ste(struct virt_smmu *smmu, uint32_t sid,
>     return 0;
> 
> bad_ste:
> +    arm_vsmmu_send_event(smmu, &ent);
>     return -EINVAL;
> }
> 
> @@ -577,7 +730,8 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
>     .write = vsmmuv3_mmio_write,
> };
> 
> -static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)
> +static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
> +                               paddr_t size, uint32_t virq)
> {
>     struct virt_smmu *smmu;
> 
> @@ -586,6 +740,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr, paddr_t size)

We are now injecting irqs due to this patch, we should vgic_reserve_virq() it as we do in patch
“xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3”, so maybe better to move these changes here.

>         return -ENOMEM;
> 
>     smmu->d = d;
> +    smmu->virq = virq;
>     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
>     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
>     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
> @@ -612,14 +767,16 @@ int domain_vsmmuv3_init(struct domain *d)
> 
>         list_for_each_entry(hw_iommu, &host_iommu_list, entry)
>         {
> -            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size);
> +            ret = vsmmuv3_init_single(d, hw_iommu->addr, hw_iommu->size,
> +                                      hw_iommu->irq);
>             if ( ret )
>                 return ret;
>         }
>     }
>     else
>     {
> -        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
> +        ret = vsmmuv3_init_single(d, GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE,
> +                                  GUEST_VSMMU_SPI);
>         if ( ret )
>             return ret;
>     }
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index ebac02ed63..1b606e20fd 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -527,9 +527,10 @@ typedef uint64_t xen_callback_t;
> #define GUEST_EVTCHN_PPI        31
> 
> #define GUEST_VPL011_SPI        32
> +#define GUEST_VSMMU_SPI         33

This change alone breaks already built toolstack, can we just
have GUEST_VSMMU_SPI as 44 and leave GUEST_VIRTIO_MMIO_SPI_FIRST and
GUEST_VIRTIO_MMIO_SPI_LAST as they are?

> 
> -#define GUEST_VIRTIO_MMIO_SPI_FIRST   33
> -#define GUEST_VIRTIO_MMIO_SPI_LAST    43
> +#define GUEST_VIRTIO_MMIO_SPI_FIRST   34
> +#define GUEST_VIRTIO_MMIO_SPI_LAST    44
> 
> /*
>  * SGI is the preferred delivery mechanism of FF-A pending notifications or
> 

Cheers,
Luca

[PATCH v3 13/23] xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

"iommus" property will be added for dom0 devices to virtual
IOMMU node to enable the dom0 linux kernel to configure the IOMMU

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain_build.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index a51563ee3d..30a4f1fd09 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -538,9 +538,12 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
             continue;
         }
 
-        if ( iommu_node )
+        /*
+         * Expose IOMMU specific properties to hwdom when vIOMMU is
+         * enabled.
+         */
+        if ( iommu_node && !is_viommu_enabled() )
         {
-            /* Don't expose IOMMU specific properties to hwdom */
             if ( dt_property_name_is_equal(prop, "iommus") )
                 continue;
 
-- 
2.43.0

Re: [PATCH v3 13/23] xen/arm: vsmmuv3: Add "iommus" property node for dom0 devices

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> "iommus" property will be added for dom0 devices to virtual
> IOMMU node to enable the dom0 linux kernel to configure the IOMMU
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/arch/arm/domain_build.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index a51563ee3d..30a4f1fd09 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -538,9 +538,12 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>             continue;
>         }
> 
> -        if ( iommu_node )
> +        /*
> +         * Expose IOMMU specific properties to hwdom when vIOMMU is
> +         * enabled.
> +         */
> +        if ( iommu_node && !is_viommu_enabled() )
>         {
> -            /* Don't expose IOMMU specific properties to hwdom */
>             if ( dt_property_name_is_equal(prop, "iommus") )
>                 continue;
> 
> 

I think this commit should be squashed with "xen/arm: vIOMMU: IOMMU device tree node for dom0”
so that we can have a consistent handling of the DT properties.

Cheers,
Luca

[PATCH v3 14/23] xen/arm: vIOMMU: IOMMU device tree node for dom0

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

XEN will create an IOMMU device tree node in the device tree
to enable the dom0 to discover the virtual SMMUv3 during dom0 boot.
IOMMU device tree node will only be created when cmdline option viommu
is enabled.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/domain_build.c       | 94 +++++++++++++++++++++++++++++++
 xen/arch/arm/include/asm/viommu.h |  1 +
 2 files changed, 95 insertions(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 30a4f1fd09..54ae3b5033 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1440,6 +1440,95 @@ int __init make_timer_node(const struct kernel_info *kinfo)
     return res;
 }
 
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+static int make_hwdom_viommu_node(const struct kernel_info *kinfo)
+{
+    uint32_t len;
+    int res;
+    char buf[24];
+    void *fdt = kinfo->fdt;
+    const void *prop = NULL;
+    const struct dt_device_node *iommu = NULL;
+    struct host_iommu *iommu_data;
+    gic_interrupt_t intr;
+
+    if ( list_empty(&host_iommu_list) )
+        return 0;
+
+    list_for_each_entry( iommu_data, &host_iommu_list, entry )
+    {
+        if ( iommu_data->hwdom_node_created )
+            return 0;
+
+        iommu = iommu_data->dt_node;
+
+        snprintf(buf, sizeof(buf), "iommu@%"PRIx64, iommu_data->addr);
+
+        res = fdt_begin_node(fdt, buf);
+        if ( res )
+            return res;
+
+        prop = dt_get_property(iommu, "compatible", &len);
+        if ( !prop )
+        {
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "compatible", prop, len);
+        if ( res )
+            return res;
+
+        if ( iommu->phandle )
+        {
+            res = fdt_property_cell(fdt, "phandle", iommu->phandle);
+            if ( res )
+                return res;
+        }
+
+        /* Use the same reg regions as the IOMMU node in host DTB. */
+        prop = dt_get_property(iommu, "reg", &len);
+        if ( !prop )
+        {
+            printk(XENLOG_ERR "vIOMMU: Can't find IOMMU reg property.\n");
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "reg", prop, len);
+        if ( res )
+            return res;
+
+        prop = dt_get_property(iommu, "#iommu-cells", &len);
+        if ( !prop )
+        {
+            res = -FDT_ERR_XEN(ENOENT);
+            return res;
+        }
+
+        res = fdt_property(fdt, "#iommu-cells", prop, len);
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "interrupt-names", "combined");
+        if ( res )
+            return res;
+
+        set_interrupt(intr, iommu_data->irq, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+        res = fdt_property_interrupts(kinfo, &intr, 1);
+        if ( res )
+            return res;
+
+        iommu_data->hwdom_node_created = true;
+
+        fdt_end_node(fdt);
+    }
+
+    return res;
+}
+#endif
+
 static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
                               struct dt_device_node *node,
                               p2m_type_t p2mt)
@@ -1508,6 +1597,11 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
     if ( dt_match_node(timer_matches, node) )
         return make_timer_node(kinfo);
 
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+    if ( device_get_class(node) == DEVICE_IOMMU && is_viommu_enabled() )
+        return make_hwdom_viommu_node(kinfo);
+#endif
+
     /* Skip nodes used by Xen */
     if ( dt_device_used_by(node) == DOMID_XEN )
     {
diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index ed338fe0ec..01d4d0dfef 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -19,6 +19,7 @@ struct host_iommu {
     paddr_t addr;
     paddr_t size;
     uint32_t irq;
+    bool hwdom_node_created;
 };
 
 struct viommu_ops {
-- 
2.43.0

Re: [PATCH v3 14/23] xen/arm: vIOMMU: IOMMU device tree node for dom0

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> XEN will create an IOMMU device tree node in the device tree
> to enable the dom0 to discover the virtual SMMUv3 during dom0 boot.
> IOMMU device tree node will only be created when cmdline option viommu
> is enabled.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/arch/arm/domain_build.c       | 94 +++++++++++++++++++++++++++++++
> xen/arch/arm/include/asm/viommu.h |  1 +
> 2 files changed, 95 insertions(+)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 30a4f1fd09..54ae3b5033 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1440,6 +1440,95 @@ int __init make_timer_node(const struct kernel_info *kinfo)
>     return res;
> }
> 
> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
> +static int make_hwdom_viommu_node(const struct kernel_info *kinfo)

__init ?

The rest looks ok to me, with that fixed:

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

Cheers,
Luca

[PATCH v3 15/23] xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

XEN will create an Emulated SMMUv3 device tree node in the device tree
to enable the dom0less domains to discover the virtual SMMUv3 during
boot. Emulated SMMUv3 device tree node will only be created when cmdline
option vsmmuv3 is enabled.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/dom0less-build.c         | 53 +++++++++++++++++++++++++++
 xen/include/public/device_tree_defs.h |  1 +
 2 files changed, 54 insertions(+)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 067835e5d0..a48edb9568 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -218,10 +218,63 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
 }
 #endif
 
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
+{
+    int res;
+    char buf[24];
+    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
+    __be32 *cells;
+    void *fdt = kinfo->fdt;
+
+    snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);
+
+    res = fdt_begin_node(fdt, buf);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", "arm,smmu-v3");
+    if ( res )
+        return res;
+
+    /* Create reg property */
+    cells = &reg[0];
+    dt_child_set_range(&cells, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+                       GUEST_VSMMUV3_BASE, GUEST_VSMMUV3_SIZE);
+    res = fdt_property(fdt, "reg", reg,
+                       (GUEST_ROOT_ADDRESS_CELLS +
+                       GUEST_ROOT_SIZE_CELLS) * sizeof(*reg));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_VSMMUV3);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#iommu-cells", 1);
+    if ( res )
+        return res;
+
+    res = fdt_end_node(fdt);
+
+    return res;
+}
+#endif
+
 int __init make_arch_nodes(struct kernel_info *kinfo)
 {
     int ret;
 
+
+#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
+    if ( is_viommu_enabled() )
+    {
+        ret = make_vsmmuv3_node(kinfo);
+        if ( ret )
+            return -EINVAL;
+    }
+#endif
+
     ret = make_psci_node(kinfo->fdt);
     if ( ret )
         return -EINVAL;
diff --git a/xen/include/public/device_tree_defs.h b/xen/include/public/device_tree_defs.h
index 9e80d0499d..7846a0425c 100644
--- a/xen/include/public/device_tree_defs.h
+++ b/xen/include/public/device_tree_defs.h
@@ -14,6 +14,7 @@
  */
 #define GUEST_PHANDLE_GIC (65000)
 #define GUEST_PHANDLE_IOMMU (GUEST_PHANDLE_GIC + 1)
+#define GUEST_PHANDLE_VSMMUV3 (GUEST_PHANDLE_IOMMU + 1)
 
 #define GUEST_ROOT_ADDRESS_CELLS 2
 #define GUEST_ROOT_SIZE_CELLS 2
-- 
2.43.0

Re: [PATCH v3 15/23] xen/arm: vsmmuv3: Emulated SMMUv3 device tree node for dom0less

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> XEN will create an Emulated SMMUv3 device tree node in the device tree
> to enable the dom0less domains to discover the virtual SMMUv3 during
> boot. Emulated SMMUv3 device tree node will only be created when cmdline
> option vsmmuv3 is enabled.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/arch/arm/dom0less-build.c         | 53 +++++++++++++++++++++++++++
> xen/include/public/device_tree_defs.h |  1 +
> 2 files changed, 54 insertions(+)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 067835e5d0..a48edb9568 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -218,10 +218,63 @@ static int __init make_vpl011_uart_node(struct kernel_info *kinfo)
> }
> #endif
> 
> +#ifdef CONFIG_VIRTUAL_ARM_SMMU_V3
> +static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
> +{
> +    int res;
> +    char buf[24];
> +    __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
> +    __be32 *cells;
> +    void *fdt = kinfo->fdt;
> +
> +    snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);

PRIx64 ?

The rest looks ok to me, with that fixed:

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

Cheers,
Luca

[PATCH v3 16/23] arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

libxl will create an Emulated SMMUv3 device tree node in the device
tree to enable the guest OS to discover the virtual SMMUv3 during guest
boot.

Emulated SMMUv3 device tree node will only be created when
"viommu=smmuv3" is set in xl domain configuration.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c | 37 ++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index a248793588..eb879473f5 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -901,6 +901,36 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
+                             const struct arch_info *ainfo,
+                             struct xc_dom_image *dom)
+{
+    int res;
+    const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);
+
+    res = fdt_begin_node(fdt, name);
+    if (res) return res;
+
+    res = fdt_property_compat(gc, fdt, 1, "arm,smmu-v3");
+    if (res) return res;
+
+    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
+                            GUEST_ROOT_SIZE_CELLS, 1, GUEST_VSMMUV3_BASE,
+                            GUEST_VSMMUV3_SIZE);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_VSMMUV3);
+    if (res) return res;
+
+    res = fdt_property_cell(fdt, "#iommu-cells", 1);
+    if (res) return res;
+
+    res = fdt_end_node(fdt);
+    if (res) return res;
+
+    return 0;
+}
+
 static int make_vpci_node(libxl__gc *gc, void *fdt,
                           const struct arch_info *ainfo,
                           struct xc_dom_image *dom)
@@ -942,6 +972,10 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
         GUEST_VPCI_PREFETCH_MEM_SIZE);
     if (res) return res;
 
+    res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
+                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
+    if (res) return res;
+
     res = fdt_end_node(fdt);
     if (res) return res;
 
@@ -1408,6 +1442,9 @@ next_resize:
         if (d_config->num_pcidevs)
             FDT( make_vpci_node(gc, fdt, ainfo, dom) );
 
+        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3)
+            FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
+
         for (i = 0; i < d_config->num_disks; i++) {
             libxl_device_disk *disk = &d_config->disks[i];
 
-- 
2.43.0

Re: [PATCH v3 16/23] arm/libxl: vsmmuv3: Emulated SMMUv3 device tree node in libxl

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> libxl will create an Emulated SMMUv3 device tree node in the device
> tree to enable the guest OS to discover the virtual SMMUv3 during guest
> boot.
> 
> Emulated SMMUv3 device tree node will only be created when
> "viommu=smmuv3" is set in xl domain configuration.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> tools/libs/light/libxl_arm.c | 37 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 37 insertions(+)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index a248793588..eb879473f5 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -901,6 +901,36 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
>     return 0;
> }
> 
> +static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
> +                             const struct arch_info *ainfo,
> +                             struct xc_dom_image *dom)
> +{
> +    int res;
> +    const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);

PRIx64 ?

> +
> +    res = fdt_begin_node(fdt, name);
> +    if (res) return res;
> +
> +    res = fdt_property_compat(gc, fdt, 1, "arm,smmu-v3");
> +    if (res) return res;
> +
> +    res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS,
> +                            GUEST_ROOT_SIZE_CELLS, 1, GUEST_VSMMUV3_BASE,
> +                            GUEST_VSMMUV3_SIZE);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "phandle", GUEST_PHANDLE_VSMMUV3);
> +    if (res) return res;
> +
> +    res = fdt_property_cell(fdt, "#iommu-cells", 1);
> +    if (res) return res;
> +
> +    res = fdt_end_node(fdt);
> +    if (res) return res;
> +
> +    return 0;
> +}
> +
> static int make_vpci_node(libxl__gc *gc, void *fdt,
>                           const struct arch_info *ainfo,
>                           struct xc_dom_image *dom)
> @@ -942,6 +972,10 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
>         GUEST_VPCI_PREFETCH_MEM_SIZE);
>     if (res) return res;
> 
> +    res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
> +                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
> +    if (res) return res;

Shouldn’t this one be protected by (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3)?

Cheers,
Luca

[PATCH v3 17/23] xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Alloc and reserve virq for event queue and global error to send event to
guests. Also Modify the libxl to accomadate the new define virq.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c           | 28 ++++++++++++++++++++++++--
 xen/arch/arm/dom0less-build.c          | 17 ++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.c | 13 ++++++++++++
 3 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index eb879473f5..803c3b39b7 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -86,8 +86,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
 {
     uint32_t nr_spis = 0, cfg_nr_spis = d_config->b_info.arch_arm.nr_spis;
     unsigned int i;
-    uint32_t vuart_irq, virtio_irq = 0;
-    bool vuart_enabled = false, virtio_enabled = false;
+    uint32_t vuart_irq, virtio_irq = 0, vsmmu_irq = 0;
+    bool vuart_enabled = false, virtio_enabled = false, vsmmu_enabled = false;
     uint64_t virtio_mmio_base = GUEST_VIRTIO_MMIO_BASE;
     uint32_t virtio_mmio_irq = GUEST_VIRTIO_MMIO_SPI_FIRST;
     int rc;
@@ -102,6 +102,16 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
         vuart_enabled = true;
     }
 
+    /*
+     * If smmuv3 viommu is enabled then increment the nr_spis to allow allocation
+     * of SPI VIRQ for VSMMU.
+     */
+    if (d_config->b_info.arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
+        nr_spis += (GUEST_VSMMU_SPI - 32) + 1;
+        vsmmu_irq = GUEST_VSMMU_SPI;
+        vsmmu_enabled = true;
+    }
+
     for (i = 0; i < d_config->num_disks; i++) {
         libxl_device_disk *disk = &d_config->disks[i];
 
@@ -170,6 +180,11 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
             return ERROR_FAIL;
         }
 
+        if (vsmmu_enabled && irq == vsmmu_irq) {
+            LOG(ERROR, "Physical IRQ %u conflicting with vSMMUv3 SPI\n", irq);
+            return ERROR_FAIL;
+        }
+
         if (irq < 32)
             continue;
 
@@ -907,6 +922,7 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
 {
     int res;
     const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);
+    gic_interrupt intr;
 
     res = fdt_begin_node(fdt, name);
     if (res) return res;
@@ -925,6 +941,14 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
     res = fdt_property_cell(fdt, "#iommu-cells", 1);
     if (res) return res;
 
+    res = fdt_property_string(fdt, "interrupt-names", "combined");
+    if (res) return res;
+
+    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property_interrupts(gc, fdt, &intr, 1);
+    if (res) return res;
+
     res = fdt_end_node(fdt);
     if (res) return res;
 
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index a48edb9568..7380753fa2 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -225,6 +225,7 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
     char buf[24];
     __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
     __be32 *cells;
+    gic_interrupt_t intr;
     void *fdt = kinfo->fdt;
 
     snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);
@@ -255,6 +256,22 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
     if ( res )
         return res;
 
+    res = fdt_property_string(fdt, "interrupt-names", "combined");
+    if ( res )
+        return res;
+
+    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
+
+    res = fdt_property(kinfo->fdt, "interrupts",
+                       intr, sizeof(intr));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(kinfo->fdt, "interrupt-parent",
+                            kinfo->phandle_intc);
+    if ( res )
+        return res;
+
     res = fdt_end_node(fdt);
 
     return res;
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 7a6c18df53..a5b9700369 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -733,6 +733,7 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
 static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
                                paddr_t size, uint32_t virq)
 {
+    int ret;
     struct virt_smmu *smmu;
 
     smmu = xzalloc(struct virt_smmu);
@@ -748,12 +749,24 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
 
     spin_lock_init(&smmu->cmd_queue_lock);
 
+    ret = vgic_reserve_virq(d, virq);
+    if ( !ret )
+    {
+        ret = -EINVAL;
+        goto out;
+    }
+
     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
 
     /* Register the vIOMMU to be able to clean it up later. */
     list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
 
     return 0;
+
+out:
+    xfree(smmu);
+    vgic_free_virq(d, virq);
+    return ret;
 }
 
 int domain_vsmmuv3_init(struct domain *d)
-- 
2.43.0

Re: [PATCH v3 17/23] xen/arm: vsmmuv3: Alloc virq for virtual SMMUv3

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Alloc and reserve virq for event queue and global error to send event to
> guests. Also Modify the libxl to accomadate the new define virq.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> tools/libs/light/libxl_arm.c           | 28 ++++++++++++++++++++++++--
> xen/arch/arm/dom0less-build.c          | 17 ++++++++++++++++
> xen/drivers/passthrough/arm/vsmmu-v3.c | 13 ++++++++++++
> 3 files changed, 56 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index eb879473f5..803c3b39b7 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -86,8 +86,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> {
>     uint32_t nr_spis = 0, cfg_nr_spis = d_config->b_info.arch_arm.nr_spis;
>     unsigned int i;
> -    uint32_t vuart_irq, virtio_irq = 0;
> -    bool vuart_enabled = false, virtio_enabled = false;
> +    uint32_t vuart_irq, virtio_irq = 0, vsmmu_irq = 0;
> +    bool vuart_enabled = false, virtio_enabled = false, vsmmu_enabled = false;
>     uint64_t virtio_mmio_base = GUEST_VIRTIO_MMIO_BASE;
>     uint32_t virtio_mmio_irq = GUEST_VIRTIO_MMIO_SPI_FIRST;
>     int rc;
> @@ -102,6 +102,16 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>         vuart_enabled = true;
>     }
> 
> +    /*
> +     * If smmuv3 viommu is enabled then increment the nr_spis to allow allocation
> +     * of SPI VIRQ for VSMMU.
> +     */
> +    if (d_config->b_info.arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
> +        nr_spis += (GUEST_VSMMU_SPI - 32) + 1;
> +        vsmmu_irq = GUEST_VSMMU_SPI;
> +        vsmmu_enabled = true;
> +    }

Now we would inflate too much nr_spis if also vPL011 is present, I think this commit should
modify that part and this part in this way:

if (d_config->b_info.arch_arm.vuart == LIBXL_VUART_TYPE_SBSA_UART) {
    nr_spis = max(nr_spis, GUEST_VPL011_SPI - 32 + 1);
    vuart_irq = GUEST_VPL011_SPI;
    vuart_enabled = true;
}

if (d_config->b_info.arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
    nr_spis = max(nr_spis, GUEST_VSMMU_SPI - 32 + 1);
    vsmmu_irq = GUEST_VSMMU_SPI;
    vsmmu_enabled = true;
}

as done here https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/light/libxl_arm.c;h=7e9f8a1bc3666ac9a3f8aa487366399dc255b149;hb=refs/heads/staging#l144


> +
>     for (i = 0; i < d_config->num_disks; i++) {
>         libxl_device_disk *disk = &d_config->disks[i];
> 
> @@ -170,6 +180,11 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
>             return ERROR_FAIL;
>         }
> 
> +        if (vsmmu_enabled && irq == vsmmu_irq) {
> +            LOG(ERROR, "Physical IRQ %u conflicting with vSMMUv3 SPI\n", irq);
> +            return ERROR_FAIL;
> +        }
> +
>         if (irq < 32)
>             continue;
> 
> @@ -907,6 +922,7 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
> {
>     int res;
>     const char *name = GCSPRINTF("iommu@%llx", GUEST_VSMMUV3_BASE);
> +    gic_interrupt intr;
> 
>     res = fdt_begin_node(fdt, name);
>     if (res) return res;
> @@ -925,6 +941,14 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
>     res = fdt_property_cell(fdt, "#iommu-cells", 1);
>     if (res) return res;
> 
> +    res = fdt_property_string(fdt, "interrupt-names", "combined");
> +    if (res) return res;
> +
> +    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> +
> +    res = fdt_property_interrupts(gc, fdt, &intr, 1);
> +    if (res) return res;
> +
>     res = fdt_end_node(fdt);
>     if (res) return res;
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index a48edb9568..7380753fa2 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -225,6 +225,7 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
>     char buf[24];
>     __be32 reg[GUEST_ROOT_ADDRESS_CELLS + GUEST_ROOT_SIZE_CELLS];
>     __be32 *cells;
> +    gic_interrupt_t intr;
>     void *fdt = kinfo->fdt;
> 
>     snprintf(buf, sizeof(buf), "iommu@%llx", GUEST_VSMMUV3_BASE);
> @@ -255,6 +256,22 @@ static int __init make_vsmmuv3_node(const struct kernel_info *kinfo)
>     if ( res )
>         return res;
> 
> +    res = fdt_property_string(fdt, "interrupt-names", "combined");
> +    if ( res )
> +        return res;
> +
> +    set_interrupt(intr, GUEST_VSMMU_SPI, 0xf, DT_IRQ_TYPE_LEVEL_HIGH);
> +
> +    res = fdt_property(kinfo->fdt, "interrupts",
> +                       intr, sizeof(intr));
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(kinfo->fdt, "interrupt-parent",
> +                            kinfo->phandle_intc);
> +    if ( res )
> +        return res;
> +
>     res = fdt_end_node(fdt);
> 
>     return res;
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 7a6c18df53..a5b9700369 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -733,6 +733,7 @@ static const struct mmio_handler_ops vsmmuv3_mmio_handler = {
> static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
>                                paddr_t size, uint32_t virq)
> {
> +    int ret;
>     struct virt_smmu *smmu;
> 
>     smmu = xzalloc(struct virt_smmu);
> @@ -748,12 +749,24 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
> 
>     spin_lock_init(&smmu->cmd_queue_lock);
> 
> +    ret = vgic_reserve_virq(d, virq);
> +    if ( !ret )
> +    {
> +        ret = -EINVAL;
> +        goto out;
> +    }
> +
>     register_mmio_handler(d, &vsmmuv3_mmio_handler, addr, size, smmu);
> 
>     /* Register the vIOMMU to be able to clean it up later. */
>     list_add_tail(&smmu->viommu_list, &d->arch.viommu_list);
> 
>     return 0;
> +
> +out:
> +    xfree(smmu);
> +    vgic_free_virq(d, virq);

I don’t think we should use vgic_free_virq here, if it was reserved already there is some
bug, the virq can be GUEST_VSMMU_SPI for guests and hw_iommu->irq for HW domain,
so if it’s reserved it should be a configuration issue and freeing it would remove “someone else”
reserved irq.
Probably I would print something in the if above and also I would just have the content of this
Label-path inside the if branch, since it is the only consumer of this label.

> +    return ret;
> }
> 
> int domain_vsmmuv3_init(struct domain *d)
> 

Cheers,
Luca

[PATCH v3 18/23] xen/arm: vsmmuv3: Add support to send stage-1 event to guest

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

Stage-1 translation is handled by guest, therefore stage-1 fault has to
be forwarded to guest.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c  | 48 ++++++++++++++++++++++++--
 xen/drivers/passthrough/arm/vsmmu-v3.c | 45 ++++++++++++++++++++++++
 xen/drivers/passthrough/arm/vsmmu-v3.h | 12 +++++++
 3 files changed, 103 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index cf8f638a49..4c1951d753 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -869,7 +869,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 	return 0;
 }
 
-__maybe_unused
 static struct arm_smmu_master *
 arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 {
@@ -890,10 +889,51 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 	return NULL;
 }
 
+static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
+{
+	int ret;
+	struct arm_smmu_master *master;
+	u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]);
+
+	switch (FIELD_GET(EVTQ_0_ID, evt[0])) {
+	case EVT_ID_TRANSLATION_FAULT:
+		break;
+	case EVT_ID_ADDR_SIZE_FAULT:
+		break;
+	case EVT_ID_ACCESS_FAULT:
+		break;
+	case EVT_ID_PERMISSION_FAULT:
+		break;
+	default:
+		return -EOPNOTSUPP;
+	}
+
+	/* Stage-2 event */
+	if (evt[1] & EVTQ_1_S2)
+		return -EFAULT;
+
+	mutex_lock(&smmu->streams_mutex);
+	master = arm_smmu_find_master(smmu, sid);
+	if (!master) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	ret = arm_vsmmu_handle_evt(master->domain->d, smmu->dev, evt);
+	if (ret) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+out_unlock:
+	mutex_unlock(&smmu->streams_mutex);
+	return ret;
+}
+
 /* IRQ and event handlers */
 static void arm_smmu_evtq_tasklet(void *dev)
 {
-	int i;
+	int i, ret;
 	struct arm_smmu_device *smmu = dev;
 	struct arm_smmu_queue *q = &smmu->evtq.q;
 	struct arm_smmu_ll_queue *llq = &q->llq;
@@ -903,6 +943,10 @@ static void arm_smmu_evtq_tasklet(void *dev)
 		while (!queue_remove_raw(q, evt)) {
 			u8 id = FIELD_GET(EVTQ_0_ID, evt[0]);
 
+			ret = arm_smmu_handle_evt(smmu, evt);
+			if (!ret)
+				continue;
+
 			dev_info(smmu->dev, "event 0x%02x received:\n", id);
 			for (i = 0; i < ARRAY_SIZE(evt); ++i)
 				dev_info(smmu->dev, "\t0x%016llx\n",
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index a5b9700369..5d0dabd2b2 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -103,6 +103,7 @@ struct arm_vsmmu_queue {
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
+    paddr_t     addr;
     uint8_t     sid_split;
     uint32_t    features;
     uint32_t    cr[3];
@@ -237,6 +238,49 @@ void arm_vsmmu_send_event(struct virt_smmu *smmu,
     return;
 }
 
+static struct virt_smmu *vsmmuv3_find_by_addr(struct domain *d, paddr_t paddr)
+{
+    struct virt_smmu *smmu;
+
+    list_for_each_entry( smmu, &d->arch.viommu_list, viommu_list )
+    {
+        if ( smmu->addr == paddr )
+            return smmu;
+    }
+
+    return NULL;
+}
+
+int arm_vsmmu_handle_evt(struct domain *d, struct device *dev, uint64_t *evt)
+{
+    int ret;
+    struct virt_smmu *smmu;
+
+    if ( is_hardware_domain(d) )
+    {
+        paddr_t paddr;
+        /* Base address */
+        ret = dt_device_get_address(dev_to_dt(dev), 0, &paddr, NULL);
+        if ( ret )
+            return -EINVAL;
+
+        smmu = vsmmuv3_find_by_addr(d, paddr);
+        if ( !smmu )
+            return -ENODEV;
+    }
+    else
+    {
+        smmu = list_entry(d->arch.viommu_list.next,
+                          struct virt_smmu, viommu_list);
+    }
+
+    ret = arm_vsmmu_write_evtq(smmu, evt);
+    if ( ret )
+        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
+
+    return 0;
+}
+
 static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
                               uint64_t *ste)
 {
@@ -742,6 +786,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
 
     smmu->d = d;
     smmu->virq = virq;
+    smmu->addr = addr;
     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
index e11f85b431..c7bfd3fb59 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.h
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
@@ -8,6 +8,12 @@
 
 void vsmmuv3_set_type(void);
 
+static inline int arm_vsmmu_handle_evt(struct domain *d,
+                                       struct device *dev, uint64_t *evt)
+{
+    return -EINVAL;
+}
+
 #else
 
 static inline void vsmmuv3_set_type(void)
@@ -15,6 +21,12 @@ static inline void vsmmuv3_set_type(void)
     return;
 }
 
+static inline int arm_vsmmu_handle_evt(struct domain *d,
+                                       struct device *dev, uint64_t *evt)
+{
+    return -EINVAL;
+}
+
 #endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
 
 #endif /* __ARCH_ARM_VSMMU_V3_H__ */
-- 
2.43.0

Re: [PATCH v3 18/23] xen/arm: vsmmuv3: Add support to send stage-1 event to guest

[Luca Fancellu]

HI Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> Stage-1 translation is handled by guest, therefore stage-1 fault has to
> be forwarded to guest.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/drivers/passthrough/arm/smmu-v3.c  | 48 ++++++++++++++++++++++++--
> xen/drivers/passthrough/arm/vsmmu-v3.c | 45 ++++++++++++++++++++++++
> xen/drivers/passthrough/arm/vsmmu-v3.h | 12 +++++++
> 3 files changed, 103 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index cf8f638a49..4c1951d753 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -869,7 +869,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
> return 0;
> }
> 
> -__maybe_unused
> static struct arm_smmu_master *
> arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
> {
> @@ -890,10 +889,51 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
> return NULL;
> }
> 
> +static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
> +{
> + int ret;
> + struct arm_smmu_master *master;
> + u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]);
> +
> + switch (FIELD_GET(EVTQ_0_ID, evt[0])) {
> + case EVT_ID_TRANSLATION_FAULT:
> + break;
> + case EVT_ID_ADDR_SIZE_FAULT:
> + break;
> + case EVT_ID_ACCESS_FAULT:
> + break;
> + case EVT_ID_PERMISSION_FAULT:
> + break;
> + default:
> + return -EOPNOTSUPP;
> + }
> +
> + /* Stage-2 event */
> + if (evt[1] & EVTQ_1_S2)
> + return -EFAULT;
> +
> + mutex_lock(&smmu->streams_mutex);
> + master = arm_smmu_find_master(smmu, sid);
> + if (!master) {
> + ret = -EINVAL;
> + goto out_unlock;
> + }
> +
> + ret = arm_vsmmu_handle_evt(master->domain->d, smmu->dev, evt);
> + if (ret) {
> + ret = -EINVAL;
> + goto out_unlock;
> + }
> +
> +out_unlock:
> + mutex_unlock(&smmu->streams_mutex);
> + return ret;
> +}
> +
> /* IRQ and event handlers */
> static void arm_smmu_evtq_tasklet(void *dev)
> {
> - int i;
> + int i, ret;
> struct arm_smmu_device *smmu = dev;
> struct arm_smmu_queue *q = &smmu->evtq.q;
> struct arm_smmu_ll_queue *llq = &q->llq;
> @@ -903,6 +943,10 @@ static void arm_smmu_evtq_tasklet(void *dev)
> while (!queue_remove_raw(q, evt)) {
> u8 id = FIELD_GET(EVTQ_0_ID, evt[0]);
> 
> + ret = arm_smmu_handle_evt(smmu, evt);
> + if (!ret)
> + continue;
> +
> dev_info(smmu->dev, "event 0x%02x received:\n", id);
> for (i = 0; i < ARRAY_SIZE(evt); ++i)
> dev_info(smmu->dev, "\t0x%016llx\n",
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index a5b9700369..5d0dabd2b2 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -103,6 +103,7 @@ struct arm_vsmmu_queue {
> struct virt_smmu {
>     struct      domain *d;
>     struct      list_head viommu_list;
> +    paddr_t     addr;
>     uint8_t     sid_split;
>     uint32_t    features;
>     uint32_t    cr[3];
> @@ -237,6 +238,49 @@ void arm_vsmmu_send_event(struct virt_smmu *smmu,
>     return;
> }
> 
> +static struct virt_smmu *vsmmuv3_find_by_addr(struct domain *d, paddr_t paddr)
> +{
> +    struct virt_smmu *smmu;
> +
> +    list_for_each_entry( smmu, &d->arch.viommu_list, viommu_list )
> +    {
> +        if ( smmu->addr == paddr )
> +            return smmu;
> +    }
> +
> +    return NULL;
> +}
> +
> +int arm_vsmmu_handle_evt(struct domain *d, struct device *dev, uint64_t *evt)
> +{
> +    int ret;
> +    struct virt_smmu *smmu;
> +
> +    if ( is_hardware_domain(d) )
> +    {
> +        paddr_t paddr;
> +        /* Base address */
> +        ret = dt_device_get_address(dev_to_dt(dev), 0, &paddr, NULL);
> +        if ( ret )
> +            return -EINVAL;
> +
> +        smmu = vsmmuv3_find_by_addr(d, paddr);
> +        if ( !smmu )
> +            return -ENODEV;
> +    }
> +    else
> +    {
> +        smmu = list_entry(d->arch.viommu_list.next,
> +                          struct virt_smmu, viommu_list);
> +    }
> +
> +    ret = arm_vsmmu_write_evtq(smmu, evt);
> +    if ( ret )
> +        arm_vsmmu_inject_irq(smmu, true, GERROR_EVTQ_ABT_ERR);
> +
> +    return 0;
> +}
> +
> static int arm_vsmmu_find_ste(struct virt_smmu *smmu, uint32_t sid,
>                               uint64_t *ste)
> {
> @@ -742,6 +786,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
> 
>     smmu->d = d;
>     smmu->virq = virq;
> +    smmu->addr = addr;
>     smmu->cmdq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_CMDQS);
>     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
>     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.h b/xen/drivers/passthrough/arm/vsmmu-v3.h
> index e11f85b431..c7bfd3fb59 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.h
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.h
> @@ -8,6 +8,12 @@
> 
> void vsmmuv3_set_type(void);
> 
> +static inline int arm_vsmmu_handle_evt(struct domain *d,
> +                                       struct device *dev, uint64_t *evt)
> +{
> +    return -EINVAL;
> +}

Shouldn’t this be the prototype of arm_vsmmu_handle_evt() instead of a stub implementation?

> +
> #else
> 
> static inline void vsmmuv3_set_type(void)
> @@ -15,6 +21,12 @@ static inline void vsmmuv3_set_type(void)
>     return;
> }
> 
> +static inline int arm_vsmmu_handle_evt(struct domain *d,
> +                                       struct device *dev, uint64_t *evt)
> +{
> +    return -EINVAL;
> +}
> +
> #endif /* CONFIG_VIRTUAL_ARM_SMMU_V3 */
> 
> #endif /* __ARCH_ARM_VSMMU_V3_H__ */
> 

Cheers,
Luca

[PATCH v3 19/23] libxl/arm: vIOMMU: Modify the partial device tree for iommus

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

To configure IOMMU in guest for passthrough devices, user will need to
copy the unmodified "iommus" property from host device tree to partial
device tree. To enable the dom0 linux kernel to confiure the IOMMU
correctly replace the phandle in partial device tree with virtual
IOMMU phandle when "iommus" property is set.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Singed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/libs/light/libxl_arm.c | 47 +++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 803c3b39b7..7b887898bb 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1326,6 +1326,41 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return 0;
 }
 
+static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+{
+    int nodeoff, proplen, i, r;
+    const fdt32_t *prop;
+    fdt32_t *prop_c;
+
+    nodeoff = fdt_path_offset(pfdt, "/passthrough");
+    if (nodeoff < 0)
+        return nodeoff;
+
+    for (nodeoff = fdt_first_subnode(pfdt, nodeoff);
+         nodeoff >= 0;
+         nodeoff = fdt_next_subnode(pfdt, nodeoff)) {
+
+        prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+        if (!prop)
+            continue;
+
+        prop_c = libxl__zalloc(gc, proplen);
+
+        for (i = 0; i < proplen / 8; ++i) {
+            prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+            prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        }
+
+        r = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
+        if (r) {
+            LOG(ERROR, "Can't set the iommus property in partial FDT");
+            return r;
+        }
+    }
+
+    return 0;
+}
+
 #else
 
 static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
@@ -1344,6 +1379,13 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return -FDT_ERR_INTERNAL;
 }
 
+static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+{
+    LOG(ERROR, "partial device tree not supported");
+
+    return ERROR_FAIL;
+}
+
 #endif /* ENABLE_PARTIAL_DEVICE_TREE */
 
 #define FDT_MAX_SIZE (1<<20)
@@ -1466,8 +1508,11 @@ next_resize:
         if (d_config->num_pcidevs)
             FDT( make_vpci_node(gc, fdt, ainfo, dom) );
 
-        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3)
+        if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
             FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
+            if (pfdt)
+                FDT( modify_partial_fdt(gc, pfdt) );
+        }
 
         for (i = 0; i < d_config->num_disks; i++) {
             libxl_device_disk *disk = &d_config->disks[i];
-- 
2.43.0

Re: [PATCH v3 19/23] libxl/arm: vIOMMU: Modify the partial device tree for iommus

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> To configure IOMMU in guest for passthrough devices, user will need to
> copy the unmodified "iommus" property from host device tree to partial
> device tree. To enable the dom0 linux kernel to confiure the IOMMU

Typo s/confiure/configure/

The rest looks ok to me

with that addressed:

Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>

Cheers,
Luca

[PATCH v3 20/23] xen/arm: vIOMMU: Modify the partial device tree for dom0less

[Milan Djokic]

From: Rahul Singh <rahul.singh@arm.com>

To configure IOMMU in guest for passthrough devices, user will need to
copy the unmodified "iommus" property from host device tree to partial
device tree. To enable the dom0 linux kernel to confiure the IOMMU
correctly replace the phandle in partial device tree with virtual
IOMMU phandle when "iommus" property is set.

Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/common/device-tree/dom0less-build.c | 31 ++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 840d14419d..4b74d2f705 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -318,7 +318,35 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
 }
 
-static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
+static void modify_pfdt_node(void *pfdt, int nodeoff)
+{
+    int proplen, i, rc;
+    const fdt32_t *prop;
+    fdt32_t *prop_c;
+
+    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+    if ( !prop )
+        return;
+
+    prop_c = xzalloc_bytes(proplen);
+
+    for ( i = 0; i < proplen / 8; ++i )
+    {
+        prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+        prop_c[i * 2 + 1] = prop[i * 2 + 1];
+    }
+
+    rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
+    if ( rc )
+    {
+        dprintk(XENLOG_ERR, "Can't set the iommus property in partial FDT");
+        return;
+    }
+
+    return;
+}
+
+static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
                                  int nodeoff,
                                  uint32_t address_cells, uint32_t size_cells,
                                  bool scan_passthrough_prop)
@@ -344,6 +372,7 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
     node_next = fdt_first_subnode(pfdt, nodeoff);
     while ( node_next > 0 )
     {
+        modify_pfdt_node(pfdt, node_next);
         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
                             scan_passthrough_prop);
         if ( rc )
-- 
2.43.0

Re: [PATCH v3 20/23] xen/arm: vIOMMU: Modify the partial device tree for dom0less

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> From: Rahul Singh <rahul.singh@arm.com>
> 
> To configure IOMMU in guest for passthrough devices, user will need to
> copy the unmodified "iommus" property from host device tree to partial
> device tree. To enable the dom0 linux kernel to confiure the IOMMU

typo s/confiure/configure/

> correctly replace the phandle in partial device tree with virtual
> IOMMU phandle when "iommus" property is set.
> 
> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/common/device-tree/dom0less-build.c | 31 ++++++++++++++++++++++++-
> 1 file changed, 30 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
> index 840d14419d..4b74d2f705 100644
> --- a/xen/common/device-tree/dom0less-build.c
> +++ b/xen/common/device-tree/dom0less-build.c
> @@ -318,7 +318,35 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
>     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
> }
> 
> -static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
> +static void modify_pfdt_node(void *pfdt, int nodeoff)
> +{
> +    int proplen, i, rc;
> +    const fdt32_t *prop;
> +    fdt32_t *prop_c;
> +
> +    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
> +    if ( !prop )
> +        return;
> +
> +    prop_c = xzalloc_bytes(proplen);

we should check if the allocation was ok

> +
> +    for ( i = 0; i < proplen / 8; ++i )
> +    {
> +        prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
> +        prop_c[i * 2 + 1] = prop[i * 2 + 1];
> +    }
> +
> +    rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
> +    if ( rc )
> +    {
> +        dprintk(XENLOG_ERR, "Can't set the iommus property in partial FDT");
> +        return;
> +    }
> +
> +    return;
> +}
> +
> +static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
>                                  int nodeoff,
>                                  uint32_t address_cells, uint32_t size_cells,
>                                  bool scan_passthrough_prop)
> @@ -344,6 +372,7 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, const void *pfdt,
>     node_next = fdt_first_subnode(pfdt, nodeoff);
>     while ( node_next > 0 )
>     {
> +        modify_pfdt_node(pfdt, node_next);
>         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
>                             scan_passthrough_prop);
>         if ( rc )
> 

Cheers,
Luca

[PATCH v3 21/23] xen/arm: vIOMMU vSID->pSID mapping layer

[Milan Djokic]

Introduce vIOMMU mapping layer in order to support passthrough of IOMMU
devices attached to different physical IOMMUs (e.g. devices with the same streamID).
New generic vIOMMU API is added: viommu_allocate_free_vid().
This function will allocate a new guest vSID and map it to input pSID.
Once mapping is established, guest will use vSID for stage-1 commands
and xen will translate vSID->pSID and propagate it towards stage-2.
Introduced naming is generic (vID/pID), since this API could be used
for other IOMMU types in the future.
Implemented usage of the new API for dom0less guests. vSIDs are allocated
on guest device tree creation and the original pSID is
replaced with vSID which shall be used by the guest driver.

Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 xen/arch/arm/include/asm/viommu.h       | 10 ++++
 xen/common/device-tree/dom0less-build.c | 32 +++++++++---
 xen/drivers/passthrough/arm/viommu.c    |  7 +++
 xen/drivers/passthrough/arm/vsmmu-v3.c  | 67 ++++++++++++++++++++++++-
 4 files changed, 106 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
index 01d4d0dfef..6a2fc56e38 100644
--- a/xen/arch/arm/include/asm/viommu.h
+++ b/xen/arch/arm/include/asm/viommu.h
@@ -33,6 +33,15 @@ struct viommu_ops {
      * Called during domain destruction to free resources used by vIOMMU.
      */
     int (*relinquish_resources)(struct domain *d);
+
+    /*
+     * Allocate free vSID/vRID for the guest device and establish vID->pID mapping
+     * Called during domain device assignment.
+     * Returns 0 on success and sets vid argument to newly allocated vSID/vRID
+     * mapped to physical ID (id argument).
+     * Negative error code returned if allocation fails.
+     */
+    int (*allocate_free_vid)(struct domain *d, uint32_t id, uint32_t *vid);
 };
 
 struct viommu_desc {
@@ -48,6 +57,7 @@ struct viommu_desc {
 
 int domain_viommu_init(struct domain *d, uint16_t viommu_type);
 int viommu_relinquish_resources(struct domain *d);
+int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid);
 uint16_t viommu_get_type(void);
 void add_to_host_iommu_list(paddr_t addr, paddr_t size,
                             const struct dt_device_node *node);
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 4b74d2f705..f5afdf381c 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -31,6 +31,8 @@
 #include <xen/static-memory.h>
 #include <xen/static-shmem.h>
 
+#include <asm/viommu.h>
+
 #define XENSTORE_PFN_LATE_ALLOC UINT64_MAX
 
 static domid_t __initdata xs_domid = DOMID_INVALID;
@@ -318,22 +320,33 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
 }
 
-static void modify_pfdt_node(void *pfdt, int nodeoff)
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d)
 {
     int proplen, i, rc;
     const fdt32_t *prop;
     fdt32_t *prop_c;
+    uint32_t vsid;
 
-    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
+    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen); 
     if ( !prop )
         return;
 
     prop_c = xzalloc_bytes(proplen);
 
+    /* 
+     * Assign <vIOMMU vSID> pairs to iommus property and establish
+     * vSID->pSID mappings
+    */
     for ( i = 0; i < proplen / 8; ++i )
     {
         prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
-        prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        rc = viommu_allocate_free_vid(d, fdt32_to_cpu(prop[i * 2 + 1]), &vsid);
+        if( rc ) {
+            dprintk(XENLOG_ERR, "Failed to allocate new vSID for iommu device");
+            return;
+        }
+        prop_c[i * 2 + 1] = cpu_to_fdt32(vsid);
     }
 
     rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
@@ -345,11 +358,14 @@ static void modify_pfdt_node(void *pfdt, int nodeoff)
 
     return;
 }
+#else
+    static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d) {}
+#endif
 
 static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
                                  int nodeoff,
                                  uint32_t address_cells, uint32_t size_cells,
-                                 bool scan_passthrough_prop)
+                                 bool scan_passthrough_prop, struct domain *d)
 {
     int rc = 0;
     void *fdt = kinfo->fdt;
@@ -372,9 +388,9 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
     node_next = fdt_first_subnode(pfdt, nodeoff);
     while ( node_next > 0 )
     {
-        modify_pfdt_node(pfdt, node_next);
+        modify_pfdt_node(pfdt, node_next, d);
         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
-                            scan_passthrough_prop);
+                            scan_passthrough_prop, d);
         if ( rc )
             return rc;
 
@@ -443,7 +459,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
             res = scan_pfdt_node(kinfo, pfdt, node_next,
                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
-                                 false);
+                                 false, d);
             if ( res )
                 goto out;
             continue;
@@ -453,7 +469,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
             res = scan_pfdt_node(kinfo, pfdt, node_next,
                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
-                                 true);
+                                 true, d);
             if ( res )
                 goto out;
             continue;
diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
index 5f5892fbb2..4b7837a91f 100644
--- a/xen/drivers/passthrough/arm/viommu.c
+++ b/xen/drivers/passthrough/arm/viommu.c
@@ -71,6 +71,13 @@ int viommu_relinquish_resources(struct domain *d)
     return cur_viommu->ops->relinquish_resources(d);
 }
 
+int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
+    if ( !cur_viommu )
+        return -ENODEV;
+
+    return cur_viommu->ops->allocate_free_vid(d, id, vid);
+}
+
 uint16_t viommu_get_type(void)
 {
     if ( !cur_viommu )
diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
index 5d0dabd2b2..604f09e980 100644
--- a/xen/drivers/passthrough/arm/vsmmu-v3.c
+++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
@@ -53,6 +53,8 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
 #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
 
+#define MAX_VSID   (1 << SMMU_IDR1_SIDSIZE)
+
 /* event queue entry */
 struct arm_smmu_evtq_ent {
     /* Common fields */
@@ -100,6 +102,14 @@ struct arm_vsmmu_queue {
     uint8_t     max_n_shift;
 };
 
+/* vSID->pSID mapping entry */
+struct vsid_entry {
+    bool        valid;
+    uint32_t    vsid;
+    struct host_iommu *phys_smmu;
+    uint32_t    psid;
+};
+
 struct virt_smmu {
     struct      domain *d;
     struct      list_head viommu_list;
@@ -118,6 +128,7 @@ struct virt_smmu {
     uint64_t    evtq_irq_cfg0;
     struct      arm_vsmmu_queue evtq, cmdq;
     spinlock_t  cmd_queue_lock;
+    struct vsid_entry *vsids;
 };
 
 /* Queue manipulation functions */
@@ -426,6 +437,29 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
     uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
     struct iommu_guest_config guest_cfg = {0};
+    uint32_t psid;
+    struct arm_smmu_evtq_ent ent = {
+        .opcode = EVT_ID_BAD_STE,
+        .sid = sid,
+        .c_bad_ste_streamid = {
+            .ssid = 0,
+            .ssv = false,
+        },
+    };
+
+    /* SIDs identity mapped for HW domain */
+    if ( is_hardware_domain(d) )
+        psid = sid;
+    else {
+        /* vSID out of range or not mapped to pSID */
+        if ( sid >= MAX_VSID || !smmu->vsids[sid].valid )
+        {
+            arm_vsmmu_send_event(smmu, &ent);
+            return -EINVAL;
+        }
+
+        psid = smmu->vsids[sid].psid;
+    }
 
     ret = arm_vsmmu_find_ste(smmu, sid, ste);
     if ( ret )
@@ -446,7 +480,7 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
     else
         guest_cfg.config = ARM_SMMU_DOMAIN_NESTED;
 
-    ret = hd->platform_ops->attach_guest_config(d, sid, &guest_cfg);
+    ret = hd->platform_ops->attach_guest_config(d, psid, &guest_cfg);
     if ( ret )
         return ret;
 
@@ -791,6 +825,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
+    smmu->vsids = xzalloc_array(struct vsid_entry, MAX_VSID);
 
     spin_lock_init(&smmu->cmd_queue_lock);
 
@@ -850,8 +885,9 @@ int vsmmuv3_relinquish_resources(struct domain *d)
     if ( list_head_is_null(&d->arch.viommu_list) )
         return 0;
 
-    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
+    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list)
     {
+        xfree(pos->vsids);
         list_del(&pos->viommu_list);
         xfree(pos);
     }
@@ -859,8 +895,35 @@ int vsmmuv3_relinquish_resources(struct domain *d)
     return 0;
 }
 
+int vsmmuv3_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
+    uint16_t i = 0;
+    struct virt_smmu *smmu;
+
+    if ( list_head_is_null(&d->arch.viommu_list) )
+        return -ENODEV;
+
+    smmu = list_first_entry(&d->arch.viommu_list, struct virt_smmu, viommu_list);
+
+    /* Get first free vSID index */
+    while ( smmu->vsids[i].valid && i++ < MAX_VSID );
+
+    /* Max number of vSIDs already allocated? */
+    if ( i == MAX_VSID) {
+        return -ENOMEM;
+    }
+
+    /* Establish vSID->pSID mapping */
+    smmu->vsids[i].valid = true;
+    smmu->vsids[i].vsid = i;
+    smmu->vsids[i].psid = id;
+    *vid = smmu->vsids[i].vsid;
+
+    return 0;
+}
+
 static const struct viommu_ops vsmmuv3_ops = {
     .domain_init = domain_vsmmuv3_init,
+    .allocate_free_vid = vsmmuv3_allocate_free_vid,
     .relinquish_resources = vsmmuv3_relinquish_resources,
 };
 
-- 
2.43.0

Re: [PATCH v3 21/23] xen/arm: vIOMMU vSID->pSID mapping layer

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> Introduce vIOMMU mapping layer in order to support passthrough of IOMMU
> devices attached to different physical IOMMUs (e.g. devices with the same streamID).
> New generic vIOMMU API is added: viommu_allocate_free_vid().
> This function will allocate a new guest vSID and map it to input pSID.
> Once mapping is established, guest will use vSID for stage-1 commands
> and xen will translate vSID->pSID and propagate it towards stage-2.
> Introduced naming is generic (vID/pID), since this API could be used
> for other IOMMU types in the future.
> Implemented usage of the new API for dom0less guests. vSIDs are allocated
> on guest device tree creation and the original pSID is
> replaced with vSID which shall be used by the guest driver.
> 
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> xen/arch/arm/include/asm/viommu.h       | 10 ++++
> xen/common/device-tree/dom0less-build.c | 32 +++++++++---
> xen/drivers/passthrough/arm/viommu.c    |  7 +++
> xen/drivers/passthrough/arm/vsmmu-v3.c  | 67 ++++++++++++++++++++++++-
> 4 files changed, 106 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/arm/include/asm/viommu.h b/xen/arch/arm/include/asm/viommu.h
> index 01d4d0dfef..6a2fc56e38 100644
> --- a/xen/arch/arm/include/asm/viommu.h
> +++ b/xen/arch/arm/include/asm/viommu.h
> @@ -33,6 +33,15 @@ struct viommu_ops {
>      * Called during domain destruction to free resources used by vIOMMU.
>      */
>     int (*relinquish_resources)(struct domain *d);
> +
> +    /*
> +     * Allocate free vSID/vRID for the guest device and establish vID->pID mapping
> +     * Called during domain device assignment.
> +     * Returns 0 on success and sets vid argument to newly allocated vSID/vRID
> +     * mapped to physical ID (id argument).
> +     * Negative error code returned if allocation fails.
> +     */
> +    int (*allocate_free_vid)(struct domain *d, uint32_t id, uint32_t *vid);
> };
> 
> struct viommu_desc {
> @@ -48,6 +57,7 @@ struct viommu_desc {
> 
> int domain_viommu_init(struct domain *d, uint16_t viommu_type);
> int viommu_relinquish_resources(struct domain *d);
> +int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid);
> uint16_t viommu_get_type(void);
> void add_to_host_iommu_list(paddr_t addr, paddr_t size,
>                             const struct dt_device_node *node);
> diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
> index 4b74d2f705..f5afdf381c 100644
> --- a/xen/common/device-tree/dom0less-build.c
> +++ b/xen/common/device-tree/dom0less-build.c
> @@ -31,6 +31,8 @@
> #include <xen/static-memory.h>
> #include <xen/static-shmem.h>
> 
> +#include <asm/viommu.h>
> +
> #define XENSTORE_PFN_LATE_ALLOC UINT64_MAX
> 
> static domid_t __initdata xs_domid = DOMID_INVALID;
> @@ -318,22 +320,33 @@ static int __init handle_prop_pfdt(struct kernel_info *kinfo,
>     return ( propoff != -FDT_ERR_NOTFOUND ) ? propoff : 0;
> }
> 
> -static void modify_pfdt_node(void *pfdt, int nodeoff)
> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
> +static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d)
> {
>     int proplen, i, rc;
>     const fdt32_t *prop;
>     fdt32_t *prop_c;
> +    uint32_t vsid;
> 
> -    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen);
> +    prop = fdt_getprop(pfdt, nodeoff, "iommus", &proplen); 
>     if ( !prop )
>         return;
> 
>     prop_c = xzalloc_bytes(proplen);
> 
> +    /* 
> +     * Assign <vIOMMU vSID> pairs to iommus property and establish
> +     * vSID->pSID mappings
> +    */
>     for ( i = 0; i < proplen / 8; ++i )
>     {
>         prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
> -        prop_c[i * 2 + 1] = prop[i * 2 + 1];
> +        rc = viommu_allocate_free_vid(d, fdt32_to_cpu(prop[i * 2 + 1]), &vsid);
> +        if( rc ) {

NIT: codestyle, space after ‘if'

> +            dprintk(XENLOG_ERR, "Failed to allocate new vSID for iommu device");
> +            return;
> +        }
> +        prop_c[i * 2 + 1] = cpu_to_fdt32(vsid);
>     }
> 
>     rc = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
> @@ -345,11 +358,14 @@ static void modify_pfdt_node(void *pfdt, int nodeoff)
> 
>     return;
> }
> +#else
> +    static void modify_pfdt_node(void *pfdt, int nodeoff, struct domain *d) {}
> +#endif
> 
> static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
>                                  int nodeoff,
>                                  uint32_t address_cells, uint32_t size_cells,
> -                                 bool scan_passthrough_prop)
> +                                 bool scan_passthrough_prop, struct domain *d)
> {
>     int rc = 0;
>     void *fdt = kinfo->fdt;
> @@ -372,9 +388,9 @@ static int __init scan_pfdt_node(struct kernel_info *kinfo, void *pfdt,
>     node_next = fdt_first_subnode(pfdt, nodeoff);
>     while ( node_next > 0 )
>     {
> -        modify_pfdt_node(pfdt, node_next);
> +        modify_pfdt_node(pfdt, node_next, d);
>         rc = scan_pfdt_node(kinfo, pfdt, node_next, address_cells, size_cells,
> -                            scan_passthrough_prop);
> +                            scan_passthrough_prop, d);
>         if ( rc )
>             return rc;
> 
> @@ -443,7 +459,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
>             res = scan_pfdt_node(kinfo, pfdt, node_next,
>                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
>                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
> -                                 false);
> +                                 false, d);
>             if ( res )
>                 goto out;
>             continue;
> @@ -453,7 +469,7 @@ static int __init domain_handle_dtb_boot_module(struct domain *d,
>             res = scan_pfdt_node(kinfo, pfdt, node_next,
>                                  DT_ROOT_NODE_ADDR_CELLS_DEFAULT,
>                                  DT_ROOT_NODE_SIZE_CELLS_DEFAULT,
> -                                 true);
> +                                 true, d);
>             if ( res )
>                 goto out;
>             continue;
> diff --git a/xen/drivers/passthrough/arm/viommu.c b/xen/drivers/passthrough/arm/viommu.c
> index 5f5892fbb2..4b7837a91f 100644
> --- a/xen/drivers/passthrough/arm/viommu.c
> +++ b/xen/drivers/passthrough/arm/viommu.c
> @@ -71,6 +71,13 @@ int viommu_relinquish_resources(struct domain *d)
>     return cur_viommu->ops->relinquish_resources(d);
> }
> 
> +int viommu_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
> +    if ( !cur_viommu )
> +        return -ENODEV;
> +
> +    return cur_viommu->ops->allocate_free_vid(d, id, vid);
> +}
> +
> uint16_t viommu_get_type(void)
> {
>     if ( !cur_viommu )
> diff --git a/xen/drivers/passthrough/arm/vsmmu-v3.c b/xen/drivers/passthrough/arm/vsmmu-v3.c
> index 5d0dabd2b2..604f09e980 100644
> --- a/xen/drivers/passthrough/arm/vsmmu-v3.c
> +++ b/xen/drivers/passthrough/arm/vsmmu-v3.c
> @@ -53,6 +53,8 @@ extern const struct viommu_desc __read_mostly *cur_viommu;
> #define smmu_get_ste_s1ctxptr(x)    FIELD_PREP(STRTAB_STE_0_S1CTXPTR_MASK, \
>                                     FIELD_GET(STRTAB_STE_0_S1CTXPTR_MASK, x))
> 
> +#define MAX_VSID   (1 << SMMU_IDR1_SIDSIZE)
> +
> /* event queue entry */
> struct arm_smmu_evtq_ent {
>     /* Common fields */
> @@ -100,6 +102,14 @@ struct arm_vsmmu_queue {
>     uint8_t     max_n_shift;
> };
> 
> +/* vSID->pSID mapping entry */
> +struct vsid_entry {
> +    bool        valid;
> +    uint32_t    vsid;
> +    struct host_iommu *phys_smmu;
> +    uint32_t    psid;
> +};
> +
> struct virt_smmu {
>     struct      domain *d;
>     struct      list_head viommu_list;
> @@ -118,6 +128,7 @@ struct virt_smmu {
>     uint64_t    evtq_irq_cfg0;
>     struct      arm_vsmmu_queue evtq, cmdq;
>     spinlock_t  cmd_queue_lock;
> +    struct vsid_entry *vsids;
> };
> 
> /* Queue manipulation functions */
> @@ -426,6 +437,29 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
>     struct arm_vsmmu_s1_trans_cfg s1_cfg = {0};
>     uint32_t sid = smmu_cmd_get_sid(cmdptr[0]);
>     struct iommu_guest_config guest_cfg = {0};
> +    uint32_t psid;
> +    struct arm_smmu_evtq_ent ent = {
> +        .opcode = EVT_ID_BAD_STE,
> +        .sid = sid,
> +        .c_bad_ste_streamid = {
> +            .ssid = 0,
> +            .ssv = false,
> +        },
> +    };
> +
> +    /* SIDs identity mapped for HW domain */
> +    if ( is_hardware_domain(d) )
> +        psid = sid;
> +    else {
> +        /* vSID out of range or not mapped to pSID */
> +        if ( sid >= MAX_VSID || !smmu->vsids[sid].valid )
> +        {
> +            arm_vsmmu_send_event(smmu, &ent);
> +            return -EINVAL;
> +        }
> +
> +        psid = smmu->vsids[sid].psid;
> +    }
> 
>     ret = arm_vsmmu_find_ste(smmu, sid, ste);
>     if ( ret )
> @@ -446,7 +480,7 @@ static int arm_vsmmu_handle_cfgi_ste(struct virt_smmu *smmu, uint64_t *cmdptr)
>     else
>         guest_cfg.config = ARM_SMMU_DOMAIN_NESTED;
> 
> -    ret = hd->platform_ops->attach_guest_config(d, sid, &guest_cfg);
> +    ret = hd->platform_ops->attach_guest_config(d, psid, &guest_cfg);
>     if ( ret )
>         return ret;
> 
> @@ -791,6 +825,7 @@ static int vsmmuv3_init_single(struct domain *d, paddr_t addr,
>     smmu->cmdq.ent_size = CMDQ_ENT_DWORDS * DWORDS_BYTES;
>     smmu->evtq.q_base = FIELD_PREP(Q_BASE_LOG2SIZE, SMMU_EVTQS);
>     smmu->evtq.ent_size = EVTQ_ENT_DWORDS * DWORDS_BYTES;
> +    smmu->vsids = xzalloc_array(struct vsid_entry, MAX_VSID);

we should check if allocation is ok

> 
>     spin_lock_init(&smmu->cmd_queue_lock);
> 
> @@ -850,8 +885,9 @@ int vsmmuv3_relinquish_resources(struct domain *d)
>     if ( list_head_is_null(&d->arch.viommu_list) )
>         return 0;
> 
> -    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list )
> +    list_for_each_entry_safe(pos, temp, &d->arch.viommu_list, viommu_list)
>     {
> +        xfree(pos->vsids);
>         list_del(&pos->viommu_list);
>         xfree(pos);
>     }
> @@ -859,8 +895,35 @@ int vsmmuv3_relinquish_resources(struct domain *d)
>     return 0;
> }
> 
> +int vsmmuv3_allocate_free_vid(struct domain *d, uint32_t id, uint32_t *vid) {
> +    uint16_t i = 0;
> +    struct virt_smmu *smmu;
> +
> +    if ( list_head_is_null(&d->arch.viommu_list) )
> +        return -ENODEV;
> +
> +    smmu = list_first_entry(&d->arch.viommu_list, struct virt_smmu, viommu_list);
> +
> +    /* Get first free vSID index */
> +    while ( smmu->vsids[i].valid && i++ < MAX_VSID );

This will access OOB when you will reach MAX_VSID and later evaluate smmu->vsids[i].valid,
I suggest to modify like this:

while ( i < MAX_VSID && smmu->vsids[i].valid )
    i++;

> +
> +    /* Max number of vSIDs already allocated? */
> +    if ( i == MAX_VSID) {

NIT: code style, space before ‘)'

> +        return -ENOMEM;
> +    }
> +
> +    /* Establish vSID->pSID mapping */
> +    smmu->vsids[i].valid = true;
> +    smmu->vsids[i].vsid = i;
> +    smmu->vsids[i].psid = id;
> +    *vid = smmu->vsids[i].vsid;

Why is phys_smmu never populated and used?

> +
> +    return 0;
> +}
> +
> static const struct viommu_ops vsmmuv3_ops = {
>     .domain_init = domain_vsmmuv3_init,
> +    .allocate_free_vid = vsmmuv3_allocate_free_vid,
>     .relinquish_resources = vsmmuv3_relinquish_resources,
> };
> 

Cheers,
Luca

[PATCH v3 22/23] libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping

[Milan Djokic]

For guests created via control domain (xl, zephyr xenlib), partial device
tree is parsed and loaded on control domain side.
SIDs in guests device tree have to be replaced with
virtual SIDs which are mapped to physical SIDs. In order
to do that, control domain has to request from Xen to create
a new vSID and map it to original pSID for every guest device IOMMU
stream ID. For this purpose, new domctl command (XEN_DOMCTL_viommu_allocate_vid)
is introduced which control domain can use to request a new vSID mapping and
insert a new vSID into guest device tree once mapped.
Requested vSID allocation using this interface for vPCI/DT devices.

Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 tools/include/xenctrl.h             |  12 +++
 tools/libs/ctrl/xc_domain.c         |  23 +++++
 tools/libs/light/libxl_arm.c        | 127 +++++++++++++++++++++++++---
 xen/arch/arm/domctl.c               |  34 ++++++++
 xen/include/public/domctl.h         |  20 +++++
 xen/xsm/flask/hooks.c               |   4 +
 xen/xsm/flask/policy/access_vectors |   2 +
 7 files changed, 212 insertions(+), 10 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index d5dbf69c89..61be892cc8 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -2659,6 +2659,18 @@ int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
                              const uint32_t *llc_colors,
                              uint32_t num_llc_colors);
 
+/*
+ * Allocate guest IOMMU vSID and establish its mapping to pSID.
+ * It can only be used on domain DT creation.
+ * Currently used for ARM only, possibly for RISC-V in the
+ * future. Function has no effect for x86.
+ */
+int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
+                                         uint32_t domid,
+                                         uint16_t nr_sids,
+                                         uint32_t first_psid,
+                                         uint32_t *first_vsid);
+
 #if defined(__arm__) || defined(__aarch64__)
 int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
                   uint32_t overlay_fdt_size, uint8_t overlay_op);
diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
index 01c0669c88..39ffe80e6d 100644
--- a/tools/libs/ctrl/xc_domain.c
+++ b/tools/libs/ctrl/xc_domain.c
@@ -2222,6 +2222,29 @@ out:
 
     return ret;
 }
+
+int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
+                                         uint32_t domid,
+                                         uint16_t nr_sids,
+                                         uint32_t first_psid,
+                                         uint32_t *first_vsid)
+{
+    int err;
+    struct xen_domctl domctl = {};
+
+    domctl.cmd = XEN_DOMCTL_viommu_alloc_vsid_range;
+    domctl.domain = domid;
+    domctl.u.viommu_alloc_vsid_range.first_psid = first_psid;
+    domctl.u.viommu_alloc_vsid_range.nr_sids = nr_sids;
+
+    if ( (err = do_domctl(xch, &domctl)) != 0 )
+        return err;
+
+    *first_vsid = domctl.u.viommu_alloc_vsid_range.first_vsid;
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 7b887898bb..79904b746c 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -955,6 +955,13 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
     return 0;
 }
 
+/*
+ * Stores starting vSID of vPCI IOMMU SID range
+ * Used as a lookup value to avoid repeated
+ * vSID range allocation on every fdt resize.
+ */
+static int vpci_first_vsid = -1;
+
 static int make_vpci_node(libxl__gc *gc, void *fdt,
                           const struct arch_info *ainfo,
                           struct xc_dom_image *dom)
@@ -963,6 +970,9 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
     const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
     const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
     const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
+    uint16_t iommu_range_size = 0x1000;
+    uint32_t first_vsid;
+    uint32_t first_psid = 0;
 
     res = fdt_begin_node(fdt, name);
     if (res) return res;
@@ -996,8 +1006,20 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
         GUEST_VPCI_PREFETCH_MEM_SIZE);
     if (res) return res;
 
+    /* request vSID range allocation if not already allocated */
+    if (vpci_first_vsid < 0) {
+        res = xc_domain_viommu_allocate_vsid_range(CTX->xch, dom->guest_domid,
+            iommu_range_size, first_psid, &first_vsid);
+        if (res)
+            return res;
+        vpci_first_vsid = first_vsid;
+    }
+    else {
+        first_vsid = vpci_first_vsid;
+    }
+
     res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
-                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
+                             GUEST_PHANDLE_VSMMUV3, first_vsid, iommu_range_size);
     if (res) return res;
 
     res = fdt_end_node(fdt);
@@ -1326,11 +1348,92 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return 0;
 }
 
-static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+/*
+ * Store virtualized 'iommus' properties for every node attached to IOMMU
+ * and passthroughed to guest.
+ * Used as a lookup table for mapping <phandle pSID> -> <vhandle vSID>
+ */
+struct viommu_stream {
+    XEN_LIST_ENTRY(struct viommu_stream) entry;
+    char path[128];          /* DT path, stable across resizes */
+    fdt32_t *iommus;         /* fully virtualized iommus property */
+};
+
+static XEN_LIST_HEAD(, struct viommu_stream) viommu_stream_list;
+
+/*
+ * Helper function which creates mapping of dt node to
+ * to virtualized 'iommus' property
+ * Mappings stored in a global 'viommu_stream_list' to
+ * make it reusable for every fdt resize
+ */
+static int viommu_get_stream(libxl__gc *gc,
+                             uint32_t domid,
+                             const fdt32_t *prop,
+                             int proplen,
+                             const char* path, fdt32_t **iommus)
+{
+    int i, r;
+    uint32_t vsid, psid;
+    struct viommu_stream *viommu_stream;
+
+    /* Lookup if stream for target device is already allocated */
+    XEN_LIST_FOREACH(viommu_stream, &viommu_stream_list, entry)
+    {
+        if (!strcmp(viommu_stream->path, path)) {
+            *iommus = viommu_stream->iommus;
+            return 0;
+        }
+    }
+
+    /* Allocate new viommu stream */
+    viommu_stream = malloc(sizeof(struct viommu_stream));
+    if (!viommu_stream)
+        return ERROR_NOMEM;
+    memset(viommu_stream, 0, sizeof(struct viommu_stream));
+    viommu_stream->iommus = malloc(proplen);
+    if (!viommu_stream->iommus)
+        return ERROR_NOMEM;
+    memset(viommu_stream->iommus, 0, proplen);
+
+    LOG(DEBUG, "Creating vIOMMU stream for device %s",
+        path);
+
+    /*
+     * Virtualize device "iommus" property
+     * (replace pIOMMU with vIOMMU phandle and pSIDs with mapped vSIDs)
+     */
+    for (i = 0; i < proplen / 8; ++i) {
+        viommu_stream->iommus[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
+        /* Allocate new vSID mapped to pSID */
+        psid = fdt32_to_cpu(prop[i * 2 + 1]);
+        r = xc_domain_viommu_allocate_vsid_range(CTX->xch, domid, 1, psid, &vsid);
+        if (r) {
+            LOG(ERROR, "Can't allocate new vSID/vRID for guest IOMMU device");
+            return r;
+        }
+        viommu_stream->iommus[i * 2 + 1] = cpu_to_fdt32(vsid);
+        LOG(DEBUG, "Mapped vSID: %u to pSID: %u", vsid, psid);
+    }
+
+    strcpy(viommu_stream->path, path);
+    *iommus =  viommu_stream->iommus;
+
+    XEN_LIST_INSERT_HEAD(&viommu_stream_list, viommu_stream, entry);
+
+    return 0;
+}
+
+/*
+ * Used to update partial fdt when vIOMMU is enabled
+ * Maps dt properties of IOMMU devices to virtual IOMMU
+ */
+static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
 {
-    int nodeoff, proplen, i, r;
+    int nodeoff, proplen, r;
     const fdt32_t *prop;
     fdt32_t *prop_c;
+    char path[128];
 
     nodeoff = fdt_path_offset(pfdt, "/passthrough");
     if (nodeoff < 0)
@@ -1344,11 +1447,16 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
         if (!prop)
             continue;
 
-        prop_c = libxl__zalloc(gc, proplen);
+        r = fdt_get_path(pfdt, nodeoff, path, sizeof(path));
+        if ( r < 0 ) {
+            LOG(ERROR, "Can't get passthrough node path");
+            return r;
+        }
 
-        for (i = 0; i < proplen / 8; ++i) {
-            prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
-            prop_c[i * 2 + 1] = prop[i * 2 + 1];
+        r = viommu_get_stream(gc, domid, prop, proplen, path, &prop_c);
+        if (r) {
+            LOG(ERROR, "Can't get viommu stream");
+            return r;
         }
 
         r = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
@@ -1360,7 +1468,6 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
 
     return 0;
 }
-
 #else
 
 static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
@@ -1379,7 +1486,7 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
     return -FDT_ERR_INTERNAL;
 }
 
-static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
+static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
 {
     LOG(ERROR, "partial device tree not supported");
 
@@ -1511,7 +1618,7 @@ next_resize:
         if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
             FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
             if (pfdt)
-                FDT( modify_partial_fdt(gc, pfdt) );
+                FDT( viommu_modify_partial_fdt(gc, pfdt, dom->guest_domid) );
         }
 
         for (i = 0; i < d_config->num_disks; i++) {
diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index ad914c915f..c85853e4cb 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -16,6 +16,7 @@
 #include <xen/types.h>
 #include <xsm/xsm.h>
 #include <public/domctl.h>
+#include <asm/viommu.h>
 
 void arch_get_domain_info(const struct domain *d,
                           struct xen_domctl_getdomaininfo *info)
@@ -179,6 +180,39 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
     }
     case XEN_DOMCTL_dt_overlay:
         return dt_overlay_domctl(d, &domctl->u.dt_overlay);
+
+#ifdef CONFIG_ARM_VIRTUAL_IOMMU
+    case XEN_DOMCTL_viommu_alloc_vsid_range:
+    {
+        int rc = 0;
+        uint16_t i;
+        uint32_t vsid;
+        struct xen_domctl_viommu_alloc_vsid_range *viommu_alloc_vsid_range =
+            &domctl->u.viommu_alloc_vsid_range;
+
+        if ( viommu_alloc_vsid_range->pad )
+            return -EINVAL;
+
+        for ( i = 0; i < viommu_alloc_vsid_range->nr_sids; i++ )
+        {
+            rc = viommu_allocate_free_vid(d, viommu_alloc_vsid_range->first_psid
+                                            + i, &vsid);
+            if( rc )
+                return rc;
+        }
+
+        if ( !rc )
+        {
+            /* Calculate first vSID from allocated range */
+            viommu_alloc_vsid_range->first_vsid = vsid -
+                viommu_alloc_vsid_range->nr_sids + 1;
+            rc = copy_to_guest(u_domctl, domctl, 1);
+        }
+
+        return rc;
+    }
+#endif
+
     default:
         return subarch_do_domctl(domctl, d, u_domctl);
     }
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 23124547f3..190aed1e59 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1276,6 +1276,24 @@ struct xen_domctl_get_domain_state {
     uint64_t unique_id;      /* Unique domain identifier. */
 };
 
+/*
+ * XEN_DOMCTL_viommu_alloc_vsid_range
+ *
+ * Allocate guest vSID range and
+ * establish pSID->vSID mapping for target range.
+ * Allocated range is continous
+ */
+struct xen_domctl_viommu_alloc_vsid_range {
+    /* IN: Range first pSID  */
+    uint32_t first_psid;
+    /* IN: Number of vSIDs to allocate */
+    uint16_t nr_sids;
+    /* padding, must be 0 */
+    uint16_t pad;
+    /* OUT: Mapped range first vSID */
+    uint32_t first_vsid;
+};
+
 struct xen_domctl {
 /* Stable domctl ops: interface_version is required to be 0.  */
     uint32_t cmd;
@@ -1368,6 +1386,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_gsi_permission                88
 #define XEN_DOMCTL_set_llc_colors                89
 #define XEN_DOMCTL_get_domain_state              90 /* stable interface */
+#define XEN_DOMCTL_viommu_alloc_vsid_range       91
 #define XEN_DOMCTL_gdbsx_guestmemio            1000
 #define XEN_DOMCTL_gdbsx_pausevcpu             1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu           1002
@@ -1436,6 +1455,7 @@ struct xen_domctl {
 #endif
         struct xen_domctl_set_llc_colors    set_llc_colors;
         struct xen_domctl_get_domain_state  get_domain_state;
+        struct xen_domctl_viommu_alloc_vsid_range viommu_alloc_vsid_range;
         uint8_t                             pad[128];
     } u;
 };
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index b250b27065..91e80ea80d 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -820,6 +820,10 @@ static int cf_check flask_domctl(struct domain *d, unsigned int cmd,
     case XEN_DOMCTL_set_llc_colors:
         return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SET_LLC_COLORS);
 
+    case XEN_DOMCTL_viommu_alloc_vsid_range:
+        return current_has_perm(d, SECCLASS_DOMAIN2,
+            DOMAIN2__VIOMMU_ALLOC_VSID_RANGE);
+
     default:
         return avc_unknown_permission("domctl", cmd);
     }
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index ce907d50a4..e4ffe2f5db 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -255,6 +255,8 @@ class domain2
     set_llc_colors
 # XEN_DOMCTL_get_domain_state
     get_domain_state
+# XEN_DOMCTL_viommu_alloc_vsid_range
+    viommu_alloc_vsid_range
 }
 
 # Similar to class domain, but primarily contains domctls related to HVM domains
-- 
2.43.0

Re: [PATCH v3 22/23] libxl/arm: Introduce domctl command for IOMMU vSID/vRID mapping

[Luca Fancellu]

Hi Milan,

> On 31 Mar 2026, at 02:52, Milan Djokic <milan_djokic@epam.com> wrote:
> 
> For guests created via control domain (xl, zephyr xenlib), partial device
> tree is parsed and loaded on control domain side.
> SIDs in guests device tree have to be replaced with
> virtual SIDs which are mapped to physical SIDs. In order
> to do that, control domain has to request from Xen to create
> a new vSID and map it to original pSID for every guest device IOMMU
> stream ID. For this purpose, new domctl command (XEN_DOMCTL_viommu_allocate_vid)
> is introduced which control domain can use to request a new vSID mapping and
> insert a new vSID into guest device tree once mapped.
> Requested vSID allocation using this interface for vPCI/DT devices.
> 
> Signed-off-by: Milan Djokic <milan_djokic@epam.com>
> ---
> tools/include/xenctrl.h             |  12 +++
> tools/libs/ctrl/xc_domain.c         |  23 +++++
> tools/libs/light/libxl_arm.c        | 127 +++++++++++++++++++++++++---
> xen/arch/arm/domctl.c               |  34 ++++++++
> xen/include/public/domctl.h         |  20 +++++
> xen/xsm/flask/hooks.c               |   4 +
> xen/xsm/flask/policy/access_vectors |   2 +
> 7 files changed, 212 insertions(+), 10 deletions(-)
> 
> diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
> index d5dbf69c89..61be892cc8 100644
> --- a/tools/include/xenctrl.h
> +++ b/tools/include/xenctrl.h
> @@ -2659,6 +2659,18 @@ int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
>                              const uint32_t *llc_colors,
>                              uint32_t num_llc_colors);
> 
> +/*
> + * Allocate guest IOMMU vSID and establish its mapping to pSID.
> + * It can only be used on domain DT creation.
> + * Currently used for ARM only, possibly for RISC-V in the
> + * future. Function has no effect for x86.
> + */
> +int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
> +                                         uint32_t domid,
> +                                         uint16_t nr_sids,
> +                                         uint32_t first_psid,
> +                                         uint32_t *first_vsid);
> +
> #if defined(__arm__) || defined(__aarch64__)
> int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
>                   uint32_t overlay_fdt_size, uint8_t overlay_op);
> diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
> index 01c0669c88..39ffe80e6d 100644
> --- a/tools/libs/ctrl/xc_domain.c
> +++ b/tools/libs/ctrl/xc_domain.c
> @@ -2222,6 +2222,29 @@ out:
> 
>     return ret;
> }
> +
> +int xc_domain_viommu_allocate_vsid_range(xc_interface *xch,
> +                                         uint32_t domid,
> +                                         uint16_t nr_sids,
> +                                         uint32_t first_psid,
> +                                         uint32_t *first_vsid)
> +{
> +    int err;
> +    struct xen_domctl domctl = {};
> +
> +    domctl.cmd = XEN_DOMCTL_viommu_alloc_vsid_range;
> +    domctl.domain = domid;
> +    domctl.u.viommu_alloc_vsid_range.first_psid = first_psid;
> +    domctl.u.viommu_alloc_vsid_range.nr_sids = nr_sids;
> +
> +    if ( (err = do_domctl(xch, &domctl)) != 0 )
> +        return err;
> +
> +    *first_vsid = domctl.u.viommu_alloc_vsid_range.first_vsid;
> +
> +    return 0;
> +}
> +
> /*
>  * Local variables:
>  * mode: C
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 7b887898bb..79904b746c 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -955,6 +955,13 @@ static int make_vsmmuv3_node(libxl__gc *gc, void *fdt,
>     return 0;
> }
> 
> +/*
> + * Stores starting vSID of vPCI IOMMU SID range
> + * Used as a lookup value to avoid repeated
> + * vSID range allocation on every fdt resize.
> + */
> +static int vpci_first_vsid = -1;

What happen between one guest creation and another since this and viommu_stream_list
are global?

> +
> static int make_vpci_node(libxl__gc *gc, void *fdt,
>                           const struct arch_info *ainfo,
>                           struct xc_dom_image *dom)
> @@ -963,6 +970,9 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
>     const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE;
>     const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE;
>     const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base);
> +    uint16_t iommu_range_size = 0x1000;
> +    uint32_t first_vsid;
> +    uint32_t first_psid = 0;
> 
>     res = fdt_begin_node(fdt, name);
>     if (res) return res;
> @@ -996,8 +1006,20 @@ static int make_vpci_node(libxl__gc *gc, void *fdt,
>         GUEST_VPCI_PREFETCH_MEM_SIZE);
>     if (res) return res;
> 
> +    /* request vSID range allocation if not already allocated */
> +    if (vpci_first_vsid < 0) {
> +        res = xc_domain_viommu_allocate_vsid_range(CTX->xch, dom->guest_domid,
> +            iommu_range_size, first_psid, &first_vsid);
> +        if (res)
> +            return res;
> +        vpci_first_vsid = first_vsid;
> +    }
> +    else {
> +        first_vsid = vpci_first_vsid;
> +    }
> +
>     res = fdt_property_values(gc, fdt, "iommu-map", 4, 0,
> -                              GUEST_PHANDLE_VSMMUV3, 0, 0x10000);
> +                             GUEST_PHANDLE_VSMMUV3, first_vsid, iommu_range_size);
>     if (res) return res;
> 
>     res = fdt_end_node(fdt);
> @@ -1326,11 +1348,92 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
>     return 0;
> }
> 
> -static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
> +/*
> + * Store virtualized 'iommus' properties for every node attached to IOMMU
> + * and passthroughed to guest.
> + * Used as a lookup table for mapping <phandle pSID> -> <vhandle vSID>
> + */
> +struct viommu_stream {
> +    XEN_LIST_ENTRY(struct viommu_stream) entry;
> +    char path[128];          /* DT path, stable across resizes */
> +    fdt32_t *iommus;         /* fully virtualized iommus property */
> +};
> +
> +static XEN_LIST_HEAD(, struct viommu_stream) viommu_stream_list;
> +
> +/*
> + * Helper function which creates mapping of dt node to
> + * to virtualized 'iommus' property
> + * Mappings stored in a global 'viommu_stream_list' to
> + * make it reusable for every fdt resize
> + */
> +static int viommu_get_stream(libxl__gc *gc,
> +                             uint32_t domid,
> +                             const fdt32_t *prop,
> +                             int proplen,
> +                             const char* path, fdt32_t **iommus)
> +{
> +    int i, r;
> +    uint32_t vsid, psid;
> +    struct viommu_stream *viommu_stream;
> +
> +    /* Lookup if stream for target device is already allocated */
> +    XEN_LIST_FOREACH(viommu_stream, &viommu_stream_list, entry)
> +    {
> +        if (!strcmp(viommu_stream->path, path)) {
> +            *iommus = viommu_stream->iommus;
> +            return 0;
> +        }
> +    }
> +
> +    /* Allocate new viommu stream */
> +    viommu_stream = malloc(sizeof(struct viommu_stream));
> +    if (!viommu_stream)
> +        return ERROR_NOMEM;
> +    memset(viommu_stream, 0, sizeof(struct viommu_stream));
> +    viommu_stream->iommus = malloc(proplen);
> +    if (!viommu_stream->iommus)
> +        return ERROR_NOMEM;

we should free viommu_stream here

> +    memset(viommu_stream->iommus, 0, proplen);
> +
> +    LOG(DEBUG, "Creating vIOMMU stream for device %s",
> +        path);
> +
> +    /*
> +     * Virtualize device "iommus" property
> +     * (replace pIOMMU with vIOMMU phandle and pSIDs with mapped vSIDs)
> +     */
> +    for (i = 0; i < proplen / 8; ++i) {
> +        viommu_stream->iommus[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
> +        /* Allocate new vSID mapped to pSID */
> +        psid = fdt32_to_cpu(prop[i * 2 + 1]);
> +        r = xc_domain_viommu_allocate_vsid_range(CTX->xch, domid, 1, psid, &vsid);
> +        if (r) {
> +            LOG(ERROR, "Can't allocate new vSID/vRID for guest IOMMU device");
> +            return r;
> +        }
> +        viommu_stream->iommus[i * 2 + 1] = cpu_to_fdt32(vsid);
> +        LOG(DEBUG, "Mapped vSID: %u to pSID: %u", vsid, psid);
> +    }
> +
> +    strcpy(viommu_stream->path, path);
> +    *iommus =  viommu_stream->iommus;
> +
> +    XEN_LIST_INSERT_HEAD(&viommu_stream_list, viommu_stream, entry);
> +
> +    return 0;
> +}
> +
> +/*
> + * Used to update partial fdt when vIOMMU is enabled
> + * Maps dt properties of IOMMU devices to virtual IOMMU
> + */
> +static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
> {
> -    int nodeoff, proplen, i, r;
> +    int nodeoff, proplen, r;
>     const fdt32_t *prop;
>     fdt32_t *prop_c;
> +    char path[128];
> 
>     nodeoff = fdt_path_offset(pfdt, "/passthrough");
>     if (nodeoff < 0)
> @@ -1344,11 +1447,16 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
>         if (!prop)
>             continue;
> 
> -        prop_c = libxl__zalloc(gc, proplen);
> +        r = fdt_get_path(pfdt, nodeoff, path, sizeof(path));
> +        if ( r < 0 ) {
> +            LOG(ERROR, "Can't get passthrough node path");
> +            return r;
> +        }
> 
> -        for (i = 0; i < proplen / 8; ++i) {
> -            prop_c[i * 2] = cpu_to_fdt32(GUEST_PHANDLE_VSMMUV3);
> -            prop_c[i * 2 + 1] = prop[i * 2 + 1];
> +        r = viommu_get_stream(gc, domid, prop, proplen, path, &prop_c);
> +        if (r) {
> +            LOG(ERROR, "Can't get viommu stream");
> +            return r;
>         }
> 
>         r = fdt_setprop(pfdt, nodeoff, "iommus", prop_c, proplen);
> @@ -1360,7 +1468,6 @@ static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
> 
>     return 0;
> }
> -
> #else
> 
> static int check_partial_fdt(libxl__gc *gc, void *fdt, size_t size)
> @@ -1379,7 +1486,7 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt)
>     return -FDT_ERR_INTERNAL;
> }
> 
> -static int modify_partial_fdt(libxl__gc *gc, void *pfdt)
> +static int viommu_modify_partial_fdt(libxl__gc *gc, void *pfdt, uint32_t domid)
> {
>     LOG(ERROR, "partial device tree not supported");
> 
> @@ -1511,7 +1618,7 @@ next_resize:
>         if (info->arch_arm.viommu_type == LIBXL_VIOMMU_TYPE_SMMUV3) {
>             FDT( make_vsmmuv3_node(gc, fdt, ainfo, dom) );
>             if (pfdt)
> -                FDT( modify_partial_fdt(gc, pfdt) );
> +                FDT( viommu_modify_partial_fdt(gc, pfdt, dom->guest_domid) );
>         }
> 
>         for (i = 0; i < d_config->num_disks; i++) {
> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> index ad914c915f..c85853e4cb 100644
> --- a/xen/arch/arm/domctl.c
> +++ b/xen/arch/arm/domctl.c
> @@ -16,6 +16,7 @@
> #include <xen/types.h>
> #include <xsm/xsm.h>
> #include <public/domctl.h>
> +#include <asm/viommu.h>
> 
> void arch_get_domain_info(const struct domain *d,
>                           struct xen_domctl_getdomaininfo *info)
> @@ -179,6 +180,39 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
>     }
>     case XEN_DOMCTL_dt_overlay:
>         return dt_overlay_domctl(d, &domctl->u.dt_overlay);
> +
> +#ifdef CONFIG_ARM_VIRTUAL_IOMMU
> +    case XEN_DOMCTL_viommu_alloc_vsid_range:
> +    {
> +        int rc = 0;
> +        uint16_t i;
> +        uint32_t vsid;
> +        struct xen_domctl_viommu_alloc_vsid_range *viommu_alloc_vsid_range =
> +            &domctl->u.viommu_alloc_vsid_range;
> +
> +        if ( viommu_alloc_vsid_range->pad )
> +            return -EINVAL;
> +
> +        for ( i = 0; i < viommu_alloc_vsid_range->nr_sids; i++ )
> +        {
> +            rc = viommu_allocate_free_vid(d, viommu_alloc_vsid_range->first_psid
> +                                            + i, &vsid);

Since we are using like this, wouldn’t have been better to have an “viommu_allocate_free_vid_range()”?

> +            if( rc )
> +                return rc;
> +        }
> +
> +        if ( !rc )
> +        {
> +            /* Calculate first vSID from allocated range */
> +            viommu_alloc_vsid_range->first_vsid = vsid -
> +                viommu_alloc_vsid_range->nr_sids + 1;
> +            rc = copy_to_guest(u_domctl, domctl, 1);

we should use something like:
rc = copy_to_guest(u_domctl, domctl, 1);
if ( rc )
    rc = -EFAULT;

Cheers,
Luca

[PATCH v3 23/23] doc/arm: vIOMMU design document

[Milan Djokic]

This document outlines the design of the emulated IOMMU,
including security considerations and future improvements.

Signed-off-by: Milan Djokic <milan_djokic@epam.com>
---
 docs/designs/arm-viommu.rst | 390 ++++++++++++++++++++++++++++++++++++
 1 file changed, 390 insertions(+)
 create mode 100644 docs/designs/arm-viommu.rst

diff --git a/docs/designs/arm-viommu.rst b/docs/designs/arm-viommu.rst
new file mode 100644
index 0000000000..0cf55d7108
--- /dev/null
+++ b/docs/designs/arm-viommu.rst
@@ -0,0 +1,390 @@
+==========================================================
+Design Proposal: Add SMMUv3 Stage-1 Support for XEN Guests
+==========================================================
+
+:Author:     Milan Djokic <milan_djokic@epam.com>
+:Date:       2026-02-13
+:Status:     Draft
+
+Introduction
+============
+
+The SMMUv3 supports two stages of translation. Each stage of translation 
+can be
+independently enabled. An incoming address is logically translated from 
+VA to
+IPA in stage 1, then the IPA is input to stage 2 which translates the IPA to
+the output PA. Stage 1 translation support is required to provide 
+isolation between different
+devices within OS. XEN already supports Stage 2 translation but there is no
+support for Stage 1 translation.
+This design proposal outlines the introduction of Stage-1 SMMUv3 support 
+in Xen for ARM guests.
+
+Motivation
+==========
+
+ARM systems utilizing SMMUv3 require stage-1 address translation to 
+ensure secure DMA and
+guest managed I/O memory mappings.
+With stage-1 enabled, guest manages IOVA to IPA mappings through its own 
+IOMMU driver.
+
+This feature enables:
+
+- Stage-1 translation for the guest domain
+- Device passthrough with per-device I/O address space
+
+Design Overview
+===============
+
+These changes provide emulated SMMUv3 support:
+
+- **SMMUv3 Stage-1 Translation**: stage-1 and nested translation support
+  in SMMUv3 driver.
+- **vIOMMU Abstraction**: Virtual IOMMU framework for guest stage-1
+  handling.
+- **Register/Command Emulation**: SMMUv3 register emulation and command
+  queue handling.
+- **Device Tree Extensions**: Adds `iommus` and virtual SMMUv3 nodes to
+  device trees for dom0 and dom0less scenarios.
+- **Runtime Configuration**: Introduces a `viommu` boot parameter for
+  dynamic enablement.
+
+A single vIOMMU device is exposed to the guest and mapped to one or more
+physical IOMMUs through a Xen-managed translation layer.
+The vIOMMU feature provides a generic framework together with a backend
+implementation specific to the target IOMMU type. The backend is responsible
+for implementing the hardware-specific data structures and command handling
+logic (currently only SMMUv3 is supported).
+
+This modular design allows the stage-1 support to be reused
+for other IOMMU architectures in the future.
+
+vIOMMU architecture
+===================
+
+Responsibilities:
+
+Guest:
+ - Configures stage-1 via vIOMMU commands.
+ - Handles stage-1 faults received from Xen.
+
+Xen:
+ - Emulates the IOMMU interface (registers, commands, events).
+ - Provides vSID->pSID mappings.
+ - Programs stage-1/stage-2 configuration in the physical IOMMU.
+ - Propagate stage-1 faults to guest.
+
+vIOMMU commands and faults are transmitted between guest and Xen via
+command and event queues (one command/event queue created per guest).
+
+vIOMMU command Flow:
+
+::
+
+    Guest:
+        smmu_cmd(vSID, IOVA -> IPA)
+
+    Xen:
+        trap MMIO read/write
+        translate vSID->pSID
+        store stage-1 state
+        program pIOMMU for (pSID, IPA -> PA)
+
+All hardware programming of the physical IOMMU is performed exclusively by Xen.
+
+vIOMMU Stage-1 fault handling flow:
+
+::
+
+    Xen:
+        receives stage-1 fault
+        triggers vIOMMU callback
+        injects virtual fault
+
+    Guest:
+        receives and handles fault
+
+vSID Mapping Layer
+------------------
+
+Each guest-visible Stream ID (vSID) is mapped by Xen to a physical Stream ID
+(pSID). The mapping is maintained per-domain. The allocation policy guarantees
+vSID uniqueness within a domain while allowing reuse of pSIDs for different
+pIOMMUs.
+
+* Platform devices receive individually allocated vSIDs.
+* PCI devices receive a contiguous vSID range derived from RID space.
+
+
+Supported Device Model
+======================
+
+Currently, the vIOMMU framework supports only devices described via the
+Device Tree (DT) model. This includes platform devices and basic PCI
+devices support instantiated through the vPCI DT node. ACPI-described
+devices are not supported.
+
+Guest assigned platform devices are mapped via `iommus` property:
+
+::
+
+    <&pIOMMU pSID> -> <&vIOMMU vSID>
+
+PCI devices use RID-based mapping via the root complex `iommu-map`:
+
+::
+
+    <RID-base &viommu vSID-base length>
+
+PCI Topology Assumptions and Constraints:
+
+- RID space must be contiguous
+- Pre-defined continuous pSID space (0-0x1000)
+- No runtime PCI reconfiguration
+- Single root complex assumed
+- Mapping is fixed at guest DT construction
+
+Constraints for PCI devices will be addressed as part of the future work on
+this feature.
+
+Security Considerations
+=======================
+
+Stage-1 translation provides isolation between guest devices by
+enforcing a per-device I/O address space, preventing unauthorized DMA.
+With the introduction of emulated IOMMU, additional protection
+mechanisms are required to minimize security risks.
+
+1. Observation:
+---------------
+Support for Stage-1 translation in SMMUv3 introduces new data structures 
+(`s1_cfg` alongside `s2_cfg`)
+and logic to write both Stage-1 and Stage-2 entries in the Stream Table 
+Entry (STE), including an `abort`
+field to handle partial configuration states.
+
+**Risk:**
+Without proper handling, a partially applied configuration
+might leave guest DMA mappings in an inconsistent state, potentially
+enabling unauthorized access or causing cross-domain interference.
+
+**Mitigation:** *(Handled by design)*
+This feature introduces logic that writes both `s1_cfg` and `s2_cfg` to
+STE and manages the `abort` field - only considering
+configuration if fully attached. This ensures  incomplete or invalid
+device configurations are safely ignored by the hypervisor.
+
+2. Observation:
+---------------
+Guests can now invalidate Stage-1 caches; invalidation needs forwarding
+to SMMUv3 hardware to maintain coherence.
+
+**Risk:**
+Failing to propagate cache invalidation could allow stale mappings,
+enabling access to old mappings and possibly
+data leakage or misrouting between devices assigned to the same guest.
+
+**Mitigation:**
+The guest must issue appropriate invalidation commands whenever
+its stage-1 I/O mappings are modified to ensure that translation caches
+remain coherent.
+
+3. Observation:
+---------------
+Introducing optional per-guest enabled features (`viommu` argument in xl 
+guest config) means some guests
+may opt-out.
+
+**Risk:**
+Guests without vIOMMU enabled (stage-2 only) could potentially dominate
+access to the physical command and event queues, since they bypass the
+emulation layer and processing is faster comparing to vIOMMU-enabled guests.
+
+**Mitigation:**
+Audit the impact of emulation overhead effect on IOMMU processing fairness
+in a multi-guest environment.
+Consider enabling/disabling stage-1 on a system level, instead of per-domain.
+
+4. Observation:
+---------------
+Guests have the ability to issue Stage-1 IOMMU commands like cache 
+invalidation, stream table entries
+configuration, etc. An adversarial guest may issue a high volume of 
+commands in rapid succession.
+
+**Risk:**
+Excessive commands requests can cause high hypervisor CPU consumption 
+and disrupt scheduling,
+leading to degraded system responsiveness and potential 
+denial-of-service scenarios.
+
+**Mitigation:**
+
+- Implement vIOMMU commands execution restart and continuation support:
+
+  - Introduce processing budget with only a limited amount of commands
+    handled per invocation.
+  - If additional commands remain pending after the budget is exhausted,
+    defer further processing and resume it asynchronously, e.g. via a
+    per-domain tasklet.
+
+- Batch multiple commands of same type to reduce emulation overhead:
+
+  - Inspect the command queue and group commands that can be processed
+    together (e.g. multiple successive invalidation requests or STE
+    updates for the same SID).
+  - Execute the entire batch in one go, reducing repeated accesses to
+    guest memory and emulation overhead per command.
+  - This reduces CPU time spent in the vIOMMU command processing loop.
+    The optimization is applicable only when consecutive commands of the
+    same type operate on the same SID/context.
+
+5. Observation:
+---------------
+Some guest commands issued towards vIOMMU are propagated to pIOMMU 
+command queue (e.g. TLB invalidate).
+
+**Risk:**
+Excessive commands requests from abusive guest can cause flooding of 
+physical IOMMU command queue,
+leading to degraded pIOMMU responsiveness on commands issued from other 
+guests.
+
+**Mitigation:**
+
+- Batch commands that are propagated to the pIOMMU command queue and
+  implement batch execution pause/continuation.
+  Rely on the same mechanisms as in the previous observation
+  (command continuation and batching of pIOMMU-related commands of the same
+  type and context).
+- If possible, implement domain penalization by adding a per-domain budget
+  for vIOMMU/pIOMMU usage:
+
+  - Apply per-domain dynamic budgeting of allowed IOMMU commands to
+    execute per invocation, reducing the budget for guests with
+    excessive command requests over a longer period of time
+  - Combine with command continuation mechanism
+
+6. Observation:
+---------------
+The vIOMMU feature includes an event queue used to forward IOMMU events
+to the guest (e.g. translation faults, invalid Stream IDs, permission errors).
+A malicious guest may misconfigure its IOMMU state or intentionally trigger
+faults at a high rate.
+
+**Risk:**
+Occurrence of IOMMU events with high frequency can cause Xen to flood the
+event queue and disrupt scheduling with
+high hypervisor CPU load for events handling.
+
+**Mitigation:**
+
+- Implement fail-safe state by disabling events forwarding when faults 
+  are occurred with high frequency and
+  not processed by guest:
+
+  - Introduce a per-domain pending event counter.
+  - Stop forwarding events to the guest once the number of unprocessed
+    events reaches a predefined threshold.
+
+- Consider disabling the emulated event queue for untrusted guests.
+- Note that this risk is more general and may also apply to stage-2-only
+  guests. This section addresses mitigations in the emulated IOMMU layer
+  only. Mitigation of physical event queue flooding should also be considered
+  in the target pIOMMU driver.
+
+Performance Impact
+==================
+
+With iommu stage-1 and nested translation inclusion, performance 
+overhead is introduced comparing to existing,
+stage-2 only usage in Xen. Once mappings are established, translations 
+should not introduce significant overhead.
+Emulated paths may introduce moderate overhead, primarily affecting 
+device initialization and event/command handling.
+Testing is performed on Renesas R-Car platform.
+Performance is mostly impacted by emulated vIOMMU operations, results 
+shown in the following table.
+
++-------------------------------+---------------------------------+
+| vIOMMU Operation              | Execution time in guest         |
++===============================+=================================+
+| Reg read                      | median: 645ns, worst-case: 2us  |
++-------------------------------+---------------------------------+
+| Reg write                     | median: 630ns, worst-case: 1us  |
++-------------------------------+---------------------------------+
+| Invalidate TLB                | median: 2us, worst-case: 10us   |
++-------------------------------+---------------------------------+
+| Invalidate STE                | median: 5us worst_case: 100us   |
++-------------------------------+---------------------------------+
+
+With vIOMMU exposed to guest, guest OS has to initialize IOMMU device
+and configure stage-1 mappings for the devices
+attached to it.
+Following table shows initialization stages which impact stage-1 enabled 
+guest boot time and compares it with
+stage-1 disabled guest.
+
+NOTE: Device probe execution time varies depending on device complexity.
+A USB host controller was selected as the test device in this case.
+
++---------------------+-----------------------+------------------------+
+| Stage               | Stage-1 Enabled Guest | Stage-1 Disabled Guest |
++=====================+=======================+========================+
+| IOMMU Init          | ~10ms                 | /                      |
++---------------------+-----------------------+------------------------+
+| Dev Attach / Mapping| ~100ms                | ~90ms                  |
++---------------------+-----------------------+------------------------+
+
+For devices configured with dynamic DMA mappings, DMA allocate/map/unmap 
+operations performance is
+also impacted on stage-1 enabled guests.
+Dynamic DMA mapping operation trigger emulated IOMMU functions like mmio 
+write/read and TLB invalidations.
+
++---------------+---------------------------+--------------------------+
+| DMA Op        | Stage-1 Enabled Guest     | Stage-1 Disabled Guest   |
++===============+===========================+==========================+
+| dma_alloc     | median: 20us, worst: 5ms  | median: 8us, worst: 60us |
++---------------+---------------------------+--------------------------+
+| dma_free      | median: 500us, worst: 10ms| median: 6us, worst: 30us |
++---------------+---------------------------+--------------------------+
+| dma_map       | median: 12us, worst: 60us | median: 3us, worst: 20us |
++---------------+---------------------------+--------------------------+
+| dma_unmap     | median: 400us, worst: 5ms | median: 5us, worst: 20us |
++---------------+---------------------------+--------------------------+
+
+Testing
+=======
+
+- QEMU-based ARM system tests for Stage-1 translation.
+- Actual hardware validation to ensure compatibility with real SMMUv3 
+implementations.
+- Unit/Functional tests validating correct translations (not implemented).
+
+Migration and Compatibility
+===========================
+
+This optional feature defaults to disabled (`viommu=""`) for backward 
+compatibility.
+
+Future improvements
+===================
+
+- Implement the proposed mitigations to address security risks that are 
+  not covered by the current design
+  (events batching, commands execution continuation)
+- PCI support
+- Support for other IOMMU HW (Renesas, RISC-V, etc.)
+
+References
+==========
+
+- Original feature implemented by Rahul Singh:
+  
+https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/ 
+
+- SMMUv3 architecture documentation
+- Existing vIOMMU code patterns (KVM, QEMU)
\ No newline at end of file
-- 
2.43.0

Re: [PATCH v3 00/23] Add SMMUv3 Stage 1 Support for Xen guests

[Julien Grall]

Hi Milan,

On 31/03/2026 10:51, Milan Djokic wrote:
> This patch series provides emulated SMMUv3 support in Xen, enabling stage-1
> translation for the guest OS.
> 
> Stage 1 translation support is required to provide isolation between different
> devices within OS. Xen already supports Stage 2 translation but there is no
> support for Stage 1 translation. The goal of this work is to support Stage 1
> translation for Xen guests.
> 
> This patch series represents a continuation of work from Rahul Singh:
> https://patchwork.kernel.org/project/xen-devel/cover/cover.1669888522.git.rahul.singh@arm.com/
> Original patch series is aligned with the newest Xen structure, with the addition
> of translation layer which provides 1:N vIOMMU->pIOMMU mapping, in order to
> support passthrough of the devices attached to different physical IOMMUs.
> 
> We cannot trust the guest OS to control the SMMUv3 hardware directly as
> compromised guest OS can corrupt the SMMUv3 configuration and make the system
> vulnerable. The guest gets the ownership of the stage 1 page tables and also
> owns stage 1 configuration structures. The Xen handles the root configuration
> structure (for security reasons), including the stage 2 configuration.
> 
> XEN will emulate the SMMUv3 hardware and expose the virtual SMMUv3 to the
> guest. Guest can use the native SMMUv3 driver to configure the stage 1
> translation. When the guest configures the SMMUv3 for Stage 1, XEN will trap
> the access and configure hardware.
> 
> SMMUv3 Driver(Guest OS) -> Configure the Stage-1 translation ->
> XEN trap access -> XEN SMMUv3 driver configure the HW.
> 
> The final patch series commit provides a design document for the emulated
> IOMMU (arm-viommu.rst), which was previously discussed with the maintainers.
> Details regarding implementation, future work and security risks are outlined
> in this document.
> 
> ---
> Changes in v2:
>   - Updated design and implementation with vIOMMU->pIOMMU mapping layer
>   - Addressed security risks in the design, provided initial performance
>     measurements
>   - Addressed comments from previous version
>   - Tested on Renesas R-Car platform, initial performance measurements for
>     stage-1 vs stage-1-less guests
> ---
> 
> ---
> Changes in v3:
>   - Bump domctl version, added explicit padding for the new domctl structures
>   - Remove unnecessary changes according to review comments
>   - Add "ARM" prefix for vIOMMU Kconfig options, since only ARM architecture is
>     supported at this point
>   - Re-generate go code
>   - Add missing commit sign-off tags

Can you please avoid posting new revision in reply-to a previous one? 
This makes tracking quite difficult on the reviewer side. So I ended up 
starting to post comments on v2. Please have a look at them.

Cheers,

-- 
Julien Grall