[PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[Shameer Kolothum]

Hi,

Changes from v2:
 https://lore.kernel.org/qemu-devel/20260206144823.80655-1-skolothumtho@nvidia.com/

  - Addressed feedback on v2 and picked up R-by tags. Thanks!
  - Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto
    "cmdqv".
  - Introduced a probe() callback to detect and initialize CMDQV
    support during device attach (patch #7).
  - Since CMDQV initialization is now deferred to device attach time,
    added a helper to link any unmapped MMIO/IRQ resources (patch #12).
  - Moved VINTF enable handling to the CONFIG ENABLE bit write path
    instead of any read/write access path (patch #17). Hopefully this
    won't break any corner cases.
  - Added checks for VINTF and CMDQV ENABLE bits before allocating
    vCMDQ HW queues, and free any allocated resources if disabled
    (patch #19).
  - Introduced a common helper for vEVENTQ read and validation
    (patch #23).
  - Updated bios-tables test IORT blobs for SMMUv3 device identifier
    changes (patch #29).
  - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
    using tegra241-cmdqv-stubs.c (patch #6).

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3

I have sanity tested this on an NVIDIA GRACE platform and will
continue with additional tests.

Feedback and testing are very welcome.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/20260225031658.32095-1-philmd@linaro.org/

---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/20251210133737.78257-1-skolothumtho@nvidia.com/

Thanks to Nicolin for the initial patches and testing on which this
is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces"
(VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
 - Page0 – guest owned control and status registers (directly mapped
           into the VM)
 - Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not
support the full command set. Only a subset, primarily invalidation
related commands, is accepted by the CMDQV hardware. For this reason,
a distinct CMDQV device must be exposed to the guest, and the guest OS
must include a Tegra241 CMDQV aware driver to take advantage of the
hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism,
allowing QEMU to attach guest configured VCMDQ buffers to the
underlying CMDQV hardware through IOMMUFD. The Linux kernel already
supports the full CMDQV virtualisation model via IOMMUFD[0].

---

Nicolin Chen (15):
  backends/iommufd: Update iommufd_backend_get_device_info
  backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
    ptr
  backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
  backends/iommufd: Introduce iommufd_backend_viommu_mmap
  hw/arm/tegra241-cmdqv: Implement CMDQV init
  hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
  hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
  hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
  hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
  hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
  hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
  hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space
  hw/arm/tegra241-cmdqv: Add reset handler
  hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
  hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (17):
  hw/arm/smmuv3-accel: Introduce CMDQV ops interface
  hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
  hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
  hw/arm/virt: Store SMMUv3 device objects in VirtMachineState.
  hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build
  hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support
  hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus
  system/physmem: Add address_space_is_ram() helper
  hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing
  hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
  hw/arm/smmuv3-accel: Introduce common helper for veventq read
  hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
  tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
    change
  hw/arm/smmuv3: Add per-device identifier property
  tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
    change
  hw/arm/smmuv3-accel: Introduce helper to query CMDQV type
  hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

 hw/arm/smmuv3-accel.h                         |  29 +
 hw/arm/tegra241-cmdqv.h                       | 342 ++++++++
 include/hw/arm/smmuv3.h                       |   3 +
 include/hw/arm/virt.h                         |   1 +
 include/system/iommufd.h                      |  16 +
 include/system/memory.h                       |  10 +
 backends/iommufd.c                            |  64 ++
 hw/arm/smmuv3-accel-stubs.c                   |  11 +
 hw/arm/smmuv3-accel.c                         | 196 ++++-
 hw/arm/smmuv3.c                               |   9 +
 hw/arm/tegra241-cmdqv-stubs.c                 |  18 +
 hw/arm/tegra241-cmdqv.c                       | 793 ++++++++++++++++++
 hw/arm/virt-acpi-build.c                      | 126 ++-
 hw/arm/virt.c                                 |  27 +
 hw/vfio/iommufd.c                             |   4 +-
 system/physmem.c                              |  11 +
 backends/trace-events                         |   4 +-
 hw/arm/Kconfig                                |   5 +
 hw/arm/meson.build                            |   2 +
 hw/arm/trace-events                           |   5 +
 tests/data/acpi/aarch64/virt/IORT.its_off     | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m  | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-dev  | Bin 260 -> 260 bytes
 .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
 24 files changed, 1596 insertions(+), 80 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c

-- 
2.43.0

[PATCH v3 01/32] backends/iommufd: Update iommufd_backend_get_device_info

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

The updated IOMMUFD uAPI introduces the ability for userspace to request
a specific hardware info data type via IOMMU_GET_HW_INFO. Update
iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE
when a non-zero type is supplied, and adjust all callers to pass a type
value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when
no specific type is requested.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 backends/iommufd.c    | 7 +++++++
 hw/arm/smmuv3-accel.c | 2 +-
 hw/vfio/iommufd.c     | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/backends/iommufd.c b/backends/iommufd.c
index acfab907c0..5daefe505e 100644
--- a/backends/iommufd.c
+++ b/backends/iommufd.c
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
     return true;
 }
 
+/*
+ * @type can carry a desired HW info type defined in the uapi headers. If caller
+ * doesn't have one, indicating it wants the default type, then @type should be
+ * zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
+ */
 bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid,
                                      uint32_t *type, void *data, uint32_t len,
                                      uint64_t *caps, uint8_t *max_pasid_log2,
                                      Error **errp)
 {
     struct iommu_hw_info info = {
+        .flags = (*type) ? IOMMU_HW_INFO_FLAG_INPUT_TYPE : 0,
         .size = sizeof(info),
         .dev_id = devid,
         .data_len = len,
         .data_uptr = (uintptr_t)data,
+        .in_data_type = *type,
     };
 
     if (ioctl(be->fd, IOMMU_GET_HW_INFO, &info)) {
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index f2fd926160..2c4228b7bd 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                            Error **errp)
 {
     struct iommu_hw_info_arm_smmuv3 info;
-    uint32_t data_type;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint64_t caps;
 
     if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c
index 131612eb83..3854f8b9fd 100644
--- a/hw/vfio/iommufd.c
+++ b/hw/vfio/iommufd.c
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
     ERRP_GUARD();
     IOMMUFDBackend *iommufd = vbasedev->iommufd;
     VFIOContainer *bcontainer = VFIO_IOMMU(container);
-    uint32_t type, flags = 0;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT, flags = 0;
     uint64_t hw_caps;
     VendorCaps caps;
     VFIOIOASHwpt *hwpt;
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,
     HostIOMMUDeviceIOMMUFD *idev;
     HostIOMMUDeviceCaps *caps = &hiod->caps;
     VendorCaps *vendor_caps = &caps->vendor_caps;
-    enum iommu_hw_info_type type;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint8_t max_pasid_log2;
     uint64_t hw_caps;
 
-- 
2.43.0

[PATCH v3 02/32] backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data
buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend
iommufd_backend_alloc_viommu() to pass a user pointer and size to the
kernel.

Update the caller accordingly.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/system/iommufd.h | 1 +
 backends/iommufd.c       | 4 ++++
 hw/arm/smmuv3-accel.c    | 4 ++--
 backends/trace-events    | 2 +-
 4 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/include/system/iommufd.h b/include/system/iommufd.h
index 7062944fe6..e027800c91 100644
--- a/include/system/iommufd.h
+++ b/include/system/iommufd.h
@@ -89,6 +89,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id,
                                 Error **errp);
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_hwpt, Error **errp);
 
 bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
diff --git a/backends/iommufd.c b/backends/iommufd.c
index 5daefe505e..400946810d 100644
--- a/backends/iommufd.c
+++ b/backends/iommufd.c
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,
 
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_viommu_id, Error **errp)
 {
     int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
         .type = viommu_type,
         .dev_id = dev_id,
         .hwpt_id = hwpt_id,
+        .data_len = data_len,
+        .data_uptr = (uintptr_t)data_ptr,
     };
 
     ret = ioctl(be->fd, IOMMU_VIOMMU_ALLOC, &alloc_viommu);
 
     trace_iommufd_backend_alloc_viommu(be->fd, dev_id, viommu_type, hwpt_id,
+                                       (uintptr_t)data_ptr, data_len,
                                        alloc_viommu.out_viommu_id, ret);
     if (ret) {
         error_setg_errno(errp, errno, "IOMMU_VIOMMU_ALLOC failed");
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index 2c4228b7bd..ab1b0a3669 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -533,8 +533,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     IOMMUFDViommu *viommu;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
-                                      s2_hwpt_id, &viommu_id, errp)) {
+                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
+                                      NULL, 0, &viommu_id, errp)) {
         return false;
     }
 
diff --git a/backends/trace-events b/backends/trace-events
index b9365113e7..3ba0c3503c 100644
--- a/backends/trace-events
+++ b/backends/trace-events
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%
 iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)"
 iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)"
 iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)"
-iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)"
+iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 
-- 
2.43.0

[PATCH v3 03/32] backends/iommufd: Introduce iommufd_backend_alloc_hw_queue

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Add a helper to allocate an iommufd backed HW queue for a vIOMMU.

While at it, define a struct IOMMUFDHWqueue for use by vendor
implementations.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/system/iommufd.h | 11 +++++++++++
 backends/iommufd.c       | 31 +++++++++++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 43 insertions(+)

diff --git a/include/system/iommufd.h b/include/system/iommufd.h
index e027800c91..8009ce3d31 100644
--- a/include/system/iommufd.h
+++ b/include/system/iommufd.h
@@ -65,6 +65,12 @@ typedef struct IOMMUFDVeventq {
     bool event_start; /* True after first valid event; cleared on overflow */
 } IOMMUFDVeventq;
 
+/* HW queue object for a vIOMMU-specific HW-accelerated queue */
+typedef struct IOMMUFDHWqueue {
+    IOMMUFDViommu *viommu;
+    uint32_t hw_queue_id;
+} IOMMUFDHWqueue;
+
 bool iommufd_backend_connect(IOMMUFDBackend *be, Error **errp);
 void iommufd_backend_disconnect(IOMMUFDBackend *be);
 
@@ -101,6 +107,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
                                    uint32_t *out_veventq_id,
                                    uint32_t *out_veventq_fd, Error **errp);
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
diff --git a/backends/iommufd.c b/backends/iommufd.c
index 400946810d..86f9c9f4cc 100644
--- a/backends/iommufd.c
+++ b/backends/iommufd.c
@@ -546,6 +546,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp)
+{
+    int ret;
+    struct iommu_hw_queue_alloc alloc_hw_queue = {
+        .size = sizeof(alloc_hw_queue),
+        .flags = 0,
+        .viommu_id = viommu_id,
+        .type = queue_type,
+        .index = index,
+        .nesting_parent_iova = addr,
+        .length = length,
+    };
+
+    ret = ioctl(be->fd, IOMMU_HW_QUEUE_ALLOC, &alloc_hw_queue);
+
+    trace_iommufd_backend_alloc_hw_queue(be->fd, viommu_id, queue_type,
+                                         index, addr, length,
+                                         alloc_hw_queue.out_hw_queue_id, ret);
+    if (ret) {
+        error_setg_errno(errp, errno, "IOMMU_HW_QUEUE_ALLOC failed");
+        return false;
+    }
+
+    g_assert(out_hw_queue_id);
+    *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id;
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
diff --git a/backends/trace-events b/backends/trace-events
index 3ba0c3503c..c5c1d95aad 100644
--- a/backends/trace-events
+++ b/backends/trace-events
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u
 iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
+iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
-- 
2.43.0

[PATCH v3 04/32] backends/iommufd: Introduce iommufd_backend_viommu_mmap

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Add a backend helper to mmap hardware MMIO regions exposed via iommufd for
a vIOMMU instance. This allows user space to access HW-accelerated MMIO
pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/system/iommufd.h |  4 ++++
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 27 insertions(+)

diff --git a/include/system/iommufd.h b/include/system/iommufd.h
index 8009ce3d31..38cfceca84 100644
--- a/include/system/iommufd.h
+++ b/include/system/iommufd.h
@@ -112,6 +112,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint64_t addr, uint64_t length,
                                     uint32_t *out_hw_queue_id, Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
diff --git a/backends/iommufd.c b/backends/iommufd.c
index 86f9c9f4cc..6defdf6a52 100644
--- a/backends/iommufd.c
+++ b/backends/iommufd.c
@@ -577,6 +577,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, be->fd,
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size, offset);
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
diff --git a/backends/trace-events b/backends/trace-events
index c5c1d95aad..b63420b73e 100644
--- a/backends/trace-events
+++ b/backends/trace-events
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
-- 
2.43.0

[PATCH v3 05/32] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

[Shameer Kolothum]

Command Queue Virtualization (CMDQV) is a hardware extension available
on certain platforms that allows the SMMUv3 command queue to be
virtualized and passed through to a VM, improving performance.

For example, NVIDIA Tegra241 implements CMDQV to support virtualization
of multiple command queues (VCMDQs).

The term CMDQV is used here generically to refer to any platform that
provides hardware support to virtualize the SMMUv3 command queue.

CMDQV support is a specialization of the IOMMUFD-backed accelerated
SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific
probe, initialization, and vIOMMU allocation logic from the base
implementation. The ops pointer and associated state are stored in
the accelerated SMMUv3 state.

This provides an extensible design to support future vendor-specific
CMDQV implementations.

No functional change.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3-accel.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
index 85669d0e00..5bdd01afb5 100644
--- a/hw/arm/smmuv3-accel.h
+++ b/hw/arm/smmuv3-accel.h
@@ -15,6 +15,22 @@
 #include <linux/iommufd.h>
 #endif
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*probe)(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp);
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -25,6 +41,7 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
-- 
2.43.0

[PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

[Shameer Kolothum]

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated
CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub
implementation for CMDQV probe, initialization, vIOMMU allocation
and reset handling.

Functional CMDQV support is added in follow-up patches.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
new file mode 100644
index 0000000000..07e10e86ee
--- /dev/null
+++ b/hw/arm/tegra241-cmdqv.h
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv-stubs.c b/hw/arm/tegra241-cmdqv-stubs.c
new file mode 100644
index 0000000000..eedc7bfdcd
--- /dev/null
+++ b/hw/arm/tegra241-cmdqv-stubs.c
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
+
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
new file mode 100644
index 0000000000..ad5a0d4611
--- /dev/null
+++ b/hw/arm/tegra241-cmdqv.c
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
+{
+}
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                            uint32_t *out_viommu_id, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                                 Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return &tegra241_cmdqv_ops;
+}
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index c66c452737..3305c6e76e 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
diff --git a/hw/arm/meson.build b/hw/arm/meson.build
index 8f834c32b1..fc83b635e7 100644
--- a/hw/arm/meson.build
+++ b/hw/arm/meson.build
@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
-- 
2.43.0

[PATCH v3 07/32] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

[Shameer Kolothum]

Add support for selecting and initializing a CMDQV backend based on
the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed
allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during
device setup. If probing succeeds, the selected ops are stored
in the accelerated SMMUv3 state and used. If probing fails,
QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed
during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU
allocation, free, and reset handling. Otherwise, the base
implementation is used.

The current implementation wires up the Tegra241 CMDQV backend
through the generic ops interface. Functional CMDQV behaviour is
added in subsequent patches.

No functional change.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 7 deletions(-)

diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
index 26b2fc42fd..648412cafc 100644
--- a/include/hw/arm/smmuv3.h
+++ b/include/hw/arm/smmuv3.h
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 };
 
 typedef enum {
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index ab1b0a3669..4373bbd97b 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and shared_as_sysmem
@@ -522,6 +523,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -532,10 +534,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
-        return false;
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
+            return false;
+        }
     }
 
     viommu = g_new0(IOMMUFDViommu, 1);
@@ -581,12 +590,69 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
 }
 
+static const SMMUv3AccelCmdqvOps *
+smmuv3_accel_probe_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = tegra241_cmdqv_get_ops();
+
+    if (!ops || !ops->probe) {
+        error_setg(errp, "No CMDQV ops found");
+        return NULL;
+    }
+
+    if (!ops->probe(s, idev, errp)) {
+        return NULL;
+    }
+    return ops;
+}
+
+static bool
+smmuv3_accel_select_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = NULL;
+
+    if (s->s_accel->cmdqv_ops) {
+        return true;
+    }
+
+    switch (s->cmdqv) {
+    case ON_OFF_AUTO_OFF:
+        s->s_accel->cmdqv_ops = NULL;
+        return true;
+    case ON_OFF_AUTO_AUTO:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, NULL);
+        break;
+    case ON_OFF_AUTO_ON:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, errp);
+        if (!ops) {
+            error_append_hint(errp, "CMDQV requested but not supported");
+            return false;
+        }
+        s->s_accel->cmdqv_ops = ops;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    if (ops && ops->init && !ops->init(s, errp)) {
+        return false;
+    }
+    s->s_accel->cmdqv_ops = ops;
+    return true;
+}
+
 static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
                                           HostIOMMUDevice *hiod, Error **errp)
 {
@@ -621,6 +687,10 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
         goto done;
     }
 
+    if (!smmuv3_accel_select_cmdqv(s, idev, errp)) {
+        return false;
+    }
+
     if (!smmuv3_accel_alloc_viommu(s, idev, errp)) {
         error_append_hint(errp, "Unable to alloc vIOMMU: idev devid 0x%x: ",
                           idev->devid);
@@ -867,8 +937,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
 
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)
-- 
2.43.0

[PATCH v3 08/32] hw/arm/virt: Store SMMUv3 device objects in VirtMachineState.

[Shameer Kolothum]

Introduce a GPtrArray in VirtMachineState to keep track of all
SMMUv3 devices created on the virt machine.

This will avoid relying on object_child_foreach_recursive() walks
of the object tree when accessing SMMUv3 instances. Subsequent
patches will use this list during ACPI IORT table generation and
for CMDQV-related handling.

No functional change.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/hw/arm/virt.h | 1 +
 hw/arm/virt.c         | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 8069422769..f9437e6410 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -186,6 +186,7 @@ struct VirtMachineState {
     MemoryRegion *sysmem;
     MemoryRegion *secure_sysmem;
     bool pci_preserve_config;
+    GPtrArray *smmuv3_devices;
 };
 
 #define VIRT_ECAM_ID(high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 50865e8115..292e523664 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -3232,6 +3232,7 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev,
             }
 
             create_smmuv3_dev_dtb(vms, dev, bus, errp);
+            g_ptr_array_add(vms->smmuv3_devices, dev);
         }
     }
 
@@ -3659,6 +3660,8 @@ static void virt_instance_init(Object *obj)
     vms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     vms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
     cxl_machine_init(obj, &vms->cxl_devices_state);
+
+    vms->smmuv3_devices = g_ptr_array_new_with_free_func(NULL);
 }
 
 static const TypeInfo virt_machine_info = {
-- 
2.43.0

[PATCH v3 09/32] hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build

[Shameer Kolothum]

Stop walking the object tree to discover SMMUv3 devices when building
the IORT table.

Use the SMMUv3 device array maintained by the virt machine instead,
avoiding recursive object traversal.

No functional change intended.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++----------------------
 1 file changed, 31 insertions(+), 39 deletions(-)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 544004615d..ae78e9b9e0 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
     return map_a->input_base - map_b->input_base;
 }
 
-static int iort_smmuv3_devices(Object *obj, void *opaque)
-{
-    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
-    GArray *sdev_blob = opaque;
-    AcpiIortIdMapping idmap;
-    PlatformBusDevice *pbus;
-    int min_bus, max_bus;
-    SysBusDevice *sbdev;
-    PCIBus *bus;
-
-    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
-        return 0;
-    }
-
-    bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
-    sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
-    sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
-    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
-    sbdev = SYS_BUS_DEVICE(obj);
-    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
-    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
-    sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
-    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
-    sdev.irq += ARM_SPI_BASE;
-
-    pci_bus_range(bus, &min_bus, &max_bus);
-    sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
-    idmap.input_base = min_bus << 8,
-    idmap.id_count = (max_bus - min_bus + 1) << 8,
-    g_array_append_val(sdev.rc_smmu_idmaps, idmap);
-    g_array_append_val(sdev_blob, sdev);
-    return 0;
-}
-
 /*
  * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
  * return the total number of idmaps.
  */
-static int populate_smmuv3_dev(GArray *sdev_blob)
+static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
 {
-    object_child_foreach_recursive(object_get_root(),
-                                   iort_smmuv3_devices, sdev_blob);
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        AcpiIortSMMUv3Dev sdev = {0};
+        AcpiIortIdMapping idmap;
+        PlatformBusDevice *pbus;
+        int min_bus, max_bus;
+        SysBusDevice *sbdev;
+        PCIBus *bus;
+
+        bus = PCI_BUS(object_property_get_link(obj, "primary-bus",
+                                               &error_abort));
+        sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
+        sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
+        sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
+        sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        sdev.irq += ARM_SPI_BASE;
+
+        pci_bus_range(bus, &min_bus, &max_bus);
+        sdev.rc_smmu_idmaps = g_array_new(false, true,
+                                          sizeof(AcpiIortIdMapping));
+        idmap.input_base = min_bus << 8;
+        idmap.id_count = (max_bus - min_bus + 1) << 8;
+        g_array_append_val(sdev.rc_smmu_idmaps, idmap);
+        g_array_append_val(sdev_blob, sdev);
+    }
     /* Sort the smmuv3 devices(if any) by smmu idmap input_base */
     g_array_sort(sdev_blob, smmuv3_dev_idmap_compare);
     /*
@@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     if (vms->legacy_smmuv3_present) {
         rc_smmu_idmaps_len = populate_smmuv3_legacy_dev(smmuv3_devs);
     } else {
-        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs);
+        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs, vms);
     }
 
     num_smmus = smmuv3_devs->len;
-- 
2.43.0

[PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

[Shameer Kolothum]

Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241 CMDQV.

Validate the returned data type, version, number of vCMDQs and SIDs per
VM. Fail the probe if the host does not meet the expected requirements.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h |  5 +++++
 hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 07e10e86ee..312064a081 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -10,6 +10,11 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#define TEGRA241_CMDQV_VERSION             1
+#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
+#define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index ad5a0d4611..a270fa7ce4 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
+    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
+    uint64_t caps;
+
+    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
+                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
+                                         NULL, errp)) {
+        return false;
+    }
+    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
+        error_setg(errp, "Host CMDQV: unexpected data type %u (expected %u)",
+                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
+        return false;
+    }
+    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
+        error_setg(errp, "Host CMDQV: unsupported version %u (expected %u)",
+                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
+        return false;
+    }
+    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)",
+                   cmdqv_info.log2vcmdqs, TEGRA241_CMDQV_NUM_CMDQ_LOG2);
+        return false;
+    }
+    if (cmdqv_info.log2vsids < TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >= %u)",
+                   cmdqv_info.log2vsids, TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
+        return false;
+    }
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
-- 
2.43.0

[PATCH v3 11/32] hw/arm/tegra241-cmdqv: Implement CMDQV init

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues
(VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split
into 64KB pages:

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0
0x20000: Global VCMDQ registers, Page1
0x30000: VINTF0 logical VCMDQ registers, Page0
0x40000: VINTF0 logical VCMDQ registers, Page1

This patch wires up the Tegra241 CMDQV init callback and allocates
vendor-specific CMDQV state. The state pointer is stored in
SMMUv3AccelState for use by subsequent CMDQV operations.

The CMDQV MMIO region and a dedicated IRQ line are registered with the
SMMUv3 device. The MMIO read/write handlers are currently stubs and will
be implemented in later patches.

The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF
error conditions. This patch only registers the IRQ line. Interrupt
generation and propagation to the guest will be added in a subsequent
patch.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3-accel.h   |  1 +
 hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++--
 3 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
index 5bdd01afb5..7d6e4c6b76 100644
--- a/hw/arm/smmuv3-accel.h
+++ b/hw/arm/smmuv3-accel.h
@@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
     const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;  /* vendor specific CMDQV state */
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 312064a081..46aa9e8a9f 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -15,6 +15,24 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages)
+ *
+ * 0x00000  TEGRA241_CMDQV_CFG    (Global CMDQV configuration)
+ * 0x10000  TEGRA241_VCMDQ_PAGE0  (Virtual CMDQ page 0)
+ * 0x20000  TEGRA241_VCMDQ_PAGE1  (Virtual CMDQ page 1)
+ * 0x30000  TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0)
+ * 0x40000  TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1)
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index a270fa7ce4..6959766129 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
 }
@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
-- 
2.43.0

[PATCH v3 12/32] hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

[Shameer Kolothum]

SMMUv3 devices with acceleration may enable CMDQV extensions
after device realize. In that case, additional MMIO regions and
IRQ lines may be registered but not yet mapped to the platform bus.

Ensure SMMUv3 device resources are linked to the platform bus
during machine_done().

This is safe to do unconditionally since the platform bus helpers
skip resources that are already mapped.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/virt.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 292e523664..c75a8d6e9e 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms)
     }
 }
 
+/*
+ * SMMUv3 devices with acceleration may enable CMDQV extensions
+ * after device realize. In that case, additional MMIO regions and
+ * IRQ lines may be registered but not yet mapped to the platform bus.
+ *
+ * Ensure all resources are linked to the platform bus before final
+ * machine setup.
+ */
+
+static void virt_smmuv3_dev_link_cmdqv(VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        DeviceState *dev = g_ptr_array_index(vms->smmuv3_devices, i);
+        platform_bus_link_device(PLATFORM_BUS_DEVICE(vms->platform_bus_dev),
+                                 SYS_BUS_DEVICE(dev));
+    }
+}
+
 static
 void virt_machine_done(Notifier *notifier, void *data)
 {
@@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)
     if (vms->cxl_devices_state.is_enabled) {
         cxl_fmws_link_targets(&error_fatal);
     }
+
+    virt_smmuv3_dev_link_cmdqv(vms);
+
     /*
      * If the user provided a dtb, we assume the dynamic sysbus nodes
      * already are integrated there. This corresponds to a use case where
-- 
2.43.0

[PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Replace the stub implementation with real vIOMMU allocation for
Tegra241 CMDQV.

Free the vIOMMU ID on teardown.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 6959766129..d487612ba2 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
-- 
2.43.0

[PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Tegra241 CMDQV defines a set of global control and status registers
used to configure virtual command queue allocation and interrupt
behavior.

Add read/write emulation for the global CMDQV register page
(offset 0x00000), backed by a simple register cache. This includes
CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
map and the VINTF0 related registers defined in the global CMDQV
register space.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 46aa9e8a9f..50bcecee9d 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+#define A_VI_ERR_MAP 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)                 \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)                       \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_MATCH_(i, j)                               \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index d487612ba2..a3830a02d6 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
-- 
2.43.0

[PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

  -Global VCMDQ registers at 0x10000/0x20000
  -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
IOMMU_HW_QUEUE_ALLOC.

Add read side emulation for both global VCMDQ and VI_VCMDQ register
ranges. MMIO accesses are decoded to extract the VCMDQ instance index
and normalized to a VCMDQ0_* register offset, allowing a single helper
to service all VCMDQ instances.

VI_VCMDQ accesses are translated to their equivalent global VCMDQ
offsets and reuse the same decoding path. All VCMDQ reads are currently
served from cached register state.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
 2 files changed, 255 insertions(+)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 50bcecee9d..d379b8860c 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* Omitting [0][1~2] as not being directly called */
 A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index a3830a02d6..d2e6938e44 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -14,6 +14,46 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 {
     int i;
@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ
+         * decoding logic below.
+         */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a 0x80-byte window starting at 0x10000.
+         *
+         * The MMIO offset is decoded to extract the VCMDQ index and normalized
+         * to the corresponding VCMDQ0_* register by subtracting index * 0x80.
+         *
+         * A single helper then services all VCMDQs, with @index selecting the
+         * instance.
+         */
+        index = (offset - 0x10000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
+        index = (offset - 0x20000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
                       __func__, offset);
-- 
2.43.0

[PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

This is the write side counterpart of the VCMDQ read emulation.

Add write handling for global VCMDQ and VI_VCMDQ register windows.
Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to
VCMDQ0_* offsets, reusing the same layout assumptions as the read path.

VI_VCMDQ registers are treated as a logical alias of the global VCMDQ
registers and share the same decoding logic.

Writes are backed by cached register state only; no hardware queue
mapping is performed yet.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)

diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index d2e6938e44..e1f1562c44 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void
+tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
+                           uint64_t value, unsigned size)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
                                        uint64_t value)
 {
@@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_write_vintf(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        index = (offset - 0x10000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        index = (offset - 0x20000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
-- 
2.43.0

[PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
VINTF pages expose a logical view local to a given VINTF. Although real
hardware may support multiple VINTFs, the kernel currently exposes a
single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during
vIOMMU allocation. However, the logical-to-physical association between
VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and
unmap it when VINTF is disabled. This prepares the VINTF mapping
in advance of subsequent patches that add VCMDQ allocation support.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index d379b8860c..3ce9f539ae 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index e1f1562c44..a3767a85a3 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
 
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
-- 
2.43.0

[PATCH v3 18/32] system/physmem: Add address_space_is_ram() helper

[Shameer Kolothum]

Introduce address_space_is_ram(), a helper to determine whether
a guest physical address resolves to a RAM-backed MemoryRegion within
an AddressSpace.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/system/memory.h | 10 ++++++++++
 system/physmem.c        | 11 +++++++++++
 2 files changed, 21 insertions(+)

diff --git a/include/system/memory.h b/include/system/memory.h
index 0562af3136..02b2e83fd7 100644
--- a/include/system/memory.h
+++ b/include/system/memory.h
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,
  */
 bool address_space_is_io(AddressSpace *as, hwaddr addr);
 
+/**
+ * address_space_is_ram: check whether a guest physical address whithin
+ *                       an address space is RAM.
+ *
+ * @as: #AddressSpace to be accessed
+ * @addr: address within that address space
+ */
+
+bool address_space_is_ram(AddressSpace *as, hwaddr addr);
+
 /* address_space_map: map a physical memory region into a host virtual address
  *
  * May map a subset of the requested range, given by and returned in @plen.
diff --git a/system/physmem.c b/system/physmem.c
index 2fb0c25c93..ddcf921311 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)
     return !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
 }
 
+bool address_space_is_ram(AddressSpace *as, hwaddr addr)
+{
+    MemoryRegion *mr;
+
+    RCU_READ_LOCK_GUARD();
+    mr = address_space_translate(as, addr, &addr, NULL, false,
+                                 MEMTXATTRS_UNSPECIFIED);
+
+    return memory_region_is_ram(mr);
+}
+
 static hwaddr
 flatview_extend_translation(FlatView *fv, hwaddr addr,
                             hwaddr target_len,
-- 
2.43.0

[PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Add support for allocating IOMMUFD hardware queues when the guest
programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed
through the VINTF Page0 region. This region is mapped directly into
the guest address space (introduced in a subsequent patch), so QEMU
does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the
hardware queue based on that bit. Instead, allocate the IOMMUFD
hardware queue when the guest writes a VCMDQ BASE register with a
valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ,
free it before reallocation.

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 79 insertions(+), 3 deletions(-)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 3ce9f539ae..139e14b61b 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index a3767a85a3..002dde50fc 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in the reverse order to avoid "resource busy" error */
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);
+
+    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
+                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
+                                        index, addr, size, &hw_queue_id,
+                                        errp)) {
+        return false;
+    }
+    hw_queue = g_new(IOMMUFDHWqueue, 1);
+    hw_queue->hw_queue_id = hw_queue_id;
+    hw_queue->viommu = viommu;
+    cmdqv->vcmdq[index] = hw_queue;
+
+    return true;
+}
+
 static bool
 tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
 {
@@ -192,7 +253,7 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
-                           uint64_t value, unsigned size)
+                           uint64_t value, unsigned size, Error **errp)
 {
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
@@ -220,11 +281,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                 (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
                 (value & 0xffffffffULL);
         }
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_BASE_H:
         cmdqv->vcmdq_base[index] =
             (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
             ((uint64_t)value << 32);
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
         if (size == 8) {
@@ -263,6 +326,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
             tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
@@ -302,6 +366,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
         break;
@@ -321,7 +386,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
         index = (offset - 0x10000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         /* Same decoding as read() case: See comments above */
@@ -330,7 +395,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         index = (offset - 0x20000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
-- 
2.43.0

[PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

[Shameer Kolothum]

When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the
kernel, access the VCMDQ registers directly through the VINTF page0
backing instead of using QEMU's cached register state.

VINTF page0 provides the backing memory region for VCMDQ registers
once a hardware queue is created. In that case, reads and writes
should reflect the live backing state.

If a VCMDQ is not allocated, or if VINTF page0 is not available,
continue to use the cached register values maintained by QEMU.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 002dde50fc..17b9552906 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -14,17 +14,44 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+
+    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
+                        (offset0 - 0x10000));
+}
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                            uint64_t value, unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;
-- 
2.43.0

[PATCH v3 21/32] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

VINTF page0 is backed by host memory returned by the kernel via mmap.
Instead of trapping and emulating accesses to this region, map it
directly into the guest-visible MMIO space.

The VINTF page0 MMIO region is created lazily when the first VCMDQ
hardware queue is allocated. When CMDQV or VINTF is disabled, the
region is removed.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 139e14b61b..914977c2ef 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -38,6 +38,7 @@ typedef struct Tegra241CMDQV {
     qemu_irq irq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 17b9552906..ce144add54 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -178,6 +178,38 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv->mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
 {
     SMMUv3AccelState *accel = cmdqv->s_accel;
@@ -236,6 +268,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -371,6 +404,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
             tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
@@ -411,6 +445,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
-- 
2.43.0

[PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

[Shameer Kolothum]

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the
corresponding teardown path to free the vEVENTQ during cleanup.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/smmuv3-accel.c   | 10 ++++++++-
 hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
index 7d6e4c6b76..4bff90e2c1 100644
--- a/hw/arm/smmuv3-accel.h
+++ b/hw/arm/smmuv3-accel.h
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 914977c2ef..01d446474a 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -37,6 +37,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
+    IOMMUFDVeventq *veventq;
     void *vintf_page0;
     MemoryRegion *mr_vintf_page0;
 
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index 4373bbd97b..f6602f51aa 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+
     /* Attach a HWPT based on SMMUv3 GBPA.ABORT value */
     hwpt_id = smmuv3_accel_gbpa_hwpt(s, accel);
     if (!host_iommu_device_iommufd_attach_hwpt(idev, hwpt_id, errp)) {
-        goto free_veventq;
+        goto free_cmdqv_veventq;
     }
     return true;
 
+free_cmdqv_veventq:
+    if (cmdqv_ops && cmdqv_ops->free_veventq) {
+        cmdqv_ops->free_veventq(s);
+    }
 free_veventq:
     smmuv3_accel_free_veventq(accel);
 free_bypass_hwpt:
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index ce144add54..8cde459b4f 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -487,6 +487,51 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     }
 }
 
+static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
+{
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+
+    if (!veventq) {
+        return;
+    }
+    close(veventq->veventq_fd);
+    iommufd_backend_free_id(veventq->viommu->iommufd, veventq->veventq_id);
+    g_free(veventq);
+    cmdqv->veventq = NULL;
+}
+
+static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
+{
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+    IOMMUFDVeventq *veventq;
+    uint32_t veventq_id;
+    uint32_t veventq_fd;
+
+    if (cmdqv->veventq) {
+        return true;
+    }
+
+    if (!iommufd_backend_alloc_veventq(viommu->iommufd, viommu->viommu_id,
+                                       IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                       1 << 16, &veventq_id, &veventq_fd,
+                                       errp)) {
+        error_append_hint(errp, "Tegra241 CMDQV: failed to alloc veventq");
+        return false;
+    }
+
+    veventq = g_new(IOMMUFDVeventq, 1);
+    veventq->veventq_id = veventq_id;
+    veventq->veventq_fd = veventq_fd;
+    veventq->viommu = accel->viommu;
+    cmdqv->veventq = veventq;
+
+    return true;
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
     SMMUv3AccelState *accel = s->s_accel;
@@ -580,6 +625,8 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
     .init = tegra241_cmdqv_init,
     .alloc_viommu = tegra241_cmdqv_alloc_viommu,
     .free_viommu = tegra241_cmdqv_free_viommu,
+    .alloc_veventq = tegra241_cmdqv_alloc_veventq,
+    .free_veventq = tegra241_cmdqv_free_veventq,
     .reset = tegra241_cmdqv_reset,
 };
 
-- 
2.43.0

[PATCH v3 23/32] hw/arm/smmuv3-accel: Introduce common helper for veventq read

[Shameer Kolothum]

Move the vEVENTQ read and validation logic into a common helper.

The helper performs the read(), checks for overflow and short reads,
validates the sequence number, and updates the sequence state.

This helper can be reused in the subsequent patch for Tegra241 CMDQV
vEVENTQ support.

Error handling is slightly adjusted: instead of reporting errors
directly in the read handler, the helper now returns errors via
Error **. Sequence gaps are reported as warnings.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3-accel.h       |  2 ++
 hw/arm/smmuv3-accel-stubs.c |  6 ++++
 hw/arm/smmuv3-accel.c       | 67 ++++++++++++++++++++++---------------
 3 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
index 4bff90e2c1..c349981e79 100644
--- a/hw/arm/smmuv3-accel.h
+++ b/hw/arm/smmuv3-accel.h
@@ -70,6 +70,8 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
                                 Error **errp);
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
diff --git a/hw/arm/smmuv3-accel-stubs.c b/hw/arm/smmuv3-accel-stubs.c
index 870fc2a71c..1d5d3bb10c 100644
--- a/hw/arm/smmuv3-accel-stubs.c
+++ b/hw/arm/smmuv3-accel-stubs.c
@@ -42,6 +42,12 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
     return true;
 }
 
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
+{
+    return true;
+}
+
 void smmuv3_accel_idr_override(SMMUv3State *s)
 {
 }
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index f6602f51aa..5f296ea763 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -391,47 +391,60 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *bs, void *cmd, SMMUDevice *sdev,
                    sizeof(Cmd), &entry_num, cmd, errp);
 }
 
-static void smmuv3_accel_event_read(void *opaque)
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
 {
-    SMMUv3State *s = opaque;
-    IOMMUFDVeventq *veventq = s->s_accel->veventq;
-    struct {
-        struct iommufd_vevent_header hdr;
-        struct iommu_vevent_arm_smmuv3 vevent;
-    } buf;
-    enum iommu_veventq_type type = IOMMU_VEVENTQ_TYPE_ARM_SMMUV3;
-    uint32_t id = veventq->veventq_id;
     uint32_t last_seq = veventq->last_event_seq;
+    uint32_t id = veventq->veventq_id;
+    struct iommufd_vevent_header *hdr;
     ssize_t bytes;
 
-    bytes = read(veventq->veventq_fd, &buf, sizeof(buf));
+    bytes = read(veventq->veventq_fd, buf, size);
     if (bytes <= 0) {
         if (errno == EAGAIN || errno == EINTR) {
-            return;
+            return true;
         }
-        error_report_once("vEVENTQ(type %u id %u): read failed (%m)", type, id);
-        return;
+        error_setg(errp, "vEVENTQ(type %u id %u): read failed (%m)", type, id);
+        return false;
     }
-
-    if (bytes == sizeof(buf.hdr) &&
-        (buf.hdr.flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
-        error_report_once("vEVENTQ(type %u id %u): overflowed", type, id);
+    hdr = (struct iommufd_vevent_header *)buf;
+    if (bytes == sizeof(*hdr) &&
+        (hdr->flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
+        error_setg(errp, "vEVENTQ(type %u id %u): overflowed", type, id);
         veventq->event_start = false;
-        return;
+        return false;
     }
-    if (bytes < sizeof(buf)) {
-        error_report_once("vEVENTQ(type %u id %u): short read(%zd/%zd bytes)",
-                          type, id, bytes, sizeof(buf));
-        return;
+    if (bytes < size) {
+        error_setg(errp, "vEVENTQ(type %u id %u): short read(%zd/%zd bytes)",
+                          type, id, bytes, size);
+        return false;
     }
-
     /* Check sequence in hdr for lost events if any */
-    if (veventq->event_start && (buf.hdr.sequence - last_seq != 1)) {
-        error_report_once("vEVENTQ(type %u id %u): lost %u event(s)",
-                          type, id, buf.hdr.sequence - last_seq - 1);
+    if (veventq->event_start && (hdr->sequence - last_seq != 1)) {
+        warn_report("vEVENTQ(type %u id %u): lost %u event(s)",
+                    type, id, hdr->sequence - last_seq - 1);
     }
-    veventq->last_event_seq = buf.hdr.sequence;
+    veventq->last_event_seq = hdr->sequence;
     veventq->event_start = true;
+    return true;
+}
+
+static void smmuv3_accel_event_read(void *opaque)
+{
+    SMMUv3State *s = opaque;
+    IOMMUFDVeventq *veventq = s->s_accel->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_arm_smmuv3 vevent;
+    } buf;
+    Error *local_err;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_ARM_SMMUV3, &buf,
+                                          sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
     smmuv3_propagate_event(s, (Evt *)&buf.vevent);
 }
 
-- 
2.43.0

[PATCH v3 24/32] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

[Shameer Kolothum]

Install an event handler on the CMDQV vEVENTQ fd to read and propagate
host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered
via qemu_set_fd_handler().

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.c | 58 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  3 +++
 2 files changed, 61 insertions(+)

diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 8cde459b4f..99b85e698f 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -9,8 +9,10 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "trace.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
@@ -487,6 +489,43 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     }
 }
 
+static void tegra241_cmdqv_event_read(void *opaque)
+{
+    Tegra241CMDQV *cmdqv = opaque;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    Error *local_err;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                          &buf, sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];
+        }
+        cmdqv->vi_err_map[0] |= 0x1;
+        qemu_irq_pulse(cmdqv->irq);
+        trace_tegra241_cmdqv_err_map(
+        cmdqv->vintf_cmdq_err_map[3], cmdqv->vintf_cmdq_err_map[2],
+        cmdqv->vintf_cmdq_err_map[1], cmdqv->vintf_cmdq_err_map[0]);
+    }
+}
+
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
 {
     SMMUv3AccelState *accel = s->s_accel;
@@ -496,6 +535,7 @@ static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
     if (!veventq) {
         return;
     }
+    qemu_set_fd_handler(veventq->veventq_fd, NULL, NULL, NULL);
     close(veventq->veventq_fd);
     iommufd_backend_free_id(veventq->viommu->iommufd, veventq->veventq_id);
     g_free(veventq);
@@ -510,6 +550,7 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
     IOMMUFDVeventq *veventq;
     uint32_t veventq_id;
     uint32_t veventq_fd;
+    int flags;
 
     if (cmdqv->veventq) {
         return true;
@@ -523,13 +564,30 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
         return false;
     }
 
+    flags = fcntl(veventq_fd, F_GETFL);
+    if (flags < 0) {
+        error_setg(errp, "Failed to get flags for vEVENTQ fd");
+        goto free_veventq;
+    }
+    if (fcntl(veventq_fd, F_SETFL, O_NONBLOCK | flags) < 0) {
+        error_setg(errp, "Failed to set O_NONBLOCK on vEVENTQ fd");
+        goto free_veventq;
+    }
+
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     veventq->viommu = accel->viommu;
     cmdqv->veventq = veventq;
 
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, cmdqv);
     return true;
+
+free_veventq:
+    close(veventq_fd);
+    iommufd_backend_free_id(viommu->iommufd, veventq_id);
+    return false;
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
diff --git a/hw/arm/trace-events b/hw/arm/trace-events
index 3457536fb0..76bda0efef 100644
--- a/hw/arm/trace-events
+++ b/hw/arm/trace-events
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev
 smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64
 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x"
 
+# tegra241-cmdqv
+tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
 strongarm_ssp_read_underrun(void) "SSP rx underrun"
-- 
2.43.0

[PATCH v3 25/32] hw/arm/tegra241-cmdqv: Add reset handler

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Introduce a reset handler for the Tegra241 CMDQV and initialize its
register state.

CMDQV gets initialized early during guest boot, hence the handler verifies
that at least one cold-plugged device is attached to the associated vIOMMU
before proceeding. This is required to retrieve host CMDQV info and
to validate it against the QEMU implementation support.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/smmuv3-accel.c   | 12 +++++++++-
 hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 4 files changed, 64 insertions(+), 1 deletion(-)

diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
index 01d446474a..96a737eb4e 100644
--- a/hw/arm/tegra241-cmdqv.h
+++ b/hw/arm/tegra241-cmdqv.h
@@ -72,6 +72,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index 5f296ea763..9a570b8af9 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -774,7 +774,11 @@ static void smmuv3_accel_unset_iommu_device(PCIBus *bus, void *opaque,
     QLIST_REMOVE(accel_dev, next);
     trace_smmuv3_accel_unset_iommu_device(devfn, idev->devid);
 
-    if (QLIST_EMPTY(&accel->device_list)) {
+   /*
+    * Keep the vIOMMU alive when CMDQV is present, as the vIOMMU to host
+    * SMMUv3 association cannot be changed via device hot-plug.
+    */
+    if (QLIST_EMPTY(&accel->device_list) && !accel->cmdqv) {
         smmuv3_accel_free_viommu(accel);
     }
 }
@@ -966,6 +970,12 @@ void smmuv3_accel_reset(SMMUv3State *s)
     /* Attach a HWPT based on GBPA reset value */
     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
 
+    if (s->cmdqv == ON_OFF_AUTO_ON && QLIST_EMPTY(&accel->device_list)) {
+        error_report("cmdqv=on: requires at least one cold-plugged "
+                     "vfio-pci device");
+        exit(1);
+    }
+
     if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
         accel->cmdqv_ops->reset(s);
     }
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 99b85e698f..5afdc5c8a4 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
 #include "trace.h"
 
@@ -617,8 +618,57 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     return true;
 }
 
+static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
+{
+    int i;
+
+    cmdqv->config = V_CONFIG_RESET;
+    cmdqv->param =
+        FIELD_DP32(cmdqv->param, PARAM, CMDQV_VER, TEGRA241_CMDQV_VERSION);
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_CMDQ_LOG2,
+                          TEGRA241_CMDQV_NUM_CMDQ_LOG2);
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_SID_PER_VM_LOG2,
+                          TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
+    trace_tegra241_cmdqv_init_regs(cmdqv->param);
+    cmdqv->status = R_STATUS_CMDQV_ENABLED_MASK;
+    for (i = 0; i < 2; i++) {
+        cmdqv->vi_err_map[i] = 0;
+        cmdqv->vi_int_mask[i] = 0;
+        cmdqv->cmdq_err_map[i] = 0;
+    }
+    cmdqv->vintf_config = 0;
+    cmdqv->vintf_status = 0;
+    for (i = 0; i < 4; i++) {
+        cmdqv->vintf_cmdq_err_map[i] = 0;
+    }
+    for (i = 0; i < TEGRA241_CMDQV_MAX_CMDQ; i++) {
+        cmdqv->cmdq_alloc_map[i] = 0;
+        cmdqv->vcmdq_cons_indx[i] = 0;
+        cmdqv->vcmdq_prod_indx[i] = 0;
+        cmdqv->vcmdq_config[i] = 0;
+        cmdqv->vcmdq_status[i] = 0;
+        cmdqv->vcmdq_gerror[i] = 0;
+        cmdqv->vcmdq_gerrorn[i] = 0;
+        cmdqv->vcmdq_base[i] = 0;
+        cmdqv->vcmdq_cons_indx_base[i] = 0;
+    }
+    return;
+}
+
 static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+
+    if (!cmdqv) {
+        return;
+    }
+
+    tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
+    tegra241_cmdqv_munmap_vintf_page0(cmdqv, NULL);
+    tegra241_cmdqv_free_all_vcmdq(cmdqv);
+
+    tegra241_cmdqv_init_regs(s, cmdqv);
 }
 
 static const MemoryRegionOps mmio_cmdqv_ops = {
diff --git a/hw/arm/trace-events b/hw/arm/trace-events
index 76bda0efef..ef495c040c 100644
--- a/hw/arm/trace-events
+++ b/hw/arm/trace-events
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 
 # tegra241-cmdqv
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
-- 
2.43.0

[PATCH v3 26/32] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

CMDQV HW reads guest queue memory in its host physical address setup via
IOMUUFD. This requires the guest queue memory isn't only contiguous in
guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we
must only advertise a CMDQV size that the host can safely back with
physically contiguous memory. Allowing a CMDQV larger than the host page
size could cause the hardware to DMA across page boundaries leading to
faults.

Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the
host’s contiguous backing.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 5afdc5c8a4..a379341c0a 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -11,10 +11,14 @@
 #include "qemu/error-report.h"
 #include "qemu/log.h"
 #include "trace.h"
+#include <math.h>
 
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 
 static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
@@ -618,9 +622,38 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     return true;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)
+{
+    RAMBlock *rb;
+    size_t pg, min_pg = SIZE_MAX;
+
+    RAMBLOCK_FOREACH(rb) {
+        MemoryRegion *mr = rb->mr;
+
+        /* Only consider real RAM regions */
+        if (!mr || !memory_region_is_ram(mr)) {
+            continue;
+        }
+
+        /* Skip RAM regions that are not backed by a memory-backend */
+        if (!object_dynamic_cast(mr->owner, TYPE_MEMORY_BACKEND)) {
+            continue;
+        }
+
+        pg = qemu_ram_pagesize(rb);
+        if (pg && pg < min_pg) {
+            min_pg = pg;
+        }
+    }
+
+    return (min_pg == SIZE_MAX) ? qemu_real_host_page_size() : min_pg;
+}
+
 static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
 {
     int i;
+    size_t pgsize;
+    uint32_t val;
 
     cmdqv->config = V_CONFIG_RESET;
     cmdqv->param =
@@ -652,6 +685,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
         cmdqv->vcmdq_base[i] = 0;
         cmdqv->vcmdq_cons_indx_base[i] = 0;
     }
+
+   /*
+    * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+    * queue fits entirely within the smallest backend page size, ensuring
+    * the command queue is physically contiguous in host memory.
+    */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(log2(pgsize) - 4, val));
+
     return;
 }
 
-- 
2.43.0

[PATCH v3 27/32] tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change

[Shameer Kolothum]

The IORT SMMUv3 node Identifier field will be derived from a new
per-device "identifier" property instead of relying on enumeration
order.

Add the affected IORT blobs to allowed-diff list for bios-table tests.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..df5fe417c0 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,5 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/aarch64/virt/IORT.its_off",
+"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
-- 
2.43.0

[PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

[Shameer Kolothum]

Add an "identifier" property to the SMMUv3 device and use it when
building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable
per-device identifier. A subsequent patch will use the same identifier
when generating the DSDT description for Tegra241 CMDQV, ensuring that
the IORT and DSDT entries refer to the same SMMUv3 instance.

No functional change intended.

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 include/hw/arm/smmuv3.h  | 1 +
 hw/arm/smmuv3.c          | 2 ++
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 4 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
index 648412cafc..73b8f39aaa 100644
--- a/include/hw/arm/smmuv3.h
+++ b/include/hw/arm/smmuv3.h
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
index 468135bb24..c1f84bedd4 100644
--- a/hw/arm/smmuv3.c
+++ b/hw/arm/smmuv3.c
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index ae78e9b9e0..20605185c5 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index c75a8d6e9e..44c6b99c96 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
     }
 }
 
+static uint8_t smmuv3_dev_id;
 static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                             DeviceState *dev, Error **errp)
 {
@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
+                                     NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
-- 
2.43.0

[PATCH v3 29/32] tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier change

[Shameer Kolothum]

Update the reference IORT blobs after identifier change for SMMUv3 nodes.
This affects the aarch64 'virt' IORT tests.

IORT diff is the identifier change in SMMUv3 nodes:

 /*
  * Intel ACPI Component Architecture
  * AML/ASL+ Disassembler version 20230628 (64-bit version)
  * Copyright (c) 2000 - 2023 Intel Corporation
  *
- * Disassembly of tests/data/acpi/aarch64/virt/IORT.its_off, Wed Feb 25 16:36:44 2026
+ * Disassembly of /tmp/aml-KVO1K3, Wed Feb 25 16:36:44 2026
  *
  * ACPI Data Table [IORT]
  *
  * Format: [HexOffset DecimalOffset ByteLength]  FieldName : FieldValue (in hex)
  */

 [000h 0000 004h]                   Signature : "IORT"    [IO Remapping Table]
 [004h 0004 004h]                Table Length : 000000AC
 [008h 0008 001h]                    Revision : 05
-[009h 0009 001h]                    Checksum : 95
+[009h 0009 001h]                    Checksum : 96
 [00Ah 0010 006h]                      Oem ID : "BOCHS "
 [010h 0016 008h]                Oem Table ID : "BXPC    "
 [018h 0024 004h]                Oem Revision : 00000001
 [01Ch 0028 004h]             Asl Compiler ID : "BXPC"
 [020h 0032 004h]       Asl Compiler Revision : 00000001

 [024h 0036 004h]                  Node Count : 00000002
 [028h 0040 004h]                 Node Offset : 00000030
 [02Ch 0044 004h]                    Reserved : 00000000
...

 [074h 0116 001h]                        Type : 02
 [075h 0117 002h]                      Length : 0038
 [077h 0119 001h]                    Revision : 03
-[078h 0120 004h]                  Identifier : 00000001
+[078h 0120 004h]                  Identifier : 00000000
 [07Ch 0124 004h]               Mapping Count : 00000001
 [080h 0128 004h]              Mapping Offset : 00000024

 [084h 0132 008h]           Memory Properties : [IORT Memory Access Properties]
 [084h 0132 004h]             Cache Coherency : 00000001
 [088h 0136 001h]       Hints (decoded below) : 00
                                    Transient : 0
                               Write Allocate : 0
                                Read Allocate : 0
                                     Override : 0
 [089h 0137 002h]                    Reserved : 0000
 [08Bh 0139 001h] Memory Flags (decoded below) : 03
                                    Coherency : 1
                             Device Attribute : 1
 [08Ch 0140 004h]               ATS Attribute : 00000000
 [090h 0144 004h]          PCI Segment Number : 00000000
 [094h 0148 001h]           Memory Size Limit : 40
 [095h 0149 002h]          PASID Capabilities : 0000
 [097h 0151 001h]                    Reserved : 00

 [098h 0152 004h]                  Input base : 00000000
 [09Ch 0156 004h]                    ID Count : 000000FF
 [0wA0h 0160 004h]                 Output Base : 00000000
 [0A4h 0164 004h]            Output Reference : 00000030
 [0A8h 0168 004h]       Flags (decoded below) : 00000000
                               Single Mapping : 0

...

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h     |   4 ----
 tests/data/acpi/aarch64/virt/IORT.its_off       | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m    | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-dev    | Bin 260 -> 260 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
 5 files changed, 4 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index df5fe417c0..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,5 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/aarch64/virt/IORT.its_off",
-"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
-"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
-"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
diff --git a/tests/data/acpi/aarch64/virt/IORT.its_off b/tests/data/acpi/aarch64/virt/IORT.its_off
index 0cf52b52f671637bf4dbc9e0fc80c3c73d0b01d3..295dc381063fa4c415a933c5c690ca0fc8ffd6dc 100644
GIT binary patch
delta 23
ecmZ3(xQ3C-(?2L=4FdxM>$Hhnd5jDbD~$n68wN1|

delta 23
ecmZ3(xQ3C-(?2L=4FdxM>(q%{d5nw`D~$n676vf@

diff --git a/tests/data/acpi/aarch64/virt/IORT.msi_gicv2m b/tests/data/acpi/aarch64/virt/IORT.msi_gicv2m
index 0cf52b52f671637bf4dbc9e0fc80c3c73d0b01d3..295dc381063fa4c415a933c5c690ca0fc8ffd6dc 100644
GIT binary patch
delta 23
ecmZ3(xQ3C-(?2L=4FdxM>$Hhnd5jDbD~$n68wN1|

delta 23
ecmZ3(xQ3C-(?2L=4FdxM>(q%{d5nw`D~$n676vf@

diff --git a/tests/data/acpi/aarch64/virt/IORT.smmuv3-dev b/tests/data/acpi/aarch64/virt/IORT.smmuv3-dev
index 60cfed1361976ef26b280c11ba2e233f1cfd9383..12f3e0f01864acf24fc86d4cde2de561aa92f878 100644
GIT binary patch
delta 23
ecmZo+YGLB?^bZPQVPs%n^`6MJiIHLAUUvXH(*=zH

delta 23
ecmZo+YGLB?^bZPQVPs%n^_<AHiIHjIUUvXH$pwu7

diff --git a/tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy b/tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy
index e2de4049f044bdfe5b3e656b94282ddf17832d9f..f940b696af2abcbb3b46fcc865acde7456aa3389 100644
GIT binary patch
delta 23
ecmX@Wcz}`1(?2NW00RR9tL#LsJVu6zmF@scb_NIl

delta 23
ecmX@Wcz}`1(?2NW00RR9tIR~MJVwTemF@scaRvwg

-- 
2.43.0

[PATCH v3 30/32] hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

[Shameer Kolothum]

Introduce a SMMUv3AccelCmdqvType enum and a helper to query the
CMDQV implementation type associated with an accelerated SMMUv3
instance.

A subsequent patch will use this helper when generating the
Tegra241 CMDQV DSDT.

Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3-accel.h       |  7 +++++++
 hw/arm/smmuv3-accel-stubs.c |  5 +++++
 hw/arm/smmuv3-accel.c       | 12 ++++++++++++
 hw/arm/tegra241-cmdqv.c     |  6 ++++++
 4 files changed, 30 insertions(+)

diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
index c349981e79..6d21788006 100644
--- a/hw/arm/smmuv3-accel.h
+++ b/hw/arm/smmuv3-accel.h
@@ -15,6 +15,11 @@
 #include <linux/iommufd.h>
 #endif
 
+typedef enum SMMUv3AccelCmdqvType {
+    SMMUV3_CMDQV_NONE = 0,
+    SMMUV3_CMDQV_TEGRA241,
+} SMMUv3AccelCmdqvType;
+
 /*
  * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
  * support multiple VCMDQs with virtualization capabilities.
@@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {
     void (*free_viommu)(SMMUv3State *s);
     bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
     void (*free_veventq)(SMMUv3State *s);
+    SMMUv3AccelCmdqvType (*get_type)(void);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
@@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
                                       void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
diff --git a/hw/arm/smmuv3-accel-stubs.c b/hw/arm/smmuv3-accel-stubs.c
index 1d5d3bb10c..5ca94d605f 100644
--- a/hw/arm/smmuv3-accel-stubs.c
+++ b/hw/arm/smmuv3-accel-stubs.c
@@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    return SMMUV3_CMDQV_NONE;
+}
diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
index 9a570b8af9..585b460943 100644
--- a/hw/arm/smmuv3-accel.c
+++ b/hw/arm/smmuv3-accel.c
@@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    SMMUv3State *s = ARM_SMMUV3(obj);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) {
+        return SMMUV3_CMDQV_NONE;
+    }
+
+    return accel->cmdqv_ops->get_type();
+}
+
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp)
 {
     SMMUState *bs = ARM_SMMU(s);
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index a379341c0a..42d7dbfde7 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type(void)
+{
+    return SMMUV3_CMDQV_TEGRA241;
+};
+
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
@@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
     .free_viommu = tegra241_cmdqv_free_viommu,
     .alloc_veventq = tegra241_cmdqv_alloc_veventq,
     .free_veventq = tegra241_cmdqv_free_veventq,
+    .get_type = tegra241_cmdqv_get_type,
     .reset = tegra241_cmdqv_reset,
 };
 
-- 
2.43.0

[PATCH v3 31/32] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

[Shameer Kolothum]

From: Nicolin Chen <nicolinc@nvidia.com>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is
created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches
the Identifier field of the corresponding SMMUv3 IORT node, allowing
the CMDQV DSDT device to be correctly associated with its SMMU.

Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241
CMDQV header, the Meson file entry is updated to build it as part of
arm_ss instead of arm_common_ss

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/virt-acpi-build.c | 52 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events      |  1 +
 2 files changed, 53 insertions(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 20605185c5..462faa6a2b 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -65,6 +65,9 @@
 #include "target/arm/cpu.h"
 #include "target/arm/multiprocessing.h"
 
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
 #define ARM_SPI_BASE 32
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
@@ -1113,6 +1116,51 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
     build_fadt(table_data, linker, &fadt, vms->oem_id, vms->oem_table_id);
 }
 
+static void acpi_dsdt_add_tegra241_cmdqv(Aml *scope, VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        PlatformBusDevice *pbus;
+        Aml *dev, *crs, *addr;
+        SysBusDevice *sbdev;
+        hwaddr base;
+        uint32_t id;
+        int irq;
+
+        if (smmuv3_accel_cmdqv_type(obj) != SMMUV3_CMDQV_TEGRA241) {
+            continue;
+        }
+        id = object_property_get_uint(obj, "identifier", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        base = platform_bus_get_mmio_addr(pbus, sbdev, 1);
+        base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        irq = platform_bus_get_irqn(pbus, sbdev, NUM_SMMU_IRQS);
+        irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        irq += ARM_SPI_BASE;
+
+        dev = aml_device("CV%.02u", id);
+        aml_append(dev, aml_name_decl("_HID", aml_string("NVDA200C")));
+        aml_append(dev, aml_name_decl("_UID", aml_int(id)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
+
+        crs = aml_resource_template();
+        addr = aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                                AML_CACHEABLE, AML_READ_WRITE, 0x0, base,
+                                base + TEGRA241_CMDQV_IO_LEN - 0x1, 0x0,
+                                TEGRA241_CMDQV_IO_LEN);
+        aml_append(crs, addr);
+        aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
+                                      AML_ACTIVE_HIGH, AML_EXCLUSIVE,
+                                      (uint32_t *)&irq, 1));
+        aml_append(dev, aml_name_decl("_CRS", crs));
+
+        aml_append(scope, dev);
+
+        trace_virt_acpi_dsdt_tegra241_cmdqv(id, base, irq);
+    }
+}
+
 /* DSDT */
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
@@ -1177,6 +1225,10 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_tpm(scope, vms);
 #endif
 
+    if (!vms->legacy_smmuv3_present) {
+        acpi_dsdt_add_tegra241_cmdqv(scope, vms);
+    }
+
     aml_append(dsdt, scope);
 
     pci0_scope = aml_scope("\\_SB.PCI0");
diff --git a/hw/arm/trace-events b/hw/arm/trace-events
index ef495c040c..e7e3ccfe9f 100644
--- a/hw/arm/trace-events
+++ b/hw/arm/trace-events
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
-- 
2.43.0

[PATCH v3 32/32] hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

[Shameer Kolothum]

Introduce a “cmdqv” property to enable Tegra241 CMDQV support.
This is only enabled for accelerated SMMUv3 devices.

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
---
 hw/arm/smmuv3.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
index c1f84bedd4..70135af015 100644
--- a/hw/arm/smmuv3.c
+++ b/hw/arm/smmuv3.c
@@ -1967,6 +1967,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
             error_setg(errp, "ssidsize can only be set if accel=on");
             return false;
         }
+        if (s->cmdqv == ON_OFF_AUTO_ON) {
+            error_setg(errp, "cmdqv can only be enabled if accel=on");
+            return false;
+        }
         return true;
     }
 
@@ -2119,6 +2123,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_BOOL("ats", SMMUv3State, ats, false),
     DEFINE_PROP_UINT8("oas", SMMUv3State, oas, 44),
     DEFINE_PROP_UINT8("ssidsize", SMMUv3State, ssidsize, 0),
+    DEFINE_PROP_ON_OFF_AUTO("cmdqv", SMMUv3State, cmdqv, ON_OFF_AUTO_AUTO),
 };
 
 static void smmuv3_instance_init(Object *obj)
@@ -2158,6 +2163,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to 0. A value greater than 0 is required to enable "
         "PASID support.");
+    object_class_property_set_description(klass, "cmdqv",
+        "Enable/disable CMDQ-Virtualisation support (for accel=on)");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
-- 
2.43.0

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Jonathan Cameron via qemu development]

On Thu, 26 Feb 2026 10:50:39 +0000
Shameer Kolothum <skolothumtho@nvidia.com> wrote:

> From: Nicolin Chen <nicolinc@nvidia.com>
> 
> Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
> 
>   -Global VCMDQ registers at 0x10000/0x20000
>   -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
> 
> The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> IOMMU_HW_QUEUE_ALLOC.
> 
> Add read side emulation for both global VCMDQ and VI_VCMDQ register
> ranges. MMIO accesses are decoded to extract the VCMDQ instance index
> and normalized to a VCMDQ0_* register offset, allowing a single helper
> to service all VCMDQ instances.
> 
> VI_VCMDQ accesses are translated to their equivalent global VCMDQ
> offsets and reuse the same decoding path. All VCMDQ reads are currently
> served from cached register state.
> 
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
Hi Shameer,

As noted below there are a lot of repeats of 0x80 and the register window offsets in here.
Maybe some defines would make things clearer?

>  static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  {
>      int i;
> @@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>          qemu_log_mask(LOG_UNIMP,
> @@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>          return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>          return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> +        /*
> +         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
> +         * layout as the global VCMDQ registers, but are based at 0x30000
> +         * instead of 0x10000.
> +         *
> +         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
> +         * global VCMDQ offset, then fall through to reuse the common VCMDQ
> +         * decoding logic below.
> +         */
> +        offset -= 0x20000;
There are a lot of repeated numeric values of offsets and sizes in here.
I'm a bit in two minds about whether they are clearer as numbers or you should add
a few more defines.

Jonathan

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

[Eric Auger]

Hi Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated
> CMDQV ops interface.
>
> This patch wires up the Tegra241 CMDQV backend and provides a stub
> implementation for CMDQV probe, initialization, vIOMMU allocation
> and reset handling.
>
> Functional CMDQV support is added in follow-up patches.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
>  hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
>  hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
>  hw/arm/Kconfig                |  5 ++++
>  hw/arm/meson.build            |  2 ++
>  5 files changed, 96 insertions(+)
>  create mode 100644 hw/arm/tegra241-cmdqv.h
>  create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
>  create mode 100644 hw/arm/tegra241-cmdqv.c
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> new file mode 100644
> index 0000000000..07e10e86ee
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -0,0 +1,15 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
> + *
> + * Written by Nicolin Chen, Shameer Kolothum
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#ifndef HW_ARM_TEGRA241_CMDQV_H
> +#define HW_ARM_TEGRA241_CMDQV_H
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> +
> +#endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv-stubs.c b/hw/arm/tegra241-cmdqv-stubs.c
> new file mode 100644
> index 0000000000..eedc7bfdcd
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv-stubs.c
> @@ -0,0 +1,18 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + *
> + * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
^typo
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +#include "hw/arm/smmuv3.h"
> +#include "smmuv3-accel.h"
> +#include "hw/arm/tegra241-cmdqv.h"
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
> +{
> +    return NULL;
> +}
> +
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> new file mode 100644
> index 0000000000..ad5a0d4611
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -0,0 +1,56 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
> + *
> + * Written by Nicolin Chen, Shameer Kolothum
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "hw/arm/smmuv3.h"
> +#include "smmuv3-accel.h"
> +#include "tegra241-cmdqv.h"
> +
> +static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
> +{
> +}
> +
> +static bool
> +tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                            uint32_t *out_viommu_id, Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static void tegra241_cmdqv_reset(SMMUv3State *s)
> +{
> +}
> +
> +static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                                 Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
> +    .probe = tegra241_cmdqv_probe,
> +    .init = tegra241_cmdqv_init,
> +    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
> +    .free_viommu = tegra241_cmdqv_free_viommu,
> +    .reset = tegra241_cmdqv_reset,

I think I would create a no_cmdqv_ops to convert the existing code
without any cmdqv support to the same api. This would avoid
the repeating: 

if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.

Also this looks a bit strange to have different signatures for the
functions used with both vcmdq on and off

Eric


> +};
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
> +{
> +    return &tegra241_cmdqv_ops;
> +}
> diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
> index c66c452737..3305c6e76e 100644
> --- a/hw/arm/Kconfig
> +++ b/hw/arm/Kconfig
> @@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
>      depends on TCG
>      select FSL_IMX8MP
>  
> +config TEGRA241_CMDQV
> +    bool
> +    depends on ARM_SMMUV3_ACCEL
> +
>  config ARM_SMMUV3_ACCEL
>      bool
>      depends on ARM_SMMUV3
> @@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
>  config ARM_SMMUV3
>      bool
>      select ARM_SMMUV3_ACCEL if IOMMUFD
> +    imply TEGRA241_CMDQV
>  
>  config FSL_IMX6UL
>      bool
> diff --git a/hw/arm/meson.build b/hw/arm/meson.build
> index 8f834c32b1..fc83b635e7 100644
> --- a/hw/arm/meson.build
> +++ b/hw/arm/meson.build
> @@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
>  stub_ss.add(files('smmuv3-accel-stubs.c'))
> +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
> +stub_ss.add(files('tegra241-cmdqv-stubs.c'))
>  arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
>  arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
>  arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(

Re: [PATCH v3 09/32] hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Stop walking the object tree to discover SMMUv3 devices when building
> the IORT table.
>
> Use the SMMUv3 device array maintained by the virt machine instead,
> avoiding recursive object traversal.
>
> No functional change intended.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++----------------------
>  1 file changed, 31 insertions(+), 39 deletions(-)
>
> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index 544004615d..ae78e9b9e0 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
>      return map_a->input_base - map_b->input_base;
>  }
>  
> -static int iort_smmuv3_devices(Object *obj, void *opaque)
> -{
> -    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
> -    AcpiIortSMMUv3Dev sdev = {0};
> -    GArray *sdev_blob = opaque;
> -    AcpiIortIdMapping idmap;
> -    PlatformBusDevice *pbus;
> -    int min_bus, max_bus;
> -    SysBusDevice *sbdev;
> -    PCIBus *bus;
> -
> -    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
> -        return 0;
> -    }
> -
> -    bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
> -    sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
> -    sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
> -    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
> -    sbdev = SYS_BUS_DEVICE(obj);
> -    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
> -    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
> -    sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
> -    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
> -    sdev.irq += ARM_SPI_BASE;
> -
> -    pci_bus_range(bus, &min_bus, &max_bus);
> -    sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
> -    idmap.input_base = min_bus << 8,
> -    idmap.id_count = (max_bus - min_bus + 1) << 8,
> -    g_array_append_val(sdev.rc_smmu_idmaps, idmap);
> -    g_array_append_val(sdev_blob, sdev);
> -    return 0;
> -}
> -
>  /*
>   * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
>   * return the total number of idmaps.
>   */
> -static int populate_smmuv3_dev(GArray *sdev_blob)
> +static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
nit: I would prefer vms to be the 1st arg
>  {
> -    object_child_foreach_recursive(object_get_root(),
> -                                   iort_smmuv3_devices, sdev_blob);
> +    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
> +        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
> +        AcpiIortSMMUv3Dev sdev = {0};
> +        AcpiIortIdMapping idmap;
> +        PlatformBusDevice *pbus;
> +        int min_bus, max_bus;
> +        SysBusDevice *sbdev;
> +        PCIBus *bus;
> +
> +        bus = PCI_BUS(object_property_get_link(obj, "primary-bus",
> +                                               &error_abort));
> +        sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
> +        sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
> +        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
> +        sbdev = SYS_BUS_DEVICE(obj);
> +        sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
> +        sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
> +        sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
> +        sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
> +        sdev.irq += ARM_SPI_BASE;
> +
> +        pci_bus_range(bus, &min_bus, &max_bus);
> +        sdev.rc_smmu_idmaps = g_array_new(false, true,
> +                                          sizeof(AcpiIortIdMapping));
> +        idmap.input_base = min_bus << 8;
> +        idmap.id_count = (max_bus - min_bus + 1) << 8;
> +        g_array_append_val(sdev.rc_smmu_idmaps, idmap);
> +        g_array_append_val(sdev_blob, sdev);
> +    }
>      /* Sort the smmuv3 devices(if any) by smmu idmap input_base */
>      g_array_sort(sdev_blob, smmuv3_dev_idmap_compare);
>      /*
> @@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
>      if (vms->legacy_smmuv3_present) {
>          rc_smmu_idmaps_len = populate_smmuv3_legacy_dev(smmuv3_devs);
>      } else {
> -        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs);
> +        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs, vms);
So did you check this does not break existing bios qtests (as we change
the way we iterate on the devices)? If so maybe add a comment in commit
description.
>      }
>  
>      num_smmus = smmuv3_devs->len;
I would squash previous patch in that one.

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241 CMDQV.
>
> Validate the returned data type, version, number of vCMDQs and SIDs per
> VM. Fail the probe if the host does not meet the expected requirements.

Can you explain why it is important to check 

number of vCMDQs and SIDs per VM? 

>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h |  5 +++++
>  hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
>  2 files changed, 35 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 07e10e86ee..312064a081 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -10,6 +10,11 @@
>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>  #define HW_ARM_TEGRA241_CMDQV_H
>  
> +#define TEGRA241_CMDQV_VERSION             1
> +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
> +#define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index ad5a0d4611..a270fa7ce4 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
>  static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>                                   Error **errp)
>  {
> -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> -    return false;
> +    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
> +    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
> +    uint64_t caps;
> +
> +    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
> +                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
> +                                         NULL, errp)) {
> +        return false;
> +    }
> +    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
> +        error_setg(errp, "Host CMDQV: unexpected data type %u (expected %u)",
> +                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
> +        return false;
> +    }
> +    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
> +        error_setg(errp, "Host CMDQV: unsupported version %u (expected %u)",
> +                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
> +        return false;
> +    }
> +    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
> +        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)",
> +                   cmdqv_info.log2vcmdqs, TEGRA241_CMDQV_NUM_CMDQ_LOG2);
> +        return false;
> +    }
> +    if (cmdqv_info.log2vsids < TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {
> +        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >= %u)",
> +                   cmdqv_info.log2vsids, TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
> +        return false;
> +    }
> +    return true;
>  }
>  
>  static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
Besides
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric

Re: [PATCH v3 11/32] hw/arm/tegra241-cmdqv: Implement CMDQV init

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV extends SMMUv3 with support for virtual command queues
> (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split
> into 64KB pages:
>
> 0x00000: Global CMDQV registers
> 0x10000: Global VCMDQ registers, Page0
> 0x20000: Global VCMDQ registers, Page1
> 0x30000: VINTF0 logical VCMDQ registers, Page0
> 0x40000: VINTF0 logical VCMDQ registers, Page1
>
> This patch wires up the Tegra241 CMDQV init callback and allocates
> vendor-specific CMDQV state. The state pointer is stored in
> SMMUv3AccelState for use by subsequent CMDQV operations.
>
> The CMDQV MMIO region and a dedicated IRQ line are registered with the
> SMMUv3 device. The MMIO read/write handlers are currently stubs and will
> be implemented in later patches.
>
> The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF
> error conditions. This patch only registers the IRQ line. Interrupt
> generation and propagation to the guest will be added in a subsequent
> patch.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/smmuv3-accel.h   |  1 +
>  hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++--
>  3 files changed, 47 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
> index 5bdd01afb5..7d6e4c6b76 100644
> --- a/hw/arm/smmuv3-accel.h
> +++ b/hw/arm/smmuv3-accel.h
> @@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {
>      uint32_t abort_hwpt_id;
>      QLIST_HEAD(, SMMUv3AccelDevice) device_list;
>      const SMMUv3AccelCmdqvOps *cmdqv_ops;
> +    void *cmdqv;  /* vendor specific CMDQV state */
>  } SMMUv3AccelState;
>  
>  typedef struct SMMUS1Hwpt {
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 312064a081..46aa9e8a9f 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -15,6 +15,24 @@
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>  
> +/*
> + * Tegra241 CMDQV MMIO layout (64KB pages)
> + *
> + * 0x00000  TEGRA241_CMDQV_CFG    (Global CMDQV configuration)
> + * 0x10000  TEGRA241_VCMDQ_PAGE0  (Virtual CMDQ page 0)
> + * 0x20000  TEGRA241_VCMDQ_PAGE1  (Virtual CMDQ page 1)
> + * 0x30000  TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0)
> + * 0x40000  TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1)
> + */
> +#define TEGRA241_CMDQV_IO_LEN 0x50000
> +
> +typedef struct Tegra241CMDQV {
> +    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
> +    SMMUv3AccelState *s_accel;
> +    MemoryRegion mmio_cmdqv;
> +    qemu_irq irq;
> +} Tegra241CMDQV;
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index a270fa7ce4..6959766129 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -13,6 +13,16 @@
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
> +{
> +    return 0;
> +}
> +
> +static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
> +                                 unsigned size)
> +{
> +}
> +
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
>  {
>  }
> @@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
>  {
>  }
>  
> +static const MemoryRegionOps mmio_cmdqv_ops = {
> +    .read = tegra241_cmdqv_read,
> +    .write = tegra241_cmdqv_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +};
> +
>  static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
>  {
> -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> -    return false;
> +    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
> +    SMMUv3AccelState *accel = s->s_accel;
> +    Tegra241CMDQV *cmdqv;
> +
> +    cmdqv = g_new0(Tegra241CMDQV, 1);
> +    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
> +                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
> +    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
> +    sysbus_init_irq(sbd, &cmdqv->irq);
> +    cmdqv->s_accel = accel;
> +    accel->cmdqv = cmdqv;
> +    return true;
>  }
>  
>  static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

RE: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

[Shameer Kolothum Thodi]

Hi Eric,

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 09:18
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241
> CMDQV ops backend stub
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Shameer,
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3
> > accelerated CMDQV ops interface.
> >
> > This patch wires up the Tegra241 CMDQV backend and provides a stub
> > implementation for CMDQV probe, initialization, vIOMMU allocation and
> > reset handling.
> >
> > Functional CMDQV support is added in follow-up patches.
> >
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
> >  hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
> >  hw/arm/tegra241-cmdqv.c       | 56
> +++++++++++++++++++++++++++++++++++
> >  hw/arm/Kconfig                |  5 ++++
> >  hw/arm/meson.build            |  2 ++
> >  5 files changed, 96 insertions(+)
> >  create mode 100644 hw/arm/tegra241-cmdqv.h  create mode 100644
> > hw/arm/tegra241-cmdqv-stubs.c  create mode 100644
> > hw/arm/tegra241-cmdqv.c
> >
> > diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h new
> > file mode 100644 index 0000000000..07e10e86ee
> > --- /dev/null
> > +++ b/hw/arm/tegra241-cmdqv.h
> > @@ -0,0 +1,15 @@
> > +/*
> > + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
> > +rights reserved
> > + * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
> > + *
> > + * Written by Nicolin Chen, Shameer Kolothum
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later  */
> > +
> > +#ifndef HW_ARM_TEGRA241_CMDQV_H
> > +#define HW_ARM_TEGRA241_CMDQV_H
> > +
> > +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> > +
> > +#endif /* HW_ARM_TEGRA241_CMDQV_H */
> > diff --git a/hw/arm/tegra241-cmdqv-stubs.c
> > b/hw/arm/tegra241-cmdqv-stubs.c new file mode 100644 index
> > 0000000000..eedc7bfdcd
> > --- /dev/null
> > +++ b/hw/arm/tegra241-cmdqv-stubs.c
> > @@ -0,0 +1,18 @@
> > +/*
> > + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
> > +rights reserved
> > + *
> > + * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
> ^typo
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later */
> > +
> > +#include "qemu/osdep.h"
> > +#include "hw/arm/smmuv3.h"
> > +#include "smmuv3-accel.h"
> > +#include "hw/arm/tegra241-cmdqv.h"
> > +
> > +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
> > +    return NULL;
> > +}
> > +
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c new
> > file mode 100644 index 0000000000..ad5a0d4611
> > --- /dev/null
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -0,0 +1,56 @@
> > +/*
> > + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
> > +rights reserved
> > + * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
> > + *
> > + * Written by Nicolin Chen, Shameer Kolothum
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later  */
> > +
> > +#include "qemu/osdep.h"
> > +
> > +#include "hw/arm/smmuv3.h"
> > +#include "smmuv3-accel.h"
> > +#include "tegra241-cmdqv.h"
> > +
> > +static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { }
> > +
> > +static bool
> > +tegra241_cmdqv_alloc_viommu(SMMUv3State *s,
> HostIOMMUDeviceIOMMUFD *idev,
> > +                            uint32_t *out_viommu_id, Error **errp) {
> > +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > +    return false;
> > +}
> > +
> > +static void tegra241_cmdqv_reset(SMMUv3State *s) { }
> > +
> > +static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) {
> > +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > +    return false;
> > +}
> > +
> > +static bool tegra241_cmdqv_probe(SMMUv3State *s,
> HostIOMMUDeviceIOMMUFD *idev,
> > +                                 Error **errp) {
> > +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > +    return false;
> > +}
> > +
> > +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
> > +    .probe = tegra241_cmdqv_probe,
> > +    .init = tegra241_cmdqv_init,
> > +    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
> > +    .free_viommu = tegra241_cmdqv_free_viommu,
> > +    .reset = tegra241_cmdqv_reset,
> 
> I think I would create a no_cmdqv_ops to convert the existing code without
> any cmdqv support to the same api. This would avoid the repeating:
> 
> if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.

I guess you are suggesting having something like,

static const SMMUv3AccelCmdqvOps no_cmdqv_ops = {
    .probe = no_cmdqv_probe,
    .alloc_viommu = no_cmdqv_alloc_viommu,
    ...
};

static bool no_cmdqv_probe(SMMUv3State *s,
                           HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
{
    return true;
}

static bool no_cmdqv_alloc_viommu(SMMUv3State *s,
                                  HostIOMMUDeviceIOMMUFD *idev,
                                  uint32_t *id,
                                  Error **errp)
{
   return false; /* We have to call the default vIOMMU alloc*/
}

And then call them directly. I will take a look at that approach.

> 
> Also this looks a bit strange to have different signatures for the functions used
> with both vcmdq on and off

Is the concern here is mainly around alloc_viommu( that is the one which require fallback
to default vIOMMU alloc)?

static bool
smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                          Error **errp)

and 

bool (*alloc_viommu)(SMMUv3State *s,
                         HostIOMMUDeviceIOMMUFD *idev,
                         uint32_t *out_viommu_id,
                         Error **errp);

Making those consistent would likely require some refactoring of the
existing path. I will have a play with that and see.

Thanks,
Shameer

> Eric
> 
> 
> > +};
> > +
> > +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
> > +    return &tegra241_cmdqv_ops;
> > +}
> > diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index
> > c66c452737..3305c6e76e 100644
> > --- a/hw/arm/Kconfig
> > +++ b/hw/arm/Kconfig
> > @@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
> >      depends on TCG
> >      select FSL_IMX8MP
> >
> > +config TEGRA241_CMDQV
> > +    bool
> > +    depends on ARM_SMMUV3_ACCEL
> > +
> >  config ARM_SMMUV3_ACCEL
> >      bool
> >      depends on ARM_SMMUV3
> > @@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL  config
> ARM_SMMUV3
> >      bool
> >      select ARM_SMMUV3_ACCEL if IOMMUFD
> > +    imply TEGRA241_CMDQV
> >
> >  config FSL_IMX6UL
> >      bool
> > diff --git a/hw/arm/meson.build b/hw/arm/meson.build
> > index 8f834c32b1..fc83b635e7 100644
> > --- a/hw/arm/meson.build
> > +++ b/hw/arm/meson.build
> > @@ -88,6 +88,8 @@ arm_common_ss.add(when:
> 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
> >  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:
> files('smmuv3.c'))
> >  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:
> files('smmuv3-accel.c'))
> >  stub_ss.add(files('smmuv3-accel-stubs.c'))
> > +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:
> files('tegra241-cmdqv.c'))
> > +stub_ss.add(files('tegra241-cmdqv-stubs.c'))
> >  arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-
> imx6ul.c', 'mcimx6ul-evk.c'))
> >  arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:
> files('nrf51_soc.c'))
> >  arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(

RE: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 10:31
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host
> Tegra241 CMDQV support
> 
> External email: Use caution opening links or attachments
> 
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241
> CMDQV.
> >
> > Validate the returned data type, version, number of vCMDQs and SIDs
> > per VM. Fail the probe if the host does not meet the expected requirements.
> 
> Can you explain why it is important to check
> 
> number of vCMDQs and SIDs per VM?

Right. I will add that.

The QEMU Tegra241 CMDQV implementation supports only two VCMDQs and 
16 SIDs per VM. The check ensures the host implementation is compatible with
that.

Thanks,
Shameer
> 
> >
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.h |  5 +++++
> >  hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
> >  2 files changed, 35 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h index
> > 07e10e86ee..312064a081 100644
> > --- a/hw/arm/tegra241-cmdqv.h
> > +++ b/hw/arm/tegra241-cmdqv.h
> > @@ -10,6 +10,11 @@
> >  #ifndef HW_ARM_TEGRA241_CMDQV_H
> >  #define HW_ARM_TEGRA241_CMDQV_H
> >
> > +#define TEGRA241_CMDQV_VERSION             1
> > +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
> > +#define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> > +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
> > +
> >  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> >
> >  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c index
> > ad5a0d4611..a270fa7ce4 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s,
> > Error **errp)  static bool tegra241_cmdqv_probe(SMMUv3State *s,
> HostIOMMUDeviceIOMMUFD *idev,
> >                                   Error **errp)  {
> > -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > -    return false;
> > +    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
> > +    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
> > +    uint64_t caps;
> > +
> > +    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid,
> &data_type,
> > +                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
> > +                                         NULL, errp)) {
> > +        return false;
> > +    }
> > +    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
> > +        error_setg(errp, "Host CMDQV: unexpected data type %u (expected
> %u)",
> > +                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
> > +        return false;
> > +    }
> > +    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
> > +        error_setg(errp, "Host CMDQV: unsupported version %u (expected
> %u)",
> > +                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
> > +        return false;
> > +    }
> > +    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
> > +        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >=
> %u)",
> > +                   cmdqv_info.log2vcmdqs,
> TEGRA241_CMDQV_NUM_CMDQ_LOG2);
> > +        return false;
> > +    }
> > +    if (cmdqv_info.log2vsids <
> TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {
> > +        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >=
> %u)",
> > +                   cmdqv_info.log2vsids,
> TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
> > +        return false;
> > +    }
> > +    return true;
> >  }
> >
> >  static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
> Besides
> Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric

Re: [PATCH v3 12/32] hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> SMMUv3 devices with acceleration may enable CMDQV extensions
> after device realize. In that case, additional MMIO regions and
> IRQ lines may be registered but not yet mapped to the platform bus.
>
> Ensure SMMUv3 device resources are linked to the platform bus
> during machine_done().
>
> This is safe to do unconditionally since the platform bus helpers
> skip resources that are already mapped.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/virt.c | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 292e523664..c75a8d6e9e 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms)
>      }
>  }
>  
> +/*
> + * SMMUv3 devices with acceleration may enable CMDQV extensions
> + * after device realize. In that case, additional MMIO regions and
> + * IRQ lines may be registered but not yet mapped to the platform bus.
> + *
> + * Ensure all resources are linked to the platform bus before final
> + * machine setup.
> + */
> +
> +static void virt_smmuv3_dev_link_cmdqv(VirtMachineState *vms)
> +{
> +    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
> +        DeviceState *dev = g_ptr_array_index(vms->smmuv3_devices, i);
need extra line
> +        platform_bus_link_device(PLATFORM_BUS_DEVICE(vms->platform_bus_dev),
> +                                 SYS_BUS_DEVICE(dev));
> +    }
> +}
> +
>  static
>  void virt_machine_done(Notifier *notifier, void *data)
>  {
> @@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)
>      if (vms->cxl_devices_state.is_enabled) {
>          cxl_fmws_link_targets(&error_fatal);
>      }
> +
> +    virt_smmuv3_dev_link_cmdqv(vms);
> +
>      /*
>       * If the user provided a dtb, we assume the dynamic sysbus nodes
>       * already are integrated there. This corresponds to a use case where
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

Re: [PATCH v3 07/32] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Add support for selecting and initializing a CMDQV backend based on
> the cmdqv OnOffAuto property.
>
> If set to OFF, CMDQV is not used and the default IOMMUFD-backed
> allocation path is taken.
>
> If set to AUTO, QEMU attempts to probe a CMDQV backend during
> device setup. If probing succeeds, the selected ops are stored
> in the accelerated SMMUv3 state and used. If probing fails,
> QEMU silently falls back to the default path.
>
> If set to ON, QEMU requires CMDQV support. Probing is performed
> during setup and failure results in an error.
>
> When a CMDQV backend is active, its callbacks are used for vIOMMU
> allocation, free, and reset handling. Otherwise, the base
> implementation is used.
>
> The current implementation wires up the Tegra241 CMDQV backend
> through the generic ops interface. Functional CMDQV behaviour is
> added in subsequent patches.
>
> No functional change.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  include/hw/arm/smmuv3.h |  2 +
>  hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
>  2 files changed, 88 insertions(+), 7 deletions(-)
>
> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
> index 26b2fc42fd..648412cafc 100644
> --- a/include/hw/arm/smmuv3.h
> +++ b/include/hw/arm/smmuv3.h
> @@ -73,6 +73,8 @@ struct SMMUv3State {
>      bool ats;
>      uint8_t oas;
>      uint8_t ssidsize;
> +    /* SMMU CMDQV extension */
> +    OnOffAuto cmdqv;
>  };
>  
>  typedef enum {
> diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
> index ab1b0a3669..4373bbd97b 100644
> --- a/hw/arm/smmuv3-accel.c
> +++ b/hw/arm/smmuv3-accel.c
> @@ -18,6 +18,7 @@
>  
>  #include "smmuv3-internal.h"
>  #include "smmuv3-accel.h"
> +#include "tegra241-cmdqv.h"
>  
>  /*
>   * The root region aliases the global system memory, and shared_as_sysmem
> @@ -522,6 +523,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>                            Error **errp)
>  {
>      SMMUv3AccelState *accel = s->s_accel;
> +    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
>      struct iommu_hwpt_arm_smmuv3 bypass_data = {
>          .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
>      };
> @@ -532,10 +534,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>      uint32_t viommu_id, hwpt_id;
>      IOMMUFDViommu *viommu;
>  
> -    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> -                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
> -                                      NULL, 0, &viommu_id, errp)) {
> -        return false;
> +    if (cmdqv_ops) {
> +        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
> +            return false;
> +        }
> +    } else {
> +        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> +                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
> +                                          s2_hwpt_id, NULL, 0, &viommu_id,
> +                                          errp)) {
> +            return false;
> +        }
>      }
>  
>      viommu = g_new0(IOMMUFDViommu, 1);
> @@ -581,12 +590,69 @@ free_bypass_hwpt:
>  free_abort_hwpt:
>      iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
>  free_viommu:
> -    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
> +    if (cmdqv_ops && cmdqv_ops->free_viommu) {
> +        cmdqv_ops->free_viommu(s);
> +    } else {
> +        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
> +    }
reading [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU
alloc/free

tegra241_cmdqv_free_viommu() looks identifical in both implementations

So maybe this is not needed

Eric

>      g_free(viommu);
>      accel->viommu = NULL;
>      return false;
>  }
>  
> +static const SMMUv3AccelCmdqvOps *
> +smmuv3_accel_probe_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                          Error **errp)
> +{
> +    const SMMUv3AccelCmdqvOps *ops = tegra241_cmdqv_get_ops();
> +
> +    if (!ops || !ops->probe) {
> +        error_setg(errp, "No CMDQV ops found");
> +        return NULL;
> +    }
> +
> +    if (!ops->probe(s, idev, errp)) {
> +        return NULL;
> +    }
> +    return ops;
> +}
> +
> +static bool
> +smmuv3_accel_select_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                          Error **errp)
> +{
> +    const SMMUv3AccelCmdqvOps *ops = NULL;
> +
> +    if (s->s_accel->cmdqv_ops) {
> +        return true;
> +    }
> +
> +    switch (s->cmdqv) {
> +    case ON_OFF_AUTO_OFF:
> +        s->s_accel->cmdqv_ops = NULL;
> +        return true;
> +    case ON_OFF_AUTO_AUTO:
> +        ops = smmuv3_accel_probe_cmdqv(s, idev, NULL);
> +        break;
> +    case ON_OFF_AUTO_ON:
> +        ops = smmuv3_accel_probe_cmdqv(s, idev, errp);
> +        if (!ops) {
> +            error_append_hint(errp, "CMDQV requested but not supported");
> +            return false;
> +        }
> +        s->s_accel->cmdqv_ops = ops;
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> +
> +    if (ops && ops->init && !ops->init(s, errp)) {
> +        return false;
> +    }
> +    s->s_accel->cmdqv_ops = ops;
> +    return true;
> +}
> +
>  static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
>                                            HostIOMMUDevice *hiod, Error **errp)
>  {
> @@ -621,6 +687,10 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
>          goto done;
>      }
>  
> +    if (!smmuv3_accel_select_cmdqv(s, idev, errp)) {
> +        return false;
> +    }
> +
>      if (!smmuv3_accel_alloc_viommu(s, idev, errp)) {
>          error_append_hint(errp, "Unable to alloc vIOMMU: idev devid 0x%x: ",
>                            idev->devid);
> @@ -867,8 +937,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
>  
>  void smmuv3_accel_reset(SMMUv3State *s)
>  {
> -     /* Attach a HWPT based on GBPA reset value */
> -     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
> +    SMMUv3AccelState *accel = s->s_accel;
> +
> +    if (!accel) {
> +        return;
> +    }
> +    /* Attach a HWPT based on GBPA reset value */
> +    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
> +
> +    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
> +        accel->cmdqv_ops->reset(s);
> +    }
>  }
>  
>  static void smmuv3_accel_as_init(SMMUv3State *s)

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Replace the stub implementation with real vIOMMU allocation for
> Tegra241 CMDQV.
>
> Free the vIOMMU ID on teardown.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index 6959766129..d487612ba2 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
>  {
> +    SMMUv3AccelState *accel = s->s_accel;
> +    IOMMUFDViommu *viommu = accel->viommu;
> +
> +    if (!viommu) {
> +        return;
> +    }
> +    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
>  }
>  
>  static bool
>  tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>                              uint32_t *out_viommu_id, Error **errp)
>  {
> -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> -    return false;
> +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
> +
> +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> +                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
if the only think that differs compared to no cmdq is the type, ie.

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv that is being used in the smmu_accel device. Then you can get rid of alloc and free

Eric


> +                                      idev->hwpt_id, &cmdqv->cmdqv_data,
> +                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
> +                                      errp)) {
> +        return false;
> +    }
> +    return true;
>  }
>  
>  static void tegra241_cmdqv_reset(SMMUv3State *s)

RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 11:09
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV
> vIOMMU alloc/free
> 
> External email: Use caution opening links or attachments
> 
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > From: Nicolin Chen <nicolinc@nvidia.com>
> >
> > Replace the stub implementation with real vIOMMU allocation for
> > Tegra241 CMDQV.
> >
> > Free the vIOMMU ID on teardown.
> >
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
> >  1 file changed, 17 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c index
> > 6959766129..d487612ba2 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque,
> > hwaddr offset, uint64_t value,
> >
> >  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)  {
> > +    SMMUv3AccelState *accel = s->s_accel;
> > +    IOMMUFDViommu *viommu = accel->viommu;
> > +
> > +    if (!viommu) {
> > +        return;
> > +    }
> > +    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
> >  }
> >
> >  static bool
> >  tegra241_cmdqv_alloc_viommu(SMMUv3State *s,
> HostIOMMUDeviceIOMMUFD *idev,
> >                              uint32_t *out_viommu_id, Error **errp)  {
> > -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > -    return false;
> > +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
> > +
> > +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> > +
> > + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
> if the only think that differs compared to no cmdq is the type, ie.
> 
> IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
> IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
> 
> + passing true args, maybe you can just record the type of the cmdqv
> + that is being used in the smmu_accel device. Then you can get rid of
> + alloc and free

It is not just the type. Based on the type we also need to pass,

 * @data_len: Length of the type specific data
 * @__reserved: Must be 0
 * @data_uptr: User pointer to a driver-specific virtual IOMMU data
 *

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different 
implementations in future, it probably makes sense to keep this.

Thanks,
Shameer

> 
> Eric
> 
> 
> > +                                      idev->hwpt_id, &cmdqv->cmdqv_data,
> > +                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
> > +                                      errp)) {
> > +        return false;
> > +    }
> > +    return true;
> >  }
> >
> >  static void tegra241_cmdqv_reset(SMMUv3State *s)

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Eric Auger]

On 3/9/26 12:31 PM, Shameer Kolothum Thodi wrote:
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: 09 March 2026 11:09
>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>> arm@nongnu.org; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>> <kjaju@nvidia.com>; phrdina@redhat.com
>> Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV
>> vIOMMU alloc/free
>>
>> External email: Use caution opening links or attachments
>>
>>
>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
>>> From: Nicolin Chen <nicolinc@nvidia.com>
>>>
>>> Replace the stub implementation with real vIOMMU allocation for
>>> Tegra241 CMDQV.
>>>
>>> Free the vIOMMU ID on teardown.
>>>
>>> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>> ---
>>>  hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
>>>  1 file changed, 17 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c index
>>> 6959766129..d487612ba2 100644
>>> --- a/hw/arm/tegra241-cmdqv.c
>>> +++ b/hw/arm/tegra241-cmdqv.c
>>> @@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque,
>>> hwaddr offset, uint64_t value,
>>>
>>>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)  {
>>> +    SMMUv3AccelState *accel = s->s_accel;
>>> +    IOMMUFDViommu *viommu = accel->viommu;
>>> +
>>> +    if (!viommu) {
>>> +        return;
>>> +    }
>>> +    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
>>>  }
>>>
>>>  static bool
>>>  tegra241_cmdqv_alloc_viommu(SMMUv3State *s,
>> HostIOMMUDeviceIOMMUFD *idev,
>>>                              uint32_t *out_viommu_id, Error **errp)  {
>>> -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
>>> -    return false;
>>> +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
>>> +
>>> +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
>>> +
>>> + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
>> if the only think that differs compared to no cmdq is the type, ie.
>>
>> IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
>> IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
>>
>> + passing true args, maybe you can just record the type of the cmdqv
>> + that is being used in the smmu_accel device. Then you can get rid of
>> + alloc and free
> It is not just the type. Based on the type we also need to pass,
>
>  * @data_len: Length of the type specific data
>  * @__reserved: Must be 0
>  * @data_uptr: User pointer to a driver-specific virtual IOMMU data
>  *
>
> And the above is implementation specific.
>
> If our idea of the "ops" is to allow easier support for different 
> implementations in future, it probably makes sense to keep this.

OK I missed the fact data_len and IOMMU data needed specialization.

Thanks

Eric
>
> Thanks,
> Shameer
>
>> Eric
>>
>>
>>> +                                      idev->hwpt_id, &cmdqv->cmdqv_data,
>>> +                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
>>> +                                      errp)) {
>>> +        return false;
>>> +    }
>>> +    return true;
>>>  }
>>>
>>>  static void tegra241_cmdqv_reset(SMMUv3State *s)

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

[Eric Auger]

Hi Shameer, 

On 3/9/26 11:48 AM, Shameer Kolothum Thodi wrote:
> Hi Eric,
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: 09 March 2026 09:18
>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>> arm@nongnu.org; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>> <kjaju@nvidia.com>; phrdina@redhat.com
>> Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241
>> CMDQV ops backend stub
>>
>> External email: Use caution opening links or attachments
>>
>>
>> Hi Shameer,
>>
>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
>>> Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3
>>> accelerated CMDQV ops interface.
>>>
>>> This patch wires up the Tegra241 CMDQV backend and provides a stub
>>> implementation for CMDQV probe, initialization, vIOMMU allocation and
>>> reset handling.
>>>
>>> Functional CMDQV support is added in follow-up patches.
>>>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>> ---
>>>  hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
>>>  hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
>>>  hw/arm/tegra241-cmdqv.c       | 56
>> +++++++++++++++++++++++++++++++++++
>>>  hw/arm/Kconfig                |  5 ++++
>>>  hw/arm/meson.build            |  2 ++
>>>  5 files changed, 96 insertions(+)
>>>  create mode 100644 hw/arm/tegra241-cmdqv.h  create mode 100644
>>> hw/arm/tegra241-cmdqv-stubs.c  create mode 100644
>>> hw/arm/tegra241-cmdqv.c
>>>
>>> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h new
>>> file mode 100644 index 0000000000..07e10e86ee
>>> --- /dev/null
>>> +++ b/hw/arm/tegra241-cmdqv.h
>>> @@ -0,0 +1,15 @@
>>> +/*
>>> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
>>> +rights reserved
>>> + * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
>>> + *
>>> + * Written by Nicolin Chen, Shameer Kolothum
>>> + *
>>> + * SPDX-License-Identifier: GPL-2.0-or-later  */
>>> +
>>> +#ifndef HW_ARM_TEGRA241_CMDQV_H
>>> +#define HW_ARM_TEGRA241_CMDQV_H
>>> +
>>> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>>> +
>>> +#endif /* HW_ARM_TEGRA241_CMDQV_H */
>>> diff --git a/hw/arm/tegra241-cmdqv-stubs.c
>>> b/hw/arm/tegra241-cmdqv-stubs.c new file mode 100644 index
>>> 0000000000..eedc7bfdcd
>>> --- /dev/null
>>> +++ b/hw/arm/tegra241-cmdqv-stubs.c
>>> @@ -0,0 +1,18 @@
>>> +/*
>>> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
>>> +rights reserved
>>> + *
>>> + * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
>> ^typo
>>> + *
>>> + * SPDX-License-Identifier: GPL-2.0-or-later */
>>> +
>>> +#include "qemu/osdep.h"
>>> +#include "hw/arm/smmuv3.h"
>>> +#include "smmuv3-accel.h"
>>> +#include "hw/arm/tegra241-cmdqv.h"
>>> +
>>> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
>>> +    return NULL;
>>> +}
>>> +
>>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c new
>>> file mode 100644 index 0000000000..ad5a0d4611
>>> --- /dev/null
>>> +++ b/hw/arm/tegra241-cmdqv.c
>>> @@ -0,0 +1,56 @@
>>> +/*
>>> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
>>> +rights reserved
>>> + * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
>>> + *
>>> + * Written by Nicolin Chen, Shameer Kolothum
>>> + *
>>> + * SPDX-License-Identifier: GPL-2.0-or-later  */
>>> +
>>> +#include "qemu/osdep.h"
>>> +
>>> +#include "hw/arm/smmuv3.h"
>>> +#include "smmuv3-accel.h"
>>> +#include "tegra241-cmdqv.h"
>>> +
>>> +static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { }
>>> +
>>> +static bool
>>> +tegra241_cmdqv_alloc_viommu(SMMUv3State *s,
>> HostIOMMUDeviceIOMMUFD *idev,
>>> +                            uint32_t *out_viommu_id, Error **errp) {
>>> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
>>> +    return false;
>>> +}
>>> +
>>> +static void tegra241_cmdqv_reset(SMMUv3State *s) { }
>>> +
>>> +static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) {
>>> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
>>> +    return false;
>>> +}
>>> +
>>> +static bool tegra241_cmdqv_probe(SMMUv3State *s,
>> HostIOMMUDeviceIOMMUFD *idev,
>>> +                                 Error **errp) {
>>> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
>>> +    return false;
>>> +}
>>> +
>>> +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
>>> +    .probe = tegra241_cmdqv_probe,
>>> +    .init = tegra241_cmdqv_init,
>>> +    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
>>> +    .free_viommu = tegra241_cmdqv_free_viommu,
>>> +    .reset = tegra241_cmdqv_reset,
>> I think I would create a no_cmdqv_ops to convert the existing code without
>> any cmdqv support to the same api. This would avoid the repeating:
>>
>> if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.
> I guess you are suggesting having something like,
>
> static const SMMUv3AccelCmdqvOps no_cmdqv_ops = {
>     .probe = no_cmdqv_probe,
>     .alloc_viommu = no_cmdqv_alloc_viommu,
>     ...
> };
>
> static bool no_cmdqv_probe(SMMUv3State *s,
>                            HostIOMMUDeviceIOMMUFD *idev,
>                            Error **errp)
> {
>     return true;
> }
>
> static bool no_cmdqv_alloc_viommu(SMMUv3State *s,
>                                   HostIOMMUDeviceIOMMUFD *idev,
>                                   uint32_t *id,
>                                   Error **errp)
> {
>    return false; /* We have to call the default vIOMMU alloc*/
> }
>
> And then call them directly. I will take a look at that approach.

Well forget it, I don't think it is worth the pain.
>
>> Also this looks a bit strange to have different signatures for the functions used
>> with both vcmdq on and off
> Is the concern here is mainly around alloc_viommu( that is the one which require fallback
> to default vIOMMU alloc)?
>
> static bool
> smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>                           Error **errp)
>
> and 
>
> bool (*alloc_viommu)(SMMUv3State *s,
>                          HostIOMMUDeviceIOMMUFD *idev,
>                          uint32_t *out_viommu_id,
>                          Error **errp);
>
> Making those consistent would likely require some refactoring of the
> existing path. I will have a play with that and see.

Same here. That's OK for now.

Thanks

Eric
>
> Thanks,
> Shameer
>
>> Eric
>>
>>
>>> +};
>>> +
>>> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
>>> +    return &tegra241_cmdqv_ops;
>>> +}
>>> diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index
>>> c66c452737..3305c6e76e 100644
>>> --- a/hw/arm/Kconfig
>>> +++ b/hw/arm/Kconfig
>>> @@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
>>>      depends on TCG
>>>      select FSL_IMX8MP
>>>
>>> +config TEGRA241_CMDQV
>>> +    bool
>>> +    depends on ARM_SMMUV3_ACCEL
>>> +
>>>  config ARM_SMMUV3_ACCEL
>>>      bool
>>>      depends on ARM_SMMUV3
>>> @@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL  config
>> ARM_SMMUV3
>>>      bool
>>>      select ARM_SMMUV3_ACCEL if IOMMUFD
>>> +    imply TEGRA241_CMDQV
>>>
>>>  config FSL_IMX6UL
>>>      bool
>>> diff --git a/hw/arm/meson.build b/hw/arm/meson.build
>>> index 8f834c32b1..fc83b635e7 100644
>>> --- a/hw/arm/meson.build
>>> +++ b/hw/arm/meson.build
>>> @@ -88,6 +88,8 @@ arm_common_ss.add(when:
>> 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
>>>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:
>> files('smmuv3.c'))
>>>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:
>> files('smmuv3-accel.c'))
>>>  stub_ss.add(files('smmuv3-accel-stubs.c'))
>>> +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:
>> files('tegra241-cmdqv.c'))
>>> +stub_ss.add(files('tegra241-cmdqv-stubs.c'))
>>>  arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-
>> imx6ul.c', 'mcimx6ul-evk.c'))
>>>  arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:
>> files('nrf51_soc.c'))
>>>  arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

[Eric Auger]

Hi Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated
> CMDQV ops interface.
>
> This patch wires up the Tegra241 CMDQV backend and provides a stub
> implementation for CMDQV probe, initialization, vIOMMU allocation
> and reset handling.
>
> Functional CMDQV support is added in follow-up patches.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
>  hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
>  hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
>  hw/arm/Kconfig                |  5 ++++
>  hw/arm/meson.build            |  2 ++
>  5 files changed, 96 insertions(+)
>  create mode 100644 hw/arm/tegra241-cmdqv.h
>  create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
>  create mode 100644 hw/arm/tegra241-cmdqv.c
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> new file mode 100644
> index 0000000000..07e10e86ee
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -0,0 +1,15 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
> + *
> + * Written by Nicolin Chen, Shameer Kolothum
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#ifndef HW_ARM_TEGRA241_CMDQV_H
> +#define HW_ARM_TEGRA241_CMDQV_H
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> +
> +#endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv-stubs.c b/hw/arm/tegra241-cmdqv-stubs.c
> new file mode 100644
> index 0000000000..eedc7bfdcd
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv-stubs.c
> @@ -0,0 +1,18 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + *
> + * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +#include "hw/arm/smmuv3.h"
> +#include "smmuv3-accel.h"
> +#include "hw/arm/tegra241-cmdqv.h"
not sure you need all those headers for the stub

Eric
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
> +{
> +    return NULL;
> +}
> +
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> new file mode 100644
> index 0000000000..ad5a0d4611
> --- /dev/null
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -0,0 +1,56 @@
> +/*
> + * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
> + *
> + * Written by Nicolin Chen, Shameer Kolothum
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "hw/arm/smmuv3.h"
> +#include "smmuv3-accel.h"
> +#include "tegra241-cmdqv.h"
> +
> +static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
> +{
> +}
> +
> +static bool
> +tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                            uint32_t *out_viommu_id, Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static void tegra241_cmdqv_reset(SMMUv3State *s)
> +{
> +}
> +
> +static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
> +                                 Error **errp)
> +{
> +    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> +    return false;
> +}
> +
> +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
> +    .probe = tegra241_cmdqv_probe,
> +    .init = tegra241_cmdqv_init,
> +    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
> +    .free_viommu = tegra241_cmdqv_free_viommu,
> +    .reset = tegra241_cmdqv_reset,
> +};
> +
> +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
> +{
> +    return &tegra241_cmdqv_ops;
> +}
> diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
> index c66c452737..3305c6e76e 100644
> --- a/hw/arm/Kconfig
> +++ b/hw/arm/Kconfig
> @@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
>      depends on TCG
>      select FSL_IMX8MP
>  
> +config TEGRA241_CMDQV
> +    bool
> +    depends on ARM_SMMUV3_ACCEL
> +
>  config ARM_SMMUV3_ACCEL
>      bool
>      depends on ARM_SMMUV3
> @@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
>  config ARM_SMMUV3
>      bool
>      select ARM_SMMUV3_ACCEL if IOMMUFD
> +    imply TEGRA241_CMDQV
>  
>  config FSL_IMX6UL
>      bool
> diff --git a/hw/arm/meson.build b/hw/arm/meson.build
> index 8f834c32b1..fc83b635e7 100644
> --- a/hw/arm/meson.build
> +++ b/hw/arm/meson.build
> @@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
>  arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
>  stub_ss.add(files('smmuv3-accel-stubs.c'))
> +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
> +stub_ss.add(files('tegra241-cmdqv-stubs.c'))
>  arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
>  arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
>  arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Eric Auger]

Hi Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV defines a set of global control and status registers
> used to configure virtual command queue allocation and interrupt
> behavior.
>
> Add read/write emulation for the global CMDQV register page
> (offset 0x00000), backed by a simple register cache. This includes
> CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
> map and the VINTF0 related registers defined in the global CMDQV
> register space.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 228 insertions(+), 1 deletion(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 46aa9e8a9f..50bcecee9d 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -10,6 +10,9 @@
>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>  #define HW_ARM_TEGRA241_CMDQV_H
>  
> +#include "hw/core/registerfields.h"
> +#include "smmuv3-accel.h"
> +
>  #define TEGRA241_CMDQV_VERSION             1
>  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +
> +    /* Register Cache */
> +    uint32_t config;
> +    uint32_t param;
> +    uint32_t status;
> +    uint32_t vi_err_map[2];
> +    uint32_t vi_int_mask[2];
> +    uint32_t cmdq_err_map[4];
> +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vintf_config;
> +    uint32_t vintf_status;
> +    uint32_t vintf_sid_match[16];
> +    uint32_t vintf_sid_replace[16];
> +    uint32_t vintf_cmdq_err_map[4];
>  } Tegra241CMDQV;
>  
> +/* Global CMDQV MMIO registers (offset 0x00000) */
> +REG32(CONFIG, 0x0)
> +FIELD(CONFIG, CMDQV_EN, 0, 1)
> +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
> +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
> +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
> +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
> +
> +REG32(PARAM, 0x4)
> +FIELD(PARAM, CMDQV_VER, 0, 4)
> +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
> +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
this is called CMDQV_NUM_VI_LOG2 in the spec I have access to
VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naming
> +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
same here s/VM/VI
> +
> +REG32(STATUS, 0x8)
> +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
> +
I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
> +#define A_VI_ERR_MAP 0x14
_0?
> +#define A_VI_ERR_MAP_1 0x18
> +#define V_VI_ERR_MAP_NO_ERROR (0)
> +#define V_VI_ERR_MAP_ERROR (1)
> +
same SMMU_CMDQV_VI_ING_MASK0/1
> +#define A_VI_INT_MASK 0x1c
> +#define A_VI_INT_MASK_1 0x20
> +#define V_VI_INT_MASK_NOT_MASKED (0)
> +#define V_VI_INT_MASK_MASKED (1)
> +
SMMU_CMDQV_CMDQ_ERR_MAP0-3
> +#define A_CMDQ_ERR_MAP 0x24
> +#define A_CMDQ_ERR_MAP_1 0x28
> +#define A_CMDQ_ERR_MAP_2 0x2c
> +#define A_CMDQ_ERR_MAP_3 0x30
> +
> +/* i = [0, 1] */

> +#define A_CMDQ_ALLOC_MAP_(i)   
why A_? Since you rely on the REG macros, A_ should prefix the address
offset
Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in
the spec?
>               \
> +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
> +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
> +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
> +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
> +
Please explain why we only expose 2 of those regs among 128?
> +A_CMDQ_ALLOC_MAP_(0)
> +A_CMDQ_ALLOC_MAP_(1)
> +
> +
> +/* i = [0, 0] */
> +#define A_VINTFi_CONFIG(i)     
again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0
>                   \
> +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
> +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
> +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
> +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
> +
> +A_VINTFi_CONFIG(0)
> +
> +#define A_VINTFi_STATUS(i)                       \
> +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
> +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
> +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
> +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
> +
> +A_VINTFi_STATUS(0)
> +
> +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
> +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
Nit: the spec also contains the typo but I would rather fix it in the code

Explicitly state we only expose 1 VINTF
> +
> +/* i = [0, 0], j = [0, 15] */
does that mean that we can have a max 16 SIDs per VM?
> +#define A_VINTFi_SID_MATCH_(i, j) 
if matched vi, I would prefer vi
>                               \
> +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
> +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
> +
> +A_VINTFi_SID_MATCH_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_MATCH_(0, 15)
> +
> +/* i = [0, 0], j = [0, 15] */
vint = 0, 16 identical register entries
> +#define A_VINTFi_SID_REPLACE_(i, j)                               \
> +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
s/19/20
> +
> +A_VINTFi_SID_REPLACE_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_REPLACE_(0, 15)
> +
> +/* i = [0, 0], j = [0, 3] */
vi = 0, 4 identical regs
> +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
> +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
> +
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> +/* Omitting [0][1~2] as not being directly called */
I don't get this comment
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index d487612ba2..a3830a02d6 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -8,19 +8,138 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/log.h"
>  
>  #include "hw/arm/smmuv3.h"
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        return cmdqv->vintf_config;
> +    case A_VINTF0_STATUS:
> +        return cmdqv->vintf_status;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        return cmdqv->vintf_sid_match[i];
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        return cmdqv->vintf_sid_replace[i];
> +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
> +        return cmdqv->vintf_cmdq_err_map[i];
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
> -    return 0;
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
If all those regs belong to the Global CMDQV registers page this shall
be within the first 64KB
> +        return 0;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        return cmdqv->config;
> +    case A_PARAM:
> +        return cmdqv->param;
> +    case A_STATUS:
> +        return cmdqv->status;
> +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
> +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
> +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
> +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
> +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> +                                       uint64_t value)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        /* Strip off HYP_OWN setting from guest kernel */
> +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
Can you explain why this needed?
> +
> +        cmdqv->vintf_config = value;
> +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        } else {
> +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        }
> +        break;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        cmdqv->vintf_sid_match[i] = value;
> +        break;
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        cmdqv->vintf_sid_replace[i] = value;
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return;
> +    }
>  }
nit: if you can put write_vintf just after its read fellow, I think it
would ease the comparison/review
>  
>  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        cmdqv->config = value;
> +        if (value & R_CONFIG_CMDQV_EN_MASK) {
> +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
> +        } else {
> +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
> +        }
> +        break;
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
> +        break;
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
> +        break;
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

At this stage of the reading it is unclear to me why we need to expose
all those regs to the guest. I would assume that a subset of them are
used by the hyp and not supposed to be accessed by the guest. For
instance the SID match/replace, ...

Please could you clairfy what is absolutely needed to expose to the guest?

Thanks

Eric

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV defines a set of global control and status registers
> used to configure virtual command queue allocation and interrupt
> behavior.
>
> Add read/write emulation for the global CMDQV register page
In the spec this is named CMDQ-V Config. This 64kB page contains CMDQ-V
Config + VI-n Configs

Eric
> (offset 0x00000), backed by a simple register cache. This includes
> CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
> map and the VINTF0 related registers defined in the global CMDQV
> register space.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 228 insertions(+), 1 deletion(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 46aa9e8a9f..50bcecee9d 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -10,6 +10,9 @@
>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>  #define HW_ARM_TEGRA241_CMDQV_H
>  
> +#include "hw/core/registerfields.h"
> +#include "smmuv3-accel.h"
> +
>  #define TEGRA241_CMDQV_VERSION             1
>  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +
> +    /* Register Cache */
> +    uint32_t config;
> +    uint32_t param;
> +    uint32_t status;
> +    uint32_t vi_err_map[2];
> +    uint32_t vi_int_mask[2];
> +    uint32_t cmdq_err_map[4];
> +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vintf_config;
> +    uint32_t vintf_status;
> +    uint32_t vintf_sid_match[16];
> +    uint32_t vintf_sid_replace[16];
> +    uint32_t vintf_cmdq_err_map[4];
>  } Tegra241CMDQV;
>  
> +/* Global CMDQV MMIO registers (offset 0x00000) */
> +REG32(CONFIG, 0x0)
> +FIELD(CONFIG, CMDQV_EN, 0, 1)
> +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
> +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
> +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
> +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
> +
> +REG32(PARAM, 0x4)
> +FIELD(PARAM, CMDQV_VER, 0, 4)
> +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
> +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
> +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
> +
> +REG32(STATUS, 0x8)
> +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
> +
> +#define A_VI_ERR_MAP 0x14
> +#define A_VI_ERR_MAP_1 0x18
> +#define V_VI_ERR_MAP_NO_ERROR (0)
> +#define V_VI_ERR_MAP_ERROR (1)
> +
> +#define A_VI_INT_MASK 0x1c
> +#define A_VI_INT_MASK_1 0x20
> +#define V_VI_INT_MASK_NOT_MASKED (0)
> +#define V_VI_INT_MASK_MASKED (1)
> +
> +#define A_CMDQ_ERR_MAP 0x24
> +#define A_CMDQ_ERR_MAP_1 0x28
> +#define A_CMDQ_ERR_MAP_2 0x2c
> +#define A_CMDQ_ERR_MAP_3 0x30
> +
> +/* i = [0, 1] */
> +#define A_CMDQ_ALLOC_MAP_(i)                 \
> +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
> +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
> +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
> +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
> +
> +A_CMDQ_ALLOC_MAP_(0)
> +A_CMDQ_ALLOC_MAP_(1)
> +
> +
> +/* i = [0, 0] */
> +#define A_VINTFi_CONFIG(i)                       \
> +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
> +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
> +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
> +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
> +
> +A_VINTFi_CONFIG(0)
> +
> +#define A_VINTFi_STATUS(i)                       \
> +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
> +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
> +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
> +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
> +
> +A_VINTFi_STATUS(0)
> +
> +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
> +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_MATCH_(i, j)                               \
> +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
> +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
> +
> +A_VINTFi_SID_MATCH_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_MATCH_(0, 15)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_REPLACE_(i, j)                               \
> +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
> +
> +A_VINTFi_SID_REPLACE_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_REPLACE_(0, 15)
> +
> +/* i = [0, 0], j = [0, 3] */
> +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
> +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
> +
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> +/* Omitting [0][1~2] as not being directly called */
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index d487612ba2..a3830a02d6 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -8,19 +8,138 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/log.h"
>  
>  #include "hw/arm/smmuv3.h"
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        return cmdqv->vintf_config;
> +    case A_VINTF0_STATUS:
> +        return cmdqv->vintf_status;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        return cmdqv->vintf_sid_match[i];
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        return cmdqv->vintf_sid_replace[i];
> +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
> +        return cmdqv->vintf_cmdq_err_map[i];
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
> -    return 0;
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return 0;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        return cmdqv->config;
> +    case A_PARAM:
> +        return cmdqv->param;
> +    case A_STATUS:
> +        return cmdqv->status;
> +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
> +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
> +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
> +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
> +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> +                                       uint64_t value)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        /* Strip off HYP_OWN setting from guest kernel */
> +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> +
> +        cmdqv->vintf_config = value;
> +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        } else {
> +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        }
> +        break;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        cmdqv->vintf_sid_match[i] = value;
> +        break;
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        cmdqv->vintf_sid_replace[i] = value;
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return;
> +    }
>  }
>  
>  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        cmdqv->config = value;
> +        if (value & R_CONFIG_CMDQV_EN_MASK) {
> +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
> +        } else {
> +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
> +        }
> +        break;
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
> +        break;
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
> +        break;
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

[Nicolin Chen]

On Thu, Feb 26, 2026 at 10:50:46AM +0000, Shameer Kolothum wrote:
> Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the
> corresponding teardown path to free the vEVENTQ during cleanup.
> 
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/smmuv3-accel.h   |  2 ++
>  hw/arm/tegra241-cmdqv.h |  1 +
>  hw/arm/smmuv3-accel.c   | 10 ++++++++-
>  hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 59 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
> index 7d6e4c6b76..4bff90e2c1 100644
> --- a/hw/arm/smmuv3-accel.h
> +++ b/hw/arm/smmuv3-accel.h
> @@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
>                           uint32_t *out_viommu_id,
>                           Error **errp);
>      void (*free_viommu)(SMMUv3State *s);
> +    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
> +    void (*free_veventq)(SMMUv3State *s);

As I replied in v2, this should really depend on viommu and should
be simply added to tegra241_cmdqv_alloc/free_viommu().

The alloc_viommu is an override of the standard viommu type, so it
might be useful to have an op. 

> diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
> index 4373bbd97b..f6602f51aa 100644
> --- a/hw/arm/smmuv3-accel.c
> +++ b/hw/arm/smmuv3-accel.c
> @@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>          goto free_bypass_hwpt;
>      }
>  
> +    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
> +        goto free_veventq;
> +    }
> +

But, this veventq op is completely in parallel to the standard one.
There is no need for the smmuv3-accel to initiate the allocation?

Thanks
Nicolin

RE: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: 09 March 2026 17:24
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> eric.auger@redhat.com; peter.maydell@linaro.org; clg@redhat.com;
> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ
> allocation and free
> 
> On Thu, Feb 26, 2026 at 10:50:46AM +0000, Shameer Kolothum wrote:
> > Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the
> > corresponding teardown path to free the vEVENTQ during cleanup.
> >
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/smmuv3-accel.h   |  2 ++
> >  hw/arm/tegra241-cmdqv.h |  1 +
> >  hw/arm/smmuv3-accel.c   | 10 ++++++++-
> >  hw/arm/tegra241-cmdqv.c | 47
> +++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 59 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
> > index 7d6e4c6b76..4bff90e2c1 100644
> > --- a/hw/arm/smmuv3-accel.h
> > +++ b/hw/arm/smmuv3-accel.h
> > @@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
> >                           uint32_t *out_viommu_id,
> >                           Error **errp);
> >      void (*free_viommu)(SMMUv3State *s);
> > +    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
> > +    void (*free_veventq)(SMMUv3State *s);
> 
> As I replied in v2, this should really depend on viommu and should
> be simply added to tegra241_cmdqv_alloc/free_viommu().

Yes, I remember that. That was actually my initial plan, which is why
this was not part of the original callbacks in the ops struct.

However, I then realised that we are storing the viommu pointer
in veventq,

veventq->viommu = accel->viommu;

And that struct viommu is only allocated after alloc_viommu() returns.

I didn't find an easy way to resolve that.  Need to take another look.

> 
> The alloc_viommu is an override of the standard viommu type, so it
> might be useful to have an op.
> 
> > diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
> > index 4373bbd97b..f6602f51aa 100644
> > --- a/hw/arm/smmuv3-accel.c
> > +++ b/hw/arm/smmuv3-accel.c
> > @@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,
> HostIOMMUDeviceIOMMUFD *idev,
> >          goto free_bypass_hwpt;
> >      }
> >
> > +    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
> > +        goto free_veventq;
> > +    }
> > +
> 
> But, this veventq op is completely in parallel to the standard one.
> There is no need for the smmuv3-accel to initiate the allocation?

If we can resolve the above dependency, then yes it can be tucked
inside .

Thanks,
Shameer

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Eric Auger]

Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
>
>   -Global VCMDQ registers at 0x10000/0x20000
Spec mentions CMDQV_CMDQ_BASE and CMDV_VI_CMDQ_BASE offset. You could
refer to that here and in the code (See fig 3.1).
Also there are size macros that could be used too.

By the way why do you name them "Global". Is that a spec terminology?

What I am really missing overall is the guest pov. I think I understand
the VMM is supposed to create some VIs, assign some VCMDQs to those VI
(that the guest will be able to have direct mapping to), define which
VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to
pSID. But then? what does the guest sees and what does he needs to program.


>   -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
>
> The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> IOMMU_HW_QUEUE_ALLOC.
If the guest detects he can use a single VMCDQs (I guess he can
introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if he
shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs
(0x30000-40000). How does the guest know that he is supposed to use VI0? 

Thanks

Eric
>
> Add read side emulation for both global VCMDQ and VI_VCMDQ register
> ranges. MMIO accesses are decoded to extract the VCMDQ instance index
> and normalized to a VCMDQ0_* register offset, allowing a single helper
> to service all VCMDQ instances.
>
> VI_VCMDQ accesses are translated to their equivalent global VCMDQ
> offsets and reuse the same decoding path. All VCMDQ reads are currently
> served from cached register state.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
>  2 files changed, 255 insertions(+)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 50bcecee9d..d379b8860c 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
>      uint32_t vintf_sid_match[16];
>      uint32_t vintf_sid_replace[16];
>      uint32_t vintf_cmdq_err_map[4];
> +    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
>  } Tegra241CMDQV;
>  
>  /* Global CMDQV MMIO registers (offset 0x00000) */
> @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
>  /* Omitting [0][1~2] as not being directly called */
>  A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
>  
> +/*
> + * VCMDQ register windows.
> + *
> + * Page 0 @ 0x10000: VCMDQ control and status registers
> + * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
> + */
> +#define A_VCMDQi_CONS_INDX(i)                       \
> +    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> +    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> +
> +A_VCMDQi_CONS_INDX(0)
> +A_VCMDQi_CONS_INDX(1)
> +
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
> +
> +#define A_VCMDQi_PROD_INDX(i)                             \
> +    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
> +    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
> +
> +A_VCMDQi_PROD_INDX(0)
> +A_VCMDQi_PROD_INDX(1)
> +
> +#define A_VCMDQi_CONFIG(i)                             \
> +    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> +
> +A_VCMDQi_CONFIG(0)
> +A_VCMDQi_CONFIG(1)
> +
> +#define A_VCMDQi_STATUS(i)                             \
> +    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
> +    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> +
> +A_VCMDQi_STATUS(0)
> +A_VCMDQi_STATUS(1)
> +
> +#define A_VCMDQi_GERROR(i)                               \
> +    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
> +    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> +    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VCMDQi_GERROR(0)
> +A_VCMDQi_GERROR(1)
> +
> +#define A_VCMDQi_GERRORN(i)                               \
> +    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
> +    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> +    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VCMDQi_GERRORN(0)
> +A_VCMDQi_GERRORN(1)
> +
> +#define A_VCMDQi_BASE_L(i)                       \
> +    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
> +    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> +    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
> +
> +A_VCMDQi_BASE_L(0)
> +A_VCMDQi_BASE_L(1)
> +
> +#define A_VCMDQi_BASE_H(i)                             \
> +    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
> +    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
> +
> +A_VCMDQi_BASE_H(0)
> +A_VCMDQi_BASE_H(1)
> +
> +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> +
> +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> +
> +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> +
> +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> +A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> +
> +/*
> + * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
> + *
> + * Page 0 @ 0x30000: VI_VCMDQ control and status registers
> + * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
> + */
> +#define A_VI_VCMDQi_CONS_INDX(i)                       \
> +    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> +
> +A_VI_VCMDQi_CONS_INDX(0)
> +A_VI_VCMDQi_CONS_INDX(1)
> +
> +#define A_VI_VCMDQi_PROD_INDX(i)                             \
> +    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
> +
> +A_VI_VCMDQi_PROD_INDX(0)
> +A_VI_VCMDQi_PROD_INDX(1)
> +
> +#define A_VI_VCMDQi_CONFIG(i)                             \
> +    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> +
> +A_VI_VCMDQi_CONFIG(0)
> +A_VI_VCMDQi_CONFIG(1)
> +
> +#define A_VI_VCMDQi_STATUS(i)                             \
> +    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> +
> +A_VI_VCMDQi_STATUS(0)
> +A_VI_VCMDQi_STATUS(1)
> +
> +#define A_VI_VCMDQi_GERROR(i)                               \
> +    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
> +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> +    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VI_VCMDQi_GERROR(0)
> +A_VI_VCMDQi_GERROR(1)
> +
> +#define A_VI_VCMDQi_GERRORN(i)                               \
> +    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VI_VCMDQi_GERRORN(0)
> +A_VI_VCMDQi_GERRORN(1)
> +
> +#define A_VI_VCMDQi_BASE_L(i)                       \
> +    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> +    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
> +
> +A_VI_VCMDQi_BASE_L(0)
> +A_VI_VCMDQi_BASE_L(1)
> +
> +#define A_VI_VCMDQi_BASE_H(i)                             \
> +    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
> +
> +A_VI_VCMDQi_BASE_H(0)
> +A_VI_VCMDQi_BASE_H(1)
> +
> +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> +
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> +
> +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> +
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index a3830a02d6..d2e6938e44 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -14,6 +14,46 @@
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +/*
> + * Read a VCMDQ register using VCMDQ0_* offsets.
> + *
> + * The caller normalizes the MMIO offset such that @offset0 always refers
> + * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> + *
> + * All VCMDQ accesses return cached registers.
> + */
> +static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
> +                                          int index)
> +{
> +    switch (offset0) {
> +    case A_VCMDQ0_CONS_INDX:
> +        return cmdqv->vcmdq_cons_indx[index];
> +    case A_VCMDQ0_PROD_INDX:
> +        return cmdqv->vcmdq_prod_indx[index];
> +    case A_VCMDQ0_CONFIG:
> +        return cmdqv->vcmdq_config[index];
> +    case A_VCMDQ0_STATUS:
> +        return cmdqv->vcmdq_status[index];
> +    case A_VCMDQ0_GERROR:
> +        return cmdqv->vcmdq_gerror[index];
> +    case A_VCMDQ0_GERRORN:
> +        return cmdqv->vcmdq_gerrorn[index];
> +    case A_VCMDQ0_BASE_L:
> +        return cmdqv->vcmdq_base[index];
> +    case A_VCMDQ0_BASE_H:
> +        return cmdqv->vcmdq_base[index] >> 32;
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
> +        return cmdqv->vcmdq_cons_indx_base[index];
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
> +        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
> +    default:
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset0);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  {
>      int i;
> @@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>          qemu_log_mask(LOG_UNIMP,
> @@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>          return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>          return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> +        /*
> +         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
> +         * layout as the global VCMDQ registers, but are based at 0x30000
> +         * instead of 0x10000.
> +         *
> +         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
> +         * global VCMDQ offset, then fall through to reuse the common VCMDQ
> +         * decoding logic below.
> +         */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
> +        /*
> +         * Decode a per-VCMDQ register access.
> +         *
> +         * The hardware supports up to 128 identical VCMDQ instances; we
> +         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
> +         * occupies a 0x80-byte window starting at 0x10000.
> +         *
> +         * The MMIO offset is decoded to extract the VCMDQ index and normalized
> +         * to the corresponding VCMDQ0_* register by subtracting index * 0x80.
> +         *
> +         * A single helper then services all VCMDQs, with @index selecting the
> +         * instance.
> +         */
> +        index = (offset - 0x10000) / 0x80;
> +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
> +    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
> +        index = (offset - 0x20000) / 0x80;
> +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
>      default:
>          qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
>                        __func__, offset);

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> VINTF pages expose a logical view local to a given VINTF. Although real
Again I fail to understand the guest POV. If the goal is to let the
guest have a direct access to some vcmdqs why do we need to emulate the
CMDQ regs?

Eric
> hardware may support multiple VINTFs, the kernel currently exposes a
> single VINTF per VM.
>
> The kernel provides an mmap offset for the VINTF Page0 region during
> vIOMMU allocation. However, the logical-to-physical association between
> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> to that, the mapped Page0 does not back any real VCMDQ state.
>
> When VINTF is enabled, mmap the kernel provided Page0 region and
> unmap it when VINTF is disabled. This prepares the VINTF mapping
> in advance of subsequent patches that add VCMDQ allocation support.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h |  3 +++
>  hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 45 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index d379b8860c..3ce9f539ae 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -18,6 +18,8 @@
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>  
> +#define VINTF_PAGE_SIZE 0x10000
> +
>  /*
>   * Tegra241 CMDQV MMIO layout (64KB pages)
>   *
> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +    void *vintf_page0;
>  
>      /* Register Cache */
>      uint32_t config;
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index e1f1562c44..a3767a85a3 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +static bool
> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    if (!cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> +        return false;
> +    }
> +    cmdqv->vintf_page0 = NULL;
> +    return true;
> +}
> +
> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> +
> +    if (cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
> +                                     VINTF_PAGE_SIZE,
> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
> +                                     &cmdqv->vintf_page0, errp)) {
> +        return false;
> +    }
> +
> +    return true;
> +}
> +
>  /*
>   * Write a VCMDQ register using VCMDQ0_* offsets.
>   *
> @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>  }
>  
>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> -                                       uint64_t value)
> +                                       uint64_t value, Error **errp)
>  {
>      int i;
>  
> @@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>  
>          cmdqv->vintf_config = value;
>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>          } else {
> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>          }
>          break;
> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    Error *local_err = NULL;
>      int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
>          break;
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>          break;
>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>          /* Same decoding as read() case: See comments above */
> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
>                        __func__, offset);
>      }
> +
> +    if (local_err) {
> +        error_report_err(local_err);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV defines a set of global control and status registers
> used to configure virtual command queue allocation and interrupt
> behavior.
>
> Add read/write emulation for the global CMDQV register page
> (offset 0x00000), backed by a simple register cache. This includes
> CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
> map and the VINTF0 related registers defined in the global CMDQV
> register space.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 228 insertions(+), 1 deletion(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 46aa9e8a9f..50bcecee9d 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -10,6 +10,9 @@
>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>  #define HW_ARM_TEGRA241_CMDQV_H
>  
> +#include "hw/core/registerfields.h"
> +#include "smmuv3-accel.h"
> +
>  #define TEGRA241_CMDQV_VERSION             1
>  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +
> +    /* Register Cache */
> +    uint32_t config;
> +    uint32_t param;
> +    uint32_t status;
> +    uint32_t vi_err_map[2];
> +    uint32_t vi_int_mask[2];
> +    uint32_t cmdq_err_map[4];
> +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vintf_config;
> +    uint32_t vintf_status;
> +    uint32_t vintf_sid_match[16];
> +    uint32_t vintf_sid_replace[16];
> +    uint32_t vintf_cmdq_err_map[4];
>  } Tegra241CMDQV;
>  
> +/* Global CMDQV MMIO registers (offset 0x00000) */
> +REG32(CONFIG, 0x0)
> +FIELD(CONFIG, CMDQV_EN, 0, 1)
> +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
> +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
> +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
> +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
> +
> +REG32(PARAM, 0x4)
> +FIELD(PARAM, CMDQV_VER, 0, 4)
> +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
> +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
> +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
> +
> +REG32(STATUS, 0x8)
> +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
> +
> +#define A_VI_ERR_MAP 0x14
> +#define A_VI_ERR_MAP_1 0x18
> +#define V_VI_ERR_MAP_NO_ERROR (0)
> +#define V_VI_ERR_MAP_ERROR (1)
> +
> +#define A_VI_INT_MASK 0x1c
> +#define A_VI_INT_MASK_1 0x20
> +#define V_VI_INT_MASK_NOT_MASKED (0)
> +#define V_VI_INT_MASK_MASKED (1)
> +
> +#define A_CMDQ_ERR_MAP 0x24
> +#define A_CMDQ_ERR_MAP_1 0x28
> +#define A_CMDQ_ERR_MAP_2 0x2c
> +#define A_CMDQ_ERR_MAP_3 0x30
> +
> +/* i = [0, 1] */
> +#define A_CMDQ_ALLOC_MAP_(i)                 \
> +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
> +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
> +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
> +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
> +
> +A_CMDQ_ALLOC_MAP_(0)
> +A_CMDQ_ALLOC_MAP_(1)
> +
> +
> +/* i = [0, 0] */
> +#define A_VINTFi_CONFIG(i)                       \
> +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
> +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
> +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
> +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
> +
> +A_VINTFi_CONFIG(0)
> +
> +#define A_VINTFi_STATUS(i)                       \
> +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
> +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
> +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
> +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
> +
> +A_VINTFi_STATUS(0)
> +
> +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
> +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_MATCH_(i, j)                               \
> +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
> +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
> +
> +A_VINTFi_SID_MATCH_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_MATCH_(0, 15)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_REPLACE_(i, j)                               \
> +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
> +
> +A_VINTFi_SID_REPLACE_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_REPLACE_(0, 15)
> +
> +/* i = [0, 0], j = [0, 3] */
> +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
> +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
> +
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> +/* Omitting [0][1~2] as not being directly called */
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index d487612ba2..a3830a02d6 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -8,19 +8,138 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/log.h"
>  
>  #include "hw/arm/smmuv3.h"
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        return cmdqv->vintf_config;
> +    case A_VINTF0_STATUS:
> +        return cmdqv->vintf_status;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        return cmdqv->vintf_sid_match[i];
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        return cmdqv->vintf_sid_replace[i];
> +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
> +        return cmdqv->vintf_cmdq_err_map[i];
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
> -    return 0;
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return 0;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        return cmdqv->config;
> +    case A_PARAM:
> +        return cmdqv->param;
> +    case A_STATUS:
> +        return cmdqv->status;
> +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
> +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
> +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
> +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
> +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> +                                       uint64_t value)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        /* Strip off HYP_OWN setting from guest kernel */
> +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> +
> +        cmdqv->vintf_config = value;
> +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        } else {
> +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        }
> +        break;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        cmdqv->vintf_sid_match[i] = value;
> +        break;
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        cmdqv->vintf_sid_replace[i] = value;
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return;
> +    }
>  }
>  
>  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        cmdqv->config = value;
> +        if (value & R_CONFIG_CMDQV_EN_MASK) {
> +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
> +        } else {
> +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
> +        }
> +        break;
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
> +        break;
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
> +        break;
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
Please could you add trace points for mmio access as this is generally
done for SMMU access? See trace_smmuv3_read/write_mmio()

Eric

RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Shameer Kolothum Thodi]

Hi Eric,

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 17:44
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and
> VINTF VCMDQ register reads
> 
> External email: Use caution opening links or attachments
> 
> 
> Shameer,
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > From: Nicolin Chen <nicolinc@nvidia.com>
> >
> > Tegra241 CMDQV exposes per-VCMDQ register windows through two
> MMIO views:
> >
> >   -Global VCMDQ registers at 0x10000/0x20000
> Spec mentions CMDQV_CMDQ_BASE and CMDV_VI_CMDQ_BASE offset. You
> could
> refer to that here and in the code (See fig 3.1).
> Also there are size macros that could be used too.
> 
> By the way why do you name them "Global". Is that a spec terminology?
> 
> What I am really missing overall is the guest pov. I think I understand
> the VMM is supposed to create some VIs, assign some VCMDQs to those VI
> (that the guest will be able to have direct mapping to), define which
> VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to
> pSID. But then? what does the guest sees and what does he needs to
> program.
> 
> 
> >   -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
> >
> > The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> > and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> > IOMMU_HW_QUEUE_ALLOC.
> If the guest detects he can use a single VMCDQs (I guess he can
> introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if
> he
> shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs
> (0x30000-40000). How does the guest know that he is supposed to use VI0?

The CMDQ registers can be accessed via both the so called "Global VCMDQ registers 
Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.

The difference being(from Nicolin's explanation here[0]):

"
The global page0 is programmable at any time so long as CMDQV_EN
is enabled.

The logical page0 is programmable only when SW allocates and maps
global vcmdq(s) to a VINTF. "logical" also means "local" to that
VINTF.
".

IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ
registers Page 0". 

The question is whether QEMU should also support accesses via the
"Global VCMDQ registers Page 0" if some other guest implementation
chooses to use that path.

@Nicolin, please correct me if that assumption is not right.

Thanks,
Shameer
[0]  https://lore.kernel.org/qemu-devel/aXkTzRM695oTWFaD@Asurada-Nvidia/


> Thanks
> 
> Eric
> >
> > Add read side emulation for both global VCMDQ and VI_VCMDQ register
> > ranges. MMIO accesses are decoded to extract the VCMDQ instance index
> > and normalized to a VCMDQ0_* register offset, allowing a single helper
> > to service all VCMDQ instances.
> >
> > VI_VCMDQ accesses are translated to their equivalent global VCMDQ
> > offsets and reuse the same decoding path. All VCMDQ reads are currently
> > served from cached register state.
> >
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.h | 178
> ++++++++++++++++++++++++++++++++++++++++
> >  hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
> >  2 files changed, 255 insertions(+)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> > index 50bcecee9d..d379b8860c 100644
> > --- a/hw/arm/tegra241-cmdqv.h
> > +++ b/hw/arm/tegra241-cmdqv.h
> > @@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
> >      uint32_t vintf_sid_match[16];
> >      uint32_t vintf_sid_replace[16];
> >      uint32_t vintf_cmdq_err_map[4];
> > +    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
> >  } Tegra241CMDQV;
> >
> >  /* Global CMDQV MMIO registers (offset 0x00000) */
> > @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> >  /* Omitting [0][1~2] as not being directly called */
> >  A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> >
> > +/*
> > + * VCMDQ register windows.
> > + *
> > + * Page 0 @ 0x10000: VCMDQ control and status registers
> > + * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
> > + */
> > +#define A_VCMDQi_CONS_INDX(i)                       \
> > +    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
> > +    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> > +    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> > +
> > +A_VCMDQi_CONS_INDX(0)
> > +A_VCMDQi_CONS_INDX(1)
> > +
> > +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
> > +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
> > +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
> > +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
> > +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
> > +
> > +#define A_VCMDQi_PROD_INDX(i)                             \
> > +    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
> > +    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
> > +
> > +A_VCMDQi_PROD_INDX(0)
> > +A_VCMDQi_PROD_INDX(1)
> > +
> > +#define A_VCMDQi_CONFIG(i)                             \
> > +    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
> > +    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> > +
> > +A_VCMDQi_CONFIG(0)
> > +A_VCMDQi_CONFIG(1)
> > +
> > +#define A_VCMDQi_STATUS(i)                             \
> > +    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
> > +    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> > +
> > +A_VCMDQi_STATUS(0)
> > +A_VCMDQi_STATUS(1)
> > +
> > +#define A_VCMDQi_GERROR(i)                               \
> > +    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
> > +    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> > +    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> > +    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> > +
> > +A_VCMDQi_GERROR(0)
> > +A_VCMDQi_GERROR(1)
> > +
> > +#define A_VCMDQi_GERRORN(i)                               \
> > +    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
> > +    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> > +    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> > +    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> > +
> > +A_VCMDQi_GERRORN(0)
> > +A_VCMDQi_GERRORN(1)
> > +
> > +#define A_VCMDQi_BASE_L(i)                       \
> > +    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
> > +    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> > +    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
> > +
> > +A_VCMDQi_BASE_L(0)
> > +A_VCMDQi_BASE_L(1)
> > +
> > +#define A_VCMDQi_BASE_H(i)                             \
> > +    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
> > +    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
> > +
> > +A_VCMDQi_BASE_H(0)
> > +A_VCMDQi_BASE_H(1)
> > +
> > +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> > +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i *
> 0x80) \
> > +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> > +
> > +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> > +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> > +
> > +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> > +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i *
> 0x80) \
> > +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> > +
> > +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> > +A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> > +
> > +/*
> > + * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
> > + *
> > + * Page 0 @ 0x30000: VI_VCMDQ control and status registers
> > + * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
> > + */
> > +#define A_VI_VCMDQi_CONS_INDX(i)                       \
> > +    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> > +    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> > +
> > +A_VI_VCMDQi_CONS_INDX(0)
> > +A_VI_VCMDQi_CONS_INDX(1)
> > +
> > +#define A_VI_VCMDQi_PROD_INDX(i)                             \
> > +    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
> > +
> > +A_VI_VCMDQi_PROD_INDX(0)
> > +A_VI_VCMDQi_PROD_INDX(1)
> > +
> > +#define A_VI_VCMDQi_CONFIG(i)                             \
> > +    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> > +
> > +A_VI_VCMDQi_CONFIG(0)
> > +A_VI_VCMDQi_CONFIG(1)
> > +
> > +#define A_VI_VCMDQi_STATUS(i)                             \
> > +    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> > +
> > +A_VI_VCMDQi_STATUS(0)
> > +A_VI_VCMDQi_STATUS(1)
> > +
> > +#define A_VI_VCMDQi_GERROR(i)                               \
> > +    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
> > +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> > +    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> > +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> > +
> > +A_VI_VCMDQi_GERROR(0)
> > +A_VI_VCMDQi_GERROR(1)
> > +
> > +#define A_VI_VCMDQi_GERRORN(i)                               \
> > +    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
> > +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> > +    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> > +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> > +
> > +A_VI_VCMDQi_GERRORN(0)
> > +A_VI_VCMDQi_GERRORN(1)
> > +
> > +#define A_VI_VCMDQi_BASE_L(i)                       \
> > +    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> > +    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
> > +
> > +A_VI_VCMDQi_BASE_L(0)
> > +A_VI_VCMDQi_BASE_L(1)
> > +
> > +#define A_VI_VCMDQi_BASE_H(i)                             \
> > +    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
> > +
> > +A_VI_VCMDQi_BASE_H(0)
> > +A_VI_VCMDQi_BASE_H(1)
> > +
> > +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> > +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i
> * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> > +
> > +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> > +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> > +
> > +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> > +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i
> * 0x80) \
> > +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> > +
> > +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> > +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> > +
> >  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> >
> >  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> > index a3830a02d6..d2e6938e44 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -14,6 +14,46 @@
> >  #include "smmuv3-accel.h"
> >  #include "tegra241-cmdqv.h"
> >
> > +/*
> > + * Read a VCMDQ register using VCMDQ0_* offsets.
> > + *
> > + * The caller normalizes the MMIO offset such that @offset0 always refers
> > + * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> > + *
> > + * All VCMDQ accesses return cached registers.
> > + */
> > +static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,
> hwaddr offset0,
> > +                                          int index)
> > +{
> > +    switch (offset0) {
> > +    case A_VCMDQ0_CONS_INDX:
> > +        return cmdqv->vcmdq_cons_indx[index];
> > +    case A_VCMDQ0_PROD_INDX:
> > +        return cmdqv->vcmdq_prod_indx[index];
> > +    case A_VCMDQ0_CONFIG:
> > +        return cmdqv->vcmdq_config[index];
> > +    case A_VCMDQ0_STATUS:
> > +        return cmdqv->vcmdq_status[index];
> > +    case A_VCMDQ0_GERROR:
> > +        return cmdqv->vcmdq_gerror[index];
> > +    case A_VCMDQ0_GERRORN:
> > +        return cmdqv->vcmdq_gerrorn[index];
> > +    case A_VCMDQ0_BASE_L:
> > +        return cmdqv->vcmdq_base[index];
> > +    case A_VCMDQ0_BASE_H:
> > +        return cmdqv->vcmdq_base[index] >> 32;
> > +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
> > +        return cmdqv->vcmdq_cons_indx_base[index];
> > +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
> > +        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
> > +    default:
> > +        qemu_log_mask(LOG_UNIMP,
> > +                      "%s unhandled read access at 0x%" PRIx64 "\n",
> > +                      __func__, offset0);
> > +        return 0;
> > +    }
> > +}
> > +
> >  static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,
> hwaddr offset)
> >  {
> >      int i;
> > @@ -42,6 +82,7 @@ static uint64_t
> tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
> >  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,
> unsigned size)
> >  {
> >      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> > +    int index;
> >
> >      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> >          qemu_log_mask(LOG_UNIMP,
> > @@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void
> *opaque, hwaddr offset, unsigned size)
> >          return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /
> 4];
> >      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> >          return tegra241_cmdqv_read_vintf(cmdqv, offset);
> > +    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> > +        /*
> > +         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
> > +         * layout as the global VCMDQ registers, but are based at 0x30000
> > +         * instead of 0x10000.
> > +         *
> > +         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
> > +         * global VCMDQ offset, then fall through to reuse the common VCMDQ
> > +         * decoding logic below.
> > +         */
> > +        offset -= 0x20000;
> > +        QEMU_FALLTHROUGH;
> > +    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
> > +        /*
> > +         * Decode a per-VCMDQ register access.
> > +         *
> > +         * The hardware supports up to 128 identical VCMDQ instances; we
> > +         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each
> VCMDQ
> > +         * occupies a 0x80-byte window starting at 0x10000.
> > +         *
> > +         * The MMIO offset is decoded to extract the VCMDQ index and
> normalized
> > +         * to the corresponding VCMDQ0_* register by subtracting index *
> 0x80.
> > +         *
> > +         * A single helper then services all VCMDQs, with @index selecting the
> > +         * instance.
> > +         */
> > +        index = (offset - 0x10000) / 0x80;
> > +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,
> index);
> > +    case A_VI_VCMDQ0_BASE_L ...
> A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> > +        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
> > +        offset -= 0x20000;
> > +        QEMU_FALLTHROUGH;
> > +    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> > +        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
> > +        index = (offset - 0x20000) / 0x80;
> > +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,
> index);
> >      default:
> >          qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
> PRIx64 "\n",
> >                        __func__, offset);

Re: [PATCH v3 30/32] hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Introduce a SMMUv3AccelCmdqvType enum and a helper to query the
> CMDQV implementation type associated with an accelerated SMMUv3
> instance.
>
> A subsequent patch will use this helper when generating the
> Tegra241 CMDQV DSDT.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  hw/arm/smmuv3-accel.h       |  7 +++++++
>  hw/arm/smmuv3-accel-stubs.c |  5 +++++
>  hw/arm/smmuv3-accel.c       | 12 ++++++++++++
>  hw/arm/tegra241-cmdqv.c     |  6 ++++++
>  4 files changed, 30 insertions(+)
>
> diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
> index c349981e79..6d21788006 100644
> --- a/hw/arm/smmuv3-accel.h
> +++ b/hw/arm/smmuv3-accel.h
> @@ -15,6 +15,11 @@
>  #include <linux/iommufd.h>
>  #endif
>  
> +typedef enum SMMUv3AccelCmdqvType {
> +    SMMUV3_CMDQV_NONE = 0,
> +    SMMUV3_CMDQV_TEGRA241,
> +} SMMUv3AccelCmdqvType;
> +
>  /*
>   * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
>   * support multiple VCMDQs with virtualization capabilities.
> @@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {
>      void (*free_viommu)(SMMUv3State *s);
>      bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
>      void (*free_veventq)(SMMUv3State *s);
> +    SMMUv3AccelCmdqvType (*get_type)(void);
>      void (*reset)(SMMUv3State *s);
>  } SMMUv3AccelCmdqvOps;
>  
> @@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
>  bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
>                                        void *buf, size_t size, Error **errp);
>  void smmuv3_accel_reset(SMMUv3State *s);
> +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj);
>  
>  #endif /* HW_ARM_SMMUV3_ACCEL_H */
> diff --git a/hw/arm/smmuv3-accel-stubs.c b/hw/arm/smmuv3-accel-stubs.c
> index 1d5d3bb10c..5ca94d605f 100644
> --- a/hw/arm/smmuv3-accel-stubs.c
> +++ b/hw/arm/smmuv3-accel-stubs.c
> @@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
>  void smmuv3_accel_reset(SMMUv3State *s)
>  {
>  }
> +
> +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
> +{
> +    return SMMUV3_CMDQV_NONE;
> +}
> diff --git a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c
> index 9a570b8af9..585b460943 100644
> --- a/hw/arm/smmuv3-accel.c
> +++ b/hw/arm/smmuv3-accel.c
> @@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
>      address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
>  }
>  
> +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
> +{
> +    SMMUv3State *s = ARM_SMMUV3(obj);
> +    SMMUv3AccelState *accel = s->s_accel;
> +
> +    if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) {
> +        return SMMUV3_CMDQV_NONE;
> +    }
> +
> +    return accel->cmdqv_ops->get_type();
> +}
> +
>  bool smmuv3_accel_init(SMMUv3State *s, Error **errp)
>  {
>      SMMUState *bs = ARM_SMMU(s);
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index a379341c0a..42d7dbfde7 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
>      return true;
>  }
>  
> +static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type(void)
> +{
> +    return SMMUV3_CMDQV_TEGRA241;
> +};
> +
>  static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
>                                   Error **errp)
>  {
> @@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
>      .free_viommu = tegra241_cmdqv_free_viommu,
>      .alloc_veventq = tegra241_cmdqv_alloc_veventq,
>      .free_veventq = tegra241_cmdqv_free_veventq,
> +    .get_type = tegra241_cmdqv_get_type,
>      .reset = tegra241_cmdqv_reset,
>  };
>  

Re: [PATCH v3 27/32] tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> The IORT SMMUv3 node Identifier field will be derived from a new
> per-device "identifier" property instead of relying on enumeration
> order.
>
> Add the affected IORT blobs to allowed-diff list for bios-table tests.
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> index dfb8523c8b..df5fe417c0 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1 +1,5 @@
>  /* List of comma-separated changed AML files to ignore */
> +"tests/data/acpi/aarch64/virt/IORT.its_off",
> +"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
> +"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
> +"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Nicolin Chen]

On Mon, Mar 09, 2026 at 04:31:22AM -0700, Shameer Kolothum Thodi wrote:
> > > -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > > -    return false;
> > > +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
> > > +
> > > +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> > > +
> > > + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
> > if the only think that differs compared to no cmdq is the type, ie.
> > 
> > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
> > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
> > 
> > + passing true args, maybe you can just record the type of the cmdqv
> > + that is being used in the smmu_accel device. Then you can get rid of
> > + alloc and free
> 
> It is not just the type. Based on the type we also need to pass,
> 
>  * @data_len: Length of the type specific data
>  * @__reserved: Must be 0
>  * @data_uptr: User pointer to a driver-specific virtual IOMMU data
>  *
> 
> And the above is implementation specific.
> 
> If our idea of the "ops" is to allow easier support for different 
> implementations in future, it probably makes sense to keep this.

Any future "cmdqv" will likely have a different viommu type. So,
essentially this is just a viommu type override, as Eric pointed
out above.

What we actually need to expose per viommu is type/data_len. The
data_uptr can be a union viommu_data in the accel structure?

Nicolin 

Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

[Eric Auger]

Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Add an "identifier" property to the SMMUv3 device and use it when
> building the ACPI IORT SMMUv3 node Identifier field.
>
> This avoids relying on device enumeration order and provides a stable
> per-device identifier. A subsequent patch will use the same identifier
> when generating the DSDT description for Tegra241 CMDQV, ensuring that
> the IORT and DSDT entries refer to the same SMMUv3 instance.
>
> No functional change intended.
besides changes the IORT table
>
> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  include/hw/arm/smmuv3.h  | 1 +
>  hw/arm/smmuv3.c          | 2 ++
>  hw/arm/virt-acpi-build.c | 4 +++-
>  hw/arm/virt.c            | 3 +++
>  4 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
> index 648412cafc..73b8f39aaa 100644
> --- a/include/hw/arm/smmuv3.h
> +++ b/include/hw/arm/smmuv3.h
> @@ -63,6 +63,7 @@ struct SMMUv3State {
>      qemu_irq     irq[4];
>      QemuMutex mutex;
>      char *stage;
> +    uint8_t identifier;
>  
>      /* SMMU has HW accelerator support for nested S1 + s2 */
>      bool accel;
> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
> index 468135bb24..c1f84bedd4 100644
> --- a/hw/arm/smmuv3.c
> +++ b/hw/arm/smmuv3.c
> @@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
>       * Defaults to stage 1
>       */
>      DEFINE_PROP_STRING("stage", SMMUv3State, stage),
> +    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
> +    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
>      DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
>      /* GPA of MSI doorbell, for SMMUv3 accel use. */
>      DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index ae78e9b9e0..20605185c5 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
>  typedef struct AcpiIortSMMUv3Dev {
>      int irq;
>      hwaddr base;
> +    uint8_t id;
>      GArray *rc_smmu_idmaps;
>      /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
>      size_t offset;
> @@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
>                                                 &error_abort));
>          sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
>          sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
> +        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
>          pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
>          sbdev = SYS_BUS_DEVICE(obj);
>          sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
> @@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
>                       (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
>          build_append_int_noprefix(table_data, node_size, 2); /* Length */
>          build_append_int_noprefix(table_data, 4, 1); /* Revision */
> -        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
> +        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
>          /* Number of ID mappings */
>          build_append_int_noprefix(table_data, smmu_mapping_count, 4);
>          /* Reference to ID Array */
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index c75a8d6e9e..44c6b99c96 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
>      }
>  }
>  
> +static uint8_t smmuv3_dev_id;
nit: I would rather put that at the top of the file
>  static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>                                              DeviceState *dev, Error **errp)
>  {
> @@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>                                       OBJECT(vms->sysmem), NULL);
>              object_property_set_link(OBJECT(dev), "secure-memory",
>                                       OBJECT(vms->secure_sysmem), NULL);
> +            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
> +                                     NULL);
>          }
>          if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
>              hwaddr db_start = 0;
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Add an "identifier" property to the SMMUv3 device and use it when
> building the ACPI IORT SMMUv3 node Identifier field.
>
> This avoids relying on device enumeration order and provides a stable
> per-device identifier. A subsequent patch will use the same identifier
> when generating the DSDT description for Tegra241 CMDQV, ensuring that
> the IORT and DSDT entries refer to the same SMMUv3 instance.
>
> No functional change intended.
>
> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  include/hw/arm/smmuv3.h  | 1 +
>  hw/arm/smmuv3.c          | 2 ++
>  hw/arm/virt-acpi-build.c | 4 +++-
>  hw/arm/virt.c            | 3 +++
>  4 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
> index 648412cafc..73b8f39aaa 100644
> --- a/include/hw/arm/smmuv3.h
> +++ b/include/hw/arm/smmuv3.h
> @@ -63,6 +63,7 @@ struct SMMUv3State {
>      qemu_irq     irq[4];
>      QemuMutex mutex;
>      char *stage;
> +    uint8_t identifier;
>  
>      /* SMMU has HW accelerator support for nested S1 + s2 */
>      bool accel;
> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
> index 468135bb24..c1f84bedd4 100644
> --- a/hw/arm/smmuv3.c
> +++ b/hw/arm/smmuv3.c
> @@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
>       * Defaults to stage 1
>       */
>      DEFINE_PROP_STRING("stage", SMMUv3State, stage),
> +    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
> +    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
>      DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
>      /* GPA of MSI doorbell, for SMMUv3 accel use. */
>      DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index ae78e9b9e0..20605185c5 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
>  typedef struct AcpiIortSMMUv3Dev {
>      int irq;
>      hwaddr base;
> +    uint8_t id;
>      GArray *rc_smmu_idmaps;
>      /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
>      size_t offset;
> @@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
>                                                 &error_abort));
>          sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
>          sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
> +        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
>          pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
>          sbdev = SYS_BUS_DEVICE(obj);
>          sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
> @@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
>                       (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
>          build_append_int_noprefix(table_data, node_size, 2); /* Length */
>          build_append_int_noprefix(table_data, 4, 1); /* Revision */
> -        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
> +        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
Hum actually I see that previously the id scope was shared by all nodes,
including ITS, SMMU, RC.
So now can have its and smmu node having both id=0. And this is why you
have a change in the IORT table for its=off test.
Is it legal?

IORT spec says:

Unique identifier for this node that can be used to locate it in the
parent table. This identifier enables other ACPI tables and DSDT objects
to locate this node. This field serves in the same capacity as the _UID
object associated with ACPI device objects. Note IMPLEMENTATION NOTE: In
the simplest scheme, the Identifier might be set to the node’s index in
the array of IORT nodes in the parent IORT table. Other schemes are also
possible and permitted.

So I don't think this is valid

Eric
 

>          /* Number of ID mappings */
>          build_append_int_noprefix(table_data, smmu_mapping_count, 4);
>          /* Reference to ID Array */
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index c75a8d6e9e..44c6b99c96 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
>      }
>  }
>  
> +static uint8_t smmuv3_dev_id;
>  static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>                                              DeviceState *dev, Error **errp)
>  {
> @@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>                                       OBJECT(vms->sysmem), NULL);
>              object_property_set_link(OBJECT(dev), "secure-memory",
>                                       OBJECT(vms->secure_sysmem), NULL);
> +            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
> +                                     NULL);
>          }
>          if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
>              hwaddr db_start = 0;

RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: 09 March 2026 18:10
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> Cc: eric.auger@redhat.com; qemu-arm@nongnu.org; qemu-
> devel@nongnu.org; peter.maydell@linaro.org; clg@redhat.com;
> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV
> vIOMMU alloc/free
> 
> On Mon, Mar 09, 2026 at 04:31:22AM -0700, Shameer Kolothum Thodi
> wrote:
> > > > -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > > > -    return false;
> > > > +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
> > > > +
> > > > +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> > > > +
> > > > + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
> > > if the only think that differs compared to no cmdq is the type, ie.
> > >
> > > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
> > > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
> > >
> > > + passing true args, maybe you can just record the type of the cmdqv
> > > + that is being used in the smmu_accel device. Then you can get rid of
> > > + alloc and free
> >
> > It is not just the type. Based on the type we also need to pass,
> >
> >  * @data_len: Length of the type specific data
> >  * @__reserved: Must be 0
> >  * @data_uptr: User pointer to a driver-specific virtual IOMMU data
> >  *
> >
> > And the above is implementation specific.
> >
> > If our idea of the "ops" is to allow easier support for different
> > implementations in future, it probably makes sense to keep this.
> 
> Any future "cmdqv" will likely have a different viommu type. So,
> essentially this is just a viommu type override, as Eric pointed
> out above.
> 
> What we actually need to expose per viommu is type/data_len. The
> data_uptr can be a union viommu_data in the accel structure?

So the suggestion is to get rid of alloc_viommu() and retrieve type/len 
during probe() and store the data_uptr in struct accel . Doable, I guess.

But then we need to do something similar for veventq as well, right?

Isn't that actually bloating struct accel then? 
IMHO, the ops way looks better.

Thanks,
Shameer

RE: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 18:23
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier
> property
> 
> External email: Use caution opening links or attachments
> 
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > Add an "identifier" property to the SMMUv3 device and use it when
> > building the ACPI IORT SMMUv3 node Identifier field.
> >
> > This avoids relying on device enumeration order and provides a stable
> > per-device identifier. A subsequent patch will use the same identifier
> > when generating the DSDT description for Tegra241 CMDQV, ensuring that
> > the IORT and DSDT entries refer to the same SMMUv3 instance.
> >
> > No functional change intended.
> >
> > Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  include/hw/arm/smmuv3.h  | 1 +
> >  hw/arm/smmuv3.c          | 2 ++
> >  hw/arm/virt-acpi-build.c | 4 +++-
> >  hw/arm/virt.c            | 3 +++
> >  4 files changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h
> > index 648412cafc..73b8f39aaa 100644
> > --- a/include/hw/arm/smmuv3.h
> > +++ b/include/hw/arm/smmuv3.h
> > @@ -63,6 +63,7 @@ struct SMMUv3State {
> >      qemu_irq     irq[4];
> >      QemuMutex mutex;
> >      char *stage;
> > +    uint8_t identifier;
> >
> >      /* SMMU has HW accelerator support for nested S1 + s2 */
> >      bool accel;
> > diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
> > index 468135bb24..c1f84bedd4 100644
> > --- a/hw/arm/smmuv3.c
> > +++ b/hw/arm/smmuv3.c
> > @@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
> >       * Defaults to stage 1
> >       */
> >      DEFINE_PROP_STRING("stage", SMMUv3State, stage),
> > +    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV)
> generation */
> > +    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
> >      DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
> >      /* GPA of MSI doorbell, for SMMUv3 accel use. */
> >      DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
> > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> > index ae78e9b9e0..20605185c5 100644
> > --- a/hw/arm/virt-acpi-build.c
> > +++ b/hw/arm/virt-acpi-build.c
> > @@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a,
> gconstpointer b)
> >  typedef struct AcpiIortSMMUv3Dev {
> >      int irq;
> >      hwaddr base;
> > +    uint8_t id;
> >      GArray *rc_smmu_idmaps;
> >      /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
> >      size_t offset;
> > @@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray
> *sdev_blob, VirtMachineState *vms)
> >                                                 &error_abort));
> >          sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
> >          sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
> > +        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
> >          pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
> >          sbdev = SYS_BUS_DEVICE(obj);
> >          sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
> > @@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker,
> VirtMachineState *vms)
> >                       (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
> >          build_append_int_noprefix(table_data, node_size, 2); /* Length */
> >          build_append_int_noprefix(table_data, 4, 1); /* Revision */
> > -        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
> > +        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
> Hum actually I see that previously the id scope was shared by all nodes,
> including ITS, SMMU, RC.
> So now can have its and smmu node having both id=0. And this is why you
> have a change in the IORT table for its=off test.

Hmm... I had that doubt as well. But I read it as a unique identifier for each 
type of IORT node. 

> Is it legal?
> 
> IORT spec says:
> 
> Unique identifier for this node that can be used to locate it in the
> parent table. This identifier enables other ACPI tables and DSDT objects
> to locate this node. This field serves in the same capacity as the _UID
> object associated with ACPI device objects. Note IMPLEMENTATION NOTE: In
> the simplest scheme, the Identifier might be set to the node’s index in
> the array of IORT nodes in the parent IORT table. Other schemes are also
> possible and permitted.
> 
> So I don't think this is valid

I think we need another way to sort out the identifier then.

Thanks,
Shameer

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Nicolin Chen]

On Mon, Mar 09, 2026 at 11:04:31AM -0700, Shameer Kolothum Thodi wrote:
> > > The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> > > and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> > > IOMMU_HW_QUEUE_ALLOC.

> > If the guest detects he can use a single VMCDQs (I guess he can
> > introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if
> > he
> > shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs
> > (0x30000-40000). How does the guest know that he is supposed to use VI0?
> 
> The CMDQ registers can be accessed via both the so called "Global VCMDQ registers 
> Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
> 
> The difference being(from Nicolin's explanation here[0]):
> 
> "
> The global page0 is programmable at any time so long as CMDQV_EN
> is enabled.
> 
> The logical page0 is programmable only when SW allocates and maps
> global vcmdq(s) to a VINTF. "logical" also means "local" to that
> VINTF.
> ".
> 
> IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ
> registers Page 0". 

Yes.

> The question is whether QEMU should also support accesses via the
> "Global VCMDQ registers Page 0" if some other guest implementation
> chooses to use that path.

Yes.

The point is that we can't reject guest's access to the global MMIO
space. So, QEMU has to support both. But guest OS is allowed to do a
partial map/alloc, so global MMIO page isn't identical to the VINTF
logical page in this case.

In an extreme corner case, guest OS might try to enable a VCMDQ via
the global MMIO space, though the vcmdq isn't allocated/mapped to a
VINTF. QEMU should still allow these MMIOs but does not essentially
make the vcmdq functional, since this would be "unpredictable" in a
physical HW situation.

Maybe we should elaborate these in the commit message.

Nicolin

> @Nicolin, please correct me if that assumption is not right.

RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: 09 March 2026 18:41
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> Cc: eric.auger@redhat.com; qemu-arm@nongnu.org; qemu-
> devel@nongnu.org; peter.maydell@linaro.org; clg@redhat.com;
> alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and
> VINTF VCMDQ register reads
> 
> On Mon, Mar 09, 2026 at 11:04:31AM -0700, Shameer Kolothum Thodi
> wrote:
> > > > The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> > > > and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> > > > IOMMU_HW_QUEUE_ALLOC.
> 
> > > If the guest detects he can use a single VMCDQs (I guess he can
> > > introspect this by reading the SMMU_CMDQ_PARAM_0) how does he
> know if
> > > he
> > > shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs
> > > (0x30000-40000). How does the guest know that he is supposed to use
> VI0?
> >
> > The CMDQ registers can be accessed via both the so called "Global VCMDQ
> registers
> > Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
> >
> > The difference being(from Nicolin's explanation here[0]):
> >
> > "
> > The global page0 is programmable at any time so long as CMDQV_EN
> > is enabled.
> >
> > The logical page0 is programmable only when SW allocates and maps
> > global vcmdq(s) to a VINTF. "logical" also means "local" to that
> > VINTF.
> > ".
> >
> > IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ
> > registers Page 0".
> 
> Yes.
> 
> > The question is whether QEMU should also support accesses via the
> > "Global VCMDQ registers Page 0" if some other guest implementation
> > chooses to use that path.
> 
> Yes.
> 
> The point is that we can't reject guest's access to the global MMIO
> space. So, QEMU has to support both. But guest OS is allowed to do a
> partial map/alloc, so global MMIO page isn't identical to the VINTF
> logical page in this case.

Are there any benefits for a guest implementation making use of the
"Global VCMDQ registers Page 0"?

If not, the VINTF-based access path seems preferable from a hardware
isolation point of view. In that case, would it make sense to restrict
usage to the VINTF logical page and recommend that guests use that
interface?

Thanks,
Shameer

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

[Nicolin Chen]

On Mon, Mar 09, 2026 at 11:25:21AM -0700, Shameer Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Nicolin Chen <nicolinc@nvidia.com>
> > Sent: 09 March 2026 18:10
> > To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> > Cc: eric.auger@redhat.com; qemu-arm@nongnu.org; qemu-
> > devel@nongnu.org; peter.maydell@linaro.org; clg@redhat.com;
> > alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> > <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> > <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> > zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> > <kjaju@nvidia.com>; phrdina@redhat.com
> > Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV
> > vIOMMU alloc/free
> > 
> > On Mon, Mar 09, 2026 at 04:31:22AM -0700, Shameer Kolothum Thodi
> > wrote:
> > > > > -    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
> > > > > -    return false;
> > > > > +    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
> > > > > +
> > > > > +    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
> > > > > +
> > > > > + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
> > > > if the only think that differs compared to no cmdq is the type, ie.
> > > >
> > > > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
> > > > IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
> > > >
> > > > + passing true args, maybe you can just record the type of the cmdqv
> > > > + that is being used in the smmu_accel device. Then you can get rid of
> > > > + alloc and free
> > >
> > > It is not just the type. Based on the type we also need to pass,
> > >
> > >  * @data_len: Length of the type specific data
> > >  * @__reserved: Must be 0
> > >  * @data_uptr: User pointer to a driver-specific virtual IOMMU data
> > >  *
> > >
> > > And the above is implementation specific.
> > >
> > > If our idea of the "ops" is to allow easier support for different
> > > implementations in future, it probably makes sense to keep this.
> > 
> > Any future "cmdqv" will likely have a different viommu type. So,
> > essentially this is just a viommu type override, as Eric pointed
> > out above.
> > 
> > What we actually need to expose per viommu is type/data_len. The
> > data_uptr can be a union viommu_data in the accel structure?
> 
> So the suggestion is to get rid of alloc_viommu() and retrieve type/len 
> during probe() and store the data_uptr in struct accel . Doable, I guess.
> 
> But then we need to do something similar for veventq as well, right?
> 
> Isn't that actually bloating struct accel then? 
> IMHO, the ops way looks better.

The data_uptr could be dynamically allocated corresponding to the
data_len, to avoid bloating.

But you are right we do need a place to alloc/free a 2nd veventq.

Maybe we can have a pair of alloc/free_viommu for now.

Thanks
Nicolin

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Nicolin Chen]

On Mon, Mar 09, 2026 at 11:53:59AM -0700, Shameer Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Nicolin Chen <nicolinc@nvidia.com>
> > Sent: 09 March 2026 18:41
> > To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> > Cc: eric.auger@redhat.com; qemu-arm@nongnu.org; qemu-
> > devel@nongnu.org; peter.maydell@linaro.org; clg@redhat.com;
> > alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> > <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> > <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> > zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> > <kjaju@nvidia.com>; phrdina@redhat.com
> > Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and
> > VINTF VCMDQ register reads
> > 
> > On Mon, Mar 09, 2026 at 11:04:31AM -0700, Shameer Kolothum Thodi
> > wrote:
> > > > > The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
> > > > > and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> > > > > IOMMU_HW_QUEUE_ALLOC.
> > 
> > > > If the guest detects he can use a single VMCDQs (I guess he can
> > > > introspect this by reading the SMMU_CMDQ_PARAM_0) how does he
> > know if
> > > > he
> > > > shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs
> > > > (0x30000-40000). How does the guest know that he is supposed to use
> > VI0?
> > >
> > > The CMDQ registers can be accessed via both the so called "Global VCMDQ
> > registers
> > > Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
> > >
> > > The difference being(from Nicolin's explanation here[0]):
> > >
> > > "
> > > The global page0 is programmable at any time so long as CMDQV_EN
> > > is enabled.
> > >
> > > The logical page0 is programmable only when SW allocates and maps
> > > global vcmdq(s) to a VINTF. "logical" also means "local" to that
> > > VINTF.
> > > ".
> > >
> > > IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ
> > > registers Page 0".
> > 
> > Yes.
> > 
> > > The question is whether QEMU should also support accesses via the
> > > "Global VCMDQ registers Page 0" if some other guest implementation
> > > chooses to use that path.
> > 
> > Yes.
> > 
> > The point is that we can't reject guest's access to the global MMIO
> > space. So, QEMU has to support both. But guest OS is allowed to do a
> > partial map/alloc, so global MMIO page isn't identical to the VINTF
> > logical page in this case.
> 
> Are there any benefits for a guest implementation making use of the
> "Global VCMDQ registers Page 0"?
> 
> If not, the VINTF-based access path seems preferable from a hardware
> isolation point of view.

Yea. That's why Linux was implemented in that way.

> In that case, would it make sense to restrict
> usage to the VINTF logical page and recommend that guests use that
> interface?

I don't think QEMU can recommend that... Linux kernel made a good
example but it doesn't make a lot of sense to recommend or force
all guest SWs to follow.

From hardware emulation perspective, QEMU should just re-act like
a real HW. It's okay to make it unfunctional v.s. unpredictable.
But it shouldn't reject any MMIO access, since the HW wouldn't.

Nicolin

Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

[Nicolin Chen]

On Mon, Mar 09, 2026 at 10:41:24AM -0700, Shameer Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Nicolin Chen <nicolinc@nvidia.com>
> > Sent: 09 March 2026 17:24
> > To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> > Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org;
> > eric.auger@redhat.com; peter.maydell@linaro.org; clg@redhat.com;
> > alex@shazbot.org; Nathan Chen <nathanc@nvidia.com>; Matt Ochs
> > <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> > <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> > zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> > <kjaju@nvidia.com>; phrdina@redhat.com
> > Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ
> > allocation and free
> > 
> > On Thu, Feb 26, 2026 at 10:50:46AM +0000, Shameer Kolothum wrote:
> > > Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the
> > > corresponding teardown path to free the vEVENTQ during cleanup.
> > >
> > > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > > ---
> > >  hw/arm/smmuv3-accel.h   |  2 ++
> > >  hw/arm/tegra241-cmdqv.h |  1 +
> > >  hw/arm/smmuv3-accel.c   | 10 ++++++++-
> > >  hw/arm/tegra241-cmdqv.c | 47
> > +++++++++++++++++++++++++++++++++++++++++
> > >  4 files changed, 59 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hw/arm/smmuv3-accel.h b/hw/arm/smmuv3-accel.h
> > > index 7d6e4c6b76..4bff90e2c1 100644
> > > --- a/hw/arm/smmuv3-accel.h
> > > +++ b/hw/arm/smmuv3-accel.h
> > > @@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
> > >                           uint32_t *out_viommu_id,
> > >                           Error **errp);
> > >      void (*free_viommu)(SMMUv3State *s);
> > > +    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
> > > +    void (*free_veventq)(SMMUv3State *s);
> > 
> > As I replied in v2, this should really depend on viommu and should
> > be simply added to tegra241_cmdqv_alloc/free_viommu().
> 
> Yes, I remember that. That was actually my initial plan, which is why
> this was not part of the original callbacks in the ops struct.
> 
> However, I then realised that we are storing the viommu pointer
> in veventq,
> 
> veventq->viommu = accel->viommu;
> 
> And that struct viommu is only allocated after alloc_viommu() returns.
> 
> I didn't find an easy way to resolve that.  Need to take another look.

Looks like we only needed iommufd pointer v.s. accel->viommu? This
should allow us to merge alloc_veventq into alloc_viommu.

Alternatively, as we discussed on the other side, we could drop the
alloc/free_viommu. Then, have a pair of viommu_init/destroy to have
the veventq part (and any viommu specific routine in the future).

I feel okay with either approach. So, pick one that works the best.

Nicolin

RE: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Shameer Kolothum Thodi]

Hi Eric,

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 09 March 2026 16:33
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global
> CMDQV registers
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Shameer,
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > From: Nicolin Chen <nicolinc@nvidia.com>
> >
> > Tegra241 CMDQV defines a set of global control and status registers
> > used to configure virtual command queue allocation and interrupt
> > behavior.
> >
> > Add read/write emulation for the global CMDQV register page
> > (offset 0x00000), backed by a simple register cache. This includes
> > CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
> > map and the VINTF0 related registers defined in the global CMDQV
> > register space.
> >
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.h | 108
> +++++++++++++++++++++++++++++++++++
> >  hw/arm/tegra241-cmdqv.c | 121
> +++++++++++++++++++++++++++++++++++++++-
> >  2 files changed, 228 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> > index 46aa9e8a9f..50bcecee9d 100644
> > --- a/hw/arm/tegra241-cmdqv.h
> > +++ b/hw/arm/tegra241-cmdqv.h
> > @@ -10,6 +10,9 @@
> >  #ifndef HW_ARM_TEGRA241_CMDQV_H
> >  #define HW_ARM_TEGRA241_CMDQV_H
> >
> > +#include "hw/core/registerfields.h"
> > +#include "smmuv3-accel.h"
> > +
> >  #define TEGRA241_CMDQV_VERSION             1
> >  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
> >  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> > @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
> >      SMMUv3AccelState *s_accel;
> >      MemoryRegion mmio_cmdqv;
> >      qemu_irq irq;
> > +
> > +    /* Register Cache */
> > +    uint32_t config;
> > +    uint32_t param;
> > +    uint32_t status;
> > +    uint32_t vi_err_map[2];
> > +    uint32_t vi_int_mask[2];
> > +    uint32_t cmdq_err_map[4];
> > +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
> > +    uint32_t vintf_config;
> > +    uint32_t vintf_status;
> > +    uint32_t vintf_sid_match[16];
> > +    uint32_t vintf_sid_replace[16];
> > +    uint32_t vintf_cmdq_err_map[4];
> >  } Tegra241CMDQV;
> >
> > +/* Global CMDQV MMIO registers (offset 0x00000) */
> > +REG32(CONFIG, 0x0)
> > +FIELD(CONFIG, CMDQV_EN, 0, 1)
> > +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
> > +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
> > +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
> > +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
> > +
> > +REG32(PARAM, 0x4)
> > +FIELD(PARAM, CMDQV_VER, 0, 4)
> > +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
> > +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
> this is called CMDQV_NUM_VI_LOG2 in the spec I have access to
> VI = virtual interface
> I guess there is 1-1 mapping vetween VI and VM but I would keep spec
> naming
> > +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
> same here s/VM/VI
> > +
> > +REG32(STATUS, 0x8)
> > +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
> > +
> I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
> > +#define A_VI_ERR_MAP 0x14
> _0?
> > +#define A_VI_ERR_MAP_1 0x18
> > +#define V_VI_ERR_MAP_NO_ERROR (0)
> > +#define V_VI_ERR_MAP_ERROR (1)
> > +
> same SMMU_CMDQV_VI_ING_MASK0/1
> > +#define A_VI_INT_MASK 0x1c
> > +#define A_VI_INT_MASK_1 0x20
> > +#define V_VI_INT_MASK_NOT_MASKED (0)
> > +#define V_VI_INT_MASK_MASKED (1)
> > +
> SMMU_CMDQV_CMDQ_ERR_MAP0-3
> > +#define A_CMDQ_ERR_MAP 0x24
> > +#define A_CMDQ_ERR_MAP_1 0x28
> > +#define A_CMDQ_ERR_MAP_2 0x2c
> > +#define A_CMDQ_ERR_MAP_3 0x30
> > +
> > +/* i = [0, 1] */
> 
> > +#define A_CMDQ_ALLOC_MAP_(i)
> why A_? Since you rely on the REG macros, A_ should prefix the address
> offset
> Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in
> the spec?

I will incorporate all suggestions related to naming etc and will make it same as
used in the spec.

> >               \
> > +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
> > +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
> > +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
> > +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
> > +
> Please explain why we only expose 2 of those regs among 128?

QEMU currently models only two VCMDQs as we expose that by setting
TEGRA241_CMDQV_NUM_CMDQ_LOG2 = 1 via SMMU_CMDQV_PARAM_0.
Therefore, only the corresponding CMDQ allocation map entries are
implemented. I will add a comment to make it explicit here.

> > +A_CMDQ_ALLOC_MAP_(0)
> > +A_CMDQ_ALLOC_MAP_(1)
> > +
> > +
> > +/* i = [0, 0] */
> > +#define A_VINTFi_CONFIG(i)
> again why don't we use the spec terminology
> SMMU_CMDQV_VINTF0_CONFIG_0
> >                   \
> > +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
> > +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
> > +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
> > +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
> > +
> > +A_VINTFi_CONFIG(0)
> > +
> > +#define A_VINTFi_STATUS(i)                       \
> > +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
> > +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
> > +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
> > +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
> > +
> > +A_VINTFi_STATUS(0)
> > +
> > +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
> > +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
> Nit: the spec also contains the typo but I would rather fix it in the code
> 
> Explicitly state we only expose 1 VINTF
> > +
> > +/* i = [0, 0], j = [0, 15] */
> does that mean that we can have a max 16 SIDs per VM?
> > +#define A_VINTFi_SID_MATCH_(i, j)
> if matched vi, I would prefer vi
> >                               \
> > +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
> > +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
> > +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
> > +
> > +A_VINTFi_SID_MATCH_(0, 0)
> > +/* Omitting [0][1~14] as not being directly called */
> > +A_VINTFi_SID_MATCH_(0, 15)
> > +
> > +/* i = [0, 0], j = [0, 15] */
> vint = 0, 16 identical register entries
> > +#define A_VINTFi_SID_REPLACE_(i, j)                               \
> > +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
> > +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
> s/19/20
> > +
> > +A_VINTFi_SID_REPLACE_(0, 0)
> > +/* Omitting [0][1~14] as not being directly called */
> > +A_VINTFi_SID_REPLACE_(0, 15)
> > +
> > +/* i = [0, 0], j = [0, 3] */
> vi = 0, 4 identical regs
> > +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
> > +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
> > +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
> > +
> > +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> > +/* Omitting [0][1~2] as not being directly called */
> I don't get this comment

the hardware defines four registers ([0..3]). They are handled using
the range based switch case,

case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:

And the index to arary is calculated using:
i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4

I will reword the comment something like:

Only the first and last offsets are declared explicitly since the
intermediate registers are handled via the range based switch case.
The register index is derived as:
   (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4

> > +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> > +
> >  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
> >
> >  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> > index d487612ba2..a3830a02d6 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -8,19 +8,138 @@
> >   */
> >
> >  #include "qemu/osdep.h"
> > +#include "qemu/log.h"
> >
> >  #include "hw/arm/smmuv3.h"
> >  #include "smmuv3-accel.h"
> >  #include "tegra241-cmdqv.h"
> >
> > +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,
> hwaddr offset)
> > +{
> > +    int i;
> > +
> > +    switch (offset) {
> > +    case A_VINTF0_CONFIG:
> > +        return cmdqv->vintf_config;
> > +    case A_VINTF0_STATUS:
> > +        return cmdqv->vintf_status;
> > +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> > +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> > +        return cmdqv->vintf_sid_match[i];
> > +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> > +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> > +        return cmdqv->vintf_sid_replace[i];
> > +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
> A_VINTF0_LVCMDQ_ERR_MAP_3:
> > +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
> > +        return cmdqv->vintf_cmdq_err_map[i];
> > +    default:
> > +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
> PRIx64 "\n",
> > +                      __func__, offset);
> > +        return 0;
> > +    }
> > +}
> > +
> >  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,
> unsigned size)
> >  {
> > -    return 0;
> > +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> > +
> > +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> > +        qemu_log_mask(LOG_UNIMP,
> > +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> > +                      offset);
> If all those regs belong to the Global CMDQV registers page this shall
> be within the first 64KB

Yes, in this patch that is the case. However, subsequent patches in the
series will add support for the other page windows as well. Hence the
full MMIO range check here.

> > +        return 0;
> > +    }
> > +
> > +    switch (offset) {
> > +    case A_CONFIG:
> > +        return cmdqv->config;
> > +    case A_PARAM:
> > +        return cmdqv->param;
> > +    case A_STATUS:
> > +        return cmdqv->status;
> > +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
> > +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
> > +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> > +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
> > +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
> > +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
> > +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> > +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /
> 4];
> > +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> > +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
> > +    default:
> > +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
> PRIx64 "\n",
> > +                      __func__, offset);
> > +        return 0;
> > +    }
> > +}
> > +
> > +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr
> offset,
> > +                                       uint64_t value)
> > +{
> > +    int i;
> > +
> > +    switch (offset) {
> > +    case A_VINTF0_CONFIG:
> > +        /* Strip off HYP_OWN setting from guest kernel */
> > +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> Can you explain why this needed?

The HYP_OWN bit is not guest controllable. It is owned by the host
kernel/hypervisor and is wired to zero when running in guest kernel.

Please see tegra241_vintf_hw_init() in kernel,
drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c

The guest cannot enable this bit and the effective value seen by the
guest remains zero.

> > +
> > +        cmdqv->vintf_config = value;
> > +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> > +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> > +        } else {
> > +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> > +        }
> > +        break;
> > +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> > +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> > +        cmdqv->vintf_sid_match[i] = value;
> > +        break;
> > +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> > +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> > +        cmdqv->vintf_sid_replace[i] = value;
> > +        break;
> > +    default:
> > +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
> PRIx64 "\n",
> > +                      __func__, offset);
> > +        return;
> > +    }
> >  }
> nit: if you can put write_vintf just after its read fellow, I think it
> would ease the comparison/review
> >
> >  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t
> value,
> >                                   unsigned size)
> >  {
> > +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> > +
> > +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> > +        qemu_log_mask(LOG_UNIMP,
> > +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> > +                      offset);
> > +        return;
> > +    }
> > +
> > +    switch (offset) {
> > +    case A_CONFIG:
> > +        cmdqv->config = value;
> > +        if (value & R_CONFIG_CMDQV_EN_MASK) {
> > +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
> > +        } else {
> > +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
> > +        }
> > +        break;
> > +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> > +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
> > +        break;
> > +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> > +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
> value;
> > +        break;
> > +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> > +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> > +        break;
> > +    default:
> > +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
> PRIx64 "\n",
> > +                      __func__, offset);
> > +    }
> >  }
> >
> >  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
> 
> At this stage of the reading it is unclear to me why we need to expose
> all those regs to the guest. I would assume that a subset of them are
> used by the hyp and not supposed to be accessed by the guest. For
> instance the SID match/replace, ...
> 
> Please could you clairfy what is absolutely needed to expose to the guest?

The intention here is to model the architectural register interface
defined by the CMDQV specification so the guest visible MMIO space
matches the hardware.

And some of the registers are backed by a simple register cache(like the
SID match/replace).  Access to the actual hardware state remains controlled
by QEMU, so guest writes do not necessarily translate to direct hardware
programming.

Thanks,
Shameer

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Eric Auger]

Hi Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV defines a set of global control and status registers
> used to configure virtual command queue allocation and interrupt
> behavior.
>
> Add read/write emulation for the global CMDQV register page
> (offset 0x00000), backed by a simple register cache. This includes
> CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
> map and the VINTF0 related registers defined in the global CMDQV
> register space.
This is called CMDQ-V Config within [CMDQV_BASE, CMDQV-CMDQ_BASE] in the
spec.

Precise we only support a single VINTF0 and VINTF1-63 are not supported.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 228 insertions(+), 1 deletion(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 46aa9e8a9f..50bcecee9d 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -10,6 +10,9 @@
>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>  #define HW_ARM_TEGRA241_CMDQV_H
>  
> +#include "hw/core/registerfields.h"
> +#include "smmuv3-accel.h"
> +
>  #define TEGRA241_CMDQV_VERSION             1
>  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +
> +    /* Register Cache */
> +    uint32_t config;
> +    uint32_t param;
> +    uint32_t status;
> +    uint32_t vi_err_map[2];
> +    uint32_t vi_int_mask[2];
> +    uint32_t cmdq_err_map[4];
> +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vintf_config;
> +    uint32_t vintf_status;
> +    uint32_t vintf_sid_match[16];
> +    uint32_t vintf_sid_replace[16];
> +    uint32_t vintf_cmdq_err_map[4];
>  } Tegra241CMDQV;
>  
> +/* Global CMDQV MMIO registers (offset 0x00000) */
> +REG32(CONFIG, 0x0)
> +FIELD(CONFIG, CMDQV_EN, 0, 1)
> +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
> +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
> +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
> +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
> +
> +REG32(PARAM, 0x4)
> +FIELD(PARAM, CMDQV_VER, 0, 4)
> +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
> +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
> +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
> +
> +REG32(STATUS, 0x8)
> +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
> +
> +#define A_VI_ERR_MAP 0x14
> +#define A_VI_ERR_MAP_1 0x18
> +#define V_VI_ERR_MAP_NO_ERROR (0)
> +#define V_VI_ERR_MAP_ERROR (1)
> +
> +#define A_VI_INT_MASK 0x1c
> +#define A_VI_INT_MASK_1 0x20
> +#define V_VI_INT_MASK_NOT_MASKED (0)
> +#define V_VI_INT_MASK_MASKED (1)
> +
> +#define A_CMDQ_ERR_MAP 0x24
> +#define A_CMDQ_ERR_MAP_1 0x28
> +#define A_CMDQ_ERR_MAP_2 0x2c
> +#define A_CMDQ_ERR_MAP_3 0x30
> +
> +/* i = [0, 1] */
> +#define A_CMDQ_ALLOC_MAP_(i)                 \
> +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
> +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
> +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
> +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
> +
> +A_CMDQ_ALLOC_MAP_(0)
> +A_CMDQ_ALLOC_MAP_(1)
> +
> +
> +/* i = [0, 0] */
> +#define A_VINTFi_CONFIG(i)                       \
> +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
> +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
> +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
> +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
> +
> +A_VINTFi_CONFIG(0)
> +
> +#define A_VINTFi_STATUS(i)                       \
> +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
> +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
> +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
> +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
> +
> +A_VINTFi_STATUS(0)
> +
> +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
> +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_MATCH_(i, j)                               \
> +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
> +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
> +
> +A_VINTFi_SID_MATCH_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_MATCH_(0, 15)
> +
> +/* i = [0, 0], j = [0, 15] */
> +#define A_VINTFi_SID_REPLACE_(i, j)                               \
> +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
> +
> +A_VINTFi_SID_REPLACE_(0, 0)
> +/* Omitting [0][1~14] as not being directly called */
> +A_VINTFi_SID_REPLACE_(0, 15)
> +
> +/* i = [0, 0], j = [0, 3] */
> +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
> +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
> +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
> +
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
> +/* Omitting [0][1~2] as not being directly called */
> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index d487612ba2..a3830a02d6 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -8,19 +8,138 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/log.h"
>  
>  #include "hw/arm/smmuv3.h"
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
since it belong to the Config region, can we normalize the names

tegra241_cmdqv_config_vintf_read()


> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        return cmdqv->vintf_config;
> +    case A_VINTF0_STATUS:
> +        return cmdqv->vintf_status;
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        return cmdqv->vintf_sid_match[i];
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        return cmdqv->vintf_sid_replace[i];
> +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
> +        return cmdqv->vintf_cmdq_err_map[i];
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

tegra241_cmdqv_config_read
and same for the rest

>  {
> -    return 0;
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return 0;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        return cmdqv->config;
> +    case A_PARAM:
> +        return cmdqv->param;
> +    case A_STATUS:
> +        return cmdqv->status;
> +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
> +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
> +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
> +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return 0;
> +    }
> +}
> +
> +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> +                                       uint64_t value)
> +{
> +    int i;
> +
> +    switch (offset) {
> +    case A_VINTF0_CONFIG:
> +        /* Strip off HYP_OWN setting from guest kernel */
> +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> +
> +        cmdqv->vintf_config = value;
> +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        } else {
> +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> +        }
> +        break;
what about the GLB_FILT_CFG_0 and FILT_DATA-0, at least document why you
don't support it neither in R nor W
> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
> +        cmdqv->vintf_sid_match[i] = value;
> +        break;
> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
> +        cmdqv->vintf_sid_replace[i] = value;
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +        return;
> +    }
>  }
>  
>  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +
> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
> +                      offset);
> +        return;
> +    }
> +
> +    switch (offset) {
> +    case A_CONFIG:
> +        cmdqv->config = value;
> +        if (value & R_CONFIG_CMDQV_EN_MASK) {
> +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
> +        } else {
> +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
> +        }
> +        break;
> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
> +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
> +        break;
> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
> +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
> +        break;
> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        break;
> +    default:
> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
Otherwise I reviewed the macros and they look good to me

Eric

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> VINTF pages expose a logical view local to a given VINTF. Although real
> hardware may support multiple VINTFs, the kernel currently exposes a
> single VINTF per VM.
>
> The kernel provides an mmap offset for the VINTF Page0 region during
> vIOMMU allocation. However, the logical-to-physical association between
> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> to that, the mapped Page0 does not back any real VCMDQ state.
>
> When VINTF is enabled, mmap the kernel provided Page0 region and
> unmap it when VINTF is disabled. This prepares the VINTF mapping
> in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated
(page 1 global cmdq) to something that is mmapped on a host page. How do
we transfer the state of the cmdq from one to the other?

Thanks

Eric
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h |  3 +++
>  hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 45 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index d379b8860c..3ce9f539ae 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -18,6 +18,8 @@
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>  
> +#define VINTF_PAGE_SIZE 0x10000
> +
>  /*
>   * Tegra241 CMDQV MMIO layout (64KB pages)
>   *
> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +    void *vintf_page0;
>  
>      /* Register Cache */
>      uint32_t config;
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index e1f1562c44..a3767a85a3 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +static bool
> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    if (!cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> +        return false;
> +    }
> +    cmdqv->vintf_page0 = NULL;
> +    return true;
> +}
> +
> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> +
> +    if (cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
> +                                     VINTF_PAGE_SIZE,
> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
> +                                     &cmdqv->vintf_page0, errp)) {
> +        return false;
> +    }
> +
> +    return true;
> +}
> +
>  /*
>   * Write a VCMDQ register using VCMDQ0_* offsets.
>   *
> @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>  }
>  
>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> -                                       uint64_t value)
> +                                       uint64_t value, Error **errp)
>  {
>      int i;
>  
> @@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>  
>          cmdqv->vintf_config = value;
>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>          } else {
> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>          }
>          break;
> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    Error *local_err = NULL;
>      int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
>          break;
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>          break;
>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>          /* Same decoding as read() case: See comments above */
> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
>                        __func__, offset);
>      }
> +
> +    if (local_err) {
> +        error_report_err(local_err);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
>
>   -Global VCMDQ registers at 0x10000/0x20000
>   -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
>
> The VI_VCMDQ register ranges are an alias of the global VCMDQ registers
I would recommend to use the same terminology anywhere, either VINTF or
VI and stick to it.

Why do we need emulation for VI_VCMDQ. I thought this was mmapped?

Explicitly state we only support 2 VCMDQs

> and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
> IOMMU_HW_QUEUE_ALLOC.
>
> Add read side emulation for both global VCMDQ and VI_VCMDQ register
> ranges. MMIO accesses are decoded to extract the VCMDQ instance index
> and normalized to a VCMDQ0_* register offset, allowing a single helper
> to service all VCMDQ instances.
>
> VI_VCMDQ accesses are translated to their equivalent global VCMDQ
> offsets and reuse the same decoding path. All VCMDQ reads are currently
> served from cached register state.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
>  hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
>  2 files changed, 255 insertions(+)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 50bcecee9d..d379b8860c 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
>      uint32_t vintf_sid_match[16];
>      uint32_t vintf_sid_replace[16];
>      uint32_t vintf_cmdq_err_map[4];
> +    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
> +    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
>  } Tegra241CMDQV;
>  
>  /* Global CMDQV MMIO registers (offset 0x00000) */
> @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
>  /* Omitting [0][1~2] as not being directly called */
>  A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
>  
> +/*
> + * VCMDQ register windows.
> + *
> + * Page 0 @ 0x10000: VCMDQ control and status registers
> + * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
> + */
> +#define A_VCMDQi_CONS_INDX(i)                       \
> +    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> +    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> +
> +A_VCMDQi_CONS_INDX(0)
> +A_VCMDQi_CONS_INDX(1)
> +
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
> +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
> +
> +#define A_VCMDQi_PROD_INDX(i)                             \
> +    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
> +    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
> +
> +A_VCMDQi_PROD_INDX(0)
> +A_VCMDQi_PROD_INDX(1)
> +
> +#define A_VCMDQi_CONFIG(i)                             \
> +    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> +
> +A_VCMDQi_CONFIG(0)
> +A_VCMDQi_CONFIG(1)
> +
> +#define A_VCMDQi_STATUS(i)                             \
> +    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
> +    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> +
> +A_VCMDQi_STATUS(0)
> +A_VCMDQi_STATUS(1)
> +
> +#define A_VCMDQi_GERROR(i)                               \
> +    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
> +    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> +    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VCMDQi_GERROR(0)
> +A_VCMDQi_GERROR(1)
> +
> +#define A_VCMDQi_GERRORN(i)                               \
> +    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
> +    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> +    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VCMDQi_GERRORN(0)
> +A_VCMDQi_GERRORN(1)
> +
> +#define A_VCMDQi_BASE_L(i)                       \
> +    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
> +    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> +    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
> +
> +A_VCMDQi_BASE_L(0)
> +A_VCMDQi_BASE_L(1)
> +
> +#define A_VCMDQi_BASE_H(i)                             \
> +    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
> +    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
> +
> +A_VCMDQi_BASE_H(0)
> +A_VCMDQi_BASE_H(1)
> +
> +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> +
> +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> +
> +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> +    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
> +    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> +
> +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> +A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> +
> +/*
> + * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
> + *
> + * Page 0 @ 0x30000: VI_VCMDQ control and status registers
> + * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
I don't get why it is needed.
> + */
> +#define A_VI_VCMDQi_CONS_INDX(i)                       \
> +    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
> +
> +A_VI_VCMDQi_CONS_INDX(0)
> +A_VI_VCMDQi_CONS_INDX(1)
> +
> +#define A_VI_VCMDQi_PROD_INDX(i)                             \
> +    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
> +
> +A_VI_VCMDQi_PROD_INDX(0)
> +A_VI_VCMDQi_PROD_INDX(1)
> +
> +#define A_VI_VCMDQi_CONFIG(i)                             \
> +    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
> +
> +A_VI_VCMDQi_CONFIG(0)
> +A_VI_VCMDQi_CONFIG(1)
> +
> +#define A_VI_VCMDQi_STATUS(i)                             \
> +    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
> +
> +A_VI_VCMDQi_STATUS(0)
> +A_VI_VCMDQi_STATUS(1)
> +
> +#define A_VI_VCMDQi_GERROR(i)                               \
> +    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
> +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
> +    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VI_VCMDQi_GERROR(0)
> +A_VI_VCMDQi_GERROR(1)
> +
> +#define A_VI_VCMDQi_GERRORN(i)                               \
> +    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
> +    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
> +
> +A_VI_VCMDQi_GERRORN(0)
> +A_VI_VCMDQi_GERRORN(1)
> +
> +#define A_VI_VCMDQi_BASE_L(i)                       \
> +    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
> +    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
> +
> +A_VI_VCMDQi_BASE_L(0)
> +A_VI_VCMDQi_BASE_L(1)
> +
> +#define A_VI_VCMDQi_BASE_H(i)                             \
> +    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
> +
> +A_VI_VCMDQi_BASE_H(0)
> +A_VI_VCMDQi_BASE_H(1)
> +
> +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
> +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
> +
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
> +
> +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
> +    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
> +    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
> +
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
> +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index a3830a02d6..d2e6938e44 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -14,6 +14,46 @@
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +/*
> + * Read a VCMDQ register using VCMDQ0_* offsets.
> + *
> + * The caller normalizes the MMIO offset such that @offset0 always refers
> + * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> + *
> + * All VCMDQ accesses return cached registers.
> + */
> +static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
> +                                          int index)
> +{
> +    switch (offset0) {
> +    case A_VCMDQ0_CONS_INDX:
> +        return cmdqv->vcmdq_cons_indx[index];
> +    case A_VCMDQ0_PROD_INDX:
> +        return cmdqv->vcmdq_prod_indx[index];
> +    case A_VCMDQ0_CONFIG:
> +        return cmdqv->vcmdq_config[index];
> +    case A_VCMDQ0_STATUS:
> +        return cmdqv->vcmdq_status[index];
> +    case A_VCMDQ0_GERROR:
> +        return cmdqv->vcmdq_gerror[index];
> +    case A_VCMDQ0_GERRORN:
> +        return cmdqv->vcmdq_gerrorn[index];
> +    case A_VCMDQ0_BASE_L:
> +        return cmdqv->vcmdq_base[index];
> +    case A_VCMDQ0_BASE_H:
> +        return cmdqv->vcmdq_base[index] >> 32;
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
> +        return cmdqv->vcmdq_cons_indx_base[index];
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
> +        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
> +    default:
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s unhandled read access at 0x%" PRIx64 "\n",
> +                      __func__, offset0);
> +        return 0;
> +    }
> +}
> +
>  static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  {
>      int i;
> @@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>          qemu_log_mask(LOG_UNIMP,
> @@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>          return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>          return tegra241_cmdqv_read_vintf(cmdqv, offset);
> +    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> +        /*
> +         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
> +         * layout as the global VCMDQ registers, but are based at 0x30000
> +         * instead of 0x10000.
> +         *
> +         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
> +         * global VCMDQ offset, then fall through to reuse the common VCMDQ
> +         * decoding logic below.
> +         */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
> +        /*
> +         * Decode a per-VCMDQ register access.
> +         *
> +         * The hardware supports up to 128 identical VCMDQ instances; we
> +         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
> +         * occupies a 0x80-byte window starting at 0x10000.
> +         *
> +         * The MMIO offset is decoded to extract the VCMDQ index and normalized
> +         * to the corresponding VCMDQ0_* register by subtracting index * 0x80.
> +         *
> +         * A single helper then services all VCMDQs, with @index selecting the
> +         * instance.
> +         */
> +        index = (offset - 0x10000) / 0x80;
> +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
> +    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
> +        index = (offset - 0x20000) / 0x80;
> +        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
>      default:
>          qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
>                        __func__, offset);
Thanks

Eric

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 11 March 2026 07:55
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
> for CMDQV
> 
> External email: Use caution opening links or attachments
> 
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > From: Nicolin Chen <nicolinc@nvidia.com>
> >
> > Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> > VINTF pages expose a logical view local to a given VINTF. Although real
> > hardware may support multiple VINTFs, the kernel currently exposes a
> > single VINTF per VM.
> >
> > The kernel provides an mmap offset for the VINTF Page0 region during
> > vIOMMU allocation. However, the logical-to-physical association between
> > VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> > to that, the mapped Page0 does not back any real VCMDQ state.
> >
> > When VINTF is enabled, mmap the kernel provided Page0 region and
> > unmap it when VINTF is disabled. This prepares the VINTF mapping
> > in advance of subsequent patches that add VCMDQ allocation support.
> So at some point we transition from something that is purely emulated
> (page 1 global cmdq) to something that is mmapped on a host page. How do
> we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the
"VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
nothing in the spec that forbids this), then we need to keep the two
views in sync.

So when the mapping between a global VCMDQ and a logical VCMDQ is
created through the HW_QUEUE ioctl, we should sync the state between
the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.

I will double check this.

Thanks,
Shameer

> Thanks
> 
> Eric
> >
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.h |  3 +++
> >  hw/arm/tegra241-cmdqv.c | 44
> +++++++++++++++++++++++++++++++++++++++--
> >  2 files changed, 45 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> > index d379b8860c..3ce9f539ae 100644
> > --- a/hw/arm/tegra241-cmdqv.h
> > +++ b/hw/arm/tegra241-cmdqv.h
> > @@ -18,6 +18,8 @@
> >  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> >  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
> >
> > +#define VINTF_PAGE_SIZE 0x10000
> > +
> >  /*
> >   * Tegra241 CMDQV MMIO layout (64KB pages)
> >   *
> > @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
> >      SMMUv3AccelState *s_accel;
> >      MemoryRegion mmio_cmdqv;
> >      qemu_irq irq;
> > +    void *vintf_page0;
> >
> >      /* Register Cache */
> >      uint32_t config;
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> > index e1f1562c44..a3767a85a3 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void
> *opaque, hwaddr offset, unsigned size)
> >      }
> >  }
> >
> > +static bool
> > +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error
> **errp)
> > +{
> > +    if (!cmdqv->vintf_page0) {
> > +        return true;
> > +    }
> > +
> > +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> > +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> > +        return false;
> > +    }
> > +    cmdqv->vintf_page0 = NULL;
> > +    return true;
> > +}
> > +
> > +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV
> *cmdqv, Error **errp)
> > +{
> > +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> > +
> > +    if (cmdqv->vintf_page0) {
> > +        return true;
> > +    }
> > +
> > +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
> >viommu_id,
> > +                                     VINTF_PAGE_SIZE,
> > +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
> > +                                     &cmdqv->vintf_page0, errp)) {
> > +        return false;
> > +    }
> > +
> > +    return true;
> > +}
> > +
> >  /*
> >   * Write a VCMDQ register using VCMDQ0_* offsets.
> >   *
> > @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
> *cmdqv, hwaddr offset0, int index,
> >  }
> >
> >  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr
> offset,
> > -                                       uint64_t value)
> > +                                       uint64_t value, Error **errp)
> >  {
> >      int i;
> >
> > @@ -227,8 +260,10 @@ static void
> tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> >
> >          cmdqv->vintf_config = value;
> >          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> > +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
> >              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> >          } else {
> > +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
> >              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> >          }
> >          break;
> > @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,
> hwaddr offset, uint64_t value,
> >                                   unsigned size)
> >  {
> >      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> > +    Error *local_err = NULL;
> >      int index;
> >
> >      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> > @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,
> hwaddr offset, uint64_t value,
> >          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
> value;
> >          break;
> >      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> > -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> > +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
> >          break;
> >      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> >          /* Same decoding as read() case: See comments above */
> > @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,
> hwaddr offset, uint64_t value,
> >          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
> PRIx64 "\n",
> >                        __func__, offset);
> >      }
> > +
> > +    if (local_err) {
> > +        error_report_err(local_err);
> > +    }
> >  }
> >
> >  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 3/11/26 10:26 AM, Shameer Kolothum Thodi wrote:
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: 11 March 2026 07:55
>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>> arm@nongnu.org; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>> <kjaju@nvidia.com>; phrdina@redhat.com
>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
>> for CMDQV
>>
>> External email: Use caution opening links or attachments
>>
>>
>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
>>> From: Nicolin Chen <nicolinc@nvidia.com>
>>>
>>> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
>>> VINTF pages expose a logical view local to a given VINTF. Although real
>>> hardware may support multiple VINTFs, the kernel currently exposes a
>>> single VINTF per VM.
>>>
>>> The kernel provides an mmap offset for the VINTF Page0 region during
>>> vIOMMU allocation. However, the logical-to-physical association between
>>> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
>>> to that, the mapped Page0 does not back any real VCMDQ state.
>>>
>>> When VINTF is enabled, mmap the kernel provided Page0 region and
>>> unmap it when VINTF is disabled. This prepares the VINTF mapping
>>> in advance of subsequent patches that add VCMDQ allocation support.
>> So at some point we transition from something that is purely emulated
>> (page 1 global cmdq) to something that is mmapped on a host page. How do
>> we transfer the state of the cmdq from one to the other?
> Right. If a guest uses both the "Global VCMDQ registers Page0" and the
> "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
> nothing in the spec that forbids this), then we need to keep the two
> views in sync.

Also assuming the global VCMDQs are accessible and thus used, I guess we
shall properly handle the actual commands (equivalent of
smmuv3_cmdq_consume()), no? 
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I guess
it would be sensible?

Thanks

Eric
>
> So when the mapping between a global VCMDQ and a logical VCMDQ is
> created through the HW_QUEUE ioctl, we should sync the state between
> the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.
>
> I will double check this.
>
> Thanks,
> Shameer
>
>> Thanks
>>
>> Eric
>>> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>> ---
>>>  hw/arm/tegra241-cmdqv.h |  3 +++
>>>  hw/arm/tegra241-cmdqv.c | 44
>> +++++++++++++++++++++++++++++++++++++++--
>>>  2 files changed, 45 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
>>> index d379b8860c..3ce9f539ae 100644
>>> --- a/hw/arm/tegra241-cmdqv.h
>>> +++ b/hw/arm/tegra241-cmdqv.h
>>> @@ -18,6 +18,8 @@
>>>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
>> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>>>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>>>
>>> +#define VINTF_PAGE_SIZE 0x10000
>>> +
>>>  /*
>>>   * Tegra241 CMDQV MMIO layout (64KB pages)
>>>   *
>>> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>>>      SMMUv3AccelState *s_accel;
>>>      MemoryRegion mmio_cmdqv;
>>>      qemu_irq irq;
>>> +    void *vintf_page0;
>>>
>>>      /* Register Cache */
>>>      uint32_t config;
>>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
>>> index e1f1562c44..a3767a85a3 100644
>>> --- a/hw/arm/tegra241-cmdqv.c
>>> +++ b/hw/arm/tegra241-cmdqv.c
>>> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void
>> *opaque, hwaddr offset, unsigned size)
>>>      }
>>>  }
>>>
>>> +static bool
>>> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error
>> **errp)
>>> +{
>>> +    if (!cmdqv->vintf_page0) {
>>> +        return true;
>>> +    }
>>> +
>>> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
>>> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
>>> +        return false;
>>> +    }
>>> +    cmdqv->vintf_page0 = NULL;
>>> +    return true;
>>> +}
>>> +
>>> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV
>> *cmdqv, Error **errp)
>>> +{
>>> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
>>> +
>>> +    if (cmdqv->vintf_page0) {
>>> +        return true;
>>> +    }
>>> +
>>> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
>>> viommu_id,
>>> +                                     VINTF_PAGE_SIZE,
>>> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
>>> +                                     &cmdqv->vintf_page0, errp)) {
>>> +        return false;
>>> +    }
>>> +
>>> +    return true;
>>> +}
>>> +
>>>  /*
>>>   * Write a VCMDQ register using VCMDQ0_* offsets.
>>>   *
>>> @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
>> *cmdqv, hwaddr offset0, int index,
>>>  }
>>>
>>>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr
>> offset,
>>> -                                       uint64_t value)
>>> +                                       uint64_t value, Error **errp)
>>>  {
>>>      int i;
>>>
>>> @@ -227,8 +260,10 @@ static void
>> tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>>>          cmdqv->vintf_config = value;
>>>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
>>> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>>>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>>>          } else {
>>> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>>>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>>>          }
>>>          break;
>>> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,
>> hwaddr offset, uint64_t value,
>>>                                   unsigned size)
>>>  {
>>>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
>>> +    Error *local_err = NULL;
>>>      int index;
>>>
>>>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>>> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,
>> hwaddr offset, uint64_t value,
>>>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
>> value;
>>>          break;
>>>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>>> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
>>> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>>>          break;
>>>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>>>          /* Same decoding as read() case: See comments above */
>>> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,
>> hwaddr offset, uint64_t value,
>>>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
>> PRIx64 "\n",
>>>                        __func__, offset);
>>>      }
>>> +
>>> +    if (local_err) {
>>> +        error_report_err(local_err);
>>> +    }
>>>  }
>>>
>>>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

[Eric Auger]

On 3/10/26 12:37 PM, Shameer Kolothum Thodi wrote:
> Hi Eric,
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: 09 March 2026 16:33
>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>> arm@nongnu.org; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>> <kjaju@nvidia.com>; phrdina@redhat.com
>> Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global
>> CMDQV registers
>>
>> External email: Use caution opening links or attachments
>>
>>
>> Hi Shameer,
>>
>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
>>> From: Nicolin Chen <nicolinc@nvidia.com>
>>>
>>> Tegra241 CMDQV defines a set of global control and status registers
>>> used to configure virtual command queue allocation and interrupt
>>> behavior.
>>>
>>> Add read/write emulation for the global CMDQV register page
>>> (offset 0x00000), backed by a simple register cache. This includes
>>> CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation
>>> map and the VINTF0 related registers defined in the global CMDQV
>>> register space.
>>>
>>> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>> ---
>>>  hw/arm/tegra241-cmdqv.h | 108
>> +++++++++++++++++++++++++++++++++++
>>>  hw/arm/tegra241-cmdqv.c | 121
>> +++++++++++++++++++++++++++++++++++++++-
>>>  2 files changed, 228 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
>>> index 46aa9e8a9f..50bcecee9d 100644
>>> --- a/hw/arm/tegra241-cmdqv.h
>>> +++ b/hw/arm/tegra241-cmdqv.h
>>> @@ -10,6 +10,9 @@
>>>  #ifndef HW_ARM_TEGRA241_CMDQV_H
>>>  #define HW_ARM_TEGRA241_CMDQV_H
>>>
>>> +#include "hw/core/registerfields.h"
>>> +#include "smmuv3-accel.h"
>>> +
>>>  #define TEGRA241_CMDQV_VERSION             1
>>>  #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
>>>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
>> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>>> @@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
>>>      SMMUv3AccelState *s_accel;
>>>      MemoryRegion mmio_cmdqv;
>>>      qemu_irq irq;
>>> +
>>> +    /* Register Cache */
>>> +    uint32_t config;
>>> +    uint32_t param;
>>> +    uint32_t status;
>>> +    uint32_t vi_err_map[2];
>>> +    uint32_t vi_int_mask[2];
>>> +    uint32_t cmdq_err_map[4];
>>> +    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
>>> +    uint32_t vintf_config;
>>> +    uint32_t vintf_status;
>>> +    uint32_t vintf_sid_match[16];
>>> +    uint32_t vintf_sid_replace[16];
>>> +    uint32_t vintf_cmdq_err_map[4];
>>>  } Tegra241CMDQV;
>>>
>>> +/* Global CMDQV MMIO registers (offset 0x00000) */
>>> +REG32(CONFIG, 0x0)
>>> +FIELD(CONFIG, CMDQV_EN, 0, 1)
>>> +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
>>> +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
>>> +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
>>> +FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
>>> +
>>> +REG32(PARAM, 0x4)
>>> +FIELD(PARAM, CMDQV_VER, 0, 4)
>>> +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
>>> +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
>> this is called CMDQV_NUM_VI_LOG2 in the spec I have access to
>> VI = virtual interface
>> I guess there is 1-1 mapping vetween VI and VM but I would keep spec
>> naming
>>> +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
>> same here s/VM/VI
>>> +
>>> +REG32(STATUS, 0x8)
>>> +FIELD(STATUS, CMDQV_ENABLED, 0, 1)
>>> +
>> I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
>>> +#define A_VI_ERR_MAP 0x14
>> _0?
>>> +#define A_VI_ERR_MAP_1 0x18
>>> +#define V_VI_ERR_MAP_NO_ERROR (0)
>>> +#define V_VI_ERR_MAP_ERROR (1)
>>> +
>> same SMMU_CMDQV_VI_ING_MASK0/1
>>> +#define A_VI_INT_MASK 0x1c
>>> +#define A_VI_INT_MASK_1 0x20
>>> +#define V_VI_INT_MASK_NOT_MASKED (0)
>>> +#define V_VI_INT_MASK_MASKED (1)
>>> +
>> SMMU_CMDQV_CMDQ_ERR_MAP0-3
>>> +#define A_CMDQ_ERR_MAP 0x24
>>> +#define A_CMDQ_ERR_MAP_1 0x28
>>> +#define A_CMDQ_ERR_MAP_2 0x2c
>>> +#define A_CMDQ_ERR_MAP_3 0x30
>>> +
>>> +/* i = [0, 1] */
>>> +#define A_CMDQ_ALLOC_MAP_(i)
>> why A_? Since you rely on the REG macros, A_ should prefix the address
>> offset
>> Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in
>> the spec?
> I will incorporate all suggestions related to naming etc and will make it same as
> used in the spec.
>
>>>               \
>>> +    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
>>> +    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
>>> +    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
>>> +    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
>>> +
>> Please explain why we only expose 2 of those regs among 128?
> QEMU currently models only two VCMDQs as we expose that by setting
> TEGRA241_CMDQV_NUM_CMDQ_LOG2 = 1 via SMMU_CMDQV_PARAM_0.
> Therefore, only the corresponding CMDQ allocation map entries are
> implemented. I will add a comment to make it explicit here.
>
>>> +A_CMDQ_ALLOC_MAP_(0)
>>> +A_CMDQ_ALLOC_MAP_(1)
>>> +
>>> +
>>> +/* i = [0, 0] */
>>> +#define A_VINTFi_CONFIG(i)
>> again why don't we use the spec terminology
>> SMMU_CMDQV_VINTF0_CONFIG_0
>>>                   \
>>> +    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
>>> +    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
>>> +    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
>>> +    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
>>> +
>>> +A_VINTFi_CONFIG(0)
>>> +
>>> +#define A_VINTFi_STATUS(i)                       \
>>> +    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
>>> +    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
>>> +    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
>>> +    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
>>> +
>>> +A_VINTFi_STATUS(0)
>>> +
>>> +#define V_VINTF_STATUS_NO_ERROR (0 << 1)
>>> +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
>> Nit: the spec also contains the typo but I would rather fix it in the code
>>
>> Explicitly state we only expose 1 VINTF
>>> +
>>> +/* i = [0, 0], j = [0, 15] */
>> does that mean that we can have a max 16 SIDs per VM?
>>> +#define A_VINTFi_SID_MATCH_(i, j)
>> if matched vi, I would prefer vi
>>>                               \
>>> +    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
>>> +    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
>>> +    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
>>> +
>>> +A_VINTFi_SID_MATCH_(0, 0)
>>> +/* Omitting [0][1~14] as not being directly called */
>>> +A_VINTFi_SID_MATCH_(0, 15)
>>> +
>>> +/* i = [0, 0], j = [0, 15] */
>> vint = 0, 16 identical register entries
>>> +#define A_VINTFi_SID_REPLACE_(i, j)                               \
>>> +    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
>>> +    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
>> s/19/20
>>> +
>>> +A_VINTFi_SID_REPLACE_(0, 0)
>>> +/* Omitting [0][1~14] as not being directly called */
>>> +A_VINTFi_SID_REPLACE_(0, 15)
>>> +
>>> +/* i = [0, 0], j = [0, 3] */
>> vi = 0, 4 identical regs
>>> +#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
>>> +    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
>>> +    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
>>> +
>>> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
>>> +/* Omitting [0][1~2] as not being directly called */
>> I don't get this comment
> the hardware defines four registers ([0..3]). They are handled using
> the range based switch case,
>
> case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>
> And the index to arary is calculated using:
> i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4
>
> I will reword the comment something like:
>
> Only the first and last offsets are declared explicitly since the
> intermediate registers are handled via the range based switch case.
> The register index is derived as:
>    (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4
>
>>> +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
>>> +
>>>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>>>
>>>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
>>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
>>> index d487612ba2..a3830a02d6 100644
>>> --- a/hw/arm/tegra241-cmdqv.c
>>> +++ b/hw/arm/tegra241-cmdqv.c
>>> @@ -8,19 +8,138 @@
>>>   */
>>>
>>>  #include "qemu/osdep.h"
>>> +#include "qemu/log.h"
>>>
>>>  #include "hw/arm/smmuv3.h"
>>>  #include "smmuv3-accel.h"
>>>  #include "tegra241-cmdqv.h"
>>>
>>> +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,
>> hwaddr offset)
>>> +{
>>> +    int i;
>>> +
>>> +    switch (offset) {
>>> +    case A_VINTF0_CONFIG:
>>> +        return cmdqv->vintf_config;
>>> +    case A_VINTF0_STATUS:
>>> +        return cmdqv->vintf_status;
>>> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
>>> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
>>> +        return cmdqv->vintf_sid_match[i];
>>> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
>>> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
>>> +        return cmdqv->vintf_sid_replace[i];
>>> +    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
>> A_VINTF0_LVCMDQ_ERR_MAP_3:
>>> +        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
>>> +        return cmdqv->vintf_cmdq_err_map[i];
>>> +    default:
>>> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
>> PRIx64 "\n",
>>> +                      __func__, offset);
>>> +        return 0;
>>> +    }
>>> +}
>>> +
>>>  static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,
>> unsigned size)
>>>  {
>>> -    return 0;
>>> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
>>> +
>>> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
>>> +        qemu_log_mask(LOG_UNIMP,
>>> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
>>> +                      offset);
>> If all those regs belong to the Global CMDQV registers page this shall
>> be within the first 64KB
> Yes, in this patch that is the case. However, subsequent patches in the
> series will add support for the other page windows as well. Hence the
> full MMIO range check here.
>
>>> +        return 0;
>>> +    }
>>> +
>>> +    switch (offset) {
>>> +    case A_CONFIG:
>>> +        return cmdqv->config;
>>> +    case A_PARAM:
>>> +        return cmdqv->param;
>>> +    case A_STATUS:
>>> +        return cmdqv->status;
>>> +    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
>>> +        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
>>> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
>>> +        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
>>> +    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
>>> +        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
>>> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
>>> +        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /
>> 4];
>>> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>>> +        return tegra241_cmdqv_read_vintf(cmdqv, offset);
>>> +    default:
>>> +        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
>> PRIx64 "\n",
>>> +                      __func__, offset);
>>> +        return 0;
>>> +    }
>>> +}
>>> +
>>> +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr
>> offset,
>>> +                                       uint64_t value)
>>> +{
>>> +    int i;
>>> +
>>> +    switch (offset) {
>>> +    case A_VINTF0_CONFIG:
>>> +        /* Strip off HYP_OWN setting from guest kernel */
>>> +        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
>> Can you explain why this needed?
> The HYP_OWN bit is not guest controllable. It is owned by the host
> kernel/hypervisor and is wired to zero when running in guest kernel.
>
> Please see tegra241_vintf_hw_init() in kernel,
> drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
>
> The guest cannot enable this bit and the effective value seen by the
> guest remains zero.
Well nothing in the spec tells you you cannot set this bit from a guest.

When you allocate a vcmdq for passthrough to a VM you need to set that
bit to 0 for additional checks to be done. When host is asked to prepare
the vcmdq for assignment, on tegra241_cmdqv_init_vintf_user(), hyp_owned
is set to false.

However in usual tegra241_cmdqv_hw_reset() call, hyp_own is set to true.
So I expect the guest to set the bit.

What do I miss?

Eric

>
>>> +
>>> +        cmdqv->vintf_config = value;
>>> +        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
>>> +            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>>> +        } else {
>>> +            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>>> +        }
>>> +        break;
>>> +    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
>>> +        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
>>> +        cmdqv->vintf_sid_match[i] = value;
>>> +        break;
>>> +    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
>>> +        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
>>> +        cmdqv->vintf_sid_replace[i] = value;
>>> +        break;
>>> +    default:
>>> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
>> PRIx64 "\n",
>>> +                      __func__, offset);
>>> +        return;
>>> +    }
>>>  }
>> nit: if you can put write_vintf just after its read fellow, I think it
>> would ease the comparison/review
>>>  static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t
>> value,
>>>                                   unsigned size)
>>>  {
>>> +    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
>>> +
>>> +    if (offset >= TEGRA241_CMDQV_IO_LEN) {
>>> +        qemu_log_mask(LOG_UNIMP,
>>> +                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
>>> +                      offset);
>>> +        return;
>>> +    }
>>> +
>>> +    switch (offset) {
>>> +    case A_CONFIG:
>>> +        cmdqv->config = value;
>>> +        if (value & R_CONFIG_CMDQV_EN_MASK) {
>>> +            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
>>> +        } else {
>>> +            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
>>> +        }
>>> +        break;
>>> +    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
>>> +        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
>>> +        break;
>>> +    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
>>> +        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
>> value;
>>> +        break;
>>> +    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>>> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
>>> +        break;
>>> +    default:
>>> +        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
>> PRIx64 "\n",
>>> +                      __func__, offset);
>>> +    }
>>>  }
>>>
>>>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
>> At this stage of the reading it is unclear to me why we need to expose
>> all those regs to the guest. I would assume that a subset of them are
>> used by the hyp and not supposed to be accessed by the guest. For
>> instance the SID match/replace, ...
>>
>> Please could you clairfy what is absolutely needed to expose to the guest?
> The intention here is to model the architectural register interface
> defined by the CMDQV specification so the guest visible MMIO space
> matches the hardware.
>
> And some of the registers are backed by a simple register cache(like the
> SID match/replace).  Access to the actual hardware state remains controlled
> by QEMU, so guest writes do not necessarily translate to direct hardware
> programming.
>
> Thanks,
> Shameer

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 11 March 2026 10:05
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
> for CMDQV
> 
> External email: Use caution opening links or attachments
> 
> 
> On 3/11/26 10:26 AM, Shameer Kolothum Thodi wrote:
> >
> >> -----Original Message-----
> >> From: Eric Auger <eric.auger@redhat.com>
> >> Sent: 11 March 2026 07:55
> >> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >> arm@nongnu.org; qemu-devel@nongnu.org
> >> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> >> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> >> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> >> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> >> <kjaju@nvidia.com>; phrdina@redhat.com
> >> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
> Page0
> >> for CMDQV
> >>
> >> External email: Use caution opening links or attachments
> >>
> >>
> >> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> >>> From: Nicolin Chen <nicolinc@nvidia.com>
> >>>
> >>> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> >>> VINTF pages expose a logical view local to a given VINTF. Although real
> >>> hardware may support multiple VINTFs, the kernel currently exposes a
> >>> single VINTF per VM.
> >>>
> >>> The kernel provides an mmap offset for the VINTF Page0 region during
> >>> vIOMMU allocation. However, the logical-to-physical association between
> >>> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> >>> to that, the mapped Page0 does not back any real VCMDQ state.
> >>>
> >>> When VINTF is enabled, mmap the kernel provided Page0 region and
> >>> unmap it when VINTF is disabled. This prepares the VINTF mapping
> >>> in advance of subsequent patches that add VCMDQ allocation support.
> >> So at some point we transition from something that is purely emulated
> >> (page 1 global cmdq) to something that is mmapped on a host page. How
> do
> >> we transfer the state of the cmdq from one to the other?
> > Right. If a guest uses both the "Global VCMDQ registers Page0" and the
> > "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
> > nothing in the spec that forbids this), then we need to keep the two
> > views in sync.
> 
> Also assuming the global VCMDQs are accessible and thus used, I guess we
> shall properly handle the actual commands (equivalent of
> smmuv3_cmdq_consume()), no?
> I don't see it done at the moment.
> 
> By the way, do we want support of VCMDQs in fully emulated mode. I guess
> it would be sensible?

Yes, command handling is not implemented now.

Since this is part of the accelerated SMMUv3 feature, I am not sure it
makes sense to implement full command handling in QEMU and then forward
the supported invalidation commands back to the host again.

The idea here is to accelerate the CMDQ processing through the hardware.
I am not sure a guest would have a real use case for VCMDQ in a fully
emulated mode.

Thanks,
Shameer

> Thanks
> 
> Eric
> >
> > So when the mapping between a global VCMDQ and a logical VCMDQ is
> > created through the HW_QUEUE ioctl, we should sync the state between
> > the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.
> >
> > I will double check this.
> >
> > Thanks,
> > Shameer
> >
> >> Thanks
> >>
> >> Eric
> >>> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> >>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> >>> ---
> >>>  hw/arm/tegra241-cmdqv.h |  3 +++
> >>>  hw/arm/tegra241-cmdqv.c | 44
> >> +++++++++++++++++++++++++++++++++++++++--
> >>>  2 files changed, 45 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> >>> index d379b8860c..3ce9f539ae 100644
> >>> --- a/hw/arm/tegra241-cmdqv.h
> >>> +++ b/hw/arm/tegra241-cmdqv.h
> >>> @@ -18,6 +18,8 @@
> >>>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
> >> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
> >>>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
> >>>
> >>> +#define VINTF_PAGE_SIZE 0x10000
> >>> +
> >>>  /*
> >>>   * Tegra241 CMDQV MMIO layout (64KB pages)
> >>>   *
> >>> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
> >>>      SMMUv3AccelState *s_accel;
> >>>      MemoryRegion mmio_cmdqv;
> >>>      qemu_irq irq;
> >>> +    void *vintf_page0;
> >>>
> >>>      /* Register Cache */
> >>>      uint32_t config;
> >>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> >>> index e1f1562c44..a3767a85a3 100644
> >>> --- a/hw/arm/tegra241-cmdqv.c
> >>> +++ b/hw/arm/tegra241-cmdqv.c
> >>> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void
> >> *opaque, hwaddr offset, unsigned size)
> >>>      }
> >>>  }
> >>>
> >>> +static bool
> >>> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,
> Error
> >> **errp)
> >>> +{
> >>> +    if (!cmdqv->vintf_page0) {
> >>> +        return true;
> >>> +    }
> >>> +
> >>> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> >>> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> >>> +        return false;
> >>> +    }
> >>> +    cmdqv->vintf_page0 = NULL;
> >>> +    return true;
> >>> +}
> >>> +
> >>> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV
> >> *cmdqv, Error **errp)
> >>> +{
> >>> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> >>> +
> >>> +    if (cmdqv->vintf_page0) {
> >>> +        return true;
> >>> +    }
> >>> +
> >>> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
> >>> viommu_id,
> >>> +                                     VINTF_PAGE_SIZE,
> >>> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
> >>> +                                     &cmdqv->vintf_page0, errp)) {
> >>> +        return false;
> >>> +    }
> >>> +
> >>> +    return true;
> >>> +}
> >>> +
> >>>  /*
> >>>   * Write a VCMDQ register using VCMDQ0_* offsets.
> >>>   *
> >>> @@ -216,7 +249,7 @@
> tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
> >> *cmdqv, hwaddr offset0, int index,
> >>>  }
> >>>
> >>>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,
> hwaddr
> >> offset,
> >>> -                                       uint64_t value)
> >>> +                                       uint64_t value, Error **errp)
> >>>  {
> >>>      int i;
> >>>
> >>> @@ -227,8 +260,10 @@ static void
> >> tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> >>>          cmdqv->vintf_config = value;
> >>>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> >>> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
> >>>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
> >>>          } else {
> >>> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
> >>>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
> >>>          }
> >>>          break;
> >>> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,
> >> hwaddr offset, uint64_t value,
> >>>                                   unsigned size)
> >>>  {
> >>>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> >>> +    Error *local_err = NULL;
> >>>      int index;
> >>>
> >>>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> >>> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,
> >> hwaddr offset, uint64_t value,
> >>>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
> >> value;
> >>>          break;
> >>>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> >>> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> >>> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
> >>>          break;
> >>>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> >>>          /* Same decoding as read() case: See comments above */
> >>> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void
> *opaque,
> >> hwaddr offset, uint64_t value,
> >>>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
> >> PRIx64 "\n",
> >>>                        __func__, offset);
> >>>      }
> >>> +
> >>> +    if (local_err) {
> >>> +        error_report_err(local_err);
> >>> +    }
> >>>  }
> >>>
> >>>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 3/11/26 1:34 PM, Shameer Kolothum Thodi wrote:
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: 11 March 2026 10:05
>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>> arm@nongnu.org; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>> <kjaju@nvidia.com>; phrdina@redhat.com
>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
>> for CMDQV
>>
>> External email: Use caution opening links or attachments
>>
>>
>> On 3/11/26 10:26 AM, Shameer Kolothum Thodi wrote:
>>>> -----Original Message-----
>>>> From: Eric Auger <eric.auger@redhat.com>
>>>> Sent: 11 March 2026 07:55
>>>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
>>>> arm@nongnu.org; qemu-devel@nongnu.org
>>>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
>>>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
>>>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
>>>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
>>>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
>>>> <kjaju@nvidia.com>; phrdina@redhat.com
>>>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
>> Page0
>>>> for CMDQV
>>>>
>>>> External email: Use caution opening links or attachments
>>>>
>>>>
>>>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
>>>>> From: Nicolin Chen <nicolinc@nvidia.com>
>>>>>
>>>>> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
>>>>> VINTF pages expose a logical view local to a given VINTF. Although real
>>>>> hardware may support multiple VINTFs, the kernel currently exposes a
>>>>> single VINTF per VM.
>>>>>
>>>>> The kernel provides an mmap offset for the VINTF Page0 region during
>>>>> vIOMMU allocation. However, the logical-to-physical association between
>>>>> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
>>>>> to that, the mapped Page0 does not back any real VCMDQ state.
>>>>>
>>>>> When VINTF is enabled, mmap the kernel provided Page0 region and
>>>>> unmap it when VINTF is disabled. This prepares the VINTF mapping
>>>>> in advance of subsequent patches that add VCMDQ allocation support.
>>>> So at some point we transition from something that is purely emulated
>>>> (page 1 global cmdq) to something that is mmapped on a host page. How
>> do
>>>> we transfer the state of the cmdq from one to the other?
>>> Right. If a guest uses both the "Global VCMDQ registers Page0" and the
>>> "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
>>> nothing in the spec that forbids this), then we need to keep the two
>>> views in sync.
>> Also assuming the global VCMDQs are accessible and thus used, I guess we
>> shall properly handle the actual commands (equivalent of
>> smmuv3_cmdq_consume()), no?
>> I don't see it done at the moment.
>>
>> By the way, do we want support of VCMDQs in fully emulated mode. I guess
>> it would be sensible?
> Yes, command handling is not implemented now.
>
> Since this is part of the accelerated SMMUv3 feature, I am not sure it
> makes sense to implement full command handling in QEMU and then forward
> the supported invalidation commands back to the host again.
>
> The idea here is to accelerate the CMDQ processing through the hardware.
> I am not sure a guest would have a real use case for VCMDQ in a fully
> emulated mode.

OK but really the whole user model is really confusing. One starts using
a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are
not. Also when the guest is using the global vcmdq, it can
access the registers but they actually do not trigger any commands. This
really needs to be clarified.

Then I understand we want to focus on accel mode but above points need
to be clatified. I just wanted to mention that vcmdq should logically
also work in fully emulated mode (and you seemed to implement a mix
actually when you have emulated access to vi vcmdq) as it replaces the
SMMU standard cmdq, if I understand correctly

Eric


>
> Thanks,
> Shameer
>
>> Thanks
>>
>> Eric
>>> So when the mapping between a global VCMDQ and a logical VCMDQ is
>>> created through the HW_QUEUE ioctl, we should sync the state between
>>> the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.
>>>
>>> I will double check this.
>>>
>>> Thanks,
>>> Shameer
>>>
>>>> Thanks
>>>>
>>>> Eric
>>>>> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
>>>>> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
>>>>> ---
>>>>>  hw/arm/tegra241-cmdqv.h |  3 +++
>>>>>  hw/arm/tegra241-cmdqv.c | 44
>>>> +++++++++++++++++++++++++++++++++++++++--
>>>>>  2 files changed, 45 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
>>>>> index d379b8860c..3ce9f539ae 100644
>>>>> --- a/hw/arm/tegra241-cmdqv.h
>>>>> +++ b/hw/arm/tegra241-cmdqv.h
>>>>> @@ -18,6 +18,8 @@
>>>>>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<
>>>> TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>>>>>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>>>>>
>>>>> +#define VINTF_PAGE_SIZE 0x10000
>>>>> +
>>>>>  /*
>>>>>   * Tegra241 CMDQV MMIO layout (64KB pages)
>>>>>   *
>>>>> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>>>>>      SMMUv3AccelState *s_accel;
>>>>>      MemoryRegion mmio_cmdqv;
>>>>>      qemu_irq irq;
>>>>> +    void *vintf_page0;
>>>>>
>>>>>      /* Register Cache */
>>>>>      uint32_t config;
>>>>> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
>>>>> index e1f1562c44..a3767a85a3 100644
>>>>> --- a/hw/arm/tegra241-cmdqv.c
>>>>> +++ b/hw/arm/tegra241-cmdqv.c
>>>>> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void
>>>> *opaque, hwaddr offset, unsigned size)
>>>>>      }
>>>>>  }
>>>>>
>>>>> +static bool
>>>>> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,
>> Error
>>>> **errp)
>>>>> +{
>>>>> +    if (!cmdqv->vintf_page0) {
>>>>> +        return true;
>>>>> +    }
>>>>> +
>>>>> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
>>>>> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
>>>>> +        return false;
>>>>> +    }
>>>>> +    cmdqv->vintf_page0 = NULL;
>>>>> +    return true;
>>>>> +}
>>>>> +
>>>>> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV
>>>> *cmdqv, Error **errp)
>>>>> +{
>>>>> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
>>>>> +
>>>>> +    if (cmdqv->vintf_page0) {
>>>>> +        return true;
>>>>> +    }
>>>>> +
>>>>> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
>>>>> viommu_id,
>>>>> +                                     VINTF_PAGE_SIZE,
>>>>> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
>>>>> +                                     &cmdqv->vintf_page0, errp)) {
>>>>> +        return false;
>>>>> +    }
>>>>> +
>>>>> +    return true;
>>>>> +}
>>>>> +
>>>>>  /*
>>>>>   * Write a VCMDQ register using VCMDQ0_* offsets.
>>>>>   *
>>>>> @@ -216,7 +249,7 @@
>> tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
>>>> *cmdqv, hwaddr offset0, int index,
>>>>>  }
>>>>>
>>>>>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,
>> hwaddr
>>>> offset,
>>>>> -                                       uint64_t value)
>>>>> +                                       uint64_t value, Error **errp)
>>>>>  {
>>>>>      int i;
>>>>>
>>>>> @@ -227,8 +260,10 @@ static void
>>>> tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>>>>>          cmdqv->vintf_config = value;
>>>>>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
>>>>> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>>>>>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>>>>>          } else {
>>>>> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>>>>>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>>>>>          }
>>>>>          break;
>>>>> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,
>>>> hwaddr offset, uint64_t value,
>>>>>                                   unsigned size)
>>>>>  {
>>>>>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
>>>>> +    Error *local_err = NULL;
>>>>>      int index;
>>>>>
>>>>>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>>>>> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,
>>>> hwaddr offset, uint64_t value,
>>>>>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =
>>>> value;
>>>>>          break;
>>>>>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>>>>> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
>>>>> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>>>>>          break;
>>>>>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>>>>>          /* Same decoding as read() case: See comments above */
>>>>> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void
>> *opaque,
>>>> hwaddr offset, uint64_t value,
>>>>>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"
>>>> PRIx64 "\n",
>>>>>                        __func__, offset);
>>>>>      }
>>>>> +
>>>>> +    if (local_err) {
>>>>> +        error_report_err(local_err);
>>>>> +    }
>>>>>  }
>>>>>
>>>>>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> This is the write side counterpart of the VCMDQ read emulation.
>
> Add write handling for global VCMDQ and VI_VCMDQ register windows.
> Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to
> VCMDQ0_* offsets, reusing the same layout assumptions as the read path.
>
> VI_VCMDQ registers are treated as a logical alias of the global VCMDQ
> registers and share the same decoding logic.
>
> Writes are backed by cached register state only; no hardware queue
> mapping is performed yet.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 83 insertions(+)
>
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index d2e6938e44..e1f1562c44 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +/*
> + * Write a VCMDQ register using VCMDQ0_* offsets.
> + *
> + * The caller normalizes the MMIO offset such that @offset0 always refers
> + * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> + */
> +static void
> +tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
> +                           uint64_t value, unsigned size)
> +{
> +    switch (offset0) {
> +    case A_VCMDQ0_CONS_INDX:
> +        cmdqv->vcmdq_cons_indx[index] = value;
> +        return;
> +    case A_VCMDQ0_PROD_INDX:
> +        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
> +        return;
> +    case A_VCMDQ0_CONFIG:
> +        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
> +            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
> +        } else {
> +            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
> +        }
> +        cmdqv->vcmdq_config[index] = (uint32_t)value;
> +        return;
> +    case A_VCMDQ0_GERRORN:
> +        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
> +        return;
> +    case A_VCMDQ0_BASE_L:
> +        if (size == 8) {
> +            cmdqv->vcmdq_base[index] = value;
> +        } else if (size == 4) {
> +            cmdqv->vcmdq_base[index] =
> +                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
> +                (value & 0xffffffffULL);
> +        }
If we compare to smmuv3 std command queue API, we miss some checks. See
smmu_writel()
        if (!smmu_cmdq_base_writable(s, reg_sec_sid)) {
for CONS
        if (!smmu_cmdq_disabled_stable(s, reg_sec_sid)) {

In the spec I have access to I failed to find any details about
equivalent restrictions

Can you check if those write accesses are always allowed?

Thanks

Eric

> +        return;
> +    case A_VCMDQ0_BASE_H:
> +        cmdqv->vcmdq_base[index] =
> +            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
> +            ((uint64_t)value << 32);
> +        return;
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
> +        if (size == 8) {
> +            cmdqv->vcmdq_cons_indx_base[index] = value;
> +        } else if (size == 4) {
> +            cmdqv->vcmdq_cons_indx_base[index] =
> +                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
> +                (value & 0xffffffffULL);
> +        }
> +        return;
> +    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
> +        cmdqv->vcmdq_cons_indx_base[index] =
> +            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
> +            ((uint64_t)value << 32);
> +        return;
> +    default:
> +        qemu_log_mask(LOG_UNIMP,
> +                      "%s unhandled write access at 0x%" PRIx64 "\n",
> +                      __func__, offset0);
> +        return;
> +    }
> +}
> +
>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>                                         uint64_t value)
>  {
> @@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
>          qemu_log_mask(LOG_UNIMP,
> @@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
>          tegra241_cmdqv_write_vintf(cmdqv, offset, value);
>          break;
> +    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
> +        /* Same decoding as read() case: See comments above */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
> +        index = (offset - 0x10000) / 0x80;
> +        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
> +                                   size);
> +        break;
> +    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        /* Same decoding as read() case: See comments above */
> +        offset -= 0x20000;
> +        QEMU_FALLTHROUGH;
> +    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
> +        index = (offset - 0x20000) / 0x80;
> +        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
> +                                   size);
> +        break;
>      default:
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
>                        __func__, offset);

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> VINTF pages expose a logical view local to a given VINTF. Although real
> hardware may support multiple VINTFs, the kernel currently exposes a
> single VINTF per VM.
>
> The kernel provides an mmap offset for the VINTF Page0 region during
> vIOMMU allocation. However, the logical-to-physical association between
> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> to that, the mapped Page0 does not back any real VCMDQ state.
So what does happen if the guest attempts to access the VI CMDQ before
its mmapping.

At the moment MMIO accesses are valid as VI_VCMDQ0 MMIOs are handled in
the same MMIO region as the Config page. 
Is it the expected behavior? I guess you should signal an error and
maaybe signal an interrupt if it is not masked for this VI?

Spec says:"While the SW can program the virtual CMDQ(s) directly using
the direct VCMDQ aperture (and not though the Virtual Interface), it is
required hat the VCMDQ be allocated to a VI before it is used to send
commands to the SMMU

Note the terminology used here: direct VCMDQ aparture (not Global)

Thanks

Eric
>
> When VINTF is enabled, mmap the kernel provided Page0 region and
> unmap it when VINTF is disabled. This prepares the VINTF mapping
> in advance of subsequent patches that add VCMDQ allocation support.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h |  3 +++
>  hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 45 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index d379b8860c..3ce9f539ae 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -18,6 +18,8 @@
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>  
> +#define VINTF_PAGE_SIZE 0x10000
> +
>  /*
>   * Tegra241 CMDQV MMIO layout (64KB pages)
>   *
> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +    void *vintf_page0;
>  
>      /* Register Cache */
>      uint32_t config;
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index e1f1562c44..a3767a85a3 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +static bool
> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    if (!cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> +        return false;
> +    }
> +    cmdqv->vintf_page0 = NULL;
> +    return true;
> +}
> +
> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> +
> +    if (cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
> +                                     VINTF_PAGE_SIZE,
> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
> +                                     &cmdqv->vintf_page0, errp)) {
> +        return false;
> +    }
> +
> +    return true;
> +}
> +
>  /*
>   * Write a VCMDQ register using VCMDQ0_* offsets.
>   *
> @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>  }
>  
>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> -                                       uint64_t value)
> +                                       uint64_t value, Error **errp)
>  {
>      int i;
>  
> @@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>  
>          cmdqv->vintf_config = value;
>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>          } else {
> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>          }
>          break;
> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    Error *local_err = NULL;
>      int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
>          break;
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>          break;
>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>          /* Same decoding as read() case: See comments above */
> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
>                        __func__, offset);
>      }
> +
> +    if (local_err) {
> +        error_report_err(local_err);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> VINTF pages expose a logical view local to a given VINTF. Although real
> hardware may support multiple VINTFs, the kernel currently exposes a
> single VINTF per VM.
>
> The kernel provides an mmap offset for the VINTF Page0 region during
> vIOMMU allocation. However, the logical-to-physical association between
> VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior
> to that, the mapped Page0 does not back any real VCMDQ state.
>
> When VINTF is enabled, mmap the kernel provided Page0 region and
> unmap it when VINTF is disabled. This prepares the VINTF mapping
> in advance of subsequent patches that add VCMDQ allocation support.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h |  3 +++
>  hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 45 insertions(+), 2 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index d379b8860c..3ce9f539ae 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -18,6 +18,8 @@
>  #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
>  #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
>  
> +#define VINTF_PAGE_SIZE 0x10000
> +
>  /*
>   * Tegra241 CMDQV MMIO layout (64KB pages)
>   *
> @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +    void *vintf_page0;
>  
>      /* Register Cache */
>      uint32_t config;
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index e1f1562c44..a3767a85a3 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +static bool
> +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    if (!cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
> +        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
> +        return false;
> +    }
> +    cmdqv->vintf_page0 = NULL;
> +    return true;
> +}
> +
> +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
> +{
> +    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
> +
> +    if (cmdqv->vintf_page0) {
> +        return true;
> +    }
> +
> +    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
> +                                     VINTF_PAGE_SIZE,
> +                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
If I am not wrong the mmap_offset is only available after
iommufd_backend_alloc_hw_queue(). So I would suggest to first introduce
the patch that does the iommufd_backend_alloc_hw_queue (ie. 19/32) and
then this one

Eric
> +                                     &cmdqv->vintf_page0, errp)) {
> +        return false;
> +    }
> +
> +    return true;
> +}
> +
>  /*
>   * Write a VCMDQ register using VCMDQ0_* offsets.
>   *
> @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>  }
>  
>  static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
> -                                       uint64_t value)
> +                                       uint64_t value, Error **errp)
>  {
>      int i;
>  
> @@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>  
>          cmdqv->vintf_config = value;
>          if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
> +            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>          } else {
> +            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>          }
>          break;
> @@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>                                   unsigned size)
>  {
>      Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
> +    Error *local_err = NULL;
>      int index;
>  
>      if (offset >= TEGRA241_CMDQV_IO_LEN) {
> @@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
>          break;
>      case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
> -        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
> +        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
>          break;
>      case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
>          /* Same decoding as read() case: See comments above */
> @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
>                        __func__, offset);
>      }
> +
> +    if (local_err) {
> +        error_report_err(local_err);
> +    }
>  }
>  
>  static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 11 March 2026 13:19
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
> for CMDQV
> 
> External email: Use caution opening links or attachments
> 
> 
> On 3/11/26 1:34 PM, Shameer Kolothum Thodi wrote:
> >
> >> -----Original Message-----
> >> From: Eric Auger <eric.auger@redhat.com>
> >> Sent: 11 March 2026 10:05
> >> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >> arm@nongnu.org; qemu-devel@nongnu.org
> >> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> >> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> >> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> >> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> >> <kjaju@nvidia.com>; phrdina@redhat.com
> >> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
> Page0
> >> for CMDQV
> >>
> >> External email: Use caution opening links or attachments
> >>
> >>
> >> On 3/11/26 10:26 AM, Shameer Kolothum Thodi wrote:
> >>>> -----Original Message-----
> >>>> From: Eric Auger <eric.auger@redhat.com>
> >>>> Sent: 11 March 2026 07:55
> >>>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >>>> arm@nongnu.org; qemu-devel@nongnu.org
> >>>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org;
> Nicolin
> >>>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>;
> Matt
> >>>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason
> Gunthorpe
> >>>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >>>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> >>>> <kjaju@nvidia.com>; phrdina@redhat.com
> >>>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
> >> Page0
> >>>> for CMDQV
> >>>>
> >>>> External email: Use caution opening links or attachments
> >>>>
> >>>>
> >>>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> >>>>> From: Nicolin Chen <nicolinc@nvidia.com>
> >>>>>
> >>>>> Global VCMDQ pages provide a VM wide view of all VCMDQs, while the
> >>>>> VINTF pages expose a logical view local to a given VINTF. Although real
> >>>>> hardware may support multiple VINTFs, the kernel currently exposes a
> >>>>> single VINTF per VM.
> >>>>>
> >>>>> The kernel provides an mmap offset for the VINTF Page0 region during
> >>>>> vIOMMU allocation. However, the logical-to-physical association
> between
> >>>>> VCMDQs and a VINTF is only established after HW_QUEUE allocation.
> Prior
> >>>>> to that, the mapped Page0 does not back any real VCMDQ state.
> >>>>>
> >>>>> When VINTF is enabled, mmap the kernel provided Page0 region and
> >>>>> unmap it when VINTF is disabled. This prepares the VINTF mapping
> >>>>> in advance of subsequent patches that add VCMDQ allocation support.
> >>>> So at some point we transition from something that is purely emulated
> >>>> (page 1 global cmdq) to something that is mmapped on a host page.
> How
> >> do
> >>>> we transfer the state of the cmdq from one to the other?
> >>> Right. If a guest uses both the "Global VCMDQ registers Page0" and the
> >>> "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
> >>> nothing in the spec that forbids this), then we need to keep the two
> >>> views in sync.
> >> Also assuming the global VCMDQs are accessible and thus used, I guess we
> >> shall properly handle the actual commands (equivalent of
> >> smmuv3_cmdq_consume()), no?
> >> I don't see it done at the moment.
> >>
> >> By the way, do we want support of VCMDQs in fully emulated mode. I
> guess
> >> it would be sensible?
> > Yes, command handling is not implemented now.
> >
> > Since this is part of the accelerated SMMUv3 feature, I am not sure it
> > makes sense to implement full command handling in QEMU and then
> forward
> > the supported invalidation commands back to the host again.
> >
> > The idea here is to accelerate the CMDQ processing through the hardware.
> > I am not sure a guest would have a real use case for VCMDQ in a fully
> > emulated mode.
> 
> OK but really the whole user model is really confusing. One starts using
> a "global" vcmdq and then switches to a logical one.
> On real HW I guess they are synced. But on emulation, currently they are
> not. Also when the guest is using the global vcmdq, it can
> access the registers but they actually do not trigger any commands. This
> really needs to be clarified.

Yes, this could use better documentation. I can clarify the model in the
commit message and code comments.

One option we discussed earlier to avoid the confusion was to restrict
usage to the "VINTF logical VCMDQ" interface and return an error if the
guest accesses the "Global VCMDQ registers".

However, since this is still MMIO space, the thinking was that it may be
better to allow the access even if it is effectively non-functional, rather
than making the behaviour unpredictable.

See the discussion:
https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/

> Then I understand we want to focus on accel mode but above points need
> to be clatified. I just wanted to mention that vcmdq should logically
> also work in fully emulated mode (and you seemed to implement a mix
> actually when you have emulated access to vi vcmdq) as it replaces the
> SMMU standard cmdq, if I understand correctly

IIUC, in the current Linux implementation VCMDQ does not replace the
standard SMMU CMDQ from the guest perspective. This is also why we do
not expose the HYP_OWN bit to the guest.

When HYP_OWN is not set, guest still uses the normal SMMU CMDQ. 
CMDQ is only used as a fast-path for a small set of invalidation commands
that can be sent directly to the host hardware.

See:

static bool tegra241_guest_vcmdq_supports_cmd(struct arm_smmu_cmdq_ent *ent)
{
        switch (ent->opcode) {
        case CMDQ_OP_TLBI_NH_ASID:
        case CMDQ_OP_TLBI_NH_VA:
        case CMDQ_OP_ATC_INV:
                return true;
        default:
                return false;
        }
}

So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is
only used to accelerate these invalidation commands.

Thanks,
Shameer

Re: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> From: Nicolin Chen <nicolinc@nvidia.com>
>
> Add support for allocating IOMMUFD hardware queues when the guest
> programs the VCMDQ BASE registers.
>
> VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed
> through the VINTF Page0 region. This region is mapped directly into
> the guest address space (introduced in a subsequent patch), so QEMU
> does not trap writes to VCMDQ_CONFIG.
>
> Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the
> hardware queue based on that bit. Instead, allocate the IOMMUFD
> hardware queue when the guest writes a VCMDQ BASE register with a
> valid RAM-backed address and when CMDQV and VINTF are enabled.
>
> If a hardware queue was previously allocated for the same VCMDQ,
> free it before reallocation.
>
> Writes with invalid addresses are ignored.
>
> All allocated VCMDQs are freed when CMDQV or VINTF is disabled.
>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.h | 11 +++++++
>  hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 79 insertions(+), 3 deletions(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.h b/hw/arm/tegra241-cmdqv.h
> index 3ce9f539ae..139e14b61b 100644
> --- a/hw/arm/tegra241-cmdqv.h
> +++ b/hw/arm/tegra241-cmdqv.h
> @@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {
>      SMMUv3AccelState *s_accel;
>      MemoryRegion mmio_cmdqv;
>      qemu_irq irq;
> +    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
>      void *vintf_page0;
>  
>      /* Register Cache */
> @@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
>  A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
>  A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
>  
> +static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdq)
> +{
> +    return cmdq->status & R_STATUS_CMDQV_ENABLED_MASK;
> +}
> +
> +static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdq)
> +{
> +    return cmdq->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
> +}
> +
>  const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
>  
>  #endif /* HW_ARM_TEGRA241_CMDQV_H */
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index a3767a85a3..002dde50fc 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
>      }
>  }
>  
> +static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
> +{
> +    SMMUv3AccelState *accel = cmdqv->s_accel;
> +    IOMMUFDViommu *viommu = accel->viommu;
> +    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
> +
> +    if (!vcmdq) {
> +        return;
> +    }
> +    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
> +    g_free(vcmdq);
> +    cmdqv->vcmdq[index] = NULL;
> +}
> +
> +static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
> +{
> +    /* Free in the reverse order to avoid "resource busy" error */
> +    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
Is it future proof to restrict the number of queues to 2? Shouldn't it
be configurable?

Eric
> +        tegra241_cmdqv_free_vcmdq(cmdqv, i);
> +    }
> +}
> +
> +static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
> +                                       Error **errp)
> +{
> +    SMMUv3AccelState *accel = cmdqv->s_accel;
> +    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
> +                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
> +    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
> +    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
> +    uint64_t size = 1ULL << (log2 + 4);
> +    IOMMUFDViommu *viommu = accel->viommu;
> +    IOMMUFDHWqueue *hw_queue;
> +    uint32_t hw_queue_id;
> +
> +    /* Ignore any invalid address. This may come as part of reset etc */
> +    if (!address_space_is_ram(&address_space_memory, addr) ||
> +        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
> +        return true;
> +    }
> +
> +    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
> +        return true;
> +    }
> +
> +    tegra241_cmdqv_free_vcmdq(cmdqv, index);
> +
> +    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
> +                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
> +                                        index, addr, size, &hw_queue_id,
> +                                        errp)) {
> +        return false;
> +    }
> +    hw_queue = g_new(IOMMUFDHWqueue, 1);
> +    hw_queue->hw_queue_id = hw_queue_id;
> +    hw_queue->viommu = viommu;
> +    cmdqv->vcmdq[index] = hw_queue;
> +
> +    return true;
> +}
> +
>  static bool
>  tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
>  {
> @@ -192,7 +253,7 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
>   */
>  static void
>  tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
> -                           uint64_t value, unsigned size)
> +                           uint64_t value, unsigned size, Error **errp)
>  {
>      switch (offset0) {
>      case A_VCMDQ0_CONS_INDX:
> @@ -220,11 +281,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>                  (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
>                  (value & 0xffffffffULL);
>          }
> +        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
>          return;
>      case A_VCMDQ0_BASE_H:
>          cmdqv->vcmdq_base[index] =
>              (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
>              ((uint64_t)value << 32);
> +        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
>          return;
>      case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
>          if (size == 8) {
> @@ -263,6 +326,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
>              tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
>          } else {
> +            tegra241_cmdqv_free_all_vcmdq(cmdqv);
>              tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
>              cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
>          }
> @@ -302,6 +366,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>          if (value & R_CONFIG_CMDQV_EN_MASK) {
>              cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
>          } else {
> +            tegra241_cmdqv_free_all_vcmdq(cmdqv);
>              cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
>          }
>          break;
> @@ -321,7 +386,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>      case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
>          index = (offset - 0x10000) / 0x80;
>          tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
> -                                   size);
> +                                   size, &local_err);
>          break;
>      case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
>          /* Same decoding as read() case: See comments above */
> @@ -330,7 +395,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
>      case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
>          index = (offset - 0x20000) / 0x80;
>          tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
> -                                   size);
> +                                   size, &local_err);
>          break;
>      default:
>          qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",

Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

[Eric Auger]

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the
> kernel, access the VCMDQ registers directly through the VINTF page0
> backing instead of using QEMU's cached register state.
>
> VINTF page0 provides the backing memory region for VCMDQ registers
> once a hardware queue is created. In that case, reads and writes
> should reflect the live backing state.
>
> If a VCMDQ is not allocated, or if VINTF page0 is not available,
> continue to use the cached register values maintained by QEMU.
But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV.
Somehow this means that you have 3 states of the same CMDQV: the
global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV (host
backed).
Is this understanding correct?

Thanks

Eric
>
> Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> ---
>  hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 46 insertions(+), 1 deletion(-)
>
> diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> index 002dde50fc..17b9552906 100644
> --- a/hw/arm/tegra241-cmdqv.c
> +++ b/hw/arm/tegra241-cmdqv.c
> @@ -14,17 +14,44 @@
>  #include "smmuv3-accel.h"
>  #include "tegra241-cmdqv.h"
>  
> +static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
> +                                                 int index, hwaddr offset0)
> +{
> +    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
> +        return NULL;
> +    }
> +
> +    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
> +                        (offset0 - 0x10000));
> +}
>  /*
>   * Read a VCMDQ register using VCMDQ0_* offsets.
>   *
>   * The caller normalizes the MMIO offset such that @offset0 always refers
>   * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
>   *
> - * All VCMDQ accesses return cached registers.
> + * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
> + * from the VINTF page0 backing. Otherwise, fall back to cached state.
>   */
>  static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
>                                            int index)
>  {
> +    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
> +
> +    if (ptr) {
> +        switch (offset0) {
> +        case A_VCMDQ0_CONS_INDX:
> +        case A_VCMDQ0_PROD_INDX:
> +        case A_VCMDQ0_CONFIG:
> +        case A_VCMDQ0_STATUS:
> +        case A_VCMDQ0_GERROR:
> +        case A_VCMDQ0_GERRORN:
> +            return *ptr;
> +        default:
> +            break;
> +        }
> +    }
> +
>      switch (offset0) {
>      case A_VCMDQ0_CONS_INDX:
>          return cmdqv->vcmdq_cons_indx[index];
> @@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
>   *
>   * The caller normalizes the MMIO offset such that @offset0 always refers
>   * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> + *
> + * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
> + * to the VINTF page0 backing. Otherwise, update cached state.
>   */
>  static void
>  tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
>                             uint64_t value, unsigned size, Error **errp)
>  {
> +    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
> +
> +    if (ptr) {
> +        switch (offset0) {
> +        case A_VCMDQ0_CONS_INDX:
> +        case A_VCMDQ0_PROD_INDX:
> +        case A_VCMDQ0_CONFIG:
> +        case A_VCMDQ0_GERRORN:
> +            *ptr = (uint32_t)value;
> +            return;
> +        default:
> +            break;
> +        }
> +    }
> +
>      switch (offset0) {
>      case A_VCMDQ0_CONS_INDX:
>          cmdqv->vcmdq_cons_indx[index] = value;

RE: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 11 March 2026 14:53
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed
> VINTF page0 as VCMDQ backing
> 
> External email: Use caution opening links or attachments
> 
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the
> > kernel, access the VCMDQ registers directly through the VINTF page0
> > backing instead of using QEMU's cached register state.
> >
> > VINTF page0 provides the backing memory region for VCMDQ registers
> > once a hardware queue is created. In that case, reads and writes
> > should reflect the live backing state.
> >
> > If a VCMDQ is not allocated, or if VINTF page0 is not available,
> > continue to use the cached register values maintained by QEMU.
> But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV.
> Somehow this means that you have 3 states of the same CMDQV: the
> global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV
> (host
> backed).
> Is this understanding correct?

Yes. However, in the current implementation the VCMDQ is functionally
valid only if QEMU successfully sets up the VINTF page0 mmap and creates
a backing VCMDQ through the HW_QUEUE ioctl.

All other cases it just uses the cached registers for MMIO read/write.

Maybe we should expose such failures to the guest, by injecting an error
or by setting STATUS/CONFIG to indicate that the VCMDQ is not functional.

Regarding keeping the Global VCMDQ and the VI CMDQ in sync always, this
would only be possible if we do not map (tegra241_cmdqv_guest_map_vintf_page0)
the VINTF Page0 into guest memory. Once it is mapped, accesses to the 
VI CMDQ (control/status registers) no longer trap to QEMU.

However, not mapping it is not advisable as it would hurt performance,
since every CONS/PROD index update would trap to QEMU.

Thanks,
Shameer

> Thanks
> 
> Eric
> >
> > Signed-off-by: Shameer Kolothum <skolothumtho@nvidia.com>
> > ---
> >  hw/arm/tegra241-cmdqv.c | 47
> ++++++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 46 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
> > index 002dde50fc..17b9552906 100644
> > --- a/hw/arm/tegra241-cmdqv.c
> > +++ b/hw/arm/tegra241-cmdqv.c
> > @@ -14,17 +14,44 @@
> >  #include "smmuv3-accel.h"
> >  #include "tegra241-cmdqv.h"
> >
> > +static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV
> *cmdqv,
> > +                                                 int index, hwaddr offset0)
> > +{
> > +    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
> > +        return NULL;
> > +    }
> > +
> > +    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
> > +                        (offset0 - 0x10000));
> > +}
> >  /*
> >   * Read a VCMDQ register using VCMDQ0_* offsets.
> >   *
> >   * The caller normalizes the MMIO offset such that @offset0 always refers
> >   * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> >   *
> > - * All VCMDQ accesses return cached registers.
> > + * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
> > + * from the VINTF page0 backing. Otherwise, fall back to cached state.
> >   */
> >  static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,
> hwaddr offset0,
> >                                            int index)
> >  {
> > +    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
> > +
> > +    if (ptr) {
> > +        switch (offset0) {
> > +        case A_VCMDQ0_CONS_INDX:
> > +        case A_VCMDQ0_PROD_INDX:
> > +        case A_VCMDQ0_CONFIG:
> > +        case A_VCMDQ0_STATUS:
> > +        case A_VCMDQ0_GERROR:
> > +        case A_VCMDQ0_GERRORN:
> > +            return *ptr;
> > +        default:
> > +            break;
> > +        }
> > +    }
> > +
> >      switch (offset0) {
> >      case A_VCMDQ0_CONS_INDX:
> >          return cmdqv->vcmdq_cons_indx[index];
> > @@ -250,11 +277,29 @@ static bool
> tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error
> **errp)
> >   *
> >   * The caller normalizes the MMIO offset such that @offset0 always refers
> >   * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
> > + *
> > + * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
> > + * to the VINTF page0 backing. Otherwise, update cached state.
> >   */
> >  static void
> >  tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
> int index,
> >                             uint64_t value, unsigned size, Error **errp)
> >  {
> > +    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
> > +
> > +    if (ptr) {
> > +        switch (offset0) {
> > +        case A_VCMDQ0_CONS_INDX:
> > +        case A_VCMDQ0_PROD_INDX:
> > +        case A_VCMDQ0_CONFIG:
> > +        case A_VCMDQ0_GERRORN:
> > +            *ptr = (uint32_t)value;
> > +            return;
> > +        default:
> > +            break;
> > +        }
> > +    }
> > +
> >      switch (offset0) {
> >      case A_VCMDQ0_CONS_INDX:
> >          cmdqv->vcmdq_cons_indx[index] = value;

Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[Eric Auger]

Hi Shameer,

On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> Hi,
>
> Changes from v2:
>  https://lore.kernel.org/qemu-devel/20260206144823.80655-1-skolothumtho@nvidia.com/
>
>   - Addressed feedback on v2 and picked up R-by tags. Thanks!
>   - Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto
>     "cmdqv".
>   - Introduced a probe() callback to detect and initialize CMDQV
>     support during device attach (patch #7).
>   - Since CMDQV initialization is now deferred to device attach time,
>     added a helper to link any unmapped MMIO/IRQ resources (patch #12).
>   - Moved VINTF enable handling to the CONFIG ENABLE bit write path
>     instead of any read/write access path (patch #17). Hopefully this
>     won't break any corner cases.
>   - Added checks for VINTF and CMDQV ENABLE bits before allocating
>     vCMDQ HW queues, and free any allocated resources if disabled
>     (patch #19).
>   - Introduced a common helper for vEVENTQ read and validation
>     (patch #23).
>   - Updated bios-tables test IORT blobs for SMMUv3 device identifier
>     changes (patch #29).
>   - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
>     using tegra241-cmdqv-stubs.c (patch #6).
>
> Please find the complete branch here:
> https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3

with this version I still get those traces:

iommufd_backend_map_dma  iommufd=28 ioas=2 iova=0xc0b0000 size=0x10000
addr=0xffff0bd30000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
qemu-system-aarch64: warning: vfio_container_dma_map(0xaaaea11e2870,
0xc0b0000, 0x10000, 0xffff0bd30000) = -14 (Bad address)

Eric

>
> I have sanity tested this on an NVIDIA GRACE platform and will
> continue with additional tests.
>
> Feedback and testing are very welcome.
>
> Thanks,
> Shameer
> [0] https://lore.kernel.org/qemu-devel/20260225031658.32095-1-philmd@linaro.org/
>
> ---
> Background(from RFCv1):
> https://lore.kernel.org/qemu-devel/20251210133737.78257-1-skolothumtho@nvidia.com/
>
> Thanks to Nicolin for the initial patches and testing on which this
> is based.
>
> Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces"
> (VINTFs), each hosting up to 128 VCMDQs.
>
> Each VINTF exposes two 64KB MMIO pages:
>  - Page0 – guest owned control and status registers (directly mapped
>            into the VM)
>  - Page1 – queue configuration registers (trapped/emulated by QEMU)
>
> Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not
> support the full command set. Only a subset, primarily invalidation
> related commands, is accepted by the CMDQV hardware. For this reason,
> a distinct CMDQV device must be exposed to the guest, and the guest OS
> must include a Tegra241 CMDQV aware driver to take advantage of the
> hardware acceleration.
>
> VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism,
> allowing QEMU to attach guest configured VCMDQ buffers to the
> underlying CMDQV hardware through IOMMUFD. The Linux kernel already
> supports the full CMDQV virtualisation model via IOMMUFD[0].
>
> ---
>
> Nicolin Chen (15):
>   backends/iommufd: Update iommufd_backend_get_device_info
>   backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
>     ptr
>   backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
>   backends/iommufd: Introduce iommufd_backend_viommu_mmap
>   hw/arm/tegra241-cmdqv: Implement CMDQV init
>   hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
>   hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
>   hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
>   hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
>   hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
>   hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
>   hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space
>   hw/arm/tegra241-cmdqv: Add reset handler
>   hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
>   hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
>
> Shameer Kolothum (17):
>   hw/arm/smmuv3-accel: Introduce CMDQV ops interface
>   hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
>   hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
>   hw/arm/virt: Store SMMUv3 device objects in VirtMachineState.
>   hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build
>   hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support
>   hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus
>   system/physmem: Add address_space_is_ram() helper
>   hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing
>   hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
>   hw/arm/smmuv3-accel: Introduce common helper for veventq read
>   hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
>   tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
>     change
>   hw/arm/smmuv3: Add per-device identifier property
>   tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
>     change
>   hw/arm/smmuv3-accel: Introduce helper to query CMDQV type
>   hw/arm/smmuv3: Add cmdqv property for SMMUv3 device
>
>  hw/arm/smmuv3-accel.h                         |  29 +
>  hw/arm/tegra241-cmdqv.h                       | 342 ++++++++
>  include/hw/arm/smmuv3.h                       |   3 +
>  include/hw/arm/virt.h                         |   1 +
>  include/system/iommufd.h                      |  16 +
>  include/system/memory.h                       |  10 +
>  backends/iommufd.c                            |  64 ++
>  hw/arm/smmuv3-accel-stubs.c                   |  11 +
>  hw/arm/smmuv3-accel.c                         | 196 ++++-
>  hw/arm/smmuv3.c                               |   9 +
>  hw/arm/tegra241-cmdqv-stubs.c                 |  18 +
>  hw/arm/tegra241-cmdqv.c                       | 793 ++++++++++++++++++
>  hw/arm/virt-acpi-build.c                      | 126 ++-
>  hw/arm/virt.c                                 |  27 +
>  hw/vfio/iommufd.c                             |   4 +-
>  system/physmem.c                              |  11 +
>  backends/trace-events                         |   4 +-
>  hw/arm/Kconfig                                |   5 +
>  hw/arm/meson.build                            |   2 +
>  hw/arm/trace-events                           |   5 +
>  tests/data/acpi/aarch64/virt/IORT.its_off     | Bin 172 -> 172 bytes
>  tests/data/acpi/aarch64/virt/IORT.msi_gicv2m  | Bin 172 -> 172 bytes
>  tests/data/acpi/aarch64/virt/IORT.smmuv3-dev  | Bin 260 -> 260 bytes
>  .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
>  24 files changed, 1596 insertions(+), 80 deletions(-)
>  create mode 100644 hw/arm/tegra241-cmdqv.h
>  create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
>  create mode 100644 hw/arm/tegra241-cmdqv.c
>

RE: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 11 March 2026 18:25
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV
> support for accelerated SMMUv3
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Shameer,
> 
> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> > Hi,
> >
> > Changes from v2:
> >
> > https://lore.kernel.org/qemu-devel/20260206144823.80655-1-
> skolothumtho
> > @nvidia.com/
> >
> >   - Addressed feedback on v2 and picked up R-by tags. Thanks!
> >   - Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto
> >     "cmdqv".
> >   - Introduced a probe() callback to detect and initialize CMDQV
> >     support during device attach (patch #7).
> >   - Since CMDQV initialization is now deferred to device attach time,
> >     added a helper to link any unmapped MMIO/IRQ resources (patch #12).
> >   - Moved VINTF enable handling to the CONFIG ENABLE bit write path
> >     instead of any read/write access path (patch #17). Hopefully this
> >     won't break any corner cases.
> >   - Added checks for VINTF and CMDQV ENABLE bits before allocating
> >     vCMDQ HW queues, and free any allocated resources if disabled
> >     (patch #19).
> >   - Introduced a common helper for vEVENTQ read and validation
> >     (patch #23).
> >   - Updated bios-tables test IORT blobs for SMMUv3 device identifier
> >     changes (patch #29).
> >   - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
> >     using tegra241-cmdqv-stubs.c (patch #6).
> >
> > Please find the complete branch here:
> > https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-
> vcm
> > dq-v3
> 
> with this version I still get those traces:
> 
> iommufd_backend_map_dma  iommufd=28 ioas=2 iova=0xc0b0000
> size=0x10000
> addr=0xffff0bd30000 readonly=0 (-1)
> qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI
> BAR?
> qemu-system-aarch64: warning: vfio_container_dma_map(0xaaaea11e2870,
> 0xc0b0000, 0x10000, 0xffff0bd30000) = -14 (Bad address)

Yes. Please see my reply to Nicolin on v2:
https://lore.kernel.org/qemu-devel/CH3PR12MB7548C2DCA4D321D7FDE23754AB6CA@CH3PR12MB7548.namprd12.prod.outlook.com/

I do not have a way to suppress that warning currently. It should be
safe to ignore, similar to the behaviour before the dma-buf support
series for vfio-pci PCI BAR mappings.

Thanks,
Shameer

Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[Jason Gunthorpe]

On Wed, Mar 11, 2026 at 06:34:38PM +0000, Shameer Kolothum Thodi wrote:
> I do not have a way to suppress that warning currently. It should be
> safe to ignore, similar to the behaviour before the dma-buf support
> series for vfio-pci PCI BAR mappings.

qemu should have the concept of memory regions that are not available
for DMA peer to peer operation - ie are not mapped to the iommu.

Now that the kernel strictly enforces this it is essential qemu itself
keep track if it thinks p2p should work or not.

Jason

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: 12 March 2026 15:11
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> arm@nongnu.org; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0
> for CMDQV
> 
> External email: Use caution opening links or attachments
> 
> 
> On 3/11/26 2:59 PM, Shameer Kolothum Thodi wrote:
> >
> >> -----Original Message-----
> >> From: Eric Auger <eric.auger@redhat.com>
> >> Sent: 11 March 2026 13:19
> >> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >> arm@nongnu.org; qemu-devel@nongnu.org
> >> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org; Nicolin
> >> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>; Matt
> >> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason Gunthorpe
> >> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> >> <kjaju@nvidia.com>; phrdina@redhat.com
> >> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
> Page0
> >> for CMDQV
> >>
> >> External email: Use caution opening links or attachments
> >>
> >>
> >> On 3/11/26 1:34 PM, Shameer Kolothum Thodi wrote:
> >>>> -----Original Message-----
> >>>> From: Eric Auger <eric.auger@redhat.com>
> >>>> Sent: 11 March 2026 10:05
> >>>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >>>> arm@nongnu.org; qemu-devel@nongnu.org
> >>>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org;
> Nicolin
> >>>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>;
> Matt
> >>>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason
> Gunthorpe
> >>>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >>>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> >>>> <kjaju@nvidia.com>; phrdina@redhat.com
> >>>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF
> >> Page0
> >>>> for CMDQV
> >>>>
> >>>> External email: Use caution opening links or attachments
> >>>>
> >>>>
> >>>> On 3/11/26 10:26 AM, Shameer Kolothum Thodi wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: Eric Auger <eric.auger@redhat.com>
> >>>>>> Sent: 11 March 2026 07:55
> >>>>>> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>; qemu-
> >>>>>> arm@nongnu.org; qemu-devel@nongnu.org
> >>>>>> Cc: peter.maydell@linaro.org; clg@redhat.com; alex@shazbot.org;
> >> Nicolin
> >>>>>> Chen <nicolinc@nvidia.com>; Nathan Chen <nathanc@nvidia.com>;
> >> Matt
> >>>>>> Ochs <mochs@nvidia.com>; Jiandi An <jan@nvidia.com>; Jason
> >> Gunthorpe
> >>>>>> <jgg@nvidia.com>; jonathan.cameron@huawei.com;
> >>>>>> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant
> Jaju
> >>>>>> <kjaju@nvidia.com>; phrdina@redhat.com
> >>>>>> Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap
> VINTF
> >>>> Page0
> >>>>>> for CMDQV
> >>>>>>
> >>>>>> External email: Use caution opening links or attachments
> >>>>>>
> >>>>>>
> >>>>>> On 2/26/26 11:50 AM, Shameer Kolothum wrote:
> >>>>>>> From: Nicolin Chen <nicolinc@nvidia.com>
> >>>>>>>
> >>>>>>> Global VCMDQ pages provide a VM wide view of all VCMDQs, while
> the
> >>>>>>> VINTF pages expose a logical view local to a given VINTF. Although real
> >>>>>>> hardware may support multiple VINTFs, the kernel currently exposes
> a
> >>>>>>> single VINTF per VM.
> >>>>>>>
> >>>>>>> The kernel provides an mmap offset for the VINTF Page0 region
> during
> >>>>>>> vIOMMU allocation. However, the logical-to-physical association
> >> between
> >>>>>>> VCMDQs and a VINTF is only established after HW_QUEUE allocation.
> >> Prior
> >>>>>>> to that, the mapped Page0 does not back any real VCMDQ state.
> >>>>>>>
> >>>>>>> When VINTF is enabled, mmap the kernel provided Page0 region and
> >>>>>>> unmap it when VINTF is disabled. This prepares the VINTF mapping
> >>>>>>> in advance of subsequent patches that add VCMDQ allocation
> support.
> >>>>>> So at some point we transition from something that is purely emulated
> >>>>>> (page 1 global cmdq) to something that is mmapped on a host page.
> >> How
> >>>> do
> >>>>>> we transfer the state of the cmdq from one to the other?
> >>>>> Right. If a guest uses both the "Global VCMDQ registers Page0" and the
> >>>>> "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see
> >>>>> nothing in the spec that forbids this), then we need to keep the two
> >>>>> views in sync.
> >>>> Also assuming the global VCMDQs are accessible and thus used, I guess
> we
> >>>> shall properly handle the actual commands (equivalent of
> >>>> smmuv3_cmdq_consume()), no?
> >>>> I don't see it done at the moment.
> >>>>
> >>>> By the way, do we want support of VCMDQs in fully emulated mode. I
> >> guess
> >>>> it would be sensible?
> >>> Yes, command handling is not implemented now.
> >>>
> >>> Since this is part of the accelerated SMMUv3 feature, I am not sure it
> >>> makes sense to implement full command handling in QEMU and then
> >> forward
> >>> the supported invalidation commands back to the host again.
> >>>
> >>> The idea here is to accelerate the CMDQ processing through the hardware.
> >>> I am not sure a guest would have a real use case for VCMDQ in a fully
> >>> emulated mode.
> >> OK but really the whole user model is really confusing. One starts using
> >> a "global" vcmdq and then switches to a logical one.
> >> On real HW I guess they are synced. But on emulation, currently they are
> >> not. Also when the guest is using the global vcmdq, it can
> >> access the registers but they actually do not trigger any commands. This
> >> really needs to be clarified.
> > Yes, this could use better documentation. I can clarify the model in the
> > commit message and code comments.
> >
> > One option we discussed earlier to avoid the confusion was to restrict
> > usage to the "VINTF logical VCMDQ" interface and return an error if the
> > guest accesses the "Global VCMDQ registers".
> >
> > However, since this is still MMIO space, the thinking was that it may be
> > better to allow the access even if it is effectively non-functional, rather
> > than making the behaviour unpredictable.
> >
> > See the discussion:
> > https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/
> >
> >> Then I understand we want to focus on accel mode but above points need
> >> to be clatified. I just wanted to mention that vcmdq should logically
> >> also work in fully emulated mode (and you seemed to implement a mix
> >> actually when you have emulated access to vi vcmdq) as it replaces the
> >> SMMU standard cmdq, if I understand correctly
> > IIUC, in the current Linux implementation VCMDQ does not replace the
> > standard SMMU CMDQ from the guest perspective. This is also why we do
> > not expose the HYP_OWN bit to the guest.
> > When HYP_OWN is not set, guest still uses the normal SMMU CMDQ.
> > CMDQ is only used as a fast-path for a small set of invalidation commands
> > that can be sent directly to the host hardware.
> >
> > See:
> >
> > static bool tegra241_guest_vcmdq_supports_cmd(struct
> arm_smmu_cmdq_ent *ent)
> > {
> >         switch (ent->opcode) {
> >         case CMDQ_OP_TLBI_NH_ASID:
> >         case CMDQ_OP_TLBI_NH_VA:
> >         case CMDQ_OP_ATC_INV:
> >                 return true;
> >         default:
> >                 return false;
> >         }
> > }
> >
> > So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is
> > only used to accelerate these invalidation commands.
> 
> I understand. But guest works in HYP_OWN mode as it sets the HYP_OWN
> bit. So resetting it looks like a hack to me, not documented in any way
> in the spec. Besides, as tegra241_guest_vcmdq_supports_cmd() forces to
> send on the cmdqv cmdq only legal cmds I don't see any problem leaving
> the bit to 1. What is important is that the host does the check (and the
> host sets HYP_OWN to 0 for that cmdq). I did the trial to comment
>         //value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> 
> and it seems to work properly

Ah.. that’s interesting. .I was under the impression that we are toggling
the HYP_OWN so that guest does sets the tegra241_guest_vcmdq_supports_cmd()
correctly and delegates the supported cmds through that queue.

See, 
    tegra241_cmdqv_init_structures()
        tegra241_vintf_alloc_lvcmdq()
          tegra241_vcmdq_alloc_smmu_cmdq()
          {
               .... 
                if (!vcmdq->vintf->hyp_own)
                      cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;
            
However, it looks like the above happens before tegra241_vintf_hw_init(),
which sets HYP_OWN. That may explain why toggling the bit here does not
make any difference.

I need to double check why we did the toggling in the QEMU then.

@Nicolin, do you remember?

Thanks,
Shameer

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

[Nicolin Chen]

On Thu, Mar 12, 2026 at 11:05:36AM -0700, Shameer Kolothum Thodi wrote:
> > I understand. But guest works in HYP_OWN mode as it sets the HYP_OWN
> > bit. So resetting it looks like a hack to me, not documented in any way
> > in the spec. Besides, as tegra241_guest_vcmdq_supports_cmd() forces to
> > send on the cmdqv cmdq only legal cmds I don't see any problem leaving
> > the bit to 1. What is important is that the host does the check (and the
> > host sets HYP_OWN to 0 for that cmdq). I did the trial to comment
> >         //value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
> > 
> > and it seems to work properly
> 
> Ah.. that’s interesting. .I was under the impression that we are toggling
> the HYP_OWN so that guest does sets the tegra241_guest_vcmdq_supports_cmd()
> correctly and delegates the supported cmds through that queue.
> 
> See, 
>     tegra241_cmdqv_init_structures()
>         tegra241_vintf_alloc_lvcmdq()
>           tegra241_vcmdq_alloc_smmu_cmdq()
>           {
>                .... 
>                 if (!vcmdq->vintf->hyp_own)
>                       cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;
>             
> However, it looks like the above happens before tegra241_vintf_hw_init(),
> which sets HYP_OWN. That may explain why toggling the bit here does not
> make any difference.

Well, looks like we exposed a kernel bug...

It seems that the sequence of setting hyp_own and supports_cmd got
inverted during a rework when I got the driver upstream.

It turned out that both host and guest kernels sets supports_cmd to
tegra241_guest_vcmdq_supports_cmd().. IOW, the host is sending non-
invalidation commands to the SMMU CMDQ rather than the HYP_OWNed
VCMDQ, although this is completely harmless and probably does not
affect host performance at all since non-invalidation commands are
not that frequently issued.

This explains why Eric's experiment didn't fail. I'll submit a fix.

With that being said, in QEMU we do need to wire HYP_OWN bit to 0.
This is not decided by the spec, but by the host SW implementation:

 550         /*
 551          * Note that HYP_OWN bit is wired to zero when running in guest kernel,
 552          * whether enabling it here or not, as !HYP_OWN cmdq HWs only support a
 553          * restricted set of supported commands.
 554          */
 555         regval = FIELD_PREP(VINTF_HYP_OWN, hyp_own) | 
 556                  FIELD_PREP(VINTF_VMID, vintf->vsmmu.vmid);
 557         writel(regval, REG_VINTF(vintf, CONFIG));
 558
 559         ret = vintf_write_config(vintf, regval | VINTF_EN);
 560         if (ret)
 561                 return ret;
 562         /*
 563          * As being mentioned above, HYP_OWN bit is wired to zero for a guest
 564          * kernel, so read it back from HW to ensure that reflects in hyp_own
 565          */
 566         vintf->hyp_own = !!(VINTF_HYP_OWN & readl(REG_VINTF(vintf, CONFIG)));

This should have been mentioned in the uAPI header. I'll submit a
patch to add this line near IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.

One thing we need to get our head around is that a guest hardware
is different than the host hardware:
 - Host HW is backed by a VINTF (HYP_OWN=1)
 - Guest HW is backed by a VINTF (HYP_OWN=0)
This is a fixed configuration, decided by the hardware and the spec
also.

Thus, the guest version of the VINTF must align with its underlying
hardware, so QEMU cannot allow setting HYP_OWN=1.

Otherwise, guest OS would have no idea this is a guest version of
hardware and would issue unsupported commands that will fail.

Nicolin

RE: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

[Shameer Kolothum Thodi]

> -----Original Message-----
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: 11 March 2026 20:00
> To: Shameer Kolothum Thodi <skolothumtho@nvidia.com>
> Cc: eric.auger@redhat.com; qemu-arm@nongnu.org; qemu-
> devel@nongnu.org; peter.maydell@linaro.org; clg@redhat.com;
> alex@shazbot.org; Nicolin Chen <nicolinc@nvidia.com>; Nathan Chen
> <nathanc@nvidia.com>; Matt Ochs <mochs@nvidia.com>; Jiandi An
> <jan@nvidia.com>; jonathan.cameron@huawei.com;
> zhangfei.gao@linaro.org; zhenzhong.duan@intel.com; Krishnakant Jaju
> <kjaju@nvidia.com>; phrdina@redhat.com
> Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV
> support for accelerated SMMUv3
> 
> On Wed, Mar 11, 2026 at 06:34:38PM +0000, Shameer Kolothum Thodi wrote:
> > I do not have a way to suppress that warning currently. It should be
> > safe to ignore, similar to the behaviour before the dma-buf support
> > series for vfio-pci PCI BAR mappings.
> 
> qemu should have the concept of memory regions that are not available
> for DMA peer to peer operation - ie are not mapped to the iommu.
> 
> Now that the kernel strictly enforces this it is essential qemu itself
> keep track if it thinks p2p should work or not.

We could introduce something like the below to address this. I will
include it in the next respin.

Thanks,
Shameer

---
 include/system/memory.h | 2 ++
 hw/arm/tegra241-cmdqv.c | 1 +
 hw/vfio/listener.c      | 5 +++++
 3 files changed, 8 insertions(+)

diff --git a/include/system/memory.h b/include/system/memory.h
index 02b2e83fd7..fb37616503 100644
--- a/include/system/memory.h
+++ b/include/system/memory.h
@@ -874,6 +874,8 @@ struct MemoryRegion {

     /* For devices designed to perform re-entrant IO into their own IO MRs */
     bool disable_reentrancy_guard;
+    /* RAM device region that does not require IOMMU mapping for P2P */
+    bool ram_device_skip_iommu_map;
 };

 struct IOMMUMemoryRegion {
diff --git a/hw/arm/tegra241-cmdqv.c b/hw/arm/tegra241-cmdqv.c
index 42d7dbfde7..4647b4db54 100644
--- a/hw/arm/tegra241-cmdqv.c
+++ b/hw/arm/tegra241-cmdqv.c
@@ -212,6 +212,7 @@ static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
                                       memory_region_owner(&cmdqv->mmio_cmdqv),
                                       name, VINTF_PAGE_SIZE,
                                       cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;
     memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
                                         cmdqv->mr_vintf_page0, 1);
     g_free(name);
diff --git a/hw/vfio/listener.c b/hw/vfio/listener.c
index 1087fdc142..d47439befb 100644
--- a/hw/vfio/listener.c
+++ b/hw/vfio/listener.c
@@ -614,6 +614,11 @@ void vfio_container_region_add(VFIOContainer *bcontainer,
         }
     }

+    if (memory_region_is_ram_device(section->mr) &&
+        section->mr->ram_device_skip_iommu_map) {
+        return;
+    }
+
     ret = vfio_container_dma_map(bcontainer, iova, int128_get64(llsize),
                                  vaddr, section->readonly, section->mr);
     if (ret) {
--
2.43.0