* [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework
@ 2025-09-30 16:53 tangtao1634
2025-09-30 16:53 ` [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device tangtao1634
` (4 more replies)
0 siblings, 5 replies; 16+ messages in thread
From: tangtao1634 @ 2025-09-30 16:53 UTC (permalink / raw)
To: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell
Cc: qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier,
Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh,
Tao Tang
From: Tao Tang <tangtao1634@phytium.com.cn>
This patch series (V2) introduces several cleanups and improvements to the smmu-testdev device. The main goals are to refactor shared code, enhance robustness, and significantly clarify the complex page table construction used for testing.
Motivation
----------
Currently, thoroughly testing the SMMUv3 emulation requires a significant
software stack. We need to boot a full guest operating system (like Linux)
with the appropriate drivers (e.g., IOMMUFD) and rely on firmware (e.g.,
ACPI with IORT tables or Hafnium) to correctly configure the SMMU and
orchestrate DMA from a peripheral device.
This dependency on a complex software stack presents several challenges:
* High Barrier to Entry: Writing targeted tests for specific SMMU
features (like fault handling, specific translation regimes, etc.)
becomes cumbersome.
* Difficult to Debug: It's hard to distinguish whether a bug originates
from the SMMU emulation itself, the guest driver, the firmware
tables, or the guest kernel's configuration.
* Slow Iteration: The need to boot a full guest OS slows down the
development and testing cycle.
The primary goal of this work is to create a lightweight, self-contained
testing environment that allows us to exercise the SMMU's core logic
directly at the qtest level, removing the need for any guest-side software.
Our Approach: A Dedicated Test Device
-------------------------------------
To achieve this, we introduce two main components:
* A new, minimal hardware device: smmu-testdev.
* A corresponding qtest that drives this device to generate SMMU-bound
traffic.
A key question is, "Why introduce a new smmu-testdev instead of using an
existing PCIe or platform device?"
The answer lies in our goal to minimize complexity. Standard devices,
whether PCIe or platform, come with their own intricate initialization
protocols and often require a complex driver state machine to function.
Using them would re-introduce the very driver-level complexity we aim to
avoid.
The smmu-testdev is intentionally not a conformant, general-purpose PCIe
or platform device. It is a purpose-built, highly simplified "DMA engine."
I've designed it to be analogous to a minimal PCIe Root Complex that
bypasses the full, realistic topology (Host Bridges, Switches, Endpoints)
to provide a direct, programmable path for a DMA request to reach the SMMU.
Its sole purpose is to trigger a DMA transaction when its registers are
written to, making it perfectly suited for direct control from a test
environment like qtest.
The Qtest Framework
-------------------
The new qtest (smmu-testdev-qtest.c) serves as the "bare-metal driver"
for both the SMMU and the smmu-testdev. It manually performs all the
setup that would typically be handled by the guest kernel and firmware,
but in a completely controlled and predictable manner:
1. SMMU Configuration: It directly initializes the SMMU's registers to a
known state.
2. Translation Structure Setup: It manually constructs the necessary
translation structures in memory, including Stream Table Entries
(STEs), Context Descriptors (CDs), and Page Tables (PTEs).
3. DMA Trigger: It programs the smmu-testdev to initiate a DMA operation
targeting a specific IOVA.
4. Verification: It waits for the transaction to complete and verifies
that the memory was accessed correctly after address translation by
the SMMU.
This framework provides a solid and extensible foundation for validating
the SMMU's core translation paths. The initial test included in this
series covers a basic DMA completion path in the Non-Secure bank,
serving as a smoke test and a proof of concept.
It is worth noting that this series currently only includes tests for the
Non-Secure SMMU. I am aware of the ongoing discussions and RFC patches
for Secure SMMU support. To avoid a dependency on unmerged work, this
submission does not include tests for the Secure world. However, I have
already implemented these tests locally, and I am prepared to submit
them for review as soon as the core Secure SMMU support is merged
upstream.
Changes from v1 RFC:
- Clarify Page Table Construction:
Detailed comments have been added to the page table construction logic. This is a key improvement, as the test setup extensively re-uses the same set of page tables for multiple translation stages and purposes (e.g., nested S1/S2 walks, CD fetch). The new comments explain this sharing mechanism, which can otherwise be confusing to follow.
- Refactor Shared Helpers:
The helper functions std_space_offset and std_space_to_str are now moved to a common header file. This allows them to be used by both the main device implementation (hw/misc/smmu-testdev.c) and its qtest (tests/qtest/smmu-testdev-qtest.c), improving code re-use and maintainability.
- Enhance Robustness:
Assertions have been added to ensure the device operates only in the expected Non-secure context. Additional conditional checks are also included to prevent potential runtime errors and make the test device more stable.
- Code Simplification and Cleanup:
Several functions that were redundant with existing macros for constructing Context Descriptors (CD) and Stream Table Entries (STE) have been removed. This simplifies the test data setup and reduces code duplication.
Other unused code fragments have also been removed to improve overall code clarity and hygiene.
Tao Tang (2):
hw/misc/smmu-testdev: introduce minimal SMMUv3 test device
tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source
docs/specs/index.rst | 1 +
docs/specs/smmu-testdev.rst | 45 ++
hw/misc/Kconfig | 5 +
hw/misc/meson.build | 1 +
hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++
include/hw/misc/smmu-testdev.h | 402 +++++++++++++
tests/qtest/meson.build | 1 +
tests/qtest/smmu-testdev-qtest.c | 238 ++++++++
8 files changed, 1636 insertions(+)
create mode 100644 docs/specs/smmu-testdev.rst
create mode 100644 hw/misc/smmu-testdev.c
create mode 100644 include/hw/misc/smmu-testdev.h
create mode 100644 tests/qtest/smmu-testdev-qtest.c
--
2.49.0
^ permalink raw reply [flat|nested] 16+ messages in thread* [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 @ 2025-09-30 16:53 ` tangtao1634 2025-10-23 10:31 ` Alex Bennée 2025-09-30 16:53 ` [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source tangtao1634 ` (3 subsequent siblings) 4 siblings, 1 reply; 16+ messages in thread From: tangtao1634 @ 2025-09-30 16:53 UTC (permalink / raw) To: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell Cc: qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Tao Tang From: Tao Tang <tangtao1634@phytium.com.cn> Add a tiny, test-only DMA source dedicated to exercising the SMMUv3 model. The device purposefully avoids a realistic PCIe/platform implementation and instead routes DMA requests straight into the SMMU, so that qtests can populate STE/CD/PTE with known values and observe translation and data movement deterministically, without booting any firmware or guest kernel. Motivation ---------- Bringing up and regression-testing the SMMU in emulation often depends on a large and flaky software stack (enumeration, drivers, PCIe fabric). For the class of tests that only need to (1) program translation structures and (2) trigger DMA at a precise time, that stack adds noise, slows CI, and makes failures harder to attribute to the SMMU itself. A hermetic DMA source keeps the surface area small and the results reproducible. What this device is (and is not) -------------------------------- * It is a minimal DMA producer solely for SMMU tests. * It is NOT a faithful PCIe Endpoint nor a platform device. * It is NOT added to any machine by default and remains test-only. Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> --- hw/misc/Kconfig | 5 + hw/misc/meson.build | 1 + hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++++ include/hw/misc/smmu-testdev.h | 402 ++++++++++++++ 4 files changed, 1351 insertions(+) create mode 100644 hw/misc/smmu-testdev.c create mode 100644 include/hw/misc/smmu-testdev.h diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig index 4e35657468..c83a0872ef 100644 --- a/hw/misc/Kconfig +++ b/hw/misc/Kconfig @@ -25,6 +25,11 @@ config PCI_TESTDEV default y if TEST_DEVICES depends on PCI +config SMMU_TESTDEV + bool + default y if TEST_DEVICES + depends on PCI && ARM_SMMUV3 + config EDU bool default y if TEST_DEVICES diff --git a/hw/misc/meson.build b/hw/misc/meson.build index b1d8d8e5d2..862a9895c3 100644 --- a/hw/misc/meson.build +++ b/hw/misc/meson.build @@ -4,6 +4,7 @@ system_ss.add(when: 'CONFIG_FW_CFG_DMA', if_true: files('vmcoreinfo.c')) system_ss.add(when: 'CONFIG_ISA_DEBUG', if_true: files('debugexit.c')) system_ss.add(when: 'CONFIG_ISA_TESTDEV', if_true: files('pc-testdev.c')) system_ss.add(when: 'CONFIG_PCI_TESTDEV', if_true: files('pci-testdev.c')) +system_ss.add(when: 'CONFIG_SMMU_TESTDEV', if_true: files('smmu-testdev.c')) system_ss.add(when: 'CONFIG_UNIMP', if_true: files('unimp.c')) system_ss.add(when: 'CONFIG_EMPTY_SLOT', if_true: files('empty_slot.c')) system_ss.add(when: 'CONFIG_LED', if_true: files('led.c')) diff --git a/hw/misc/smmu-testdev.c b/hw/misc/smmu-testdev.c new file mode 100644 index 0000000000..156f4cd714 --- /dev/null +++ b/hw/misc/smmu-testdev.c @@ -0,0 +1,943 @@ +/* + * A test device for the SMMU + * + * This test device is a minimal SMMU-aware device used to test the SMMU. + * + * Copyright (c) 2025 Phytium Technology + * + * Author: + * Tao Tang <tangtao1634@phytium.com.cn> + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "qemu/units.h" +#include "system/address-spaces.h" +#include "exec/memattrs.h" +#include "system/dma.h" +#include "hw/pci/pci.h" +#include "hw/pci/pci_device.h" +#include "hw/arm/smmu-common.h" +#include "hw/qdev-properties.h" +#include "qom/object.h" +#include "hw/misc/smmu-testdev.h" + +#define TYPE_SMMU_TESTDEV "smmu-testdev" +OBJECT_DECLARE_SIMPLE_TYPE(SMMUTestDevState, SMMU_TESTDEV) + +struct SMMUTestDevState { + PCIDevice parent_obj; + MemoryRegion bar0; + uint32_t attr_ns; /* Track Non-Secure for now; reserve room for more. */ + + uint64_t smmu_base; + uint64_t dma_iova; + uint32_t dma_len; + uint32_t dma_dir; + uint32_t dma_result; + bool dma_pending; + + /* Future-proof DMA config */ + AddressSpace *dma_as; /* IOMMU-mediated DMA AS for this device */ + uint32_t dma_mode; /* 0=legacy pci_dma, 1=attrs via dma_memory_* */ + uint32_t dma_attrs_cfg; /* bit0 secure, bits[2:1] space, bit3 unspecified */ + + /* Translation build configuration */ + uint32_t trans_mode; /* 0=S1, 1=S2, 2=Nested */ + SMMUTestDevSpace s1_space; + SMMUTestDevSpace s2_space; + uint32_t trans_status; /* 0=ok; non-zero=error */ + + /* User-configurable BDF (device/function) */ + uint32_t cfg_dev; /* PCI device/slot number (0..31) */ + uint32_t cfg_fn; /* PCI function number (0..7) */ + + bool debug_log; /* Enable verbose debug output */ +}; + +/* BAR0 layout */ +enum { + REG_ID = 0x00, + REG_ATTR_NS = 0x04, + REG_SMMU_BASE_LO = 0x20, + REG_SMMU_BASE_HI = 0x24, + REG_DMA_IOVA_LO = 0x28, + REG_DMA_IOVA_HI = 0x2C, + REG_DMA_LEN = 0x30, + REG_DMA_DIR = 0x34, + REG_DMA_RESULT = 0x38, + REG_DMA_DOORBELL = 0x3C, + /* Extended controls for DMA attributes/mode (kept after legacy regs) */ + REG_DMA_MODE = 0x40, /* 0: legacy; 1: attrs path */ + REG_DMA_ATTRS = 0x44, /* [0] secure, [2:1] space, [3] unspecified */ + /* Translation config & builder */ + REG_TRANS_MODE = 0x48, /* 0=S1 only, 1=S2 only, 2=Nested */ + REG_S1_SPACE = 0x4C, /* SMMUTestDevSpace for stage-1 path */ + REG_S2_SPACE = 0x50, /* SMMUTestDevSpace for stage-2 path */ + REG_TRANS_DBELL = 0x54, /* bit0=build, bit1=clear status */ + REG_TRANS_STATUS = 0x58, /* 0=ok else error */ + REG_TRANS_CLEAR = 0x5C, /* write-any: clear helper-built CD/STE/PTE */ + BAR0_SIZE = 0x1000, +}; + +#define DMA_DIR_DEV2HOST 0u +#define DMA_DIR_HOST2DEV 1u +#define DMA_RESULT_IDLE 0xffffffffu +#define DMA_RESULT_BUSY 0xfffffffeu +#define DMA_ERR_BAD_LEN 0xdead0001u +#define DMA_ERR_TX_FAIL 0xdead0002u +#define DMA_MAX_LEN (64 * KiB) + +/* + * Compile-time calculated STE field setting macros. + */ + + #define STD_STE_OR_CD_ENTRY_BYTES 64 /* 64 bytes per STE entry */ + #define STD_STE_S2T0SZ_VAL 0x14 + +#define STD_STE_SET_VALID(ste, val) \ + ((ste)->word[0] = ((ste)->word[0] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) +#define STD_STE_SET_CONFIG(ste, val) \ + ((ste)->word[0] = ((ste)->word[0] & ~(0x7 << 1)) | (((val) & 0x7) << 1)) +#define STD_STE_SET_S1FMT(ste, val) \ + ((ste)->word[0] = ((ste)->word[0] & ~(0x3 << 4)) | (((val) & 0x3) << 4)) + +#define STD_STE_SET_CTXPTR(ste, val) \ +do { \ + /* Lower address bits (31:6) occupy the upper 26 bits of word[0]. */ \ + (ste)->word[0] = ((ste)->word[0] & 0x0000003FU) | \ + ((uint32_t)(val) & 0xFFFFFFC0U); \ + \ + /* Upper address bits (47:32) occupy the low 16 bits of word[1]. */ \ + (ste)->word[1] = ((ste)->word[1] & 0xFFFF0000U) | \ + ((uint32_t)(((uint64_t)(val)) >> 32) & 0x0000FFFFU); \ +} while (0) + +#define STD_STE_SET_S1CDMAX(ste, val) \ + ((ste)->word[1] = ((ste)->word[1] & ~(0x1f << 27)) | (((val) & 0x1f) << 27)) +#define STD_STE_SET_S1STALLD(ste, val) \ + ((ste)->word[2] = ((ste)->word[2] & ~(0x1 << 27)) | (((val) & 0x1) << 27)) +#define STD_STE_SET_EATS(ste, val) \ + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 28)) | (((val) & 0x3) << 28)) +#define STD_STE_SET_STRW(ste, val) \ + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 30)) | (((val) & 0x3) << 30)) +#define STD_STE_SET_NSCFG(ste, val) \ + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 14)) | (((val) & 0x3) << 14)) +#define STD_STE_SET_S2VMID(ste, val) \ + ((ste)->word[4] = ((ste)->word[4] & ~0xffff) | ((val) & 0xffff)) +#define STD_STE_SET_S2T0SZ(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~0x3f) | ((val) & 0x3f)) +#define STD_STE_SET_S2SL0(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x3 << 6)) | (((val) & 0x3) << 6)) +#define STD_STE_SET_S2TG(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x3 << 14)) | (((val) & 0x3) << 14)) +#define STD_STE_SET_S2PS(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x7 << 16)) | (((val) & 0x7) << 16)) +#define STD_STE_SET_S2AA64(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 19)) | (((val) & 0x1) << 19)) +#define STD_STE_SET_S2ENDI(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 20)) | (((val) & 0x1) << 20)) +#define STD_STE_SET_S2AFFD(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 21)) | (((val) & 0x1) << 21)) +#define STD_STE_SET_S2HD(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 23)) | (((val) & 0x1) << 23)) +#define STD_STE_SET_S2HA(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 24)) | (((val) & 0x1) << 24)) +#define STD_STE_SET_S2S(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 25)) | (((val) & 0x1) << 25)) +#define STD_STE_SET_S2R(ste, val) \ + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 26)) | (((val) & 0x1) << 26)) + +#define STD_STE_SET_S2TTB(ste, val) \ +do { \ + /* Lower address bits (31:4) occupy the upper 28 bits of word[6]. */ \ + (ste)->word[6] = ((ste)->word[6] & 0x0000000FU) | \ + ((uint32_t)(val) & 0xFFFFFFF0U); \ + \ + /* Upper address bits (51:32) occupy the low 20 bits of word[7]. */ \ + (ste)->word[7] = ((ste)->word[7] & 0xFFF00000U) | \ + ((uint32_t)(((uint64_t)(val)) >> 32) & \ + 0x000FFFFFU); \ +} while (0) + +#define STE_S2TTB(x) \ + ((extract64((x)->word[7], 0, 16) << 32) | \ + ((x)->word[6] & 0xfffffff0)) + +/* + * Compile-time calculated CD field setting macros. + */ +#define STD_CD_SET_VALID(cd, val) \ + ((cd)->word[0] = ((cd)->word[0] & ~(0x1 << 31)) | \ + (((val) & 0x1) << 31)) +#define STD_CD_SET_TSZ(cd, sel, val) \ + ((cd)->word[0] = ((cd)->word[0] & \ + ~(0x3F << ((sel) * 16 + 0))) | \ + (((val) & 0x3F) << ((sel) * 16 + 0))) +#define STD_CD_SET_TG(cd, sel, val) \ + ((cd)->word[0] = ((cd)->word[0] & \ + ~(0x3 << ((sel) * 16 + 6))) | \ + (((val) & 0x3) << ((sel) * 16 + 6))) +#define STD_CD_SET_EPD(cd, sel, val) \ + ((cd)->word[0] = ((cd)->word[0] & \ + ~(0x1 << ((sel) * 16 + 14))) | \ + (((val) & 0x1) << ((sel) * 16 + 14))) +#define STD_CD_SET_ENDI(cd, val) \ + ((cd)->word[0] = ((cd)->word[0] & ~(0x1 << 15)) | \ + (((val) & 0x1) << 15)) +#define STD_CD_SET_IPS(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x7 << 0)) | \ + (((val) & 0x7) << 0)) +#define STD_CD_SET_AFFD(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 3)) | \ + (((val) & 0x1) << 3)) +#define STD_CD_SET_HD(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 10)) | \ + (((val) & 0x1) << 10)) +#define STD_CD_SET_HA(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 11)) | \ + (((val) & 0x1) << 11)) +#define STD_CD_SET_TTB(cd, sel, val) do { \ + (cd)->word[(sel) * 2 + 2] = ((cd)->word[(sel) * 2 + 2] & 0x0000000F) | \ + ((val) & 0xFFFFFFF0); \ + (cd)->word[(sel) * 2 + 3] = ((cd)->word[(sel) * 2 + 3] & 0xFFF80000) | \ + ((((uint64_t)(val)) >> 32) & 0x0007FFFF); \ +} while (0) +#define STD_CD_SET_HAD(cd, sel, val) \ + ((cd)->word[(sel) * 2 + 2] = ((cd)->word[(sel) * 2 + 2] & \ + ~(0x1 << 1)) | \ + (((val) & 0x1) << 1)) +#define STD_CD_SET_MAIR0(cd, val) ((cd)->word[6] = (val)) +#define STD_CD_SET_MAIR1(cd, val) ((cd)->word[7] = (val)) +#define STD_CD_SET_TCR_T0SZ(cd, val) \ + ((cd)->word[4] = ((cd)->word[4] & ~0x3F) | ((val) & 0x3F)) +#define STD_CD_SET_ASID(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0xFFFF << 16)) | \ + (((val) & 0xFFFF) << 16)) +#define STD_CD_SET_S(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 12)) | \ + (((val) & 0x1) << 12)) +#define STD_CD_SET_R(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 13)) | \ + (((val) & 0x1) << 13)) +#define STD_CD_SET_A(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 14)) | \ + (((val) & 0x1) << 14)) +#define STD_CD_SET_AARCH64(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 9)) | \ + (((val) & 0x1) << 9)) +#define STD_CD_SET_TBI(cd, val) \ + ((cd)->word[1] = ((cd)->word[1] & ~(0x3 << 6)) | \ + (((val) & 0x3) << 6)) +#define STD_CD_SET_NSCFG0(cd, val) \ + ((cd)->word[2] = ((cd)->word[2] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) +#define STD_CD_SET_NSCFG1(cd, val) \ + ((cd)->word[4] = ((cd)->word[4] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) + +typedef enum TransMode { + TM_S1_ONLY = 0, + TM_S2_ONLY = 1, + TM_NESTED = 2, +} TransMode; + +/* Minimal STE/CD images (bit layout derived from test helpers) */ +typedef struct { + uint32_t word[8]; +} STEImg; + +typedef struct { + uint32_t word[8]; +} CDImg; + +/* ---- Debug helpers for printing current translation configuration ---- */ +static void G_GNUC_PRINTF(2, 3) +smmu_testdev_debug(const SMMUTestDevState *s, const char *fmt, ...) +{ + va_list ap; + g_autofree char *msg = NULL; + + if (!s->debug_log) { + return; + } + + va_start(ap, fmt); + msg = g_strdup_vprintf(fmt, ap); + va_end(ap); + + if (qemu_log_enabled()) { + qemu_log("%s", msg); + } else { + fprintf(stderr, "%s", msg); + } +} + +/* Only support Non-Secure space for now. */ +static bool smmu_testdev_space_supported(SMMUTestDevSpace sp) +{ + return sp == STD_SPACE_NONSECURE; +} + +static MemTxAttrs mk_attrs_from_space(SMMUTestDevSpace space) +{ + MemTxAttrs a = {0}; + if (!smmu_testdev_space_supported(space)) { + g_assert_not_reached(); + } else { + a.space = space; + } + a.secure = 0; + return a; +} + +/* Convert SMMUTestDevSpace to AddressSpace */ +static inline AddressSpace *space_to_as(SMMUTestDevSpace sp) +{ + /* Future work can dispatch Secure/Realm/Root address spaces here. */ + if (!smmu_testdev_space_supported(sp)) { + g_assert_not_reached(); + } + return &address_space_memory; +} + +/* Apply per-space offset for addresses or values that encode addresses. */ +static inline uint64_t std_apply_space_offs(SMMUTestDevSpace sp, uint64_t x) +{ + return x + std_space_offset(sp); +} + +/* Direct write helpers (no mirroring) */ +static void std_write64(SMMUTestDevSpace sp, uint64_t pa, uint64_t val, + uint32_t *status) +{ + MemTxAttrs a = mk_attrs_from_space(sp); + AddressSpace *as = space_to_as(sp); + if (!as) { + *status = 0xdead2011u; + return; + } + MemTxResult r = address_space_write(as, pa, a, &val, sizeof(val)); + if (r != MEMTX_OK && status) { + *status = 0xdead2011u; + return; + } + *status = 0; +} + +static void std_write32(SMMUTestDevSpace sp, uint64_t pa, uint32_t val, + uint32_t *status) +{ + MemTxAttrs a = mk_attrs_from_space(sp); + AddressSpace *as = space_to_as(sp); + if (!as) { + *status = 0xdead2012u; + return; + } + MemTxResult r = address_space_write(as, pa, a, &val, sizeof(val)); + if (r != MEMTX_OK && status) { + *status = 0xdead2012u; + return; + } + *status = 0; +} + +/* Build the translation tables with specified stage and security spaces. */ +static void smmu_testdev_build_translation(SMMUTestDevState *s) +{ + smmu_testdev_debug(s, "smmu_testdev_build_translation: stage=%s s1_space=%s" + " s2_space=%s\n", std_mode_to_str(s->trans_mode), + std_space_to_str(s->s1_space), + std_space_to_str(s->s2_space)); + uint32_t st = 0; + SMMUTestDevSpace build_space = + (s->trans_mode == TM_S1_ONLY) ? s->s1_space : s->s2_space; + + if (!smmu_testdev_space_supported(build_space) || + (s->trans_mode != TM_S2_ONLY && + !smmu_testdev_space_supported(s->s1_space))) { + /* Only the Non-Secure space is supported until more domains land. */ + s->trans_status = 0xdead3001u; + return; + } + + /* + * Build base page tables (L0..L3) in the chosen space. + * For Non-Secure, place tables at Secure-base + space offset and + * update descriptor values by the same offset to keep internal + * relationships identical across spaces. + */ + uint64_t L0_pa = std_apply_space_offs(build_space, STD_L0_ADDR); + uint64_t L1_pa = std_apply_space_offs(build_space, STD_L1_ADDR); + uint64_t L2_pa = std_apply_space_offs(build_space, STD_L2_ADDR); + uint64_t L3_pa = std_apply_space_offs(build_space, STD_L3_ADDR); + uint64_t L0_val = std_apply_space_offs(build_space, STD_L0_VAL); + uint64_t L1_val = std_apply_space_offs(build_space, STD_L1_VAL); + uint64_t L2_val = std_apply_space_offs(build_space, STD_L2_VAL); + uint64_t L3_val = std_apply_space_offs(build_space, STD_L3_VAL); + std_write64(build_space, L0_pa, L0_val, &st); + std_write64(build_space, L1_pa, L1_val, &st); + std_write64(build_space, L2_pa, L2_val, &st); + std_write64(build_space, L3_pa, L3_val, &st); + + /* Build STE image */ + STEImg ste = {0}; + switch (s->trans_mode) { + case TM_S1_ONLY: + STD_STE_SET_CONFIG(&ste, 0x5); + break; + case TM_S2_ONLY: + STD_STE_SET_CONFIG(&ste, 0x6); + break; + case TM_NESTED: + default: + STD_STE_SET_CONFIG(&ste, 0x7); + break; + } + + uint64_t vttb = STD_VTTB; + STD_STE_SET_VALID(&ste, 1); + STD_STE_SET_S2T0SZ(&ste, STD_STE_S2T0SZ_VAL); + STD_STE_SET_S2SL0(&ste, 0x2); /* Start level 0*/ + STD_STE_SET_S2TG(&ste, 0); /* 4KB */ + STD_STE_SET_S2PS(&ste, 0x5); /* 48 bits */ + STD_STE_SET_S2AA64(&ste, 1); /* Enable S2AA64 (64-bit address format). */ + STD_STE_SET_S2ENDI(&ste, 0); /* Little Endian */ + STD_STE_SET_S2AFFD(&ste, 0); /* AF Fault Disable */ + + STD_STE_SET_S2T0SZ(&ste, STD_STE_S2T0SZ_VAL); + STD_STE_SET_S2SL0(&ste, 0x2); /* Start level */ + STD_STE_SET_S2TG(&ste, 0); /* 4KB */ + STD_STE_SET_S2PS(&ste, 0x5); /* 48 bits ?*/ + /* Set Context Pointer (S1ContextPtr) */ + STD_STE_SET_CTXPTR(&ste, std_apply_space_offs(build_space, STD_CD_GPA)); + + STD_STE_SET_S2TTB(&ste, std_apply_space_offs(build_space, vttb)); + + /* + * Start assembling the STE, which is 64 bytes in total. + */ + for (int i = 0; i < 8; i++) { + std_write32(build_space, + std_apply_space_offs(build_space, STD_STE_GPA) + i * 4, + ste.word[i], &st); + if (st != 0) { + printf("Writing STE error! status: %x\n", st); + return; + } + } + + /* Build CD image for S1 path if needed */ + if (s->trans_mode != TM_S2_ONLY) { + CDImg cd = {0}; + + STD_CD_SET_ASID(&cd, 0x1e20); /* ASID */ + STD_CD_SET_AARCH64(&cd, 1); /* AA64 */ + STD_CD_SET_VALID(&cd, 1); + STD_CD_SET_A(&cd, 1); + STD_CD_SET_S(&cd, 0); + STD_CD_SET_HD(&cd, 0); + STD_CD_SET_HA(&cd, 0); + STD_CD_SET_IPS(&cd, 0x4); + STD_CD_SET_TBI(&cd, 0x0); + STD_CD_SET_AFFD(&cd, 0x0); + /* Disable TTB0 translation table walk */ + STD_CD_SET_EPD(&cd, 0, 0x0); + /* Enable TTB1 translation table walk */ + STD_CD_SET_EPD(&cd, 1, 0x1); + STD_CD_SET_TSZ(&cd, 0, 0x10); + STD_CD_SET_TG(&cd, 0, 0x0); + STD_CD_SET_ENDI(&cd, 0x0); + STD_CD_SET_NSCFG0(&cd, 0x0); + STD_CD_SET_NSCFG1(&cd, 0x0); + STD_CD_SET_R(&cd, 0x1); + + /* + * CD belongs to S1 path: compute offsets using s1_space so the + * GPA and embedded addresses are consistent with that space. + */ + uint64_t cd_ttb = std_apply_space_offs(build_space, vttb); + smmu_testdev_debug(s, "STD_CD_SET_TTB: 0x%llx\n", + (unsigned long long)cd_ttb); + STD_CD_SET_TTB(&cd, 0, cd_ttb); + + for (int i = 0; i < 8; i++) { + std_write32(s->s1_space, + std_apply_space_offs(s->s1_space, STD_CD_GPA) + i * 4, + cd.word[i], &st); + } + + L3_val = std_apply_space_offs(build_space, STD_L3_S1_VAL); + std_write64(build_space, L3_pa, L3_val, &st); + } + + /* Nested extras: CD S2 tables, CD.TTB S2 tables, shared entries. */ + if (s->trans_mode == TM_NESTED) { + /* CD.S2 tables */ + std_write64(build_space, + std_apply_space_offs(build_space, STD_CD_S2_L0_ADDR), + std_apply_space_offs(build_space, STD_L0_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_CD_S2_L1_ADDR), + std_apply_space_offs(build_space, STD_L1_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_CD_S2_L2_ADDR), + std_apply_space_offs(build_space, STD_L2_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_CD_S2_L3_ADDR), + std_apply_space_offs(build_space, + STD_CD_S2_L3_VAL), &st); + + /* CD.TTB S2 tables */ + std_write64(build_space, + std_apply_space_offs(build_space, STD_CDTTB_S2_L2_ADDR), + std_apply_space_offs(build_space, STD_L2_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_CDTTB_S2_L3_ADDR), + std_apply_space_offs(build_space, + STD_CDTTB_S2_L3_VAL), &st); + + /* Shared mappings between S1 and S2 page tables */ + std_write64(build_space, + std_apply_space_offs(build_space, STD_S1L0_IN_S2L3_ADDR), + std_apply_space_offs(build_space, + STD_S1L0_IN_S2L3_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_S1L1_IN_S2L3_ADDR), + std_apply_space_offs(build_space, + STD_S1L1_IN_S2L3_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_S1L2_IN_S2L3_ADDR), + std_apply_space_offs(build_space, + STD_S1L2_IN_S2L3_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_S1L3_IN_S2L2_ADDR), + std_apply_space_offs(build_space, + STD_S1L3_IN_S2L2_VAL), &st); + std_write64(build_space, + std_apply_space_offs(build_space, STD_S1L3_IN_S2L3_ADDR), + std_apply_space_offs(build_space, + STD_S1L3_IN_S2L3_VAL), &st); + } + + s->trans_status = st; +} + +/* Manipulate SMMU command to invalidate caches */ +static void push_cfgi_cmd(SMMUTestDevState *s, + SMMUTestDevSpace bank_sp, + uint32_t type, + uint32_t sid, + bool ssec) +{ + MemTxResult res = 0; + g_assert(smmu_testdev_space_supported(bank_sp)); + g_assert(!ssec); + hwaddr bank_off = 0; + uint32_t base_lo = address_space_ldl_le(&address_space_memory, + s->smmu_base + bank_off + 0x90, + MEMTXATTRS_UNSPECIFIED, &res); + uint32_t base_hi = address_space_ldl_le(&address_space_memory, + s->smmu_base + bank_off + 0x94, + MEMTXATTRS_UNSPECIFIED, &res); + uint64_t base = ((uint64_t)base_hi << 32) | base_lo; + uint32_t log2size = base & 0x1f; + uint64_t qbase = base & 0xfffffffffffc0ULL; + uint32_t prod = address_space_ldl_le(&address_space_memory, + s->smmu_base + bank_off + 0x98, + MEMTXATTRS_UNSPECIFIED, &res); + uint32_t index_mask = (1u << log2size) - 1u; + uint32_t slot = prod & index_mask; + uint64_t entry_pa = qbase + (uint64_t)slot * 16u; + + uint32_t words[4] = {0}; + words[0] = (type & 0xff) | (ssec ? (1u << 10) : 0u); + words[1] = sid; + + /* push command to the command queue */ + MemTxAttrs a = mk_attrs_from_space(bank_sp); + AddressSpace *as = space_to_as(bank_sp); + if (!as) { + printf("push_cfgi_cmd: space %d not supported\n", bank_sp); + return; + } + int ret = address_space_write(as, entry_pa, a, + words, sizeof(words)); + smmu_testdev_debug(s, "push_cfgi_cmd ret %d\n", ret); + + /* update PROD to trigger command handler */ + uint32_t new_prod = (prod + 1) & ((1u << (log2size + 1)) - 1u); + address_space_stl_le(&address_space_memory, + s->smmu_base + bank_off + 0x98, + new_prod, MEMTXATTRS_UNSPECIFIED, &res); + smmu_testdev_debug(s, "last res %d\n", res); +} + +/* Clear all the cache to avoid the incorrect cache hits using SMMU commands */ +static void smmu_testdev_clear_caches(SMMUTestDevState *s) +{ + uint32_t st = 0; + static const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; + + for (size_t idx = 0; idx < ARRAY_SIZE(spaces); idx++) { + SMMUTestDevSpace build_space = spaces[idx]; + if (!smmu_testdev_space_supported(build_space)) { + continue; + } + /* Clear L0..L3 entries written by the builder. */ + uint64_t L0_pa = std_apply_space_offs(build_space, STD_L0_ADDR); + uint64_t L1_pa = std_apply_space_offs(build_space, STD_L1_ADDR); + uint64_t L2_pa = std_apply_space_offs(build_space, STD_L2_ADDR); + uint64_t L3_pa = std_apply_space_offs(build_space, STD_L3_ADDR); + uint64_t S2_L0_pa = std_apply_space_offs(build_space, + STD_CD_S2_L0_ADDR); + uint64_t S2_L1_pa = std_apply_space_offs(build_space, + STD_CD_S2_L1_ADDR); + uint64_t S2_L2_pa = std_apply_space_offs(build_space, + STD_CD_S2_L2_ADDR); + uint64_t S2_L3_pa = std_apply_space_offs(build_space, + STD_CD_S2_L3_ADDR); + + std_write64(build_space, L0_pa, 0ull, &st); + std_write64(build_space, L1_pa, 0ull, &st); + std_write64(build_space, L2_pa, 0ull, &st); + std_write64(build_space, L3_pa, 0ull, &st); + std_write64(build_space, S2_L0_pa, 0ull, &st); + std_write64(build_space, S2_L1_pa, 0ull, &st); + std_write64(build_space, S2_L2_pa, 0ull, &st); + std_write64(build_space, S2_L3_pa, 0ull, &st); + + /* Clear STE image where it was placed. */ + for (int i = 0; i < 8; i++) { + std_write32(build_space, + std_apply_space_offs(build_space, STD_STE_GPA) + i * 4, + 0u, &st); + } + + /* Clear CD image in S1 space (matches builder placement). */ + for (int i = 0; i < 8; i++) { + std_write32(build_space, + std_apply_space_offs(build_space, STD_CD_GPA) + i * 4, + 0u, &st); + } + } + + /* Invalidate configuration cache via CFGI_STE and CFGI_CD commands */ + if (s->smmu_base) { + /* Compute this PCI function's StreamID: bus 0, current devfn. */ + uint8_t devfn = PCI_DEVICE(&s->parent_obj)->devfn; + uint32_t sid = PCI_BUILD_BDF(0, devfn); + + /* Non-secure bank invalidations (SSEC=0). */ + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_CFGI_STE, sid, false); + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_CFGI_CD, sid, false); + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_TLBI_NSNH_ALL, + sid, false); + + /* Add Secure/Realm/Root invalidations here once those domains exist. */ + } +} + +static void smmu_testdev_refresh_attrs(SMMUTestDevState *s) +{ + /* Report the baked-in Non-Secure attributes until more exist. */ + s->attr_ns = (STD_SPACE_NONSECURE << 1); +} + +/* Trigger a DMA operation */ +static void smmu_testdev_maybe_run_dma(SMMUTestDevState *s) +{ + if (!s->dma_pending) { + return; + } + smmu_testdev_debug(s, "smmu_testdev_maybe_run_dma: dma_pending: %d\n", + s->dma_pending); + + s->dma_pending = false; + + if (!s->dma_len || s->dma_len > DMA_MAX_LEN) { + s->dma_result = DMA_ERR_BAD_LEN; + return; + } + + g_autofree uint8_t *buf = g_malloc(s->dma_len); + MemTxResult res; + + if (s->dma_mode == 0) { + if (s->dma_dir == DMA_DIR_HOST2DEV) { + res = pci_dma_read(PCI_DEVICE(&s->parent_obj), s->dma_iova, + buf, s->dma_len); + } else { + for (uint32_t i = 0; i < s->dma_len; i++) { + buf[i] = 0xA0u + (i & 0x1fu); + } + res = pci_dma_write(PCI_DEVICE(&s->parent_obj), s->dma_iova, + buf, s->dma_len); + } + } else { + SMMUTestDevSpace dma_space = + (SMMUTestDevSpace)((s->dma_attrs_cfg >> 1) & 0x3); + if (!smmu_testdev_space_supported(dma_space)) { + /* Default to Non-Secure until other spaces are modeled. */ + dma_space = STD_SPACE_NONSECURE; + } + MemTxAttrs attrs = { + .secure = 0, + .space = dma_space, + .unspecified = (s->dma_attrs_cfg & (1u << 3)) ? 1 : 0, + }; + /* + * If 'unspecified' is set, bypass IOMMU AS and use system memory. + * This helps tests that want deterministic success without full + * IOMMU programming. + */ + AddressSpace *as = (s->dma_as && !attrs.unspecified) + ? s->dma_as + : &address_space_memory; + if (s->dma_dir == DMA_DIR_HOST2DEV) { + res = dma_memory_read(as, s->dma_iova, buf, s->dma_len, attrs); + } else { + for (uint32_t i = 0; i < s->dma_len; i++) { + buf[i] = 0xA0u + (i & 0x1fu); + } + res = dma_memory_write(as, s->dma_iova, buf, s->dma_len, attrs); + } + } + s->dma_result = (res == MEMTX_OK) ? 0 : DMA_ERR_TX_FAIL; + smmu_testdev_debug(s, "iommu ret %d , dma_result: 0x%x\n", + res, s->dma_result); +} + +static uint64_t smmu_testdev_mmio_read(void *opaque, hwaddr addr, unsigned size) +{ + SMMUTestDevState *s = opaque; + switch (addr) { + case REG_ID: + /* + * Only reads of REG_ID intentionally trigger the side effects + * (SMMU CR0 write and pending DMA). This lets tests poll + * REG_DMA_RESULT to observe BUSY before consuming the DMA. + */ + smmu_testdev_maybe_run_dma(s); + return 0x53544d4du; /* 'STMM' */ + case REG_ATTR_NS: + return s->attr_ns; + case REG_SMMU_BASE_LO: + return (uint32_t)(s->smmu_base & 0xffffffffu); + case REG_SMMU_BASE_HI: + return (uint32_t)(s->smmu_base >> 32); + case REG_DMA_IOVA_LO: + return (uint32_t)(s->dma_iova & 0xffffffffu); + case REG_DMA_IOVA_HI: + return (uint32_t)(s->dma_iova >> 32); + case REG_DMA_LEN: + return s->dma_len; + case REG_DMA_DIR: + return s->dma_dir; + case REG_DMA_RESULT: + return s->dma_result; + case REG_DMA_MODE: + return s->dma_mode; + case REG_DMA_ATTRS: + return s->dma_attrs_cfg; + case REG_TRANS_MODE: + return s->trans_mode; + case REG_S1_SPACE: + return s->s1_space; + case REG_S2_SPACE: + return s->s2_space; + case REG_TRANS_STATUS: + return s->trans_status; + default: + return 0; + } +} + +static void smmu_testdev_mmio_write(void *opaque, hwaddr addr, uint64_t val, + unsigned size) +{ + SMMUTestDevState *s = opaque; + uint32_t data = val; + + switch (addr) { + case REG_ID: + if (data == 0x1) { + smmu_testdev_refresh_attrs(s); + } + break; + case REG_SMMU_BASE_LO: + s->smmu_base = (s->smmu_base & ~0xffffffffull) | data; + break; + case REG_SMMU_BASE_HI: + s->smmu_base = (s->smmu_base & 0xffffffffull) | + ((uint64_t)data << 32); + break; + case REG_DMA_IOVA_LO: + s->dma_iova = (s->dma_iova & ~0xffffffffull) | data; + break; + case REG_DMA_IOVA_HI: + s->dma_iova = (s->dma_iova & 0xffffffffull) | + ((uint64_t)data << 32); + break; + case REG_DMA_LEN: + s->dma_len = data; + break; + case REG_DMA_DIR: + s->dma_dir = data ? DMA_DIR_HOST2DEV : DMA_DIR_DEV2HOST; + break; + case REG_DMA_RESULT: + s->dma_result = data; + break; + case REG_DMA_DOORBELL: + if (data & 0x1) { + s->dma_pending = true; + s->dma_result = DMA_RESULT_BUSY; + } else { + s->dma_pending = false; + s->dma_result = DMA_RESULT_IDLE; + } + break; + case REG_DMA_MODE: + s->dma_mode = data & 0x1; + break; + case REG_DMA_ATTRS: + s->dma_attrs_cfg = data; + break; + case REG_TRANS_MODE: + s->trans_mode = data & 0x3; + break; + case REG_S1_SPACE: + s->s1_space = (SMMUTestDevSpace)(data & 0x3); + break; + case REG_S2_SPACE: + s->s2_space = (SMMUTestDevSpace)(data & 0x3); + break; + case REG_TRANS_DBELL: + if (data & 0x2) { + s->trans_status = 0; + } + if (data & 0x1) { + smmu_testdev_build_translation(s); + } + break; + case REG_TRANS_CLEAR: + /* Clear helper caches so the next iteration rebuilds cleanly. */ + smmu_testdev_clear_caches(s); + break; + default: + break; + } +} + +static const MemoryRegionOps smmu_testdev_mmio_ops = { + .read = smmu_testdev_mmio_read, + .write = smmu_testdev_mmio_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + }, +}; + +static void smmu_testdev_realize(PCIDevice *pdev, Error **errp) +{ + SMMUTestDevState *s = SMMU_TESTDEV(pdev); + + /* Apply user-configurable BDF (default 0:1). */ + uint8_t dev = s->cfg_dev & 0x1f; + uint8_t fn = s->cfg_fn & 0x7; + pdev->devfn = (dev << 3) | fn; + + smmu_testdev_refresh_attrs(s); + s->smmu_base = 0; + s->dma_iova = 0; + s->dma_len = 0; + s->dma_dir = DMA_DIR_DEV2HOST; + s->dma_result = DMA_RESULT_IDLE; + s->dma_pending = false; + s->dma_mode = 0; + s->dma_attrs_cfg = 0; + s->dma_as = pci_device_iommu_address_space(pdev); + s->trans_mode = TM_S2_ONLY; + s->s1_space = STD_SPACE_NONSECURE; + s->s2_space = STD_SPACE_NONSECURE; + s->trans_status = 0; + + if (s->debug_log) { + smmu_testdev_debug(s, "[smmu-testdev] debug logging enabled\n"); + } + + memory_region_init_io(&s->bar0, OBJECT(pdev), &smmu_testdev_mmio_ops, s, + TYPE_SMMU_TESTDEV ".bar0", BAR0_SIZE); + pci_register_bar(pdev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY, &s->bar0); +} + +static void smmu_testdev_reset(DeviceState *dev) +{ + SMMUTestDevState *s = SMMU_TESTDEV(dev); + + smmu_testdev_refresh_attrs(s); + s->smmu_base = 0; + s->dma_iova = 0; + s->dma_len = 0; + s->dma_dir = DMA_DIR_DEV2HOST; + s->dma_result = DMA_RESULT_IDLE; + s->dma_pending = false; + s->dma_mode = 0; + s->dma_attrs_cfg = 0; + s->trans_mode = TM_S2_ONLY; + s->s1_space = STD_SPACE_NONSECURE; + s->s2_space = STD_SPACE_NONSECURE; + s->trans_status = 0; + /* Keep cfg_dev/cfg_fn as-is across reset */ +} + +static const Property smmu_testdev_properties[] = { + DEFINE_PROP_UINT32("device", SMMUTestDevState, cfg_dev, 0), + DEFINE_PROP_UINT32("function", SMMUTestDevState, cfg_fn, 1), + DEFINE_PROP_BOOL("debug-log", SMMUTestDevState, debug_log, false), +}; + +static void smmu_testdev_class_init(ObjectClass *klass, const void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PCIDeviceClass *pc = PCI_DEVICE_CLASS(klass); + + pc->realize = smmu_testdev_realize; + pc->vendor_id = PCI_VENDOR_ID_REDHAT; + pc->device_id = PCI_DEVICE_ID_REDHAT_TEST; + pc->revision = 0; + pc->class_id = PCI_CLASS_OTHERS; + dc->desc = "A test device for the SMMU"; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + device_class_set_legacy_reset(dc, smmu_testdev_reset); + device_class_set_props(dc, smmu_testdev_properties); +} + +static void smmu_testdev_instance_init(Object *obj) +{ + SMMUTestDevState *s = SMMU_TESTDEV(obj); + s->cfg_dev = 0; + s->cfg_fn = 1; /* default StreamID = 1 (slot 0, fn 1) */ + s->debug_log = false; +} + +static const TypeInfo smmu_testdev_info = { + .name = TYPE_SMMU_TESTDEV, + .parent = TYPE_PCI_DEVICE, + .instance_size = sizeof(SMMUTestDevState), + .instance_init = smmu_testdev_instance_init, + .class_init = smmu_testdev_class_init, + .interfaces = (const InterfaceInfo[]) { + { INTERFACE_CONVENTIONAL_PCI_DEVICE }, + { } + }, +}; + +static void smmu_testdev_register_types(void) +{ + type_register_static(&smmu_testdev_info); +} + +type_init(smmu_testdev_register_types); diff --git a/include/hw/misc/smmu-testdev.h b/include/hw/misc/smmu-testdev.h new file mode 100644 index 0000000000..6d97f6c704 --- /dev/null +++ b/include/hw/misc/smmu-testdev.h @@ -0,0 +1,402 @@ +/* + * A test device for the SMMU + * + * This test device is a minimal SMMU-aware device used to test the SMMU. + * + * Copyright (c) 2025 Phytium Technology + * + * Author: + * Tao Tang <tangtao1634@phytium.com.cn> + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_MISC_SMMU_TESTDEV_H +#define HW_MISC_SMMU_TESTDEV_H + +#include "qemu/osdep.h" +typedef enum SMMUTestDevSpace { + STD_SPACE_SECURE = 0, + STD_SPACE_NONSECURE = 1, + STD_SPACE_ROOT = 2, + STD_SPACE_REALM = 3, +} SMMUTestDevSpace; + +/* Only the Non-Secure space is implemented; leave room for future domains. */ +#define STD_SUPPORTED_SPACES 1 + +/* BAR0 registers (offsets) */ +enum { + STD_REG_ID = 0x00, + STD_REG_ATTR_NS = 0x04, + STD_REG_SMMU_BASE_LO = 0x20, + STD_REG_SMMU_BASE_HI = 0x24, + STD_REG_DMA_IOVA_LO = 0x28, + STD_REG_DMA_IOVA_HI = 0x2C, + STD_REG_DMA_LEN = 0x30, + STD_REG_DMA_DIR = 0x34, + STD_REG_DMA_RESULT = 0x38, + STD_REG_DMA_DBELL = 0x3C, + /* Extended controls for DMA attributes/mode */ + STD_REG_DMA_MODE = 0x40, + STD_REG_DMA_ATTRS = 0x44, + /* Translation controls */ + STD_REG_TRANS_MODE = 0x48, + STD_REG_S1_SPACE = 0x4C, + STD_REG_S2_SPACE = 0x50, + STD_REG_TRANS_DBELL = 0x54, + STD_REG_TRANS_STATUS = 0x58, + /* Clear helper-built tables/descriptors (write-any to trigger) */ + STD_REG_TRANS_CLEAR = 0x5C, +}; + +/* DMA result/status values shared with tests */ +#define STD_DMA_RESULT_IDLE 0xffffffffu +#define STD_DMA_RESULT_BUSY 0xfffffffeu +#define STD_DMA_ERR_BAD_LEN 0xdead0001u +#define STD_DMA_ERR_TX_FAIL 0xdead0002u + +/* DMA attributes layout (for STD_REG_DMA_ATTRS) */ +#define STD_DMA_ATTR_SECURE (1u << 0) +#define STD_DMA_ATTR_SPACE_SHIFT 1 +#define STD_DMA_ATTR_SPACE_MASK (0x3u << STD_DMA_ATTR_SPACE_SHIFT) +#define STD_DMA_ATTR_UNSPECIFIED (1u << 3) + +/* Command type */ +#define STD_CMD_CFGI_STE 0x03 +#define STD_CMD_CFGI_CD 0x05 +#define STD_CMD_TLBI_NSNH_ALL 0x30 + +/* + * Translation tables and descriptors for a mapping of IOVA to GPA. + * + * This file defines a set of constants used to construct a static page table + * for an smmu-testdev device. The goal is to translate a specific STD_IOVA + * into a final GPA. + * The translation is based on the Arm architecture with the following + * prerequisites: + * - Granule size: 4KB pages. + * - Page table levels: 4 levels (L0, L1, L2, L3), starting at level 0. + * - IOVA size: The walk resolves a 39-bit IOVA (0x8080604567). + * - Address space: The 4-level lookup with 4KB granules supports up to a + * 48-bit (256TB) virtual address space. Each level uses a 9-bit index + * (512 entries per table). The breakdown is: + * - L0 index: IOVA bits [47:39] + * - L1 index: IOVA bits [38:30] + * - L2 index: IOVA bits [29:21] + * - L3 index: IOVA bits [20:12] + * - Page offset: IOVA bits [11:0] + * + * NOTE: All physical addresses defined here (STD_VTTB, table addresses, etc.) + * appear to be within a secure RAM region. In practice, an offset is added + * to these values to place them in non-secure RAM. For example, when running + * in a virt machine type, the RAM base address (e.g., 0x40000000) is added to + * these constants. + * + * The page table walk for STD_IOVA (0x8080604567) proceeds as follows: + * + * The Translation Table Base (for both Stage 1 CD_TTB and Stage 2 STE_S2TTB) + * is set to STD_VTTB (0xe4d0000). + * + * 1. Level 0 (L0) Table Walk: + * l0_index = (0x8080604567 >> 39) & 0x1ff = 1 + * STD_L0_ADDR = STD_VTTB + (l0_index * 8) = 0xe4d0000 + 8 = 0xe4d0008 + * STD_L0_VAL = 0xe4d1003 + * The next level table base address = STD_L0_VAL & ~0xfff = 0xe4d1000 + * + * 2. Level 1 (L1) Table Walk: + * l1_index = (0x8080604567 >> 30) & 0x1ff = 2 + * STD_L1_ADDR = 0xe4d1000 + (l1_index * 8) = 0xe4d1000 + 16 = 0xe4d1010 + * STD_L1_VAL = 0xe4d2003 + * The next level table base address = STD_L1_VAL & ~0xfff = 0xe4d2000 + * + * 3. Level 2 (L2) Table Walk: + * l2_index = (0x8080604567 >> 21) & 0x1ff = 3 + * STD_L2_ADDR = 0xe4d2000 + (l2_index * 8) = 0xe4d2000 + 24 = 0xe4d2018 + * STD_L2_VAL = 0xe4d3003 + * The next level table base address = STD_L2_VAL & ~0xfff = 0xe4d3000 + * + * 4. Level 3 (L3) Table Walk (Leaf): + * l3_index = (0x8080604567 >> 12) & 0x1ff = 4 + * STD_L3_ADDR = 0xe4d3000 + (l3_index * 8) = 0xe4d3000 + 32 = 0xe4d3020 + * STD_L3_VAL = 0x040000000ECBA7C3 + * The next level table base address = STD_L3_VAL & ~0xfff = 0xecba000 + * + * 5. Final GPA Calculation: + * - The final output physical address is formed by combining the address from + * the leaf descriptor with the original IOVA's page offset. + * - Output Page Base Address = (STD_L3_VAL & ~0xFFFULL) = 0xECBA000 + * - Page Offset = (STD_IOVA & 0xFFFULL) = 0x567. + * - Final GPA = Output Page Base Address + Page Offset + * = 0xECBA000 + 0x567 = 0xECBA567 + */ + +#define STD_IOVA 0x0000008080604567ULL + +#define STD_VTTB 0xe4d0000 + +#define STD_STR_TAB_BASE 0x000000000E179000ULL +#define STD_STE_GPA (STD_STR_TAB_BASE + 0x40ULL) +#define STD_CD_GPA (STD_STR_TAB_BASE + 0x80ULL) + +/* Page table structures */ +#define STD_L0_ADDR 0x000000000E4D0008ULL +#define STD_L1_ADDR 0x000000000E4D1010ULL +#define STD_L2_ADDR 0x000000000E4D2018ULL +#define STD_L3_ADDR 0x000000000E4D3020ULL +#define STD_L0_VAL 0x000000000E4D1003ULL +#define STD_L1_VAL 0x000000000E4D2003ULL +#define STD_L2_VAL 0x000000000E4D3003ULL +#define STD_L3_VAL 0x040000000ECBA7C3ULL + +/* + * Nested stage PTW maybe a bit more complex. We share the page tables in + * nested stage 2 to avoid complicated definitions here. That is to say: + * + * At each level of the Stage 1 page table walk, a corresponding 4-level Stage 2 + * page table walk is performed. The intermediate Stage 2 page tables are shared + * across these walks, with the key connecting PTE values being: + * - l0_pte_val=0x4e4d1003 + * - l1_pte_val=0x4e4d2003 + * - l2_pte_val=0x4e4d3003 + * + * + * ====================================================================== + * Nested Page Table Walk (Stage 1 + Stage 2) Example + * ====================================================================== + * + * Goal: Translate IOVA 0x8080604567 to a final Physical Address (PA). + * + * Prerequisites: + * - Stage 1 VTTB (as IPA): 0x4e4d0000 + * - Stage 2 VTTB (as PA): 0x4e4d0000 + * + * ---------------------------------------------------------------------- + * 1. Stage 1 Page Table Walk (IOVA -> IPA) + * ---------------------------------------------------------------------- + * + * Level 0 (L0) Walk (IPA as PA) + * ============================= + * iova = 0x8080604567 + * l0_index = (0x8080604567 >> 39) & 0x1ff = 1 + * s1_l0_pte_addr (IPA) = 0x4e4d0000 + (1 * 8) = 0x4e4d0008 + * + * --> Nested Stage 2 Walk for S1 L0 Table (IPA 0x4e4d0000 -> PA 0x4e4d0000) + * ------------------------------------------------------------------------- + * ipa_to_translate = 0x4e4d0000 + * s2_vttb = 0x4e4d0000 + * s2_l0_index = (0x4e4d0000 >> 39) & 0x1ff = 0 + * s2_l0_pte_addr (PA) = 0x4e4d0000 + (0 * 8) = 0x4e4d0000 + * s2_l0_pte_val = 0x4e4d1003 + * s2_l1_table_base (PA) = 0x4e4d1003 & ~0xfff = 0x4e4d1000 + * s2_l1_index = (0x4e4d0000 >> 30) & 0x1ff = 1 + * s2_l1_pte_addr (PA) = 0x4e4d1000 + (1 * 8) = 0x4e4d1008 + * s2_l1_pte_val = 0x4e4d2003 + * s2_l2_table_base (PA) = 0x4e4d2003 & ~0xfff = 0x4e4d2000 + * s2_l2_index = (0x4e4d0000 >> 21) & 0x1ff = 114 (0x72) + * s2_l2_pte_addr (PA) = 0x4e4d2000 + (114 * 8) = 0x4e4d2390 + * s2_l2_pte_val = 0x4e4d3003 + * s2_l3_table_base (PA) = 0x4e4d3003 & ~0xfff = 0x4e4d3000 + * s2_l3_index = (0x4e4d0000 >> 12) & 0x1ff = 208 (0xd0) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (208 * 8) = 0x4e4d3680 + * s2_l3_pte_val = 0x040000000E4D0743ULL + * output_page_base (PA) = 0x040000000E4D0743ULL & ~0xfff = 0x4e4d0000 + * Final PA for table = 0x4e4d0000 + (0x0000 & 0xfff) = 0x4e4d0000 + * + * s1_l0_pte_val (read from PA 0x4e4d0008) = 0x4e4d1003 + * s1_l1_table_base (IPA) = 0x4e4d1003 & ~0xfff = 0x4e4d1000 + * + * Level 1 (L1) Walk (IPA as PA) + * ============================= + * iova = 0x8080604567 + * l1_index = (0x8080604567 >> 30) & 0x1ff = 2 + * s1_l1_pte_addr (IPA) = 0x4e4d1000 + (2 * 8) = 0x4e4d1010 + * + * --> Nested Stage 2 Walk for S1 L1 Table (IPA 0x4e4d1000 -> PA 0x4e4d1000) + * ------------------------------------------------------------------------- + * ipa_to_translate = 0x4e4d1000 + * s2_vttb = 0x4e4d0000 + * s2_l0_index = (0x4e4d1000 >> 39) & 0x1ff = 0 + * s2_l1_table_base (PA) = 0x4e4d1000 + * s2_l1_index = (0x4e4d1000 >> 30) & 0x1ff = 1 + * s2_l2_table_base (PA) = 0x4e4d2000 + * s2_l2_index = (0x4e4d1000 >> 21) & 0x1ff = 114 (0x72) + * s2_l3_table_base (PA) = 0x4e4d3000 + * s2_l3_index = (0x4e4d1000 >> 12) & 0x1ff = 209 (0xd1) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (209 * 8) = 0x4e4d3688 + * s2_l3_pte_val = 0x40000004e4d1743 + * output_page_base (PA) = 0x40000004e4d1743 & ~0xfff = 0x4e4d1000 + * Final PA for table = 0x4e4d1000 + (0x1000 & 0xfff) = 0x4e4d1000 + * + * s1_l1_pte_val (read from PA 0x4e4d1010) = 0x4e4d2003 + * s1_l2_table_base (IPA) = 0x4e4d2003 & ~0xfff = 0x4e4d2000 + * + * Level 2 (L2) Walk (IPA as PA) + * ============================= + * l2_index = (0x8080604567 >> 21) & 0x1ff = 3 + * s1_l2_pte_addr (IPA) = 0x4e4d2000 + (3 * 8) = 0x4e4d2018 + * + * --> Nested Stage 2 Walk for S1 L2 Table (IPA 0x4e4d2000 -> PA 0x4e4d2000) + * ------------------------------------------------------------------------- + * ipa_to_translate = 0x4e4d2000 + * s2_l0_index = (0x4e4d2000 >> 39) & 0x1ff = 0 + * s2_l1_table_base (PA) = 0x4e4d1000 + * s2_l1_index = (0x4e4d2000 >> 30) & 0x1ff = 1 + * s2_l2_table_base (PA) = 0x4e4d2000 + * s2_l2_index = (0x4e4d2000 >> 21) & 0x1ff = 114 (0x72) + * s2_l3_table_base (PA) = 0x4e4d3000 + * s2_l3_index = (0x4e4d2000 >> 12) & 0x1ff = 210 (0xd2) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (210 * 8) = 0x4e4d3690 + * s2_l3_pte_val = 0x40000004e4d2743 + * output_page_base (PA) = 0x40000004e4d2743 & ~0xfff = 0x4e4d2000 + * Final PA for table = 0x4e4d2000 + (0x2000 & 0xfff) = 0x4e4d2000 + * + * s1_l2_pte_val (read from PA 0x4e4d2018) = 0x4e4d3003 + * s1_l3_table_base (IPA) = 0x4e4d3003 & ~0xfff = 0x4e4d3000 + * + * Level 3 (L3) Walk (Leaf, IPA as PA) + * =================================== + * l3_index = (0x8080604567 >> 12) & 0x1ff = 4 + * s1_l3_pte_addr (IPA) = 0x4e4d3000 + (4 * 8) = 0x4e4d3020 + * + * --> Nested Stage 2 Walk for S1 L3 Table (IPA 0x4e4d3000 -> PA 0x4e4d3000) + * ------------------------------------------------------------------------- + * ipa_to_translate = 0x4e4d3000 + * s2_l0_index = (0x4e4d3000 >> 39) & 0x1ff = 0 + * s2_l1_table_base (PA) = 0x4e4d1000 + * s2_l1_index = (0x4e4d3000 >> 30) & 0x1ff = 1 + * s2_l2_table_base (PA) = 0x4e4d2000 + * s2_l2_index = (0x4e4d3000 >> 21) & 0x1ff = 114 (0x72) + * s2_l3_table_base (PA) = 0x4e4d3000 + * s2_l3_index = (0x4e4d3000 >> 12) & 0x1ff = 211 (0xd3) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (211 * 8) = 0x4e4d3698 + * s2_l3_pte_val = 0x40000004e4d3743 + * output_page_base (PA) = 0x40000004e4d3743 & ~0xfff = 0x4e4d3000 + * Final PA for table = 0x4e4d3000 + (0x3000 & 0xfff) = 0x4e4d3000 + * + * s1_l3_pte_val (read from PA 0x4e4d3020) = 0x40000004ecba743 + * output_page_base (IPA) = 0x40000004ecba743 & ~0xfff = 0x4ecba000 + * page_offset = 0x8080604567 & 0xfff = 0x567 + * Final IPA = 0x4ecba000 + 0x567 = 0x4ecba567 + * + * ---------------------------------------------------------------------- + * 2. Final Stage 2 Page Table Walk (Final IPA -> PA) + * ---------------------------------------------------------------------- + * + * ipa = 0x4ecba567 + * + * s2_l0_index = (0x4ecba567 >> 39) & 0x1ff = 0 + * s2_l1_table_base (PA) = 0x4e4d1000 + * + * s2_l1_index = (0x4ecba567 >> 30) & 0x1ff = 1 + * s2_l2_table_base (PA) = 0x4e4d2000 + * + * s2_l2_index = (0x4ecba567 >> 21) & 0x1ff = 118 (0x76) + * s2_l3_table_base (PA) = 0x4e4d3000 + * + * s2_l3_index = (0x4ecba567 >> 12) & 0x1ff = 186 (0xba) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (186 * 8) = 0x4e4d35d0 + * s2_l3_pte_val = 0x40000004ecba7c3 + * output_page_base (PA) = 0x40000004ecba7c3 & ~0xfff = 0x4ecba000 + * page_offset = 0x4ecba567 & 0xfff = 0x567 + * + * ---------------------------------------------------------------------- + * 3. Final Result + * ---------------------------------------------------------------------- + * Final PA = 0x4ecba000 + 0x567 = 0x4ecba567 + * + * ---------------------------------------------------------------------- + * 4. Appendix: Context Descriptor (CD) Fetch Walk + * ---------------------------------------------------------------------- + * Before any S1 walk can begin, the SMMU must fetch the Context Descriptor. + * The CD's address is an IPA, so it also requires a full S2 walk. This + * walk RE-USES the exact same S2 page tables shown above. + * + * ipa = 0x4e179080 (Address of the CD) + * + * s2_l0_index = (0x4e179080 >> 39) & 0x1ff = 0 + * s2_l0_pte_val = 0x4e4d1003 + * s2_l1_table_base (PA) = 0x4e4d1000 (*RE-USED*) + * + * s2_l1_index = (0x4e179080 >> 30) & 0x1ff = 1 + * s2_l1_pte_val = 0x4e4d2003 + * s2_l2_table_base (PA) = 0x4e4d2000 (*RE-USED*) + * + * s2_l2_index = (0x4e179080 >> 21) & 0x1ff = 112 (0x70) + * s2_l2_pte_val = 0x4e4d3003 + * s2_l3_table_base (PA) = 0x4e4d3000 (*RE-USED*) + * + * s2_l3_index = (0x4e179080 >> 12) & 0x1ff = 377 (0x179) + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (377 * 8) = 0x4e4d3bc8 + * s2_l3_pte_val = 0x40000004e179743 + * output_page_base (PA) = 0x40000004e179743 & ~0xfff = 0x4e179000 + * page_offset = 0x4e179080 & 0xfff = 0x080 + * + * Final PA for CD = 0x4e179000 + 0x080 = 0x4e179080 + * + */ +#define STD_CD_S2_L0_ADDR 0x000000000E4D0000ULL +#define STD_CD_S2_L1_ADDR 0x000000000E4D1008ULL +#define STD_CD_S2_L2_ADDR 0x000000000E4D2380ULL +#define STD_CD_S2_L3_ADDR 0x000000000E4D3BC8ULL +#define STD_CD_S2_L3_VAL 0x040000000E179743ULL + +#define STD_CDTTB_S2_L2_ADDR 0x000000000E4D2390ULL +#define STD_CDTTB_S2_L3_ADDR 0x000000000E4D3680ULL +#define STD_CDTTB_S2_L3_VAL 0x040000000E4D0743ULL + +#define STD_L3_S1_VAL 0x040000000ECBA743ULL + +#define STD_S1L0_IN_S2L3_ADDR 0x000000000E4D3688ULL +#define STD_S1L0_IN_S2L3_VAL 0x040000000E4D1743ULL +#define STD_S1L1_IN_S2L3_ADDR 0x000000000E4D3690ULL +#define STD_S1L1_IN_S2L3_VAL 0x040000000E4D2743ULL +#define STD_S1L2_IN_S2L3_ADDR 0x000000000E4D3698ULL +#define STD_S1L2_IN_S2L3_VAL 0x040000000E4D3743ULL +#define STD_S1L3_IN_S2L2_ADDR 0x000000000E4D23B0ULL +#define STD_S1L3_IN_S2L2_VAL 0x000000000E4D3003ULL +#define STD_S1L3_IN_S2L3_ADDR 0x000000000E4D35D0ULL +#define STD_S1L3_IN_S2L3_VAL 0x040000000ECBA7C3ULL + +/* + * Address-space base offsets for test tables. + * - Non-Secure uses a fixed offset, keeping internal layout identical. + * + * Note: Future spaces (e.g. Secure/Realm/Root) are not implemented here. + * When needed, introduce new offsets and reuse the helpers below so + * relative layout stays identical across spaces. + */ +#define STD_SPACE_OFFS_NS 0x40000000ULL + +static inline uint64_t std_space_offset(SMMUTestDevSpace sp) +{ + switch (sp) { + case STD_SPACE_NONSECURE: + return STD_SPACE_OFFS_NS; + default: + g_assert_not_reached(); + } +} + +static const char *std_space_to_str(SMMUTestDevSpace sp) +{ + switch (sp) { + case STD_SPACE_NONSECURE: + return "Non-Secure"; + default: + g_assert_not_reached(); + } +} + +static const char *std_mode_to_str(uint32_t m) +{ + switch (m & 0x3) { + case 0: return "S1-only"; + case 1: return "S2-only"; + case 2: return "Nested"; + default: + g_assert_not_reached(); + } +} + +#endif /* HW_MISC_SMMU_TESTDEV_H */ -- 2.49.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device 2025-09-30 16:53 ` [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device tangtao1634 @ 2025-10-23 10:31 ` Alex Bennée 2025-10-27 14:54 ` Tao Tang 0 siblings, 1 reply; 16+ messages in thread From: Alex Bennée @ 2025-10-23 10:31 UTC (permalink / raw) To: tangtao1634 Cc: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh tangtao1634 <tangtao1634@phytium.com.cn> writes: > From: Tao Tang <tangtao1634@phytium.com.cn> > > Add a tiny, test-only DMA source dedicated to exercising the SMMUv3 model. > The device purposefully avoids a realistic PCIe/platform implementation and > instead routes DMA requests straight into the SMMU, so that qtests can > populate STE/CD/PTE with known values and observe translation and data > movement deterministically, without booting any firmware or guest kernel. > > Motivation > ---------- > Bringing up and regression-testing the SMMU in emulation often depends on a > large and flaky software stack (enumeration, drivers, PCIe fabric). For the > class of tests that only need to (1) program translation structures and (2) > trigger DMA at a precise time, that stack adds noise, slows CI, and makes > failures harder to attribute to the SMMU itself. A hermetic DMA source > keeps the surface area small and the results reproducible. > > What this device is (and is not) > -------------------------------- > * It is a minimal DMA producer solely for SMMU tests. > * It is NOT a faithful PCIe Endpoint nor a platform device. > * It is NOT added to any machine by default and remains test-only. > > Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> > --- > hw/misc/Kconfig | 5 + > hw/misc/meson.build | 1 + > hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++++ > include/hw/misc/smmu-testdev.h | 402 ++++++++++++++ > 4 files changed, 1351 insertions(+) > create mode 100644 hw/misc/smmu-testdev.c > create mode 100644 include/hw/misc/smmu-testdev.h > > diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig > index 4e35657468..c83a0872ef 100644 > --- a/hw/misc/Kconfig > +++ b/hw/misc/Kconfig > @@ -25,6 +25,11 @@ config PCI_TESTDEV > default y if TEST_DEVICES > depends on PCI > > +config SMMU_TESTDEV > + bool > + default y if TEST_DEVICES > + depends on PCI && ARM_SMMUV3 > + > config EDU > bool > default y if TEST_DEVICES > diff --git a/hw/misc/meson.build b/hw/misc/meson.build > index b1d8d8e5d2..862a9895c3 100644 > --- a/hw/misc/meson.build > +++ b/hw/misc/meson.build > @@ -4,6 +4,7 @@ system_ss.add(when: 'CONFIG_FW_CFG_DMA', if_true: files('vmcoreinfo.c')) > system_ss.add(when: 'CONFIG_ISA_DEBUG', if_true: files('debugexit.c')) > system_ss.add(when: 'CONFIG_ISA_TESTDEV', if_true: files('pc-testdev.c')) > system_ss.add(when: 'CONFIG_PCI_TESTDEV', if_true: files('pci-testdev.c')) > +system_ss.add(when: 'CONFIG_SMMU_TESTDEV', if_true: files('smmu-testdev.c')) > system_ss.add(when: 'CONFIG_UNIMP', if_true: files('unimp.c')) > system_ss.add(when: 'CONFIG_EMPTY_SLOT', if_true: files('empty_slot.c')) > system_ss.add(when: 'CONFIG_LED', if_true: files('led.c')) > diff --git a/hw/misc/smmu-testdev.c b/hw/misc/smmu-testdev.c > new file mode 100644 > index 0000000000..156f4cd714 > --- /dev/null > +++ b/hw/misc/smmu-testdev.c > @@ -0,0 +1,943 @@ > +/* > + * A test device for the SMMU > + * > + * This test device is a minimal SMMU-aware device used to test the SMMU. > + * > + * Copyright (c) 2025 Phytium Technology > + * > + * Author: > + * Tao Tang <tangtao1634@phytium.com.cn> > + * > + * SPDX-License-Identifier: GPL-2.0-or-later > + */ > + > +#include "qemu/osdep.h" > +#include "qemu/log.h" > +#include "qemu/module.h" > +#include "qemu/units.h" > +#include "system/address-spaces.h" > +#include "exec/memattrs.h" > +#include "system/dma.h" > +#include "hw/pci/pci.h" > +#include "hw/pci/pci_device.h" > +#include "hw/arm/smmu-common.h" > +#include "hw/qdev-properties.h" > +#include "qom/object.h" > +#include "hw/misc/smmu-testdev.h" > + > +#define TYPE_SMMU_TESTDEV "smmu-testdev" > +OBJECT_DECLARE_SIMPLE_TYPE(SMMUTestDevState, SMMU_TESTDEV) > + > +struct SMMUTestDevState { > + PCIDevice parent_obj; > + MemoryRegion bar0; > + uint32_t attr_ns; /* Track Non-Secure for now; reserve room for more. */ > + > + uint64_t smmu_base; > + uint64_t dma_iova; > + uint32_t dma_len; > + uint32_t dma_dir; > + uint32_t dma_result; > + bool dma_pending; > + > + /* Future-proof DMA config */ > + AddressSpace *dma_as; /* IOMMU-mediated DMA AS for this device */ > + uint32_t dma_mode; /* 0=legacy pci_dma, 1=attrs via dma_memory_* */ > + uint32_t dma_attrs_cfg; /* bit0 secure, bits[2:1] space, bit3 unspecified */ > + > + /* Translation build configuration */ > + uint32_t trans_mode; /* 0=S1, 1=S2, 2=Nested */ > + SMMUTestDevSpace s1_space; > + SMMUTestDevSpace s2_space; > + uint32_t trans_status; /* 0=ok; non-zero=error */ > + > + /* User-configurable BDF (device/function) */ > + uint32_t cfg_dev; /* PCI device/slot number (0..31) */ > + uint32_t cfg_fn; /* PCI function number (0..7) */ > + > + bool debug_log; /* Enable verbose debug output */ > +}; > + > +/* BAR0 layout */ > +enum { > + REG_ID = 0x00, > + REG_ATTR_NS = 0x04, > + REG_SMMU_BASE_LO = 0x20, > + REG_SMMU_BASE_HI = 0x24, > + REG_DMA_IOVA_LO = 0x28, > + REG_DMA_IOVA_HI = 0x2C, > + REG_DMA_LEN = 0x30, > + REG_DMA_DIR = 0x34, > + REG_DMA_RESULT = 0x38, > + REG_DMA_DOORBELL = 0x3C, > + /* Extended controls for DMA attributes/mode (kept after legacy regs) */ > + REG_DMA_MODE = 0x40, /* 0: legacy; 1: attrs path */ > + REG_DMA_ATTRS = 0x44, /* [0] secure, [2:1] space, [3] unspecified */ > + /* Translation config & builder */ > + REG_TRANS_MODE = 0x48, /* 0=S1 only, 1=S2 only, 2=Nested */ > + REG_S1_SPACE = 0x4C, /* SMMUTestDevSpace for stage-1 path */ > + REG_S2_SPACE = 0x50, /* SMMUTestDevSpace for stage-2 path */ > + REG_TRANS_DBELL = 0x54, /* bit0=build, bit1=clear status */ > + REG_TRANS_STATUS = 0x58, /* 0=ok else error */ > + REG_TRANS_CLEAR = 0x5C, /* write-any: clear helper-built CD/STE/PTE */ > + BAR0_SIZE = 0x1000, > +}; > + > +#define DMA_DIR_DEV2HOST 0u > +#define DMA_DIR_HOST2DEV 1u > +#define DMA_RESULT_IDLE 0xffffffffu > +#define DMA_RESULT_BUSY 0xfffffffeu > +#define DMA_ERR_BAD_LEN 0xdead0001u > +#define DMA_ERR_TX_FAIL 0xdead0002u > +#define DMA_MAX_LEN (64 * KiB) > + > +/* > + * Compile-time calculated STE field setting macros. > + */ > + > + #define STD_STE_OR_CD_ENTRY_BYTES 64 /* 64 bytes per STE entry */ > + #define STD_STE_S2T0SZ_VAL 0x14 > + > +#define STD_STE_SET_VALID(ste, val) \ > + ((ste)->word[0] = ((ste)->word[0] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) > +#define STD_STE_SET_CONFIG(ste, val) \ > + ((ste)->word[0] = ((ste)->word[0] & ~(0x7 << 1)) | (((val) & 0x7) << 1)) > +#define STD_STE_SET_S1FMT(ste, val) \ > + ((ste)->word[0] = ((ste)->word[0] & ~(0x3 << 4)) | (((val) & 0x3) << 4)) > + > +#define STD_STE_SET_CTXPTR(ste, val) \ > +do { \ > + /* Lower address bits (31:6) occupy the upper 26 bits of word[0]. */ \ > + (ste)->word[0] = ((ste)->word[0] & 0x0000003FU) | \ > + ((uint32_t)(val) & 0xFFFFFFC0U); \ > + \ > + /* Upper address bits (47:32) occupy the low 16 bits of word[1]. */ \ > + (ste)->word[1] = ((ste)->word[1] & 0xFFFF0000U) | \ > + ((uint32_t)(((uint64_t)(val)) >> 32) & 0x0000FFFFU); \ > +} while (0) > + > +#define STD_STE_SET_S1CDMAX(ste, val) \ > + ((ste)->word[1] = ((ste)->word[1] & ~(0x1f << 27)) | (((val) & 0x1f) << 27)) > +#define STD_STE_SET_S1STALLD(ste, val) \ > + ((ste)->word[2] = ((ste)->word[2] & ~(0x1 << 27)) | (((val) & 0x1) << 27)) > +#define STD_STE_SET_EATS(ste, val) \ > + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 28)) | (((val) & 0x3) << 28)) > +#define STD_STE_SET_STRW(ste, val) \ > + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 30)) | (((val) & 0x3) << 30)) > +#define STD_STE_SET_NSCFG(ste, val) \ > + ((ste)->word[2] = ((ste)->word[2] & ~(0x3 << 14)) | (((val) & 0x3) << 14)) > +#define STD_STE_SET_S2VMID(ste, val) \ > + ((ste)->word[4] = ((ste)->word[4] & ~0xffff) | ((val) & 0xffff)) > +#define STD_STE_SET_S2T0SZ(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~0x3f) | ((val) & 0x3f)) > +#define STD_STE_SET_S2SL0(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x3 << 6)) | (((val) & 0x3) << 6)) > +#define STD_STE_SET_S2TG(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x3 << 14)) | (((val) & 0x3) << 14)) > +#define STD_STE_SET_S2PS(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x7 << 16)) | (((val) & 0x7) << 16)) > +#define STD_STE_SET_S2AA64(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 19)) | (((val) & 0x1) << 19)) > +#define STD_STE_SET_S2ENDI(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 20)) | (((val) & 0x1) << 20)) > +#define STD_STE_SET_S2AFFD(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 21)) | (((val) & 0x1) << 21)) > +#define STD_STE_SET_S2HD(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 23)) | (((val) & 0x1) << 23)) > +#define STD_STE_SET_S2HA(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 24)) | (((val) & 0x1) << 24)) > +#define STD_STE_SET_S2S(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 25)) | (((val) & 0x1) << 25)) > +#define STD_STE_SET_S2R(ste, val) \ > + ((ste)->word[5] = ((ste)->word[5] & ~(0x1 << 26)) | (((val) & 0x1) << 26)) > + > +#define STD_STE_SET_S2TTB(ste, val) \ > +do { \ > + /* Lower address bits (31:4) occupy the upper 28 bits of word[6]. */ \ > + (ste)->word[6] = ((ste)->word[6] & 0x0000000FU) | \ > + ((uint32_t)(val) & 0xFFFFFFF0U); \ > + \ > + /* Upper address bits (51:32) occupy the low 20 bits of word[7]. */ \ > + (ste)->word[7] = ((ste)->word[7] & 0xFFF00000U) | \ > + ((uint32_t)(((uint64_t)(val)) >> 32) & \ > + 0x000FFFFFU); \ > +} while (0) > + > +#define STE_S2TTB(x) \ > + ((extract64((x)->word[7], 0, 16) << 32) | \ > + ((x)->word[6] & 0xfffffff0)) > + > +/* > + * Compile-time calculated CD field setting macros. > + */ > +#define STD_CD_SET_VALID(cd, val) \ > + ((cd)->word[0] = ((cd)->word[0] & ~(0x1 << 31)) | \ > + (((val) & 0x1) << 31)) > +#define STD_CD_SET_TSZ(cd, sel, val) \ > + ((cd)->word[0] = ((cd)->word[0] & \ > + ~(0x3F << ((sel) * 16 + 0))) | \ > + (((val) & 0x3F) << ((sel) * 16 + 0))) > +#define STD_CD_SET_TG(cd, sel, val) \ > + ((cd)->word[0] = ((cd)->word[0] & \ > + ~(0x3 << ((sel) * 16 + 6))) | \ > + (((val) & 0x3) << ((sel) * 16 + 6))) > +#define STD_CD_SET_EPD(cd, sel, val) \ > + ((cd)->word[0] = ((cd)->word[0] & \ > + ~(0x1 << ((sel) * 16 + 14))) | \ > + (((val) & 0x1) << ((sel) * 16 + 14))) > +#define STD_CD_SET_ENDI(cd, val) \ > + ((cd)->word[0] = ((cd)->word[0] & ~(0x1 << 15)) | \ > + (((val) & 0x1) << 15)) > +#define STD_CD_SET_IPS(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x7 << 0)) | \ > + (((val) & 0x7) << 0)) > +#define STD_CD_SET_AFFD(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 3)) | \ > + (((val) & 0x1) << 3)) > +#define STD_CD_SET_HD(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 10)) | \ > + (((val) & 0x1) << 10)) > +#define STD_CD_SET_HA(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 11)) | \ > + (((val) & 0x1) << 11)) > +#define STD_CD_SET_TTB(cd, sel, val) do { \ > + (cd)->word[(sel) * 2 + 2] = ((cd)->word[(sel) * 2 + 2] & 0x0000000F) | \ > + ((val) & 0xFFFFFFF0); \ > + (cd)->word[(sel) * 2 + 3] = ((cd)->word[(sel) * 2 + 3] & 0xFFF80000) | \ > + ((((uint64_t)(val)) >> 32) & 0x0007FFFF); \ > +} while (0) > +#define STD_CD_SET_HAD(cd, sel, val) \ > + ((cd)->word[(sel) * 2 + 2] = ((cd)->word[(sel) * 2 + 2] & \ > + ~(0x1 << 1)) | \ > + (((val) & 0x1) << 1)) > +#define STD_CD_SET_MAIR0(cd, val) ((cd)->word[6] = (val)) > +#define STD_CD_SET_MAIR1(cd, val) ((cd)->word[7] = (val)) > +#define STD_CD_SET_TCR_T0SZ(cd, val) \ > + ((cd)->word[4] = ((cd)->word[4] & ~0x3F) | ((val) & 0x3F)) > +#define STD_CD_SET_ASID(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0xFFFF << 16)) | \ > + (((val) & 0xFFFF) << 16)) > +#define STD_CD_SET_S(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 12)) | \ > + (((val) & 0x1) << 12)) > +#define STD_CD_SET_R(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 13)) | \ > + (((val) & 0x1) << 13)) > +#define STD_CD_SET_A(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 14)) | \ > + (((val) & 0x1) << 14)) > +#define STD_CD_SET_AARCH64(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x1 << 9)) | \ > + (((val) & 0x1) << 9)) > +#define STD_CD_SET_TBI(cd, val) \ > + ((cd)->word[1] = ((cd)->word[1] & ~(0x3 << 6)) | \ > + (((val) & 0x3) << 6)) > +#define STD_CD_SET_NSCFG0(cd, val) \ > + ((cd)->word[2] = ((cd)->word[2] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) > +#define STD_CD_SET_NSCFG1(cd, val) \ > + ((cd)->word[4] = ((cd)->word[4] & ~(0x1 << 0)) | (((val) & 0x1) << 0)) > + > +typedef enum TransMode { > + TM_S1_ONLY = 0, > + TM_S2_ONLY = 1, > + TM_NESTED = 2, > +} TransMode; > + > +/* Minimal STE/CD images (bit layout derived from test helpers) */ > +typedef struct { > + uint32_t word[8]; > +} STEImg; > + > +typedef struct { > + uint32_t word[8]; > +} CDImg; > + > +/* ---- Debug helpers for printing current translation configuration ---- */ > +static void G_GNUC_PRINTF(2, 3) > +smmu_testdev_debug(const SMMUTestDevState *s, const char *fmt, ...) > +{ > + va_list ap; > + g_autofree char *msg = NULL; > + > + if (!s->debug_log) { > + return; > + } > + > + va_start(ap, fmt); > + msg = g_strdup_vprintf(fmt, ap); > + va_end(ap); > + > + if (qemu_log_enabled()) { > + qemu_log("%s", msg); > + } else { > + fprintf(stderr, "%s", msg); > + } > +} Why are we re-inventing logging here? Either use qemu_log or probably better define tracepoints. > + > +/* Only support Non-Secure space for now. */ > +static bool smmu_testdev_space_supported(SMMUTestDevSpace sp) > +{ > + return sp == STD_SPACE_NONSECURE; > +} > + > +static MemTxAttrs mk_attrs_from_space(SMMUTestDevSpace space) > +{ > + MemTxAttrs a = {0}; > + if (!smmu_testdev_space_supported(space)) { > + g_assert_not_reached(); Isn't this the same as just saying: g_assert(smmu_testdev_space_supported(space)); ? > + } else { > + a.space = space; > + } > + a.secure = 0; > + return a; > +} > + > +/* Convert SMMUTestDevSpace to AddressSpace */ > +static inline AddressSpace *space_to_as(SMMUTestDevSpace sp) > +{ > + /* Future work can dispatch Secure/Realm/Root address spaces here. */ > + if (!smmu_testdev_space_supported(sp)) { > + g_assert_not_reached(); > + } > + return &address_space_memory; > +} > + > +/* Apply per-space offset for addresses or values that encode addresses. */ > +static inline uint64_t std_apply_space_offs(SMMUTestDevSpace sp, uint64_t x) > +{ > + return x + std_space_offset(sp); > +} > + > +/* Direct write helpers (no mirroring) */ > +static void std_write64(SMMUTestDevSpace sp, uint64_t pa, uint64_t val, > + uint32_t *status) > +{ > + MemTxAttrs a = mk_attrs_from_space(sp); > + AddressSpace *as = space_to_as(sp); > + if (!as) { > + *status = 0xdead2011u; > + return; > + } > + MemTxResult r = address_space_write(as, pa, a, &val, sizeof(val)); > + if (r != MEMTX_OK && status) { > + *status = 0xdead2011u; > + return; > + } > + *status = 0; > +} > + > +static void std_write32(SMMUTestDevSpace sp, uint64_t pa, uint32_t val, > + uint32_t *status) > +{ > + MemTxAttrs a = mk_attrs_from_space(sp); > + AddressSpace *as = space_to_as(sp); > + if (!as) { > + *status = 0xdead2012u; > + return; > + } > + MemTxResult r = address_space_write(as, pa, a, &val, sizeof(val)); > + if (r != MEMTX_OK && status) { > + *status = 0xdead2012u; > + return; > + } > + *status = 0; > +} > + > +/* Build the translation tables with specified stage and security spaces. */ > +static void smmu_testdev_build_translation(SMMUTestDevState *s) > +{ > + smmu_testdev_debug(s, "smmu_testdev_build_translation: stage=%s s1_space=%s" > + " s2_space=%s\n", std_mode_to_str(s->trans_mode), > + std_space_to_str(s->s1_space), > + std_space_to_str(s->s2_space)); tracepoint > + uint32_t st = 0; > + SMMUTestDevSpace build_space = > + (s->trans_mode == TM_S1_ONLY) ? s->s1_space : s->s2_space; > + > + if (!smmu_testdev_space_supported(build_space) || > + (s->trans_mode != TM_S2_ONLY && > + !smmu_testdev_space_supported(s->s1_space))) { > + /* Only the Non-Secure space is supported until more domains land. */ > + s->trans_status = 0xdead3001u; > + return; > + } > + > + /* > + * Build base page tables (L0..L3) in the chosen space. > + * For Non-Secure, place tables at Secure-base + space offset and > + * update descriptor values by the same offset to keep internal > + * relationships identical across spaces. > + */ > + uint64_t L0_pa = std_apply_space_offs(build_space, STD_L0_ADDR); > + uint64_t L1_pa = std_apply_space_offs(build_space, STD_L1_ADDR); > + uint64_t L2_pa = std_apply_space_offs(build_space, STD_L2_ADDR); > + uint64_t L3_pa = std_apply_space_offs(build_space, STD_L3_ADDR); > + uint64_t L0_val = std_apply_space_offs(build_space, STD_L0_VAL); > + uint64_t L1_val = std_apply_space_offs(build_space, STD_L1_VAL); > + uint64_t L2_val = std_apply_space_offs(build_space, STD_L2_VAL); > + uint64_t L3_val = std_apply_space_offs(build_space, STD_L3_VAL); > + std_write64(build_space, L0_pa, L0_val, &st); > + std_write64(build_space, L1_pa, L1_val, &st); > + std_write64(build_space, L2_pa, L2_val, &st); > + std_write64(build_space, L3_pa, L3_val, &st); > + > + /* Build STE image */ > + STEImg ste = {0}; > + switch (s->trans_mode) { > + case TM_S1_ONLY: > + STD_STE_SET_CONFIG(&ste, 0x5); > + break; > + case TM_S2_ONLY: > + STD_STE_SET_CONFIG(&ste, 0x6); > + break; > + case TM_NESTED: > + default: > + STD_STE_SET_CONFIG(&ste, 0x7); > + break; > + } > + > + uint64_t vttb = STD_VTTB; > + STD_STE_SET_VALID(&ste, 1); > + STD_STE_SET_S2T0SZ(&ste, STD_STE_S2T0SZ_VAL); > + STD_STE_SET_S2SL0(&ste, 0x2); /* Start level 0*/ > + STD_STE_SET_S2TG(&ste, 0); /* 4KB */ > + STD_STE_SET_S2PS(&ste, 0x5); /* 48 bits */ > + STD_STE_SET_S2AA64(&ste, 1); /* Enable S2AA64 (64-bit address format). */ > + STD_STE_SET_S2ENDI(&ste, 0); /* Little Endian */ > + STD_STE_SET_S2AFFD(&ste, 0); /* AF Fault Disable */ > + > + STD_STE_SET_S2T0SZ(&ste, STD_STE_S2T0SZ_VAL); > + STD_STE_SET_S2SL0(&ste, 0x2); /* Start level */ > + STD_STE_SET_S2TG(&ste, 0); /* 4KB */ > + STD_STE_SET_S2PS(&ste, 0x5); /* 48 bits ?*/ > + /* Set Context Pointer (S1ContextPtr) */ > + STD_STE_SET_CTXPTR(&ste, std_apply_space_offs(build_space, STD_CD_GPA)); > + > + STD_STE_SET_S2TTB(&ste, std_apply_space_offs(build_space, vttb)); > + > + /* > + * Start assembling the STE, which is 64 bytes in total. > + */ > + for (int i = 0; i < 8; i++) { > + std_write32(build_space, > + std_apply_space_offs(build_space, STD_STE_GPA) + i * 4, > + ste.word[i], &st); > + if (st != 0) { > + printf("Writing STE error! status: %x\n", st); qemu_log_mask(LOG_GUEST_ERROR, ... > + return; > + } > + } > + > + /* Build CD image for S1 path if needed */ > + if (s->trans_mode != TM_S2_ONLY) { > + CDImg cd = {0}; > + > + STD_CD_SET_ASID(&cd, 0x1e20); /* ASID */ > + STD_CD_SET_AARCH64(&cd, 1); /* AA64 */ > + STD_CD_SET_VALID(&cd, 1); > + STD_CD_SET_A(&cd, 1); > + STD_CD_SET_S(&cd, 0); > + STD_CD_SET_HD(&cd, 0); > + STD_CD_SET_HA(&cd, 0); > + STD_CD_SET_IPS(&cd, 0x4); > + STD_CD_SET_TBI(&cd, 0x0); > + STD_CD_SET_AFFD(&cd, 0x0); > + /* Disable TTB0 translation table walk */ > + STD_CD_SET_EPD(&cd, 0, 0x0); > + /* Enable TTB1 translation table walk */ > + STD_CD_SET_EPD(&cd, 1, 0x1); > + STD_CD_SET_TSZ(&cd, 0, 0x10); > + STD_CD_SET_TG(&cd, 0, 0x0); > + STD_CD_SET_ENDI(&cd, 0x0); > + STD_CD_SET_NSCFG0(&cd, 0x0); > + STD_CD_SET_NSCFG1(&cd, 0x0); > + STD_CD_SET_R(&cd, 0x1); > + > + /* > + * CD belongs to S1 path: compute offsets using s1_space so the > + * GPA and embedded addresses are consistent with that space. > + */ > + uint64_t cd_ttb = std_apply_space_offs(build_space, vttb); > + smmu_testdev_debug(s, "STD_CD_SET_TTB: 0x%llx\n", > + (unsigned long long)cd_ttb); > + STD_CD_SET_TTB(&cd, 0, cd_ttb); > + > + for (int i = 0; i < 8; i++) { > + std_write32(s->s1_space, > + std_apply_space_offs(s->s1_space, STD_CD_GPA) + i * 4, > + cd.word[i], &st); > + } > + > + L3_val = std_apply_space_offs(build_space, STD_L3_S1_VAL); > + std_write64(build_space, L3_pa, L3_val, &st); > + } > + > + /* Nested extras: CD S2 tables, CD.TTB S2 tables, shared entries. */ > + if (s->trans_mode == TM_NESTED) { > + /* CD.S2 tables */ > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CD_S2_L0_ADDR), > + std_apply_space_offs(build_space, STD_L0_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CD_S2_L1_ADDR), > + std_apply_space_offs(build_space, STD_L1_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CD_S2_L2_ADDR), > + std_apply_space_offs(build_space, STD_L2_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CD_S2_L3_ADDR), > + std_apply_space_offs(build_space, > + STD_CD_S2_L3_VAL), &st); > + > + /* CD.TTB S2 tables */ > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CDTTB_S2_L2_ADDR), > + std_apply_space_offs(build_space, STD_L2_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_CDTTB_S2_L3_ADDR), > + std_apply_space_offs(build_space, > + STD_CDTTB_S2_L3_VAL), &st); > + > + /* Shared mappings between S1 and S2 page tables */ > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_S1L0_IN_S2L3_ADDR), > + std_apply_space_offs(build_space, > + STD_S1L0_IN_S2L3_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_S1L1_IN_S2L3_ADDR), > + std_apply_space_offs(build_space, > + STD_S1L1_IN_S2L3_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_S1L2_IN_S2L3_ADDR), > + std_apply_space_offs(build_space, > + STD_S1L2_IN_S2L3_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_S1L3_IN_S2L2_ADDR), > + std_apply_space_offs(build_space, > + STD_S1L3_IN_S2L2_VAL), &st); > + std_write64(build_space, > + std_apply_space_offs(build_space, STD_S1L3_IN_S2L3_ADDR), > + std_apply_space_offs(build_space, > + STD_S1L3_IN_S2L3_VAL), &st); > + } > + > + s->trans_status = st; > +} > + > +/* Manipulate SMMU command to invalidate caches */ > +static void push_cfgi_cmd(SMMUTestDevState *s, > + SMMUTestDevSpace bank_sp, > + uint32_t type, > + uint32_t sid, > + bool ssec) > +{ > + MemTxResult res = 0; > + g_assert(smmu_testdev_space_supported(bank_sp)); > + g_assert(!ssec); > + hwaddr bank_off = 0; > + uint32_t base_lo = address_space_ldl_le(&address_space_memory, > + s->smmu_base + bank_off + 0x90, > + MEMTXATTRS_UNSPECIFIED, &res); > + uint32_t base_hi = address_space_ldl_le(&address_space_memory, > + s->smmu_base + bank_off + 0x94, > + MEMTXATTRS_UNSPECIFIED, &res); > + uint64_t base = ((uint64_t)base_hi << 32) | base_lo; > + uint32_t log2size = base & 0x1f; > + uint64_t qbase = base & 0xfffffffffffc0ULL; > + uint32_t prod = address_space_ldl_le(&address_space_memory, > + s->smmu_base + bank_off + 0x98, > + MEMTXATTRS_UNSPECIFIED, &res); > + uint32_t index_mask = (1u << log2size) - 1u; > + uint32_t slot = prod & index_mask; > + uint64_t entry_pa = qbase + (uint64_t)slot * 16u; > + > + uint32_t words[4] = {0}; > + words[0] = (type & 0xff) | (ssec ? (1u << 10) : 0u); > + words[1] = sid; > + > + /* push command to the command queue */ > + MemTxAttrs a = mk_attrs_from_space(bank_sp); > + AddressSpace *as = space_to_as(bank_sp); > + if (!as) { > + printf("push_cfgi_cmd: space %d not supported\n", bank_sp); qemu_log_mask(LOG_UNIMP? > + return; > + } > + int ret = address_space_write(as, entry_pa, a, > + words, sizeof(words)); > + smmu_testdev_debug(s, "push_cfgi_cmd ret %d\n", ret); tracepoint > + > + /* update PROD to trigger command handler */ > + uint32_t new_prod = (prod + 1) & ((1u << (log2size + 1)) - 1u); > + address_space_stl_le(&address_space_memory, > + s->smmu_base + bank_off + 0x98, > + new_prod, MEMTXATTRS_UNSPECIFIED, &res); > + smmu_testdev_debug(s, "last res %d\n", res); tracepoint but really see if it can be merged with above. > +} > + > +/* Clear all the cache to avoid the incorrect cache hits using SMMU commands */ > +static void smmu_testdev_clear_caches(SMMUTestDevState *s) > +{ > + uint32_t st = 0; > + static const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; > + > + for (size_t idx = 0; idx < ARRAY_SIZE(spaces); idx++) { > + SMMUTestDevSpace build_space = spaces[idx]; > + if (!smmu_testdev_space_supported(build_space)) { > + continue; > + } > + /* Clear L0..L3 entries written by the builder. */ > + uint64_t L0_pa = std_apply_space_offs(build_space, STD_L0_ADDR); > + uint64_t L1_pa = std_apply_space_offs(build_space, STD_L1_ADDR); > + uint64_t L2_pa = std_apply_space_offs(build_space, STD_L2_ADDR); > + uint64_t L3_pa = std_apply_space_offs(build_space, STD_L3_ADDR); > + uint64_t S2_L0_pa = std_apply_space_offs(build_space, > + STD_CD_S2_L0_ADDR); > + uint64_t S2_L1_pa = std_apply_space_offs(build_space, > + STD_CD_S2_L1_ADDR); > + uint64_t S2_L2_pa = std_apply_space_offs(build_space, > + STD_CD_S2_L2_ADDR); > + uint64_t S2_L3_pa = std_apply_space_offs(build_space, > + STD_CD_S2_L3_ADDR); > + > + std_write64(build_space, L0_pa, 0ull, &st); > + std_write64(build_space, L1_pa, 0ull, &st); > + std_write64(build_space, L2_pa, 0ull, &st); > + std_write64(build_space, L3_pa, 0ull, &st); > + std_write64(build_space, S2_L0_pa, 0ull, &st); > + std_write64(build_space, S2_L1_pa, 0ull, &st); > + std_write64(build_space, S2_L2_pa, 0ull, &st); > + std_write64(build_space, S2_L3_pa, 0ull, &st); > + > + /* Clear STE image where it was placed. */ > + for (int i = 0; i < 8; i++) { > + std_write32(build_space, > + std_apply_space_offs(build_space, STD_STE_GPA) + i * 4, > + 0u, &st); > + } > + > + /* Clear CD image in S1 space (matches builder placement). */ > + for (int i = 0; i < 8; i++) { > + std_write32(build_space, > + std_apply_space_offs(build_space, STD_CD_GPA) + i * 4, > + 0u, &st); > + } > + } > + > + /* Invalidate configuration cache via CFGI_STE and CFGI_CD commands */ > + if (s->smmu_base) { > + /* Compute this PCI function's StreamID: bus 0, current devfn. */ > + uint8_t devfn = PCI_DEVICE(&s->parent_obj)->devfn; > + uint32_t sid = PCI_BUILD_BDF(0, devfn); > + > + /* Non-secure bank invalidations (SSEC=0). */ > + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_CFGI_STE, sid, false); > + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_CFGI_CD, sid, false); > + push_cfgi_cmd(s, STD_SPACE_NONSECURE, STD_CMD_TLBI_NSNH_ALL, > + sid, false); > + > + /* Add Secure/Realm/Root invalidations here once those domains exist. */ > + } > +} > + > +static void smmu_testdev_refresh_attrs(SMMUTestDevState *s) > +{ > + /* Report the baked-in Non-Secure attributes until more exist. */ > + s->attr_ns = (STD_SPACE_NONSECURE << 1); > +} > + > +/* Trigger a DMA operation */ > +static void smmu_testdev_maybe_run_dma(SMMUTestDevState *s) > +{ > + if (!s->dma_pending) { > + return; > + } > + smmu_testdev_debug(s, "smmu_testdev_maybe_run_dma: dma_pending: %d\n", > + s->dma_pending); tracepoint > + > + s->dma_pending = false; > + > + if (!s->dma_len || s->dma_len > DMA_MAX_LEN) { > + s->dma_result = DMA_ERR_BAD_LEN; > + return; > + } > + > + g_autofree uint8_t *buf = g_malloc(s->dma_len); > + MemTxResult res; Keep local vars at the top of the block. > + > + if (s->dma_mode == 0) { > + if (s->dma_dir == DMA_DIR_HOST2DEV) { > + res = pci_dma_read(PCI_DEVICE(&s->parent_obj), s->dma_iova, > + buf, s->dma_len); > + } else { > + for (uint32_t i = 0; i < s->dma_len; i++) { > + buf[i] = 0xA0u + (i & 0x1fu); > + } > + res = pci_dma_write(PCI_DEVICE(&s->parent_obj), s->dma_iova, > + buf, s->dma_len); > + } > + } else { > + SMMUTestDevSpace dma_space = > + (SMMUTestDevSpace)((s->dma_attrs_cfg >> 1) & 0x3); > + if (!smmu_testdev_space_supported(dma_space)) { > + /* Default to Non-Secure until other spaces are modeled. */ > + dma_space = STD_SPACE_NONSECURE; > + } > + MemTxAttrs attrs = { > + .secure = 0, > + .space = dma_space, > + .unspecified = (s->dma_attrs_cfg & (1u << 3)) ? 1 : 0, > + }; > + /* > + * If 'unspecified' is set, bypass IOMMU AS and use system memory. > + * This helps tests that want deterministic success without full > + * IOMMU programming. > + */ > + AddressSpace *as = (s->dma_as && !attrs.unspecified) > + ? s->dma_as > + : &address_space_memory; > + if (s->dma_dir == DMA_DIR_HOST2DEV) { > + res = dma_memory_read(as, s->dma_iova, buf, s->dma_len, attrs); > + } else { > + for (uint32_t i = 0; i < s->dma_len; i++) { > + buf[i] = 0xA0u + (i & 0x1fu); > + } > + res = dma_memory_write(as, s->dma_iova, buf, s->dma_len, attrs); > + } > + } > + s->dma_result = (res == MEMTX_OK) ? 0 : DMA_ERR_TX_FAIL; > + smmu_testdev_debug(s, "iommu ret %d , dma_result: 0x%x\n", > + res, s->dma_result); > +} > + > +static uint64_t smmu_testdev_mmio_read(void *opaque, hwaddr addr, unsigned size) > +{ > + SMMUTestDevState *s = opaque; > + switch (addr) { > + case REG_ID: > + /* > + * Only reads of REG_ID intentionally trigger the side effects > + * (SMMU CR0 write and pending DMA). This lets tests poll > + * REG_DMA_RESULT to observe BUSY before consuming the DMA. > + */ > + smmu_testdev_maybe_run_dma(s); > + return 0x53544d4du; /* 'STMM' */ > + case REG_ATTR_NS: > + return s->attr_ns; > + case REG_SMMU_BASE_LO: > + return (uint32_t)(s->smmu_base & 0xffffffffu); > + case REG_SMMU_BASE_HI: > + return (uint32_t)(s->smmu_base >> 32); > + case REG_DMA_IOVA_LO: > + return (uint32_t)(s->dma_iova & 0xffffffffu); > + case REG_DMA_IOVA_HI: > + return (uint32_t)(s->dma_iova >> 32); > + case REG_DMA_LEN: > + return s->dma_len; > + case REG_DMA_DIR: > + return s->dma_dir; > + case REG_DMA_RESULT: > + return s->dma_result; > + case REG_DMA_MODE: > + return s->dma_mode; > + case REG_DMA_ATTRS: > + return s->dma_attrs_cfg; > + case REG_TRANS_MODE: > + return s->trans_mode; > + case REG_S1_SPACE: > + return s->s1_space; > + case REG_S2_SPACE: > + return s->s2_space; > + case REG_TRANS_STATUS: > + return s->trans_status; > + default: > + return 0; > + } > +} > + > +static void smmu_testdev_mmio_write(void *opaque, hwaddr addr, uint64_t val, > + unsigned size) > +{ > + SMMUTestDevState *s = opaque; > + uint32_t data = val; > + > + switch (addr) { > + case REG_ID: > + if (data == 0x1) { > + smmu_testdev_refresh_attrs(s); > + } > + break; > + case REG_SMMU_BASE_LO: > + s->smmu_base = (s->smmu_base & ~0xffffffffull) | data; > + break; > + case REG_SMMU_BASE_HI: > + s->smmu_base = (s->smmu_base & 0xffffffffull) | > + ((uint64_t)data << 32); > + break; > + case REG_DMA_IOVA_LO: > + s->dma_iova = (s->dma_iova & ~0xffffffffull) | data; > + break; > + case REG_DMA_IOVA_HI: > + s->dma_iova = (s->dma_iova & 0xffffffffull) | > + ((uint64_t)data << 32); > + break; > + case REG_DMA_LEN: > + s->dma_len = data; > + break; > + case REG_DMA_DIR: > + s->dma_dir = data ? DMA_DIR_HOST2DEV : DMA_DIR_DEV2HOST; > + break; > + case REG_DMA_RESULT: > + s->dma_result = data; > + break; > + case REG_DMA_DOORBELL: > + if (data & 0x1) { > + s->dma_pending = true; > + s->dma_result = DMA_RESULT_BUSY; > + } else { > + s->dma_pending = false; > + s->dma_result = DMA_RESULT_IDLE; > + } > + break; > + case REG_DMA_MODE: > + s->dma_mode = data & 0x1; > + break; > + case REG_DMA_ATTRS: > + s->dma_attrs_cfg = data; > + break; > + case REG_TRANS_MODE: > + s->trans_mode = data & 0x3; > + break; > + case REG_S1_SPACE: > + s->s1_space = (SMMUTestDevSpace)(data & 0x3); > + break; > + case REG_S2_SPACE: > + s->s2_space = (SMMUTestDevSpace)(data & 0x3); > + break; > + case REG_TRANS_DBELL: > + if (data & 0x2) { > + s->trans_status = 0; > + } > + if (data & 0x1) { > + smmu_testdev_build_translation(s); > + } > + break; > + case REG_TRANS_CLEAR: > + /* Clear helper caches so the next iteration rebuilds cleanly. */ > + smmu_testdev_clear_caches(s); > + break; > + default: > + break; > + } > +} > + > +static const MemoryRegionOps smmu_testdev_mmio_ops = { > + .read = smmu_testdev_mmio_read, > + .write = smmu_testdev_mmio_write, > + .endianness = DEVICE_LITTLE_ENDIAN, > + .valid = { > + .min_access_size = 4, > + .max_access_size = 4, > + }, > +}; > + > +static void smmu_testdev_realize(PCIDevice *pdev, Error **errp) > +{ > + SMMUTestDevState *s = SMMU_TESTDEV(pdev); > + > + /* Apply user-configurable BDF (default 0:1). */ > + uint8_t dev = s->cfg_dev & 0x1f; > + uint8_t fn = s->cfg_fn & 0x7; > + pdev->devfn = (dev << 3) | fn; > + > + smmu_testdev_refresh_attrs(s); > + s->smmu_base = 0; > + s->dma_iova = 0; > + s->dma_len = 0; > + s->dma_dir = DMA_DIR_DEV2HOST; > + s->dma_result = DMA_RESULT_IDLE; > + s->dma_pending = false; > + s->dma_mode = 0; > + s->dma_attrs_cfg = 0; > + s->dma_as = pci_device_iommu_address_space(pdev); > + s->trans_mode = TM_S2_ONLY; > + s->s1_space = STD_SPACE_NONSECURE; > + s->s2_space = STD_SPACE_NONSECURE; > + s->trans_status = 0; > + > + if (s->debug_log) { > + smmu_testdev_debug(s, "[smmu-testdev] debug logging enabled\n"); > + } > + > + memory_region_init_io(&s->bar0, OBJECT(pdev), &smmu_testdev_mmio_ops, s, > + TYPE_SMMU_TESTDEV ".bar0", BAR0_SIZE); > + pci_register_bar(pdev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY, &s->bar0); > +} > + > +static void smmu_testdev_reset(DeviceState *dev) > +{ > + SMMUTestDevState *s = SMMU_TESTDEV(dev); > + > + smmu_testdev_refresh_attrs(s); > + s->smmu_base = 0; > + s->dma_iova = 0; > + s->dma_len = 0; > + s->dma_dir = DMA_DIR_DEV2HOST; > + s->dma_result = DMA_RESULT_IDLE; > + s->dma_pending = false; > + s->dma_mode = 0; > + s->dma_attrs_cfg = 0; > + s->trans_mode = TM_S2_ONLY; > + s->s1_space = STD_SPACE_NONSECURE; > + s->s2_space = STD_SPACE_NONSECURE; > + s->trans_status = 0; > + /* Keep cfg_dev/cfg_fn as-is across reset */ > +} > + > +static const Property smmu_testdev_properties[] = { > + DEFINE_PROP_UINT32("device", SMMUTestDevState, cfg_dev, 0), > + DEFINE_PROP_UINT32("function", SMMUTestDevState, cfg_fn, 1), > + DEFINE_PROP_BOOL("debug-log", SMMUTestDevState, debug_log, > false), We have tracepoints so we don't need this flag. > +}; > + > +static void smmu_testdev_class_init(ObjectClass *klass, const void *data) > +{ > + DeviceClass *dc = DEVICE_CLASS(klass); > + PCIDeviceClass *pc = PCI_DEVICE_CLASS(klass); > + > + pc->realize = smmu_testdev_realize; > + pc->vendor_id = PCI_VENDOR_ID_REDHAT; > + pc->device_id = PCI_DEVICE_ID_REDHAT_TEST; > + pc->revision = 0; > + pc->class_id = PCI_CLASS_OTHERS; > + dc->desc = "A test device for the SMMU"; > + set_bit(DEVICE_CATEGORY_MISC, dc->categories); > + device_class_set_legacy_reset(dc, smmu_testdev_reset); > + device_class_set_props(dc, smmu_testdev_properties); > +} > + > +static void smmu_testdev_instance_init(Object *obj) > +{ > + SMMUTestDevState *s = SMMU_TESTDEV(obj); > + s->cfg_dev = 0; > + s->cfg_fn = 1; /* default StreamID = 1 (slot 0, fn 1) */ > + s->debug_log = false; > +} > + > +static const TypeInfo smmu_testdev_info = { > + .name = TYPE_SMMU_TESTDEV, > + .parent = TYPE_PCI_DEVICE, > + .instance_size = sizeof(SMMUTestDevState), > + .instance_init = smmu_testdev_instance_init, > + .class_init = smmu_testdev_class_init, > + .interfaces = (const InterfaceInfo[]) { > + { INTERFACE_CONVENTIONAL_PCI_DEVICE }, > + { } > + }, > +}; > + > +static void smmu_testdev_register_types(void) > +{ > + type_register_static(&smmu_testdev_info); > +} > + > +type_init(smmu_testdev_register_types); > diff --git a/include/hw/misc/smmu-testdev.h b/include/hw/misc/smmu-testdev.h > new file mode 100644 > index 0000000000..6d97f6c704 > --- /dev/null > +++ b/include/hw/misc/smmu-testdev.h > @@ -0,0 +1,402 @@ > +/* > + * A test device for the SMMU > + * > + * This test device is a minimal SMMU-aware device used to test the SMMU. > + * > + * Copyright (c) 2025 Phytium Technology > + * > + * Author: > + * Tao Tang <tangtao1634@phytium.com.cn> > + * > + * SPDX-License-Identifier: GPL-2.0-or-later > + */ > + > +#ifndef HW_MISC_SMMU_TESTDEV_H > +#define HW_MISC_SMMU_TESTDEV_H > + > +#include "qemu/osdep.h" > +typedef enum SMMUTestDevSpace { > + STD_SPACE_SECURE = 0, > + STD_SPACE_NONSECURE = 1, > + STD_SPACE_ROOT = 2, > + STD_SPACE_REALM = 3, > +} SMMUTestDevSpace; > + > +/* Only the Non-Secure space is implemented; leave room for future domains. */ > +#define STD_SUPPORTED_SPACES 1 > + > +/* BAR0 registers (offsets) */ > +enum { > + STD_REG_ID = 0x00, > + STD_REG_ATTR_NS = 0x04, > + STD_REG_SMMU_BASE_LO = 0x20, > + STD_REG_SMMU_BASE_HI = 0x24, > + STD_REG_DMA_IOVA_LO = 0x28, > + STD_REG_DMA_IOVA_HI = 0x2C, > + STD_REG_DMA_LEN = 0x30, > + STD_REG_DMA_DIR = 0x34, > + STD_REG_DMA_RESULT = 0x38, > + STD_REG_DMA_DBELL = 0x3C, > + /* Extended controls for DMA attributes/mode */ > + STD_REG_DMA_MODE = 0x40, > + STD_REG_DMA_ATTRS = 0x44, > + /* Translation controls */ > + STD_REG_TRANS_MODE = 0x48, > + STD_REG_S1_SPACE = 0x4C, > + STD_REG_S2_SPACE = 0x50, > + STD_REG_TRANS_DBELL = 0x54, > + STD_REG_TRANS_STATUS = 0x58, > + /* Clear helper-built tables/descriptors (write-any to trigger) */ > + STD_REG_TRANS_CLEAR = 0x5C, > +}; Aren't we just duplicating the anonymous enum in smmu-testdev.c here? I think qtest is allowed to include qemu headers so lets just have a common set please. > + > +/* DMA result/status values shared with tests */ > +#define STD_DMA_RESULT_IDLE 0xffffffffu > +#define STD_DMA_RESULT_BUSY 0xfffffffeu > +#define STD_DMA_ERR_BAD_LEN 0xdead0001u > +#define STD_DMA_ERR_TX_FAIL 0xdead0002u > + > +/* DMA attributes layout (for STD_REG_DMA_ATTRS) */ > +#define STD_DMA_ATTR_SECURE (1u << 0) > +#define STD_DMA_ATTR_SPACE_SHIFT 1 > +#define STD_DMA_ATTR_SPACE_MASK (0x3u << STD_DMA_ATTR_SPACE_SHIFT) > +#define STD_DMA_ATTR_UNSPECIFIED (1u << 3) > + > +/* Command type */ > +#define STD_CMD_CFGI_STE 0x03 > +#define STD_CMD_CFGI_CD 0x05 > +#define STD_CMD_TLBI_NSNH_ALL 0x30 > + > +/* > + * Translation tables and descriptors for a mapping of IOVA to GPA. > + * > + * This file defines a set of constants used to construct a static page table > + * for an smmu-testdev device. The goal is to translate a specific STD_IOVA > + * into a final GPA. > + * The translation is based on the Arm architecture with the following > + * prerequisites: > + * - Granule size: 4KB pages. > + * - Page table levels: 4 levels (L0, L1, L2, L3), starting at level 0. > + * - IOVA size: The walk resolves a 39-bit IOVA (0x8080604567). > + * - Address space: The 4-level lookup with 4KB granules supports up to a > + * 48-bit (256TB) virtual address space. Each level uses a 9-bit index > + * (512 entries per table). The breakdown is: > + * - L0 index: IOVA bits [47:39] > + * - L1 index: IOVA bits [38:30] > + * - L2 index: IOVA bits [29:21] > + * - L3 index: IOVA bits [20:12] > + * - Page offset: IOVA bits [11:0] > + * > + * NOTE: All physical addresses defined here (STD_VTTB, table addresses, etc.) > + * appear to be within a secure RAM region. In practice, an offset is added > + * to these values to place them in non-secure RAM. For example, when running > + * in a virt machine type, the RAM base address (e.g., 0x40000000) is added to > + * these constants. > + * > + * The page table walk for STD_IOVA (0x8080604567) proceeds as follows: > + * > + * The Translation Table Base (for both Stage 1 CD_TTB and Stage 2 STE_S2TTB) > + * is set to STD_VTTB (0xe4d0000). > + * > + * 1. Level 0 (L0) Table Walk: > + * l0_index = (0x8080604567 >> 39) & 0x1ff = 1 > + * STD_L0_ADDR = STD_VTTB + (l0_index * 8) = 0xe4d0000 + 8 = 0xe4d0008 > + * STD_L0_VAL = 0xe4d1003 > + * The next level table base address = STD_L0_VAL & ~0xfff = 0xe4d1000 > + * > + * 2. Level 1 (L1) Table Walk: > + * l1_index = (0x8080604567 >> 30) & 0x1ff = 2 > + * STD_L1_ADDR = 0xe4d1000 + (l1_index * 8) = 0xe4d1000 + 16 = 0xe4d1010 > + * STD_L1_VAL = 0xe4d2003 > + * The next level table base address = STD_L1_VAL & ~0xfff = 0xe4d2000 > + * > + * 3. Level 2 (L2) Table Walk: > + * l2_index = (0x8080604567 >> 21) & 0x1ff = 3 > + * STD_L2_ADDR = 0xe4d2000 + (l2_index * 8) = 0xe4d2000 + 24 = 0xe4d2018 > + * STD_L2_VAL = 0xe4d3003 > + * The next level table base address = STD_L2_VAL & ~0xfff = 0xe4d3000 > + * > + * 4. Level 3 (L3) Table Walk (Leaf): > + * l3_index = (0x8080604567 >> 12) & 0x1ff = 4 > + * STD_L3_ADDR = 0xe4d3000 + (l3_index * 8) = 0xe4d3000 + 32 = 0xe4d3020 > + * STD_L3_VAL = 0x040000000ECBA7C3 > + * The next level table base address = STD_L3_VAL & ~0xfff = 0xecba000 > + * > + * 5. Final GPA Calculation: > + * - The final output physical address is formed by combining the address from > + * the leaf descriptor with the original IOVA's page offset. > + * - Output Page Base Address = (STD_L3_VAL & ~0xFFFULL) = 0xECBA000 > + * - Page Offset = (STD_IOVA & 0xFFFULL) = 0x567. > + * - Final GPA = Output Page Base Address + Page Offset > + * = 0xECBA000 + 0x567 = 0xECBA567 > + */ > + > +#define STD_IOVA 0x0000008080604567ULL > + > +#define STD_VTTB 0xe4d0000 > + > +#define STD_STR_TAB_BASE 0x000000000E179000ULL > +#define STD_STE_GPA (STD_STR_TAB_BASE + 0x40ULL) > +#define STD_CD_GPA (STD_STR_TAB_BASE + 0x80ULL) > + > +/* Page table structures */ > +#define STD_L0_ADDR 0x000000000E4D0008ULL > +#define STD_L1_ADDR 0x000000000E4D1010ULL > +#define STD_L2_ADDR 0x000000000E4D2018ULL > +#define STD_L3_ADDR 0x000000000E4D3020ULL > +#define STD_L0_VAL 0x000000000E4D1003ULL > +#define STD_L1_VAL 0x000000000E4D2003ULL > +#define STD_L2_VAL 0x000000000E4D3003ULL > +#define STD_L3_VAL 0x040000000ECBA7C3ULL > + > +/* > + * Nested stage PTW maybe a bit more complex. We share the page tables in > + * nested stage 2 to avoid complicated definitions here. That is to say: > + * > + * At each level of the Stage 1 page table walk, a corresponding 4-level Stage 2 > + * page table walk is performed. The intermediate Stage 2 page tables are shared > + * across these walks, with the key connecting PTE values being: > + * - l0_pte_val=0x4e4d1003 > + * - l1_pte_val=0x4e4d2003 > + * - l2_pte_val=0x4e4d3003 > + * > + * > + * ====================================================================== > + * Nested Page Table Walk (Stage 1 + Stage 2) Example > + * ====================================================================== > + * > + * Goal: Translate IOVA 0x8080604567 to a final Physical Address (PA). > + * > + * Prerequisites: > + * - Stage 1 VTTB (as IPA): 0x4e4d0000 > + * - Stage 2 VTTB (as PA): 0x4e4d0000 > + * > + * ---------------------------------------------------------------------- > + * 1. Stage 1 Page Table Walk (IOVA -> IPA) > + * ---------------------------------------------------------------------- > + * > + * Level 0 (L0) Walk (IPA as PA) > + * ============================= > + * iova = 0x8080604567 > + * l0_index = (0x8080604567 >> 39) & 0x1ff = 1 > + * s1_l0_pte_addr (IPA) = 0x4e4d0000 + (1 * 8) = 0x4e4d0008 > + * > + * --> Nested Stage 2 Walk for S1 L0 Table (IPA 0x4e4d0000 -> PA 0x4e4d0000) > + * ------------------------------------------------------------------------- > + * ipa_to_translate = 0x4e4d0000 > + * s2_vttb = 0x4e4d0000 > + * s2_l0_index = (0x4e4d0000 >> 39) & 0x1ff = 0 > + * s2_l0_pte_addr (PA) = 0x4e4d0000 + (0 * 8) = 0x4e4d0000 > + * s2_l0_pte_val = 0x4e4d1003 > + * s2_l1_table_base (PA) = 0x4e4d1003 & ~0xfff = 0x4e4d1000 > + * s2_l1_index = (0x4e4d0000 >> 30) & 0x1ff = 1 > + * s2_l1_pte_addr (PA) = 0x4e4d1000 + (1 * 8) = 0x4e4d1008 > + * s2_l1_pte_val = 0x4e4d2003 > + * s2_l2_table_base (PA) = 0x4e4d2003 & ~0xfff = 0x4e4d2000 > + * s2_l2_index = (0x4e4d0000 >> 21) & 0x1ff = 114 (0x72) > + * s2_l2_pte_addr (PA) = 0x4e4d2000 + (114 * 8) = 0x4e4d2390 > + * s2_l2_pte_val = 0x4e4d3003 > + * s2_l3_table_base (PA) = 0x4e4d3003 & ~0xfff = 0x4e4d3000 > + * s2_l3_index = (0x4e4d0000 >> 12) & 0x1ff = 208 (0xd0) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (208 * 8) = 0x4e4d3680 > + * s2_l3_pte_val = 0x040000000E4D0743ULL > + * output_page_base (PA) = 0x040000000E4D0743ULL & ~0xfff = 0x4e4d0000 > + * Final PA for table = 0x4e4d0000 + (0x0000 & 0xfff) = 0x4e4d0000 > + * > + * s1_l0_pte_val (read from PA 0x4e4d0008) = 0x4e4d1003 > + * s1_l1_table_base (IPA) = 0x4e4d1003 & ~0xfff = 0x4e4d1000 > + * > + * Level 1 (L1) Walk (IPA as PA) > + * ============================= > + * iova = 0x8080604567 > + * l1_index = (0x8080604567 >> 30) & 0x1ff = 2 > + * s1_l1_pte_addr (IPA) = 0x4e4d1000 + (2 * 8) = 0x4e4d1010 > + * > + * --> Nested Stage 2 Walk for S1 L1 Table (IPA 0x4e4d1000 -> PA 0x4e4d1000) > + * ------------------------------------------------------------------------- > + * ipa_to_translate = 0x4e4d1000 > + * s2_vttb = 0x4e4d0000 > + * s2_l0_index = (0x4e4d1000 >> 39) & 0x1ff = 0 > + * s2_l1_table_base (PA) = 0x4e4d1000 > + * s2_l1_index = (0x4e4d1000 >> 30) & 0x1ff = 1 > + * s2_l2_table_base (PA) = 0x4e4d2000 > + * s2_l2_index = (0x4e4d1000 >> 21) & 0x1ff = 114 (0x72) > + * s2_l3_table_base (PA) = 0x4e4d3000 > + * s2_l3_index = (0x4e4d1000 >> 12) & 0x1ff = 209 (0xd1) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (209 * 8) = 0x4e4d3688 > + * s2_l3_pte_val = 0x40000004e4d1743 > + * output_page_base (PA) = 0x40000004e4d1743 & ~0xfff = 0x4e4d1000 > + * Final PA for table = 0x4e4d1000 + (0x1000 & 0xfff) = 0x4e4d1000 > + * > + * s1_l1_pte_val (read from PA 0x4e4d1010) = 0x4e4d2003 > + * s1_l2_table_base (IPA) = 0x4e4d2003 & ~0xfff = 0x4e4d2000 > + * > + * Level 2 (L2) Walk (IPA as PA) > + * ============================= > + * l2_index = (0x8080604567 >> 21) & 0x1ff = 3 > + * s1_l2_pte_addr (IPA) = 0x4e4d2000 + (3 * 8) = 0x4e4d2018 > + * > + * --> Nested Stage 2 Walk for S1 L2 Table (IPA 0x4e4d2000 -> PA 0x4e4d2000) > + * ------------------------------------------------------------------------- > + * ipa_to_translate = 0x4e4d2000 > + * s2_l0_index = (0x4e4d2000 >> 39) & 0x1ff = 0 > + * s2_l1_table_base (PA) = 0x4e4d1000 > + * s2_l1_index = (0x4e4d2000 >> 30) & 0x1ff = 1 > + * s2_l2_table_base (PA) = 0x4e4d2000 > + * s2_l2_index = (0x4e4d2000 >> 21) & 0x1ff = 114 (0x72) > + * s2_l3_table_base (PA) = 0x4e4d3000 > + * s2_l3_index = (0x4e4d2000 >> 12) & 0x1ff = 210 (0xd2) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (210 * 8) = 0x4e4d3690 > + * s2_l3_pte_val = 0x40000004e4d2743 > + * output_page_base (PA) = 0x40000004e4d2743 & ~0xfff = 0x4e4d2000 > + * Final PA for table = 0x4e4d2000 + (0x2000 & 0xfff) = 0x4e4d2000 > + * > + * s1_l2_pte_val (read from PA 0x4e4d2018) = 0x4e4d3003 > + * s1_l3_table_base (IPA) = 0x4e4d3003 & ~0xfff = 0x4e4d3000 > + * > + * Level 3 (L3) Walk (Leaf, IPA as PA) > + * =================================== > + * l3_index = (0x8080604567 >> 12) & 0x1ff = 4 > + * s1_l3_pte_addr (IPA) = 0x4e4d3000 + (4 * 8) = 0x4e4d3020 > + * > + * --> Nested Stage 2 Walk for S1 L3 Table (IPA 0x4e4d3000 -> PA 0x4e4d3000) > + * ------------------------------------------------------------------------- > + * ipa_to_translate = 0x4e4d3000 > + * s2_l0_index = (0x4e4d3000 >> 39) & 0x1ff = 0 > + * s2_l1_table_base (PA) = 0x4e4d1000 > + * s2_l1_index = (0x4e4d3000 >> 30) & 0x1ff = 1 > + * s2_l2_table_base (PA) = 0x4e4d2000 > + * s2_l2_index = (0x4e4d3000 >> 21) & 0x1ff = 114 (0x72) > + * s2_l3_table_base (PA) = 0x4e4d3000 > + * s2_l3_index = (0x4e4d3000 >> 12) & 0x1ff = 211 (0xd3) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (211 * 8) = 0x4e4d3698 > + * s2_l3_pte_val = 0x40000004e4d3743 > + * output_page_base (PA) = 0x40000004e4d3743 & ~0xfff = 0x4e4d3000 > + * Final PA for table = 0x4e4d3000 + (0x3000 & 0xfff) = 0x4e4d3000 > + * > + * s1_l3_pte_val (read from PA 0x4e4d3020) = 0x40000004ecba743 > + * output_page_base (IPA) = 0x40000004ecba743 & ~0xfff = 0x4ecba000 > + * page_offset = 0x8080604567 & 0xfff = 0x567 > + * Final IPA = 0x4ecba000 + 0x567 = 0x4ecba567 > + * > + * ---------------------------------------------------------------------- > + * 2. Final Stage 2 Page Table Walk (Final IPA -> PA) > + * ---------------------------------------------------------------------- > + * > + * ipa = 0x4ecba567 > + * > + * s2_l0_index = (0x4ecba567 >> 39) & 0x1ff = 0 > + * s2_l1_table_base (PA) = 0x4e4d1000 > + * > + * s2_l1_index = (0x4ecba567 >> 30) & 0x1ff = 1 > + * s2_l2_table_base (PA) = 0x4e4d2000 > + * > + * s2_l2_index = (0x4ecba567 >> 21) & 0x1ff = 118 (0x76) > + * s2_l3_table_base (PA) = 0x4e4d3000 > + * > + * s2_l3_index = (0x4ecba567 >> 12) & 0x1ff = 186 (0xba) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (186 * 8) = 0x4e4d35d0 > + * s2_l3_pte_val = 0x40000004ecba7c3 > + * output_page_base (PA) = 0x40000004ecba7c3 & ~0xfff = 0x4ecba000 > + * page_offset = 0x4ecba567 & 0xfff = 0x567 > + * > + * ---------------------------------------------------------------------- > + * 3. Final Result > + * ---------------------------------------------------------------------- > + * Final PA = 0x4ecba000 + 0x567 = 0x4ecba567 > + * > + * ---------------------------------------------------------------------- > + * 4. Appendix: Context Descriptor (CD) Fetch Walk > + * ---------------------------------------------------------------------- > + * Before any S1 walk can begin, the SMMU must fetch the Context Descriptor. > + * The CD's address is an IPA, so it also requires a full S2 walk. This > + * walk RE-USES the exact same S2 page tables shown above. > + * > + * ipa = 0x4e179080 (Address of the CD) > + * > + * s2_l0_index = (0x4e179080 >> 39) & 0x1ff = 0 > + * s2_l0_pte_val = 0x4e4d1003 > + * s2_l1_table_base (PA) = 0x4e4d1000 (*RE-USED*) > + * > + * s2_l1_index = (0x4e179080 >> 30) & 0x1ff = 1 > + * s2_l1_pte_val = 0x4e4d2003 > + * s2_l2_table_base (PA) = 0x4e4d2000 (*RE-USED*) > + * > + * s2_l2_index = (0x4e179080 >> 21) & 0x1ff = 112 (0x70) > + * s2_l2_pte_val = 0x4e4d3003 > + * s2_l3_table_base (PA) = 0x4e4d3000 (*RE-USED*) > + * > + * s2_l3_index = (0x4e179080 >> 12) & 0x1ff = 377 (0x179) > + * s2_l3_pte_addr (PA) = 0x4e4d3000 + (377 * 8) = 0x4e4d3bc8 > + * s2_l3_pte_val = 0x40000004e179743 > + * output_page_base (PA) = 0x40000004e179743 & ~0xfff = 0x4e179000 > + * page_offset = 0x4e179080 & 0xfff = 0x080 > + * > + * Final PA for CD = 0x4e179000 + 0x080 = 0x4e179080 > + * > + */ > +#define STD_CD_S2_L0_ADDR 0x000000000E4D0000ULL > +#define STD_CD_S2_L1_ADDR 0x000000000E4D1008ULL > +#define STD_CD_S2_L2_ADDR 0x000000000E4D2380ULL > +#define STD_CD_S2_L3_ADDR 0x000000000E4D3BC8ULL > +#define STD_CD_S2_L3_VAL 0x040000000E179743ULL > + > +#define STD_CDTTB_S2_L2_ADDR 0x000000000E4D2390ULL > +#define STD_CDTTB_S2_L3_ADDR 0x000000000E4D3680ULL > +#define STD_CDTTB_S2_L3_VAL 0x040000000E4D0743ULL > + > +#define STD_L3_S1_VAL 0x040000000ECBA743ULL > + > +#define STD_S1L0_IN_S2L3_ADDR 0x000000000E4D3688ULL > +#define STD_S1L0_IN_S2L3_VAL 0x040000000E4D1743ULL > +#define STD_S1L1_IN_S2L3_ADDR 0x000000000E4D3690ULL > +#define STD_S1L1_IN_S2L3_VAL 0x040000000E4D2743ULL > +#define STD_S1L2_IN_S2L3_ADDR 0x000000000E4D3698ULL > +#define STD_S1L2_IN_S2L3_VAL 0x040000000E4D3743ULL > +#define STD_S1L3_IN_S2L2_ADDR 0x000000000E4D23B0ULL > +#define STD_S1L3_IN_S2L2_VAL 0x000000000E4D3003ULL > +#define STD_S1L3_IN_S2L3_ADDR 0x000000000E4D35D0ULL > +#define STD_S1L3_IN_S2L3_VAL 0x040000000ECBA7C3ULL > + > +/* > + * Address-space base offsets for test tables. > + * - Non-Secure uses a fixed offset, keeping internal layout identical. > + * > + * Note: Future spaces (e.g. Secure/Realm/Root) are not implemented here. > + * When needed, introduce new offsets and reuse the helpers below so > + * relative layout stays identical across spaces. > + */ > +#define STD_SPACE_OFFS_NS 0x40000000ULL > + > +static inline uint64_t std_space_offset(SMMUTestDevSpace sp) > +{ > + switch (sp) { > + case STD_SPACE_NONSECURE: > + return STD_SPACE_OFFS_NS; > + default: > + g_assert_not_reached(); > + } > +} > + > +static const char *std_space_to_str(SMMUTestDevSpace sp) > +{ > + switch (sp) { > + case STD_SPACE_NONSECURE: > + return "Non-Secure"; > + default: > + g_assert_not_reached(); > + } > +} > + > +static const char *std_mode_to_str(uint32_t m) > +{ > + switch (m & 0x3) { > + case 0: return "S1-only"; > + case 1: return "S2-only"; > + case 2: return "Nested"; > + default: > + g_assert_not_reached(); > + } > +} > + > +#endif /* HW_MISC_SMMU_TESTDEV_H */ -- Alex Bennée Virtualisation Tech Lead @ Linaro ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device 2025-10-23 10:31 ` Alex Bennée @ 2025-10-27 14:54 ` Tao Tang 0 siblings, 0 replies; 16+ messages in thread From: Tao Tang @ 2025-10-27 14:54 UTC (permalink / raw) To: Alex Bennée Cc: Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Hi Alex, On 2025/10/23 18:31, Alex Bennée wrote: > tangtao1634 <tangtao1634@phytium.com.cn> writes: > >> From: Tao Tang <tangtao1634@phytium.com.cn> >> >> Add a tiny, test-only DMA source dedicated to exercising the SMMUv3 model. >> The device purposefully avoids a realistic PCIe/platform implementation and >> instead routes DMA requests straight into the SMMU, so that qtests can >> populate STE/CD/PTE with known values and observe translation and data >> movement deterministically, without booting any firmware or guest kernel. >> >> Motivation >> ---------- >> Bringing up and regression-testing the SMMU in emulation often depends on a >> large and flaky software stack (enumeration, drivers, PCIe fabric). For the >> class of tests that only need to (1) program translation structures and (2) >> trigger DMA at a precise time, that stack adds noise, slows CI, and makes >> failures harder to attribute to the SMMU itself. A hermetic DMA source >> keeps the surface area small and the results reproducible. >> >> What this device is (and is not) >> -------------------------------- >> * It is a minimal DMA producer solely for SMMU tests. >> * It is NOT a faithful PCIe Endpoint nor a platform device. >> * It is NOT added to any machine by default and remains test-only. >> >> Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> >> --- >> hw/misc/Kconfig | 5 + >> hw/misc/meson.build | 1 + >> hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++++ >> include/hw/misc/smmu-testdev.h | 402 ++++++++++++++ >> 4 files changed, 1351 insertions(+) >> create mode 100644 hw/misc/smmu-testdev.c >> create mode 100644 include/hw/misc/smmu-testdev.h >> >> ------------------------------<snip>------------------------------ >> >> >> >> ------------------------------<snip>------------------------------ >> +/* ---- Debug helpers for printing current translation configuration ---- */ >> +static void G_GNUC_PRINTF(2, 3) >> +smmu_testdev_debug(const SMMUTestDevState *s, const char *fmt, ...) >> +{ >> + va_list ap; >> + g_autofree char *msg = NULL; >> + >> + if (!s->debug_log) { >> + return; >> + } >> + >> + va_start(ap, fmt); >> + msg = g_strdup_vprintf(fmt, ap); >> + va_end(ap); >> + >> + if (qemu_log_enabled()) { >> + qemu_log("%s", msg); >> + } else { >> + fprintf(stderr, "%s", msg); >> + } >> +} > Why are we re-inventing logging here? Either use qemu_log or probably > better define tracepoints. > >> + >> +static MemTxAttrs mk_attrs_from_space(SMMUTestDevSpace space) >> +{ >> + MemTxAttrs a = {0}; >> + if (!smmu_testdev_space_supported(space)) { >> + g_assert_not_reached(); > Isn't this the same as just saying: > > g_assert(smmu_testdev_space_supported(space)); > > ? > >> +static void smmu_testdev_build_translation(SMMUTestDevState *s) >> +{ >> + smmu_testdev_debug(s, "smmu_testdev_build_translation: stage=%s s1_space=%s" >> + " s2_space=%s\n", std_mode_to_str(s->trans_mode), >> + std_space_to_str(s->s1_space), >> + std_space_to_str(s->s2_space)); > tracepoint > >> + if (st != 0) { >> + printf("Writing STE error! status: %x\n", st); > > qemu_log_mask(LOG_GUEST_ERROR, ... > >> + AddressSpace *as = space_to_as(bank_sp); >> + if (!as) { >> + printf("push_cfgi_cmd: space %d not supported\n", bank_sp); > qemu_log_mask(LOG_UNIMP? > >> + return; >> + } >> + int ret = address_space_write(as, entry_pa, a, >> + words, sizeof(words)); >> + smmu_testdev_debug(s, "push_cfgi_cmd ret %d\n", ret); > tracepoint > >> + >> + /* update PROD to trigger command handler */ >> + uint32_t new_prod = (prod + 1) & ((1u << (log2size + 1)) - 1u); >> + address_space_stl_le(&address_space_memory, >> + s->smmu_base + bank_off + 0x98, >> + new_prod, MEMTXATTRS_UNSPECIFIED, &res); >> + smmu_testdev_debug(s, "last res %d\n", res); > tracepoint but really see if it can be merged with above. > >> + smmu_testdev_debug(s, "smmu_testdev_maybe_run_dma: dma_pending: %d\n", >> + s->dma_pending); > tracepoint > >> + >> + s->dma_pending = false; >> + >> + if (!s->dma_len || s->dma_len > DMA_MAX_LEN) { >> + s->dma_result = DMA_ERR_BAD_LEN; >> + return; >> + } >> + >> + g_autofree uint8_t *buf = g_malloc(s->dma_len); >> + MemTxResult res; > Keep local vars at the top of the block. > >> +static const Property smmu_testdev_properties[] = { >> + DEFINE_PROP_UINT32("device", SMMUTestDevState, cfg_dev, 0), >> + DEFINE_PROP_UINT32("function", SMMUTestDevState, cfg_fn, 1), >> + DEFINE_PROP_BOOL("debug-log", SMMUTestDevState, debug_log, >> false), > We have tracepoints so we don't need this flag. > >> +/* BAR0 registers (offsets) */ >> +enum { >> + STD_REG_ID = 0x00, >> + STD_REG_ATTR_NS = 0x04, >> + STD_REG_SMMU_BASE_LO = 0x20, >> + STD_REG_SMMU_BASE_HI = 0x24, >> + STD_REG_DMA_IOVA_LO = 0x28, >> + STD_REG_DMA_IOVA_HI = 0x2C, >> + STD_REG_DMA_LEN = 0x30, >> + STD_REG_DMA_DIR = 0x34, >> + STD_REG_DMA_RESULT = 0x38, >> + STD_REG_DMA_DBELL = 0x3C, >> + /* Extended controls for DMA attributes/mode */ >> + STD_REG_DMA_MODE = 0x40, >> + STD_REG_DMA_ATTRS = 0x44, >> + /* Translation controls */ >> + STD_REG_TRANS_MODE = 0x48, >> + STD_REG_S1_SPACE = 0x4C, >> + STD_REG_S2_SPACE = 0x50, >> + STD_REG_TRANS_DBELL = 0x54, >> + STD_REG_TRANS_STATUS = 0x58, >> + /* Clear helper-built tables/descriptors (write-any to trigger) */ >> + STD_REG_TRANS_CLEAR = 0x5C, >> +}; > Aren't we just duplicating the anonymous enum in smmu-testdev.c here? I > think qtest is allowed to include qemu headers so lets just have a > common set please. Thank you very much for the thorough and highly constructive review of this patch. I've gone through all your comments, and I agree with points you've raised. I will incorporate all your feedback into the next version of the series. To be clear, even as the broader discussion about potentially refactoring smmu-testdev into a more generic iommu-testdev continues, your review points on code quality are fundamental. I will ensure they are implemented regardless of which final architecture we decide upon. To summarize the planned changes: - Logging mechanism: I will replace the custom smmu_testdev_debug function and other printf calls with the standard tracepoint mechanism. Consequently, the debug-log device property will be removed as it will no longer be necessary. - Removing duplicated codes: Move the duplicated enum into smmu-testdev.h which will then be included by both the device implementation and the qtest file. - Code style and simplification: Simplify the g_assert checks and ensuring local variables are declared at the top of their respective blocks. Thanks again for taking the time to provide such valuable and detailed feedback. Best regards, Tao ^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 2025-09-30 16:53 ` [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device tangtao1634 @ 2025-09-30 16:53 ` tangtao1634 2025-10-23 11:02 ` Alex Bennée 2025-10-23 9:04 ` [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework Tao Tang ` (2 subsequent siblings) 4 siblings, 1 reply; 16+ messages in thread From: tangtao1634 @ 2025-09-30 16:53 UTC (permalink / raw) To: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell Cc: qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Tao Tang From: Tao Tang <tangtao1634@phytium.com.cn> Introduce a bare-metal qtest that drives the new smmu-testdev to exercise the SMMUv3 emulation without guest firmware or drivers. The test programs a minimal Non-Secure context (STE/CD/PTE), triggers a DMA, and asserts translation results. Motivation ---------- SMMU testing in emulation often requires a large software stack and a realistic PCIe fabric, which adds flakiness and obscures failures. This qtest keeps the surface small and deterministic by using a hermetic DMA source that feeds the SMMU directly. What the test covers -------------------- * Builds a Non-Secure STE/CD/PTE for a chosen stream_id/ssid. * Primes source and destination host buffers. * Kicks a DMA via smmu-testdev and waits for completion. * Verifies translated access and payload equality. Non-goals and scope limits -------------------------- * Secure bank flows are omitted because Secure SMMU support is still RFC. A local Secure test exists and can be posted once the upstream series lands. * PCIe discovery, MSI/INTx, ATS/PRI, and driver bring-up are out of scope as smmu-testdev is not a realistic PCIe Endpoint nor a platform device. Rationale for a dedicated path ------------------------------ Using a generic PCI or virtio device would still require driver init and a richer bus model, undermining determinism for this focused purpose. This qtest, paired with smmu-testdev, keeps failures attributable to the SMMU translation path. Finally we document the smmu-testdev device in docs/specs. Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> --- docs/specs/index.rst | 1 + docs/specs/smmu-testdev.rst | 45 ++++++ tests/qtest/meson.build | 1 + tests/qtest/smmu-testdev-qtest.c | 238 +++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+) create mode 100644 docs/specs/smmu-testdev.rst create mode 100644 tests/qtest/smmu-testdev-qtest.c diff --git a/docs/specs/index.rst b/docs/specs/index.rst index f19d73c9f6..47a18c48f1 100644 --- a/docs/specs/index.rst +++ b/docs/specs/index.rst @@ -39,3 +39,4 @@ guest hardware that is specific to QEMU. riscv-iommu riscv-aia aspeed-intc + smmu-testdev \ No newline at end of file diff --git a/docs/specs/smmu-testdev.rst b/docs/specs/smmu-testdev.rst new file mode 100644 index 0000000000..2599b46e4f --- /dev/null +++ b/docs/specs/smmu-testdev.rst @@ -0,0 +1,45 @@ +smmu-testdev — Minimal SMMUv3 DMA test device +============================================= + +Overview +-------- +``smmu-testdev`` is a tiny, test-only DMA source intended to exercise the +SMMUv3 emulation without booting firmware or a guest OS. It lets tests +populate STE/CD/PTE with known values and trigger a DMA that flows through +the SMMU translation path. It is **not** a faithful PCIe endpoint nor a +platform device and must be considered a QEMU-internal test vehicle. + +Status +------ +* Location: ``hw/misc/smmu-testdev.c`` +* Build guard: ``CONFIG_SMMU_TESTDEV`` +* Default machines: none (tests instantiate it explicitly) +* Intended use: qtests under ``tests/qtest/smmu-testdev-qtest.c`` + +Running the qtest +----------------- +The smoke test ships with this device and is the recommended entry point:: + + QTEST_QEMU_BINARY=qemu-system-aarch64 ./tests/qtest/smmu-testdev-qtest + --tap -k + +This programs a minimal Non-Secure SMMU context, kicks a DMA, and verifies +translation + data integrity. + +Instantiation (advanced) +------------------------ +The device is not wired into any board by default. For ad-hoc experiments, +tests (or developers) can create it dynamically via qtest or the QEMU +monitor. It exposes a single MMIO window that the test drives directly. + +Limitations +----------- +* Non-Secure bank only in this version; Secure SMMU tests are planned once + upstream Secure support lands. +* No PCIe discovery, MSI, ATS/PRI, or driver bring-up is modeled. +* The device is test-only; do not rely on it for machine realism. + +See also +-------- +* ``tests/qtest/smmu-testdev-qtest.c`` — the companion smoke test +* SMMUv3 emulation and documentation under ``hw/arm/smmu*`` diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 669d07c06b..bcdb51e141 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -263,6 +263,7 @@ qtests_aarch64 = \ config_all_devices.has_key('CONFIG_TPM_TIS_I2C') ? ['tpm-tis-i2c-test'] : []) + \ (config_all_devices.has_key('CONFIG_ASPEED_SOC') ? qtests_aspeed64 : []) + \ (config_all_devices.has_key('CONFIG_NPCM8XX') ? qtests_npcm8xx : []) + \ + (config_all_devices.has_key('CONFIG_SMMU_TESTDEV') ? ['smmu-testdev-qtest'] : []) + \ qtests_cxl + \ ['arm-cpu-features', 'numa-test', diff --git a/tests/qtest/smmu-testdev-qtest.c b/tests/qtest/smmu-testdev-qtest.c new file mode 100644 index 0000000000..d89e45757b --- /dev/null +++ b/tests/qtest/smmu-testdev-qtest.c @@ -0,0 +1,238 @@ +/* + * QTest for smmu-testdev + * + * This QTest file is used to test the smmu-testdev so that we can test SMMU + * without any guest kernel or firmware. + * + * Copyright (c) 2025 Phytium Technology + * + * Author: + * Tao Tang <tangtao1634@phytium.com.cn> + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "libqtest.h" +#include "libqos/pci.h" +#include "libqos/generic-pcihost.h" +#include "hw/pci/pci_regs.h" +#include "hw/misc/smmu-testdev.h" + +#define VIRT_SMMU_BASE 0x0000000009050000ULL +#define DMA_LEN 0x20U + +static inline uint64_t smmu_bank_base(uint64_t base, SMMUTestDevSpace sp) +{ + /* Map only the Non-Secure bank for now; future domains may offset. */ + (void)sp; + return base; +} + +static uint32_t expected_dma_result(uint32_t mode, + SMMUTestDevSpace s1_space, + SMMUTestDevSpace s2_space) +{ + (void)mode; + if (s1_space != STD_SPACE_NONSECURE || s2_space != STD_SPACE_NONSECURE) { + return STD_DMA_ERR_TX_FAIL; + } + return 0u; +} + +static void smmu_prog_bank(QTestState *qts, uint64_t B, SMMUTestDevSpace sp) +{ + g_assert_cmpuint(sp, ==, STD_SPACE_NONSECURE); + /* Program minimal SMMUv3 state in a given control bank. */ + qtest_writel(qts, B + 0x0044, 0x80000000); /* GBPA UPDATE */ + qtest_writel(qts, B + 0x0020, 0x0); /* CR0 */ + qtest_writel(qts, B + 0x0028, 0x0d75); /* CR1 */ + { + /* CMDQ_BASE: add address-space offset (S/NS/Root/Realm). */ + uint64_t v = 0x400000000e16b00aULL + std_space_offset(sp); + qtest_writeq(qts, B + 0x0090, v); + } + qtest_writel(qts, B + 0x009c, 0x0); /* CMDQ_CONS */ + qtest_writel(qts, B + 0x0098, 0x0); /* CMDQ_PROD */ + { + /* EVENTQ_BASE: add address-space offset (S/NS/Root/Realm). */ + uint64_t v = 0x400000000e17000aULL + std_space_offset(sp); + qtest_writeq(qts, B + 0x00a0, v); + } + qtest_writel(qts, B + 0x00a8, 0x0); /* EVENTQ_PROD */ + qtest_writel(qts, B + 0x00ac, 0x0); /* EVENTQ_CONS */ + qtest_writel(qts, B + 0x0088, 0x5); /* STRTAB_BASE_CFG */ + { + /* STRTAB_BASE: add address-space offset (S/NS/Root/Realm). */ + uint64_t v = 0x400000000e179000ULL + std_space_offset(sp); + qtest_writeq(qts, B + 0x0080, v); + } + qtest_writel(qts, B + 0x003C, 0x1); /* INIT */ + qtest_writel(qts, B + 0x0020, 0xD); /* CR0 */ +} + +static void smmu_prog_minimal(QTestState *qts, SMMUTestDevSpace space) +{ + /* Always program Non-Secure bank, then the requested space. */ + uint64_t ns_base = smmu_bank_base(VIRT_SMMU_BASE, STD_SPACE_NONSECURE); + smmu_prog_bank(qts, ns_base, STD_SPACE_NONSECURE); + + uint64_t sp_base = smmu_bank_base(VIRT_SMMU_BASE, space); + if (sp_base != ns_base) { + smmu_prog_bank(qts, sp_base, space); + } +} + +static uint32_t poll_dma_result(QPCIDevice *dev, QPCIBar bar, + QTestState *qts) +{ + /* Trigger side effects (DMA) via REG_ID read once. */ + (void)qpci_io_readl(dev, bar, STD_REG_ID); + + /* Poll until not BUSY, then return the result. */ + for (int i = 0; i < 1000; i++) { + uint32_t r = qpci_io_readl(dev, bar, STD_REG_DMA_RESULT); + if (r != STD_DMA_RESULT_BUSY) { + return r; + } + /* Small backoff to avoid busy spinning. */ + g_usleep(1000); + } + /* Timeout treated as failure-like non-zero. */ + return STD_DMA_RESULT_BUSY; +} + +static void test_mmio_access(void) +{ + QTestState *qts; + QGenericPCIBus gbus; + QPCIDevice *dev; + QPCIBar bar; + uint8_t buf[DMA_LEN]; + uint32_t attr_ns; + qts = qtest_init("-machine virt,acpi=off,gic-version=3,iommu=smmuv3 " \ + "-display none -smp 1 -m 512 -cpu max -net none " + "-device smmu-testdev,device=0x0,function=0x1 "); + + qpci_init_generic(&gbus, qts, NULL, false); + + /* Find device by vendor/device ID to avoid slot surprises. */ + dev = NULL; + for (int slot = 0; slot < 32 && !dev; slot++) { + for (int fn = 0; fn < 8 && !dev; fn++) { + QPCIDevice *cand = qpci_device_find(&gbus.bus, + QPCI_DEVFN(slot, fn)); + if (!cand) { + continue; + } + uint16_t vid = qpci_config_readw(cand, PCI_VENDOR_ID); + uint16_t did = qpci_config_readw(cand, PCI_DEVICE_ID); + if (vid == 0x1b36 && did == 0x0005) { + dev = cand; + } else { + g_free(cand); + } + } + } + g_assert_nonnull(dev); + + qpci_device_enable(dev); + bar = qpci_iomap(dev, 0, NULL); + g_assert_false(bar.is_io); + + /* Baseline attribute reads. */ + attr_ns = qpci_io_readl(dev, bar, STD_REG_ATTR_NS); + g_assert_cmpuint(attr_ns, ==, 0x2); + + /* Program SMMU base and DMA parameters. */ + qpci_io_writel(dev, bar, STD_REG_SMMU_BASE_LO, (uint32_t)VIRT_SMMU_BASE); + qpci_io_writel(dev, bar, STD_REG_SMMU_BASE_HI, + (uint32_t)(VIRT_SMMU_BASE >> 32)); + qpci_io_writel(dev, bar, STD_REG_DMA_IOVA_LO, (uint32_t)STD_IOVA); + qpci_io_writel(dev, bar, STD_REG_DMA_IOVA_HI, + (uint32_t)(STD_IOVA >> 32)); + qpci_io_writel(dev, bar, STD_REG_DMA_LEN, DMA_LEN); + qpci_io_writel(dev, bar, STD_REG_DMA_DIR, 0); /* device -> host */ + + qtest_memset(qts, STD_IOVA, 0x00, DMA_LEN); + qtest_memread(qts, STD_IOVA, buf, DMA_LEN); + + /* Refresh attrs via write to ensure legacy functionality still works. */ + qpci_io_writel(dev, bar, STD_REG_ID, 0x1); + /* + * invoke translation builder for multiple + * stage/security-space combinations (readable/refactored). + */ + const uint32_t modes[] = { 0u, 1u, 2u }; /* Stage1, Stage2, Nested stage */ + const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; + /* Use attrs-DMA path for end-to-end */ + qpci_io_writel(dev, bar, STD_REG_DMA_MODE, 1); + for (size_t mi = 0; mi < sizeof(modes) / sizeof(modes[0]); mi++) { + const SMMUTestDevSpace *s1_set = NULL; + size_t s1_count = 0; + const SMMUTestDevSpace *s2_set = NULL; + size_t s2_count = 0; + + switch (modes[mi]) { + case 0u: + case 1u: + case 2u: + s1_set = spaces; + s1_count = sizeof(spaces) / sizeof(spaces[0]); + s2_set = spaces; + s2_count = sizeof(spaces) / sizeof(spaces[0]); + break; + default: + g_assert_not_reached(); + } + + for (size_t si = 0; si < s1_count; si++) { + for (size_t sj = 0; sj < s2_count; sj++) { + qpci_io_writel(dev, bar, STD_REG_TRANS_MODE, modes[mi]); + qpci_io_writel(dev, bar, STD_REG_S1_SPACE, s1_set[si]); + qpci_io_writel(dev, bar, STD_REG_S2_SPACE, s2_set[sj]); + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x2); + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x1); + + uint32_t st = qpci_io_readl(dev, bar, + STD_REG_TRANS_STATUS); + g_test_message("build: stage=%s s1=%s s2=%s status=0x%x", + std_mode_to_str(modes[mi]), + std_space_to_str(s1_set[si]), + std_space_to_str(s2_set[sj]), st); + /* Program SMMU registers in selected control bank. */ + smmu_prog_minimal(qts, s1_set[si]); + + /* End-to-end DMA using tx_space per mode. */ + SMMUTestDevSpace tx_space = + (modes[mi] == 0u) ? s1_set[si] : s2_set[sj]; + uint32_t dma_attrs = ((uint32_t)tx_space << 1); + qpci_io_writel(dev, bar, STD_REG_DMA_ATTRS, + dma_attrs); + qpci_io_writel(dev, bar, STD_REG_DMA_DBELL, 1); + /* Wait for DMA completion and assert success. */ + { + uint32_t dr = poll_dma_result(dev, bar, qts); + uint32_t exp = expected_dma_result(modes[mi], + spaces[si], + spaces[sj]); + g_assert_cmpuint(dr, ==, exp); + g_test_message("polling end. attrs=0x%x res=0x%x", + dma_attrs, dr); + } + /* Clear CD/STE/PTE built by the device for next round. */ + qpci_io_writel(dev, bar, STD_REG_TRANS_CLEAR, 1); + g_test_message("clear cache end."); + } + } + } + + qtest_quit(qts); +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + qtest_add_func("/smmu-testdev/mmio", test_mmio_access); + return g_test_run(); +} -- 2.49.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source 2025-09-30 16:53 ` [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source tangtao1634 @ 2025-10-23 11:02 ` Alex Bennée 2025-10-27 15:26 ` Tao Tang 0 siblings, 1 reply; 16+ messages in thread From: Alex Bennée @ 2025-10-23 11:02 UTC (permalink / raw) To: tangtao1634 Cc: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh tangtao1634 <tangtao1634@phytium.com.cn> writes: > From: Tao Tang <tangtao1634@phytium.com.cn> > > Introduce a bare-metal qtest that drives the new smmu-testdev to exercise > the SMMUv3 emulation without guest firmware or drivers. The test programs > a minimal Non-Secure context (STE/CD/PTE), triggers a DMA, and asserts > translation results. > > Motivation > ---------- > SMMU testing in emulation often requires a large software stack and a > realistic PCIe fabric, which adds flakiness and obscures failures. This > qtest keeps the surface small and deterministic by using a hermetic DMA > source that feeds the SMMU directly. > > What the test covers > -------------------- > * Builds a Non-Secure STE/CD/PTE for a chosen stream_id/ssid. > * Primes source and destination host buffers. > * Kicks a DMA via smmu-testdev and waits for completion. > * Verifies translated access and payload equality. > > Non-goals and scope limits > -------------------------- > * Secure bank flows are omitted because Secure SMMU support is still RFC. > A local Secure test exists and can be posted once the upstream series > lands. > * PCIe discovery, MSI/INTx, ATS/PRI, and driver bring-up are out of scope > as smmu-testdev is not a realistic PCIe Endpoint nor a platform device. > > Rationale for a dedicated path > ------------------------------ > Using a generic PCI or virtio device would still require driver init and a > richer bus model, undermining determinism for this focused purpose. This > qtest, paired with smmu-testdev, keeps failures attributable to the SMMU > translation path. > > Finally we document the smmu-testdev device in docs/specs. > > Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> > --- > docs/specs/index.rst | 1 + > docs/specs/smmu-testdev.rst | 45 ++++++ > tests/qtest/meson.build | 1 + > tests/qtest/smmu-testdev-qtest.c | 238 +++++++++++++++++++++++++++++++ > 4 files changed, 285 insertions(+) > create mode 100644 docs/specs/smmu-testdev.rst > create mode 100644 tests/qtest/smmu-testdev-qtest.c > > diff --git a/docs/specs/index.rst b/docs/specs/index.rst > index f19d73c9f6..47a18c48f1 100644 > --- a/docs/specs/index.rst > +++ b/docs/specs/index.rst > @@ -39,3 +39,4 @@ guest hardware that is specific to QEMU. > riscv-iommu > riscv-aia > aspeed-intc > + smmu-testdev > \ No newline at end of file > diff --git a/docs/specs/smmu-testdev.rst b/docs/specs/smmu-testdev.rst > new file mode 100644 > index 0000000000..2599b46e4f > --- /dev/null > +++ b/docs/specs/smmu-testdev.rst > @@ -0,0 +1,45 @@ > +smmu-testdev — Minimal SMMUv3 DMA test device > +============================================= > + > +Overview > +-------- > +``smmu-testdev`` is a tiny, test-only DMA source intended to exercise the > +SMMUv3 emulation without booting firmware or a guest OS. It lets tests > +populate STE/CD/PTE with known values and trigger a DMA that flows through > +the SMMU translation path. It is **not** a faithful PCIe endpoint nor a > +platform device and must be considered a QEMU-internal test vehicle. > + > +Status > +------ > +* Location: ``hw/misc/smmu-testdev.c`` > +* Build guard: ``CONFIG_SMMU_TESTDEV`` > +* Default machines: none (tests instantiate it explicitly) > +* Intended use: qtests under ``tests/qtest/smmu-testdev-qtest.c`` > + > +Running the qtest > +----------------- > +The smoke test ships with this device and is the recommended entry point:: > + > + QTEST_QEMU_BINARY=qemu-system-aarch64 ./tests/qtest/smmu-testdev-qtest > + --tap -k > + > +This programs a minimal Non-Secure SMMU context, kicks a DMA, and verifies > +translation + data integrity. > + > +Instantiation (advanced) > +------------------------ > +The device is not wired into any board by default. For ad-hoc experiments, > +tests (or developers) can create it dynamically via qtest or the QEMU > +monitor. It exposes a single MMIO window that the test drives directly. > + > +Limitations > +----------- > +* Non-Secure bank only in this version; Secure SMMU tests are planned once > + upstream Secure support lands. > +* No PCIe discovery, MSI, ATS/PRI, or driver bring-up is modeled. > +* The device is test-only; do not rely on it for machine realism. > + > +See also > +-------- > +* ``tests/qtest/smmu-testdev-qtest.c`` — the companion smoke test > +* SMMUv3 emulation and documentation under ``hw/arm/smmu*`` > diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build > index 669d07c06b..bcdb51e141 100644 > --- a/tests/qtest/meson.build > +++ b/tests/qtest/meson.build > @@ -263,6 +263,7 @@ qtests_aarch64 = \ > config_all_devices.has_key('CONFIG_TPM_TIS_I2C') ? ['tpm-tis-i2c-test'] : []) + \ > (config_all_devices.has_key('CONFIG_ASPEED_SOC') ? qtests_aspeed64 : []) + \ > (config_all_devices.has_key('CONFIG_NPCM8XX') ? qtests_npcm8xx : []) + \ > + (config_all_devices.has_key('CONFIG_SMMU_TESTDEV') ? ['smmu-testdev-qtest'] : []) + \ > qtests_cxl + \ > ['arm-cpu-features', > 'numa-test', > diff --git a/tests/qtest/smmu-testdev-qtest.c b/tests/qtest/smmu-testdev-qtest.c > new file mode 100644 > index 0000000000..d89e45757b > --- /dev/null > +++ b/tests/qtest/smmu-testdev-qtest.c > @@ -0,0 +1,238 @@ > +/* > + * QTest for smmu-testdev > + * > + * This QTest file is used to test the smmu-testdev so that we can test SMMU > + * without any guest kernel or firmware. > + * > + * Copyright (c) 2025 Phytium Technology > + * > + * Author: > + * Tao Tang <tangtao1634@phytium.com.cn> > + * > + * SPDX-License-Identifier: GPL-2.0-or-later > + */ > + > +#include "qemu/osdep.h" > +#include "libqtest.h" > +#include "libqos/pci.h" > +#include "libqos/generic-pcihost.h" > +#include "hw/pci/pci_regs.h" > +#include "hw/misc/smmu-testdev.h" > + > +#define VIRT_SMMU_BASE 0x0000000009050000ULL > +#define DMA_LEN 0x20U > + > +static inline uint64_t smmu_bank_base(uint64_t base, SMMUTestDevSpace sp) > +{ > + /* Map only the Non-Secure bank for now; future domains may offset. */ > + (void)sp; > + return base; > +} > + > +static uint32_t expected_dma_result(uint32_t mode, > + SMMUTestDevSpace s1_space, > + SMMUTestDevSpace s2_space) > +{ > + (void)mode; > + if (s1_space != STD_SPACE_NONSECURE || s2_space != STD_SPACE_NONSECURE) { > + return STD_DMA_ERR_TX_FAIL; > + } > + return 0u; > +} > + > +static void smmu_prog_bank(QTestState *qts, uint64_t B, SMMUTestDevSpace sp) > +{ > + g_assert_cmpuint(sp, ==, STD_SPACE_NONSECURE); > + /* Program minimal SMMUv3 state in a given control bank. */ > + qtest_writel(qts, B + 0x0044, 0x80000000); /* GBPA UPDATE */ > + qtest_writel(qts, B + 0x0020, 0x0); /* CR0 */ > + qtest_writel(qts, B + 0x0028, 0x0d75); /* CR1 */ > + { > + /* CMDQ_BASE: add address-space offset (S/NS/Root/Realm). */ > + uint64_t v = 0x400000000e16b00aULL + std_space_offset(sp); > + qtest_writeq(qts, B + 0x0090, v); > + } > + qtest_writel(qts, B + 0x009c, 0x0); /* CMDQ_CONS */ > + qtest_writel(qts, B + 0x0098, 0x0); /* CMDQ_PROD */ > + { > + /* EVENTQ_BASE: add address-space offset (S/NS/Root/Realm). */ > + uint64_t v = 0x400000000e17000aULL + std_space_offset(sp); > + qtest_writeq(qts, B + 0x00a0, v); > + } > + qtest_writel(qts, B + 0x00a8, 0x0); /* EVENTQ_PROD */ > + qtest_writel(qts, B + 0x00ac, 0x0); /* EVENTQ_CONS */ > + qtest_writel(qts, B + 0x0088, 0x5); /* STRTAB_BASE_CFG */ > + { > + /* STRTAB_BASE: add address-space offset (S/NS/Root/Realm). */ > + uint64_t v = 0x400000000e179000ULL + std_space_offset(sp); > + qtest_writeq(qts, B + 0x0080, v); > + } > + qtest_writel(qts, B + 0x003C, 0x1); /* INIT */ > + qtest_writel(qts, B + 0x0020, 0xD); /* CR0 */ > +} > + > +static void smmu_prog_minimal(QTestState *qts, SMMUTestDevSpace space) > +{ > + /* Always program Non-Secure bank, then the requested space. */ > + uint64_t ns_base = smmu_bank_base(VIRT_SMMU_BASE, STD_SPACE_NONSECURE); > + smmu_prog_bank(qts, ns_base, STD_SPACE_NONSECURE); > + > + uint64_t sp_base = smmu_bank_base(VIRT_SMMU_BASE, space); > + if (sp_base != ns_base) { > + smmu_prog_bank(qts, sp_base, space); > + } > +} > + > +static uint32_t poll_dma_result(QPCIDevice *dev, QPCIBar bar, > + QTestState *qts) > +{ > + /* Trigger side effects (DMA) via REG_ID read once. */ > + (void)qpci_io_readl(dev, bar, STD_REG_ID); > + > + /* Poll until not BUSY, then return the result. */ > + for (int i = 0; i < 1000; i++) { > + uint32_t r = qpci_io_readl(dev, bar, STD_REG_DMA_RESULT); > + if (r != STD_DMA_RESULT_BUSY) { > + return r; > + } > + /* Small backoff to avoid busy spinning. */ > + g_usleep(1000); > + } > + /* Timeout treated as failure-like non-zero. */ > + return STD_DMA_RESULT_BUSY; > +} > + > +static void test_mmio_access(void) > +{ > + QTestState *qts; > + QGenericPCIBus gbus; > + QPCIDevice *dev; > + QPCIBar bar; > + uint8_t buf[DMA_LEN]; > + uint32_t attr_ns; > + qts = qtest_init("-machine virt,acpi=off,gic-version=3,iommu=smmuv3 " \ > + "-display none -smp 1 -m 512 -cpu max -net none " > + "-device smmu-testdev,device=0x0,function=0x1 "); > + > + qpci_init_generic(&gbus, qts, NULL, false); > + > + /* Find device by vendor/device ID to avoid slot surprises. */ > + dev = NULL; might as well init when you declare. > + for (int slot = 0; slot < 32 && !dev; slot++) { > + for (int fn = 0; fn < 8 && !dev; fn++) { > + QPCIDevice *cand = qpci_device_find(&gbus.bus, > + QPCI_DEVFN(slot, fn)); > + if (!cand) { > + continue; > + } > + uint16_t vid = qpci_config_readw(cand, PCI_VENDOR_ID); > + uint16_t did = qpci_config_readw(cand, PCI_DEVICE_ID); > + if (vid == 0x1b36 && did == 0x0005) { > + dev = cand; > + } else { > + g_free(cand); > + } > + } > + } > + g_assert_nonnull(dev); surely g_assert(dev) would do. > + > + qpci_device_enable(dev); > + bar = qpci_iomap(dev, 0, NULL); > + g_assert_false(bar.is_io); > + > + /* Baseline attribute reads. */ > + attr_ns = qpci_io_readl(dev, bar, STD_REG_ATTR_NS); > + g_assert_cmpuint(attr_ns, ==, 0x2); > + > + /* Program SMMU base and DMA parameters. */ > + qpci_io_writel(dev, bar, STD_REG_SMMU_BASE_LO, (uint32_t)VIRT_SMMU_BASE); > + qpci_io_writel(dev, bar, STD_REG_SMMU_BASE_HI, > + (uint32_t)(VIRT_SMMU_BASE >> 32)); > + qpci_io_writel(dev, bar, STD_REG_DMA_IOVA_LO, (uint32_t)STD_IOVA); > + qpci_io_writel(dev, bar, STD_REG_DMA_IOVA_HI, > + (uint32_t)(STD_IOVA >> 32)); > + qpci_io_writel(dev, bar, STD_REG_DMA_LEN, DMA_LEN); > + qpci_io_writel(dev, bar, STD_REG_DMA_DIR, 0); /* device -> host */ > + > + qtest_memset(qts, STD_IOVA, 0x00, DMA_LEN); > + qtest_memread(qts, STD_IOVA, buf, DMA_LEN); > + > + /* Refresh attrs via write to ensure legacy functionality still works. */ > + qpci_io_writel(dev, bar, STD_REG_ID, 0x1); > + /* > + * invoke translation builder for multiple > + * stage/security-space combinations (readable/refactored). > + */ > + const uint32_t modes[] = { 0u, 1u, 2u }; /* Stage1, Stage2, Nested stage */ > + const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; top of block. > + /* Use attrs-DMA path for end-to-end */ > + qpci_io_writel(dev, bar, STD_REG_DMA_MODE, 1); > + for (size_t mi = 0; mi < sizeof(modes) / sizeof(modes[0]); mi++) { > + const SMMUTestDevSpace *s1_set = NULL; > + size_t s1_count = 0; > + const SMMUTestDevSpace *s2_set = NULL; > + size_t s2_count = 0; > + > + switch (modes[mi]) { > + case 0u: > + case 1u: > + case 2u: > + s1_set = spaces; > + s1_count = sizeof(spaces) / sizeof(spaces[0]); > + s2_set = spaces; > + s2_count = sizeof(spaces) / sizeof(spaces[0]); > + break; > + default: > + g_assert_not_reached(); > + } > + > + for (size_t si = 0; si < s1_count; si++) { > + for (size_t sj = 0; sj < s2_count; sj++) { > + qpci_io_writel(dev, bar, STD_REG_TRANS_MODE, modes[mi]); > + qpci_io_writel(dev, bar, STD_REG_S1_SPACE, s1_set[si]); > + qpci_io_writel(dev, bar, STD_REG_S2_SPACE, s2_set[sj]); > + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x2); > + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x1); > + > + uint32_t st = qpci_io_readl(dev, bar, > + STD_REG_TRANS_STATUS); > + g_test_message("build: stage=%s s1=%s s2=%s status=0x%x", > + std_mode_to_str(modes[mi]), > + std_space_to_str(s1_set[si]), > + std_space_to_str(s2_set[sj]), st); > + /* Program SMMU registers in selected control bank. */ > + smmu_prog_minimal(qts, s1_set[si]); > + > + /* End-to-end DMA using tx_space per mode. */ > + SMMUTestDevSpace tx_space = > + (modes[mi] == 0u) ? s1_set[si] : s2_set[sj]; > + uint32_t dma_attrs = ((uint32_t)tx_space << 1); > + qpci_io_writel(dev, bar, STD_REG_DMA_ATTRS, > + dma_attrs); > + qpci_io_writel(dev, bar, STD_REG_DMA_DBELL, 1); > + /* Wait for DMA completion and assert success. */ > + { > + uint32_t dr = poll_dma_result(dev, bar, qts); > + uint32_t exp = expected_dma_result(modes[mi], > + spaces[si], > + spaces[sj]); > + g_assert_cmpuint(dr, ==, exp); > + g_test_message("polling end. attrs=0x%x res=0x%x", > + dma_attrs, dr); > + } > + /* Clear CD/STE/PTE built by the device for next round. */ > + qpci_io_writel(dev, bar, STD_REG_TRANS_CLEAR, 1); > + g_test_message("clear cache end."); > + } > + } > + } I suspect this function could be broken up a bit as new tests are added and functionality shared? > + > + qtest_quit(qts); > +} > + > +int main(int argc, char **argv) > +{ > + g_test_init(&argc, &argv, NULL); > + qtest_add_func("/smmu-testdev/mmio", test_mmio_access); > + return g_test_run(); > +} -- Alex Bennée Virtualisation Tech Lead @ Linaro ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source 2025-10-23 11:02 ` Alex Bennée @ 2025-10-27 15:26 ` Tao Tang 2025-10-27 15:52 ` Pierrick Bouvier 0 siblings, 1 reply; 16+ messages in thread From: Tao Tang @ 2025-10-27 15:26 UTC (permalink / raw) To: Alex Bennée Cc: Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Hi Alex, On 2025/10/23 19:02, Alex Bennée wrote: > tangtao1634 <tangtao1634@phytium.com.cn> writes: > >> From: Tao Tang <tangtao1634@phytium.com.cn> >> >> Introduce a bare-metal qtest that drives the new smmu-testdev to exercise >> the SMMUv3 emulation without guest firmware or drivers. The test programs >> a minimal Non-Secure context (STE/CD/PTE), triggers a DMA, and asserts >> translation results. >> >> Motivation >> ---------- >> SMMU testing in emulation often requires a large software stack and a >> realistic PCIe fabric, which adds flakiness and obscures failures. This >> qtest keeps the surface small and deterministic by using a hermetic DMA >> source that feeds the SMMU directly. >> >> What the test covers >> -------------------- >> * Builds a Non-Secure STE/CD/PTE for a chosen stream_id/ssid. >> * Primes source and destination host buffers. >> * Kicks a DMA via smmu-testdev and waits for completion. >> * Verifies translated access and payload equality. >> >> Non-goals and scope limits >> -------------------------- >> * Secure bank flows are omitted because Secure SMMU support is still RFC. >> A local Secure test exists and can be posted once the upstream series >> lands. >> * PCIe discovery, MSI/INTx, ATS/PRI, and driver bring-up are out of scope >> as smmu-testdev is not a realistic PCIe Endpoint nor a platform device. >> >> Rationale for a dedicated path >> ------------------------------ >> Using a generic PCI or virtio device would still require driver init and a >> richer bus model, undermining determinism for this focused purpose. This >> qtest, paired with smmu-testdev, keeps failures attributable to the SMMU >> translation path. >> >> Finally we document the smmu-testdev device in docs/specs. >> >> Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> >> --- >> ------------------------------<snip>------------------------------ >> >> >> >> ------------------------------<snip>------------------------------ >> + >> + /* Find device by vendor/device ID to avoid slot surprises. */ >> + dev = NULL; > might as well init when you declare. > >> + g_assert_nonnull(dev); > surely g_assert(dev) would do. > >> + const uint32_t modes[] = { 0u, 1u, 2u }; /* Stage1, Stage2, Nested stage */ >> + const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; > top of block. Thank you very much for your valuable feedback. Also I will refactor these codes with the guide of summarized plans as described in patch #1. > >> + /* Use attrs-DMA path for end-to-end */ >> + qpci_io_writel(dev, bar, STD_REG_DMA_MODE, 1); >> + for (size_t mi = 0; mi < sizeof(modes) / sizeof(modes[0]); mi++) { >> + const SMMUTestDevSpace *s1_set = NULL; >> + size_t s1_count = 0; >> + const SMMUTestDevSpace *s2_set = NULL; >> + size_t s2_count = 0; >> + >> + switch (modes[mi]) { >> + case 0u: >> + case 1u: >> + case 2u: >> + s1_set = spaces; >> + s1_count = sizeof(spaces) / sizeof(spaces[0]); >> + s2_set = spaces; >> + s2_count = sizeof(spaces) / sizeof(spaces[0]); >> + break; >> + default: >> + g_assert_not_reached(); >> + } >> + >> + for (size_t si = 0; si < s1_count; si++) { >> + for (size_t sj = 0; sj < s2_count; sj++) { >> + qpci_io_writel(dev, bar, STD_REG_TRANS_MODE, modes[mi]); >> + qpci_io_writel(dev, bar, STD_REG_S1_SPACE, s1_set[si]); >> + qpci_io_writel(dev, bar, STD_REG_S2_SPACE, s2_set[sj]); >> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x2); >> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x1); >> + >> + uint32_t st = qpci_io_readl(dev, bar, >> + STD_REG_TRANS_STATUS); >> + g_test_message("build: stage=%s s1=%s s2=%s status=0x%x", >> + std_mode_to_str(modes[mi]), >> + std_space_to_str(s1_set[si]), >> + std_space_to_str(s2_set[sj]), st); >> + /* Program SMMU registers in selected control bank. */ >> + smmu_prog_minimal(qts, s1_set[si]); >> + >> + /* End-to-end DMA using tx_space per mode. */ >> + SMMUTestDevSpace tx_space = >> + (modes[mi] == 0u) ? s1_set[si] : s2_set[sj]; >> + uint32_t dma_attrs = ((uint32_t)tx_space << 1); >> + qpci_io_writel(dev, bar, STD_REG_DMA_ATTRS, >> + dma_attrs); >> + qpci_io_writel(dev, bar, STD_REG_DMA_DBELL, 1); >> + /* Wait for DMA completion and assert success. */ >> + { >> + uint32_t dr = poll_dma_result(dev, bar, qts); >> + uint32_t exp = expected_dma_result(modes[mi], >> + spaces[si], >> + spaces[sj]); >> + g_assert_cmpuint(dr, ==, exp); >> + g_test_message("polling end. attrs=0x%x res=0x%x", >> + dma_attrs, dr); >> + } >> + /* Clear CD/STE/PTE built by the device for next round. */ >> + qpci_io_writel(dev, bar, STD_REG_TRANS_CLEAR, 1); >> + g_test_message("clear cache end."); >> + } >> + } >> + } > I suspect this function could be broken up a bit as new tests are added > and functionality shared? Sure. I've actually been thinking along the same lines. As I plan for future tests, I'm considering how best to organize the test cases given the numerous combinations of features we'll need to cover. For example, beyond iterating through security states and translation stages, we will also need to test many other parameters, such as: - Linear vs. two-level Stream Tables - Different Output Address Sizes (Although only support 44bits in current SMMU implementation) My question to you and the wider group is, how far should we go in covering these combinations for an initial smoke test? The current loops for security state and translation stage cover the basics, but I'm wondering if we should aim for more complexity at this stage, or if that's a task for future patches. I'd be very interested to hear everyone's opinion on the right scope. In any case, your suggestion to break the current test logic into smaller, shared functions is definitely the right first step to manage the structure. I will refactor the code accordingly in the next version. Thanks again for the valuable suggestion! Best regards, Tao ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source 2025-10-27 15:26 ` Tao Tang @ 2025-10-27 15:52 ` Pierrick Bouvier 2025-10-27 17:07 ` Alex Bennée 0 siblings, 1 reply; 16+ messages in thread From: Pierrick Bouvier @ 2025-10-27 15:52 UTC (permalink / raw) To: Tao Tang, Alex Bennée Cc: Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Hi Tao, On 2025-10-27 16:26, Tao Tang wrote: > Hi Alex, > > On 2025/10/23 19:02, Alex Bennée wrote: >> tangtao1634 <tangtao1634@phytium.com.cn> writes: >> >>> From: Tao Tang <tangtao1634@phytium.com.cn> >>> >>> Introduce a bare-metal qtest that drives the new smmu-testdev to exercise >>> the SMMUv3 emulation without guest firmware or drivers. The test programs >>> a minimal Non-Secure context (STE/CD/PTE), triggers a DMA, and asserts >>> translation results. >>> >>> Motivation >>> ---------- >>> SMMU testing in emulation often requires a large software stack and a >>> realistic PCIe fabric, which adds flakiness and obscures failures. This >>> qtest keeps the surface small and deterministic by using a hermetic DMA >>> source that feeds the SMMU directly. >>> >>> What the test covers >>> -------------------- >>> * Builds a Non-Secure STE/CD/PTE for a chosen stream_id/ssid. >>> * Primes source and destination host buffers. >>> * Kicks a DMA via smmu-testdev and waits for completion. >>> * Verifies translated access and payload equality. >>> >>> Non-goals and scope limits >>> -------------------------- >>> * Secure bank flows are omitted because Secure SMMU support is still RFC. >>> A local Secure test exists and can be posted once the upstream series >>> lands. >>> * PCIe discovery, MSI/INTx, ATS/PRI, and driver bring-up are out of scope >>> as smmu-testdev is not a realistic PCIe Endpoint nor a platform device. >>> >>> Rationale for a dedicated path >>> ------------------------------ >>> Using a generic PCI or virtio device would still require driver init and a >>> richer bus model, undermining determinism for this focused purpose. This >>> qtest, paired with smmu-testdev, keeps failures attributable to the SMMU >>> translation path. >>> >>> Finally we document the smmu-testdev device in docs/specs. >>> >>> Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> >>> --- >>> ------------------------------<snip>------------------------------ >>> >>> >>> >>> ------------------------------<snip>------------------------------ >>> + >>> + /* Find device by vendor/device ID to avoid slot surprises. */ >>> + dev = NULL; >> might as well init when you declare. >> >>> + g_assert_nonnull(dev); >> surely g_assert(dev) would do. >> >>> + const uint32_t modes[] = { 0u, 1u, 2u }; /* Stage1, Stage2, Nested stage */ >>> + const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; >> top of block. > > > Thank you very much for your valuable feedback. Also I will refactor > these codes with the guide of summarized plans as described in patch #1. > >> >>> + /* Use attrs-DMA path for end-to-end */ >>> + qpci_io_writel(dev, bar, STD_REG_DMA_MODE, 1); >>> + for (size_t mi = 0; mi < sizeof(modes) / sizeof(modes[0]); mi++) { >>> + const SMMUTestDevSpace *s1_set = NULL; >>> + size_t s1_count = 0; >>> + const SMMUTestDevSpace *s2_set = NULL; >>> + size_t s2_count = 0; >>> + >>> + switch (modes[mi]) { >>> + case 0u: >>> + case 1u: >>> + case 2u: >>> + s1_set = spaces; >>> + s1_count = sizeof(spaces) / sizeof(spaces[0]); >>> + s2_set = spaces; >>> + s2_count = sizeof(spaces) / sizeof(spaces[0]); >>> + break; >>> + default: >>> + g_assert_not_reached(); >>> + } >>> + >>> + for (size_t si = 0; si < s1_count; si++) { >>> + for (size_t sj = 0; sj < s2_count; sj++) { >>> + qpci_io_writel(dev, bar, STD_REG_TRANS_MODE, modes[mi]); >>> + qpci_io_writel(dev, bar, STD_REG_S1_SPACE, s1_set[si]); >>> + qpci_io_writel(dev, bar, STD_REG_S2_SPACE, s2_set[sj]); >>> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x2); >>> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x1); >>> + >>> + uint32_t st = qpci_io_readl(dev, bar, >>> + STD_REG_TRANS_STATUS); >>> + g_test_message("build: stage=%s s1=%s s2=%s status=0x%x", >>> + std_mode_to_str(modes[mi]), >>> + std_space_to_str(s1_set[si]), >>> + std_space_to_str(s2_set[sj]), st); >>> + /* Program SMMU registers in selected control bank. */ >>> + smmu_prog_minimal(qts, s1_set[si]); >>> + >>> + /* End-to-end DMA using tx_space per mode. */ >>> + SMMUTestDevSpace tx_space = >>> + (modes[mi] == 0u) ? s1_set[si] : s2_set[sj]; >>> + uint32_t dma_attrs = ((uint32_t)tx_space << 1); >>> + qpci_io_writel(dev, bar, STD_REG_DMA_ATTRS, >>> + dma_attrs); >>> + qpci_io_writel(dev, bar, STD_REG_DMA_DBELL, 1); >>> + /* Wait for DMA completion and assert success. */ >>> + { >>> + uint32_t dr = poll_dma_result(dev, bar, qts); >>> + uint32_t exp = expected_dma_result(modes[mi], >>> + spaces[si], >>> + spaces[sj]); >>> + g_assert_cmpuint(dr, ==, exp); >>> + g_test_message("polling end. attrs=0x%x res=0x%x", >>> + dma_attrs, dr); >>> + } >>> + /* Clear CD/STE/PTE built by the device for next round. */ >>> + qpci_io_writel(dev, bar, STD_REG_TRANS_CLEAR, 1); >>> + g_test_message("clear cache end."); >>> + } >>> + } >>> + } >> I suspect this function could be broken up a bit as new tests are added >> and functionality shared? > > > Sure. I've actually been thinking along the same lines. As I plan for > future tests, I'm considering how best to organize the test cases given > the numerous combinations of features we'll need to cover. For example, > beyond iterating through security states and translation stages, we will > also need to test many other parameters, such as: > > - Linear vs. two-level Stream Tables > > - Different Output Address Sizes (Although only support 44bits in > current SMMU implementation) > Reading through this, I start to wonder if we will not end up rewriting a full SMMU driver by accident. The problem with SMMU development is that from the outside, it seems to be "just a device translating DMA accesses". In reality, the "just" means we have a stateful device, configured from possibly different parts in a software stack. For example, with Realms, TF-A, RMM, and kernel all contribute to this state. A possible analogy would be if we used a QTest device to test QEMU MMU implementation, instead of simply relying on running a kernel exercising this code. That said, it's still useful for some basic scenarios, but I'm not sure it's the ultimate answer for complex use cases, and thus, it should not try to cover it. As well, this brings the question of which kind of solution we would need for that. It seems that one need would be to check the SMMU "state" from user space, which moves the problem on having a driver able to poke this state. > My question to you and the wider group is, how far should we go in > covering these combinations for an initial smoke test? The current loops > for security state and translation stage cover the basics, but I'm > wondering if we should aim for more complexity at this stage, or if > that's a task for future patches. I'd be very interested to hear > everyone's opinion on the right scope. > We have to start somewhere, so something simple and not trying to solve all use cases is the right approach. It can even just be read/write config/registers before trying to add any DMA scenario. > In any case, your suggestion to break the current test logic into > smaller, shared functions is definitely the right first step to manage > the structure. I will refactor the code accordingly in the next version. > > Thanks again for the valuable suggestion! > > Best regards, > > Tao > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source 2025-10-27 15:52 ` Pierrick Bouvier @ 2025-10-27 17:07 ` Alex Bennée 0 siblings, 0 replies; 16+ messages in thread From: Alex Bennée @ 2025-10-27 17:07 UTC (permalink / raw) To: Pierrick Bouvier Cc: Tao Tang, Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Pierrick Bouvier <pierrick.bouvier@linaro.org> writes: > Hi Tao, > > On 2025-10-27 16:26, Tao Tang wrote: >> Hi Alex, >> On 2025/10/23 19:02, Alex Bennée wrote: >>> tangtao1634 <tangtao1634@phytium.com.cn> writes: >>> >>>> From: Tao Tang <tangtao1634@phytium.com.cn> >>>> >>>> Introduce a bare-metal qtest that drives the new smmu-testdev to exercise >>>> the SMMUv3 emulation without guest firmware or drivers. The test programs >>>> a minimal Non-Secure context (STE/CD/PTE), triggers a DMA, and asserts >>>> translation results. >>>> >>>> Motivation >>>> ---------- >>>> SMMU testing in emulation often requires a large software stack and a >>>> realistic PCIe fabric, which adds flakiness and obscures failures. This >>>> qtest keeps the surface small and deterministic by using a hermetic DMA >>>> source that feeds the SMMU directly. >>>> >>>> What the test covers >>>> -------------------- >>>> * Builds a Non-Secure STE/CD/PTE for a chosen stream_id/ssid. >>>> * Primes source and destination host buffers. >>>> * Kicks a DMA via smmu-testdev and waits for completion. >>>> * Verifies translated access and payload equality. >>>> >>>> Non-goals and scope limits >>>> -------------------------- >>>> * Secure bank flows are omitted because Secure SMMU support is still RFC. >>>> A local Secure test exists and can be posted once the upstream series >>>> lands. >>>> * PCIe discovery, MSI/INTx, ATS/PRI, and driver bring-up are out of scope >>>> as smmu-testdev is not a realistic PCIe Endpoint nor a platform device. >>>> >>>> Rationale for a dedicated path >>>> ------------------------------ >>>> Using a generic PCI or virtio device would still require driver init and a >>>> richer bus model, undermining determinism for this focused purpose. This >>>> qtest, paired with smmu-testdev, keeps failures attributable to the SMMU >>>> translation path. >>>> >>>> Finally we document the smmu-testdev device in docs/specs. >>>> >>>> Signed-off-by: Tao Tang <tangtao1634@phytium.com.cn> >>>> --- >>>> ------------------------------<snip>------------------------------ >>>> >>>> >>>> >>>> ------------------------------<snip>------------------------------ >>>> + >>>> + /* Find device by vendor/device ID to avoid slot surprises. */ >>>> + dev = NULL; >>> might as well init when you declare. >>> >>>> + g_assert_nonnull(dev); >>> surely g_assert(dev) would do. >>> >>>> + const uint32_t modes[] = { 0u, 1u, 2u }; /* Stage1, Stage2, Nested stage */ >>>> + const SMMUTestDevSpace spaces[] = { STD_SPACE_NONSECURE }; >>> top of block. >> Thank you very much for your valuable feedback. Also I will refactor >> these codes with the guide of summarized plans as described in patch #1. >> >>> >>>> + /* Use attrs-DMA path for end-to-end */ >>>> + qpci_io_writel(dev, bar, STD_REG_DMA_MODE, 1); >>>> + for (size_t mi = 0; mi < sizeof(modes) / sizeof(modes[0]); mi++) { >>>> + const SMMUTestDevSpace *s1_set = NULL; >>>> + size_t s1_count = 0; >>>> + const SMMUTestDevSpace *s2_set = NULL; >>>> + size_t s2_count = 0; >>>> + >>>> + switch (modes[mi]) { >>>> + case 0u: >>>> + case 1u: >>>> + case 2u: >>>> + s1_set = spaces; >>>> + s1_count = sizeof(spaces) / sizeof(spaces[0]); >>>> + s2_set = spaces; >>>> + s2_count = sizeof(spaces) / sizeof(spaces[0]); >>>> + break; >>>> + default: >>>> + g_assert_not_reached(); >>>> + } >>>> + >>>> + for (size_t si = 0; si < s1_count; si++) { >>>> + for (size_t sj = 0; sj < s2_count; sj++) { >>>> + qpci_io_writel(dev, bar, STD_REG_TRANS_MODE, modes[mi]); >>>> + qpci_io_writel(dev, bar, STD_REG_S1_SPACE, s1_set[si]); >>>> + qpci_io_writel(dev, bar, STD_REG_S2_SPACE, s2_set[sj]); >>>> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x2); >>>> + qpci_io_writel(dev, bar, STD_REG_TRANS_DBELL, 0x1); >>>> + >>>> + uint32_t st = qpci_io_readl(dev, bar, >>>> + STD_REG_TRANS_STATUS); >>>> + g_test_message("build: stage=%s s1=%s s2=%s status=0x%x", >>>> + std_mode_to_str(modes[mi]), >>>> + std_space_to_str(s1_set[si]), >>>> + std_space_to_str(s2_set[sj]), st); >>>> + /* Program SMMU registers in selected control bank. */ >>>> + smmu_prog_minimal(qts, s1_set[si]); >>>> + >>>> + /* End-to-end DMA using tx_space per mode. */ >>>> + SMMUTestDevSpace tx_space = >>>> + (modes[mi] == 0u) ? s1_set[si] : s2_set[sj]; >>>> + uint32_t dma_attrs = ((uint32_t)tx_space << 1); >>>> + qpci_io_writel(dev, bar, STD_REG_DMA_ATTRS, >>>> + dma_attrs); >>>> + qpci_io_writel(dev, bar, STD_REG_DMA_DBELL, 1); >>>> + /* Wait for DMA completion and assert success. */ >>>> + { >>>> + uint32_t dr = poll_dma_result(dev, bar, qts); >>>> + uint32_t exp = expected_dma_result(modes[mi], >>>> + spaces[si], >>>> + spaces[sj]); >>>> + g_assert_cmpuint(dr, ==, exp); >>>> + g_test_message("polling end. attrs=0x%x res=0x%x", >>>> + dma_attrs, dr); >>>> + } >>>> + /* Clear CD/STE/PTE built by the device for next round. */ >>>> + qpci_io_writel(dev, bar, STD_REG_TRANS_CLEAR, 1); >>>> + g_test_message("clear cache end."); >>>> + } >>>> + } >>>> + } >>> I suspect this function could be broken up a bit as new tests are added >>> and functionality shared? >> Sure. I've actually been thinking along the same lines. As I plan >> for >> future tests, I'm considering how best to organize the test cases given >> the numerous combinations of features we'll need to cover. For example, >> beyond iterating through security states and translation stages, we will >> also need to test many other parameters, such as: >> - Linear vs. two-level Stream Tables >> - Different Output Address Sizes (Although only support 44bits in >> current SMMU implementation) >> > > Reading through this, I start to wonder if we will not end up > rewriting a full SMMU driver by accident. The problem with SMMU > development is that from the outside, it seems to be "just a device > translating DMA accesses". In reality, the "just" means we have a > stateful device, configured from possibly different parts in a > software stack. For example, with Realms, TF-A, RMM, and kernel all > contribute to this state. > > A possible analogy would be if we used a QTest device to test QEMU MMU > implementation, instead of simply relying on running a kernel > exercising this code. > > That said, it's still useful for some basic scenarios, but I'm not > sure it's the ultimate answer for complex use cases, and thus, it > should not try to cover it. > As well, this brings the question of which kind of solution we would > need for that. It seems that one need would be to check the SMMU > "state" from user space, which moves the problem on having a driver > able to poke this state. We should be thinking of targeted unit tests. The difference between this and a full OS is we don't need to manage multiple shifting memory maps over time. Setup a page (or two) with the permissions you expect and check that works. This would also be the place to verify edge cases that a more complex driver might get to but is hard to trigger because there are too many moving parts. IOW the scope of the qtest tests should be focused on atomic individual features and the functional tests cover making sure everything works together as a whole. >> My question to you and the wider group is, how far should we go in >> covering these combinations for an initial smoke test? The current loops >> for security state and translation stage cover the basics, but I'm >> wondering if we should aim for more complexity at this stage, or if >> that's a task for future patches. I'd be very interested to hear >> everyone's opinion on the right scope. >> > > We have to start somewhere, so something simple and not trying to > solve all use cases is the right approach. It can even just be > read/write config/registers before trying to add any DMA scenario. > >> In any case, your suggestion to break the current test logic into >> smaller, shared functions is definitely the right first step to manage >> the structure. I will refactor the code accordingly in the next version. >> Thanks again for the valuable suggestion! >> Best regards, >> Tao >> -- Alex Bennée Virtualisation Tech Lead @ Linaro ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 2025-09-30 16:53 ` [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device tangtao1634 2025-09-30 16:53 ` [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source tangtao1634 @ 2025-10-23 9:04 ` Tao Tang 2025-10-23 10:06 ` Alex Bennée 2025-10-27 13:58 ` Peter Maydell 4 siblings, 0 replies; 16+ messages in thread From: Tao Tang @ 2025-10-23 9:04 UTC (permalink / raw) To: Eric Auger, Peter Maydell, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Cc: qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh Gentle ping. Any feedback on this patch series would be appreciated. V2: https://lore.kernel.org/qemu-devel/20250930165340.42788-1-tangtao1634@phytium.com.cn/ V1: https://lore.kernel.org/qemu-devel/20250925153550.105915-1-tangtao1634@phytium.com.cn/ On 2025/10/1 00:53, tangtao1634 wrote: > From: Tao Tang <tangtao1634@phytium.com.cn> > > This patch series (V2) introduces several cleanups and improvements to the smmu-testdev device. The main goals are to refactor shared code, enhance robustness, and significantly clarify the complex page table construction used for testing. > > Motivation > ---------- > > Currently, thoroughly testing the SMMUv3 emulation requires a significant > software stack. We need to boot a full guest operating system (like Linux) > with the appropriate drivers (e.g., IOMMUFD) and rely on firmware (e.g., > ACPI with IORT tables or Hafnium) to correctly configure the SMMU and > orchestrate DMA from a peripheral device. > > This dependency on a complex software stack presents several challenges: > > * High Barrier to Entry: Writing targeted tests for specific SMMU > features (like fault handling, specific translation regimes, etc.) > becomes cumbersome. > > * Difficult to Debug: It's hard to distinguish whether a bug originates > from the SMMU emulation itself, the guest driver, the firmware > tables, or the guest kernel's configuration. > > * Slow Iteration: The need to boot a full guest OS slows down the > development and testing cycle. > > The primary goal of this work is to create a lightweight, self-contained > testing environment that allows us to exercise the SMMU's core logic > directly at the qtest level, removing the need for any guest-side software. > > Our Approach: A Dedicated Test Device > ------------------------------------- > > To achieve this, we introduce two main components: > > * A new, minimal hardware device: smmu-testdev. > * A corresponding qtest that drives this device to generate SMMU-bound > traffic. > > A key question is, "Why introduce a new smmu-testdev instead of using an > existing PCIe or platform device?" > > The answer lies in our goal to minimize complexity. Standard devices, > whether PCIe or platform, come with their own intricate initialization > protocols and often require a complex driver state machine to function. > Using them would re-introduce the very driver-level complexity we aim to > avoid. > > The smmu-testdev is intentionally not a conformant, general-purpose PCIe > or platform device. It is a purpose-built, highly simplified "DMA engine." > I've designed it to be analogous to a minimal PCIe Root Complex that > bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) > to provide a direct, programmable path for a DMA request to reach the SMMU. > Its sole purpose is to trigger a DMA transaction when its registers are > written to, making it perfectly suited for direct control from a test > environment like qtest. > > The Qtest Framework > ------------------- > > The new qtest (smmu-testdev-qtest.c) serves as the "bare-metal driver" > for both the SMMU and the smmu-testdev. It manually performs all the > setup that would typically be handled by the guest kernel and firmware, > but in a completely controlled and predictable manner: > > 1. SMMU Configuration: It directly initializes the SMMU's registers to a > known state. > > 2. Translation Structure Setup: It manually constructs the necessary > translation structures in memory, including Stream Table Entries > (STEs), Context Descriptors (CDs), and Page Tables (PTEs). > > 3. DMA Trigger: It programs the smmu-testdev to initiate a DMA operation > targeting a specific IOVA. > > 4. Verification: It waits for the transaction to complete and verifies > that the memory was accessed correctly after address translation by > the SMMU. > > This framework provides a solid and extensible foundation for validating > the SMMU's core translation paths. The initial test included in this > series covers a basic DMA completion path in the Non-Secure bank, > serving as a smoke test and a proof of concept. > > It is worth noting that this series currently only includes tests for the > Non-Secure SMMU. I am aware of the ongoing discussions and RFC patches > for Secure SMMU support. To avoid a dependency on unmerged work, this > submission does not include tests for the Secure world. However, I have > already implemented these tests locally, and I am prepared to submit > them for review as soon as the core Secure SMMU support is merged > upstream. > > > Changes from v1 RFC: > - Clarify Page Table Construction: > Detailed comments have been added to the page table construction logic. This is a key improvement, as the test setup extensively re-uses the same set of page tables for multiple translation stages and purposes (e.g., nested S1/S2 walks, CD fetch). The new comments explain this sharing mechanism, which can otherwise be confusing to follow. > > - Refactor Shared Helpers: > The helper functions std_space_offset and std_space_to_str are now moved to a common header file. This allows them to be used by both the main device implementation (hw/misc/smmu-testdev.c) and its qtest (tests/qtest/smmu-testdev-qtest.c), improving code re-use and maintainability. > > - Enhance Robustness: > Assertions have been added to ensure the device operates only in the expected Non-secure context. Additional conditional checks are also included to prevent potential runtime errors and make the test device more stable. > > - Code Simplification and Cleanup: > Several functions that were redundant with existing macros for constructing Context Descriptors (CD) and Stream Table Entries (STE) have been removed. This simplifies the test data setup and reduces code duplication. > > Other unused code fragments have also been removed to improve overall code clarity and hygiene. > > Tao Tang (2): > hw/misc/smmu-testdev: introduce minimal SMMUv3 test device > tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source > > docs/specs/index.rst | 1 + > docs/specs/smmu-testdev.rst | 45 ++ > hw/misc/Kconfig | 5 + > hw/misc/meson.build | 1 + > hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++ > include/hw/misc/smmu-testdev.h | 402 +++++++++++++ > tests/qtest/meson.build | 1 + > tests/qtest/smmu-testdev-qtest.c | 238 ++++++++ > 8 files changed, 1636 insertions(+) > create mode 100644 docs/specs/smmu-testdev.rst > create mode 100644 hw/misc/smmu-testdev.c > create mode 100644 include/hw/misc/smmu-testdev.h > create mode 100644 tests/qtest/smmu-testdev-qtest.c > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 ` (2 preceding siblings ...) 2025-10-23 9:04 ` [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework Tao Tang @ 2025-10-23 10:06 ` Alex Bennée 2025-10-23 10:11 ` Peter Maydell 2025-10-27 13:32 ` Tao Tang 2025-10-27 13:58 ` Peter Maydell 4 siblings, 2 replies; 16+ messages in thread From: Alex Bennée @ 2025-10-23 10:06 UTC (permalink / raw) To: tangtao1634 Cc: pbonzini, farosas, lvivier, Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh tangtao1634 <tangtao1634@phytium.com.cn> writes: > From: Tao Tang <tangtao1634@phytium.com.cn> > > This patch series (V2) introduces several cleanups and improvements to the smmu-testdev device. The main goals are to refactor shared code, enhance robustness, and significantly clarify the complex page table construction used for testing. > > Motivation > ---------- > > Currently, thoroughly testing the SMMUv3 emulation requires a significant > software stack. We need to boot a full guest operating system (like Linux) > with the appropriate drivers (e.g., IOMMUFD) and rely on firmware (e.g., > ACPI with IORT tables or Hafnium) to correctly configure the SMMU and > orchestrate DMA from a peripheral device. > > This dependency on a complex software stack presents several challenges: > > * High Barrier to Entry: Writing targeted tests for specific SMMU > features (like fault handling, specific translation regimes, etc.) > becomes cumbersome. > > * Difficult to Debug: It's hard to distinguish whether a bug originates > from the SMMU emulation itself, the guest driver, the firmware > tables, or the guest kernel's configuration. > > * Slow Iteration: The need to boot a full guest OS slows down the > development and testing cycle. > > The primary goal of this work is to create a lightweight, self-contained > testing environment that allows us to exercise the SMMU's core logic > directly at the qtest level, removing the need for any guest-side > software. I agree, an excellent motivation. > > Our Approach: A Dedicated Test Device > ------------------------------------- > > To achieve this, we introduce two main components: > > * A new, minimal hardware device: smmu-testdev. > * A corresponding qtest that drives this device to generate SMMU-bound > traffic. > > A key question is, "Why introduce a new smmu-testdev instead of using an > existing PCIe or platform device?" I curious what the split between PCIe and platform devices that need an SMMU are. I suspect there is a strong split between the virtualisation case and the emulation case. > The answer lies in our goal to minimize complexity. Standard devices, > whether PCIe or platform, come with their own intricate initialization > protocols and often require a complex driver state machine to function. > Using them would re-introduce the very driver-level complexity we aim to > avoid. > > The smmu-testdev is intentionally not a conformant, general-purpose PCIe > or platform device. It is a purpose-built, highly simplified "DMA engine." > I've designed it to be analogous to a minimal PCIe Root Complex that > bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) > to provide a direct, programmable path for a DMA request to reach the SMMU. > Its sole purpose is to trigger a DMA transaction when its registers are > written to, making it perfectly suited for direct control from a test > environment like qtest. > > The Qtest Framework > ------------------- > > The new qtest (smmu-testdev-qtest.c) serves as the "bare-metal driver" > for both the SMMU and the smmu-testdev. It manually performs all the > setup that would typically be handled by the guest kernel and firmware, > but in a completely controlled and predictable manner: > > 1. SMMU Configuration: It directly initializes the SMMU's registers to a > known state. > > 2. Translation Structure Setup: It manually constructs the necessary > translation structures in memory, including Stream Table Entries > (STEs), Context Descriptors (CDs), and Page Tables (PTEs). > > 3. DMA Trigger: It programs the smmu-testdev to initiate a DMA operation > targeting a specific IOVA. > > 4. Verification: It waits for the transaction to complete and verifies > that the memory was accessed correctly after address translation by > the SMMU. > > This framework provides a solid and extensible foundation for validating > the SMMU's core translation paths. The initial test included in this > series covers a basic DMA completion path in the Non-Secure bank, > serving as a smoke test and a proof of concept. > > It is worth noting that this series currently only includes tests for the > Non-Secure SMMU. I am aware of the ongoing discussions and RFC patches > for Secure SMMU support. To avoid a dependency on unmerged work, this > submission does not include tests for the Secure world. However, I have > already implemented these tests locally, and I am prepared to submit > them for review as soon as the core Secure SMMU support is merged > upstream. What about other IOMMU's? Are there any other bus mediating devices modelled in QEMU that could also benefit from the ability to trigger DMA transactions? > > > Changes from v1 RFC: > - Clarify Page Table Construction: > Detailed comments have been added to the page table construction logic. This is a key improvement, as the test setup extensively re-uses the same set of page tables for multiple translation stages and purposes (e.g., nested S1/S2 walks, CD fetch). The new comments explain this sharing mechanism, which can otherwise be confusing to follow. > > - Refactor Shared Helpers: > The helper functions std_space_offset and std_space_to_str are now moved to a common header file. This allows them to be used by both the main device implementation (hw/misc/smmu-testdev.c) and its qtest (tests/qtest/smmu-testdev-qtest.c), improving code re-use and maintainability. > > - Enhance Robustness: > Assertions have been added to ensure the device operates only in the expected Non-secure context. Additional conditional checks are also included to prevent potential runtime errors and make the test device more stable. > > - Code Simplification and Cleanup: > Several functions that were redundant with existing macros for constructing Context Descriptors (CD) and Stream Table Entries (STE) have been removed. This simplifies the test data setup and reduces code duplication. > > Other unused code fragments have also been removed to improve overall code clarity and hygiene. > > Tao Tang (2): > hw/misc/smmu-testdev: introduce minimal SMMUv3 test device > tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source > > docs/specs/index.rst | 1 + > docs/specs/smmu-testdev.rst | 45 ++ > hw/misc/Kconfig | 5 + > hw/misc/meson.build | 1 + > hw/misc/smmu-testdev.c | 943 +++++++++++++++++++++++++++++++ > include/hw/misc/smmu-testdev.h | 402 +++++++++++++ > tests/qtest/meson.build | 1 + > tests/qtest/smmu-testdev-qtest.c | 238 ++++++++ > 8 files changed, 1636 insertions(+) > create mode 100644 docs/specs/smmu-testdev.rst > create mode 100644 hw/misc/smmu-testdev.c > create mode 100644 include/hw/misc/smmu-testdev.h > create mode 100644 tests/qtest/smmu-testdev-qtest.c -- Alex Bennée Virtualisation Tech Lead @ Linaro ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-10-23 10:06 ` Alex Bennée @ 2025-10-23 10:11 ` Peter Maydell 2025-10-27 13:32 ` Tao Tang 1 sibling, 0 replies; 16+ messages in thread From: Peter Maydell @ 2025-10-23 10:11 UTC (permalink / raw) To: Alex Bennée Cc: tangtao1634, pbonzini, farosas, lvivier, Eric Auger, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh On Thu, 23 Oct 2025 at 11:06, Alex Bennée <alex.bennee@linaro.org> wrote: > tangtao1634 <tangtao1634@phytium.com.cn> writes: > > From: Tao Tang <tangtao1634@phytium.com.cn> > > A key question is, "Why introduce a new smmu-testdev instead of using an > > existing PCIe or platform device?" > > I curious what the split between PCIe and platform devices that need an > SMMU are. I suspect there is a strong split between the virtualisation > case and the emulation case. I don't think emulation vs virtualization matters much here. My impression is that SMMU is almost entirely for PCI/PCIe; the exception is that for Realm emulation we need to do granule checks for non-PCI devices like the GIC ITS, and the easy way to do that is to have the ITS accesses go through the SMMU as a "NoStreamID" client. (Linaro has a JIRA card for the Realm/GIC interaction: https://linaro.atlassian.net/browse/QEMU-606 ) thanks -- PMM ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-10-23 10:06 ` Alex Bennée 2025-10-23 10:11 ` Peter Maydell @ 2025-10-27 13:32 ` Tao Tang 2025-10-27 14:25 ` Alex Bennée 1 sibling, 1 reply; 16+ messages in thread From: Tao Tang @ 2025-10-27 13:32 UTC (permalink / raw) To: Alex Bennée Cc: Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Hi Alex, On 2025/10/23 18:06, Alex Bennée wrote: > tangtao1634 <tangtao1634@phytium.com.cn> writes: > >> From: Tao Tang <tangtao1634@phytium.com.cn> >> >> This patch series (V2) introduces several cleanups and improvements to the smmu-testdev device. The main goals are to refactor shared code, enhance robustness, and significantly clarify the complex page table construction used for testing. >> >> Motivation >> ---------- >> >> Currently, thoroughly testing the SMMUv3 emulation requires a significant >> software stack. We need to boot a full guest operating system (like Linux) >> with the appropriate drivers (e.g., IOMMUFD) and rely on firmware (e.g., >> ACPI with IORT tables or Hafnium) to correctly configure the SMMU and >> orchestrate DMA from a peripheral device. >> >> This dependency on a complex software stack presents several challenges: >> >> * High Barrier to Entry: Writing targeted tests for specific SMMU >> features (like fault handling, specific translation regimes, etc.) >> becomes cumbersome. >> >> * Difficult to Debug: It's hard to distinguish whether a bug originates >> from the SMMU emulation itself, the guest driver, the firmware >> tables, or the guest kernel's configuration. >> >> * Slow Iteration: The need to boot a full guest OS slows down the >> development and testing cycle. >> >> The primary goal of this work is to create a lightweight, self-contained >> testing environment that allows us to exercise the SMMU's core logic >> directly at the qtest level, removing the need for any guest-side >> software. > I agree, an excellent motivation. > >> Our Approach: A Dedicated Test Device >> ------------------------------------- >> >> To achieve this, we introduce two main components: >> >> * A new, minimal hardware device: smmu-testdev. >> * A corresponding qtest that drives this device to generate SMMU-bound >> traffic. >> >> A key question is, "Why introduce a new smmu-testdev instead of using an >> existing PCIe or platform device?" > I curious what the split between PCIe and platform devices that need an > SMMU are. I suspect there is a strong split between the virtualisation > case and the emulation case. Thanks again for the insightful questions and for sparking this valuable discussion. From my observation of real-world, commercially available SoCs, the SMMU is almost exclusively designed for and used with PCIe. Of course, you're right that architecturally, the SMMU specification certainly allows for non-PCIe clients. Peter's point about using the SMMU for the GIC ITS in a Realm context is an excellent example. I've also personally seen similar setups in the TF-A-Tests+FVP software stack, where platform device, named SMMUv3TestEngine, was used to test the SMMU. However, from the perspective of QEMU's current implementation, there are some significant limitations that guided the design of smmu-testdev. At the moment, the SMMU model does not really support non-PCIe devices. Two key issues are: - The IOMMU MemoryRegion, used with PCIe device, cannot be used with a platform device, which is the primary mechanism for routing DMA traffic through the IOMMU. - Internally, the SMMU code makes assumptions about its clients. For instance, the smmu_get_sid() function explicitly expects a PCIe device and has no path to acquire a StreamID for a platform device. Given this, the decision to model smmu-testdev as a minimal, PCI-like device is a pragmatic one. It aligns with the most common real-world use case while also working within the constraints of QEMU's current SMMU implementation. >> The answer lies in our goal to minimize complexity. Standard devices, >> whether PCIe or platform, come with their own intricate initialization >> protocols and often require a complex driver state machine to function. >> Using them would re-introduce the very driver-level complexity we aim to >> avoid. >> >> The smmu-testdev is intentionally not a conformant, general-purpose PCIe >> or platform device. It is a purpose-built, highly simplified "DMA engine." >> I've designed it to be analogous to a minimal PCIe Root Complex that >> bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) >> to provide a direct, programmable path for a DMA request to reach the SMMU. >> Its sole purpose is to trigger a DMA transaction when its registers are >> written to, making it perfectly suited for direct control from a test >> environment like qtest. >> >> The Qtest Framework >> ------------------- >> >> The new qtest (smmu-testdev-qtest.c) serves as the "bare-metal driver" >> for both the SMMU and the smmu-testdev. It manually performs all the >> setup that would typically be handled by the guest kernel and firmware, >> but in a completely controlled and predictable manner: >> >> 1. SMMU Configuration: It directly initializes the SMMU's registers to a >> known state. >> >> 2. Translation Structure Setup: It manually constructs the necessary >> translation structures in memory, including Stream Table Entries >> (STEs), Context Descriptors (CDs), and Page Tables (PTEs). >> >> 3. DMA Trigger: It programs the smmu-testdev to initiate a DMA operation >> targeting a specific IOVA. >> >> 4. Verification: It waits for the transaction to complete and verifies >> that the memory was accessed correctly after address translation by >> the SMMU. >> >> This framework provides a solid and extensible foundation for validating >> the SMMU's core translation paths. The initial test included in this >> series covers a basic DMA completion path in the Non-Secure bank, >> serving as a smoke test and a proof of concept. >> >> It is worth noting that this series currently only includes tests for the >> Non-Secure SMMU. I am aware of the ongoing discussions and RFC patches >> for Secure SMMU support. To avoid a dependency on unmerged work, this >> submission does not include tests for the Secure world. However, I have >> already implemented these tests locally, and I am prepared to submit >> them for review as soon as the core Secure SMMU support is merged >> upstream. > What about other IOMMU's? Are there any other bus mediating devices > modelled in QEMU that could also benefit from the ability to trigger DMA > transactions? This is a great point that I haven't fully considered. To make sure I understand correctly, are you referring to IOMMU implementations for other architectures, such as VT-d on x86 or the ongoing IOMMU work for RISC-V? I'll admit this is an area I haven't looked into. I'm very open to ideas—do you or others have suggestions on how this test-device pattern could be generalized or what would be needed to make it useful across different architectures? Thanks again for the great feedback. Best regards, Tao ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-10-27 13:32 ` Tao Tang @ 2025-10-27 14:25 ` Alex Bennée 0 siblings, 0 replies; 16+ messages in thread From: Alex Bennée @ 2025-10-27 14:25 UTC (permalink / raw) To: Tao Tang Cc: Eric Auger, Peter Maydell, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Tao Tang <tangtao1634@phytium.com.cn> writes: > Hi Alex, > > On 2025/10/23 18:06, Alex Bennée wrote: >> tangtao1634 <tangtao1634@phytium.com.cn> writes: >> >>> From: Tao Tang <tangtao1634@phytium.com.cn> >>> >>> This patch series (V2) introduces several cleanups and improvements >>> to the smmu-testdev device. The main goals are to refactor shared >>> code, enhance robustness, and significantly clarify the complex >>> page table construction used for testing. >>> >>> Motivation >>> ---------- >>> >>> Currently, thoroughly testing the SMMUv3 emulation requires a significant >>> software stack. We need to boot a full guest operating system (like Linux) >>> with the appropriate drivers (e.g., IOMMUFD) and rely on firmware (e.g., >>> ACPI with IORT tables or Hafnium) to correctly configure the SMMU and >>> orchestrate DMA from a peripheral device. >>> >>> This dependency on a complex software stack presents several challenges: >>> >>> * High Barrier to Entry: Writing targeted tests for specific SMMU >>> features (like fault handling, specific translation regimes, etc.) >>> becomes cumbersome. >>> >>> * Difficult to Debug: It's hard to distinguish whether a bug originates >>> from the SMMU emulation itself, the guest driver, the firmware >>> tables, or the guest kernel's configuration. >>> >>> * Slow Iteration: The need to boot a full guest OS slows down the >>> development and testing cycle. >>> >>> The primary goal of this work is to create a lightweight, self-contained >>> testing environment that allows us to exercise the SMMU's core logic >>> directly at the qtest level, removing the need for any guest-side >>> software. >> I agree, an excellent motivation. >> >>> Our Approach: A Dedicated Test Device >>> ------------------------------------- >>> >>> To achieve this, we introduce two main components: >>> >>> * A new, minimal hardware device: smmu-testdev. >>> * A corresponding qtest that drives this device to generate SMMU-bound >>> traffic. >>> >>> A key question is, "Why introduce a new smmu-testdev instead of using an >>> existing PCIe or platform device?" >> I curious what the split between PCIe and platform devices that need an >> SMMU are. I suspect there is a strong split between the virtualisation >> case and the emulation case. > > > Thanks again for the insightful questions and for sparking this > valuable discussion. > > > From my observation of real-world, commercially available SoCs, the > SMMU is almost exclusively designed for and used with PCIe. Of course, > you're right that architecturally, the SMMU specification certainly > allows for non-PCIe clients. Peter's point about using the SMMU for > the GIC ITS in a Realm context is an excellent example. I've also > personally seen similar setups in the TF-A-Tests+FVP software stack, > where platform device, named SMMUv3TestEngine, was used to test the > SMMU. > > However, from the perspective of QEMU's current implementation, there > are some significant limitations that guided the design of > smmu-testdev. At the moment, the SMMU model does not really support > non-PCIe devices. Two key issues are: > > - The IOMMU MemoryRegion, used with PCIe device, cannot be used with a > platform device, which is the primary mechanism for routing DMA > traffic through the IOMMU. > > - Internally, the SMMU code makes assumptions about its clients. For > instance, the smmu_get_sid() function explicitly expects a PCIe > device and has no path to acquire a StreamID for a platform device. > > Given this, the decision to model smmu-testdev as a minimal, PCI-like > device is a pragmatic one. It aligns with the most common real-world > use case while also working within the constraints of QEMU's current > SMMU implementation. OK that makes sense. One of the main use cases for a modelled SMMU in QEMU is for developing and testing FEAT_RME (Arm's Confidential Computing Realm implementation). In that case everything I've seen so far expects PCI. I guess we can put off any generalisation until we actually have some use cases that might need it. >>> The answer lies in our goal to minimize complexity. Standard devices, >>> whether PCIe or platform, come with their own intricate initialization >>> protocols and often require a complex driver state machine to function. >>> Using them would re-introduce the very driver-level complexity we aim to >>> avoid. >>> >>> The smmu-testdev is intentionally not a conformant, general-purpose PCIe >>> or platform device. It is a purpose-built, highly simplified "DMA engine." >>> I've designed it to be analogous to a minimal PCIe Root Complex that >>> bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) >>> to provide a direct, programmable path for a DMA request to reach the SMMU. >>> Its sole purpose is to trigger a DMA transaction when its registers are >>> written to, making it perfectly suited for direct control from a test >>> environment like qtest. >>> >>> The Qtest Framework >>> ------------------- >>> >>> The new qtest (smmu-testdev-qtest.c) serves as the "bare-metal driver" >>> for both the SMMU and the smmu-testdev. It manually performs all the >>> setup that would typically be handled by the guest kernel and firmware, >>> but in a completely controlled and predictable manner: >>> >>> 1. SMMU Configuration: It directly initializes the SMMU's registers to a >>> known state. >>> >>> 2. Translation Structure Setup: It manually constructs the necessary >>> translation structures in memory, including Stream Table Entries >>> (STEs), Context Descriptors (CDs), and Page Tables (PTEs). >>> >>> 3. DMA Trigger: It programs the smmu-testdev to initiate a DMA operation >>> targeting a specific IOVA. >>> >>> 4. Verification: It waits for the transaction to complete and verifies >>> that the memory was accessed correctly after address translation by >>> the SMMU. >>> >>> This framework provides a solid and extensible foundation for validating >>> the SMMU's core translation paths. The initial test included in this >>> series covers a basic DMA completion path in the Non-Secure bank, >>> serving as a smoke test and a proof of concept. >>> >>> It is worth noting that this series currently only includes tests for the >>> Non-Secure SMMU. I am aware of the ongoing discussions and RFC patches >>> for Secure SMMU support. To avoid a dependency on unmerged work, this >>> submission does not include tests for the Secure world. However, I have >>> already implemented these tests locally, and I am prepared to submit >>> them for review as soon as the core Secure SMMU support is merged >>> upstream. >> What about other IOMMU's? Are there any other bus mediating devices >> modelled in QEMU that could also benefit from the ability to trigger DMA >> transactions? > > > This is a great point that I haven't fully considered. To make sure I > understand correctly, are you referring to IOMMU implementations for > other architectures, such as VT-d on x86 or the ongoing IOMMU work for > RISC-V? Yes - generally I think having a single test device that can be used to test multiple models will be useful. I guess each qtest will be very tied to the SMMU it is modelling as it needs to program both sides but if we take care to encapsulate the programming of the test device and verification of the results we should be able to ensure good code re-use. > I'll admit this is an area I haven't looked into. I'm very > open to ideas—do you or others have suggestions on how this > test-device pattern could be generalized or what would be needed to > make it useful across different architectures? My only initial thought is the device might be better called iommu-testdev (as in a device to test IOMMUs, of which the SMMU is one). > > Thanks again for the great feedback. > > Best regards, > > Tao -- Alex Bennée Virtualisation Tech Lead @ Linaro ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 ` (3 preceding siblings ...) 2025-10-23 10:06 ` Alex Bennée @ 2025-10-27 13:58 ` Peter Maydell 2025-10-28 14:05 ` Tao Tang 4 siblings, 1 reply; 16+ messages in thread From: Peter Maydell @ 2025-10-27 13:58 UTC (permalink / raw) To: tangtao1634 Cc: pbonzini, farosas, lvivier, Eric Auger, qemu-devel, qemu-arm, Chen Baozi, Pierrick Bouvier, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh On Tue, 30 Sept 2025 at 17:53, tangtao1634 <tangtao1634@phytium.com.cn> wrote: > The smmu-testdev is intentionally not a conformant, general-purpose PCIe > or platform device. It is a purpose-built, highly simplified "DMA engine." > I've designed it to be analogous to a minimal PCIe Root Complex that > bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) > to provide a direct, programmable path for a DMA request to reach the SMMU. > Its sole purpose is to trigger a DMA transaction when its registers are > written to, making it perfectly suited for direct control from a test > environment like qtest. This makes sense to me. But looking at the code it looks like the device itself has a lot of code for setting up IOMMU page tables in guest memory when the test code writes to its registers. That surprised me, as I was expecting the test device to essentially be "do DMA on command". Is there a reason why we can't have the test code do the setting up of the IOMMU page tables itself using the qtest functions for writing guest memory? (Obviously you'd abstract this out into functions for the purpose in libqos/ somewhere.) If we did it that way, we could use the same test device as part of non-SMMUv3 iommu emulation tests too -- the qtest test case code would just set up the different IOMMU in the way that IOMMU expects before triggering DMA. thanks -- PMM ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework 2025-10-27 13:58 ` Peter Maydell @ 2025-10-28 14:05 ` Tao Tang 0 siblings, 0 replies; 16+ messages in thread From: Tao Tang @ 2025-10-28 14:05 UTC (permalink / raw) To: Peter Maydell, Alex Bennée, Pierrick Bouvier Cc: Eric Auger, qemu-devel, qemu-arm, Chen Baozi, Philippe Mathieu-Daudé, Jean-Philippe Brucker, Mostafa Saleh, Paolo Bonzini, Fabiano Rosas, Laurent Vivier Hi Alex, Peter, Pierrick: Thank you all again for the outstanding feedback. As Peter said in mail [1](https://lore.kernel.org/qemu-devel/CAFEAcA92dTDn+Zf-GZVv9zQ3_mwJHZY5hrkdgrRyE7XUio4Sjw@mail.gmail.com/): On 2025/10/27 21:58, Peter Maydell wrote: > On Tue, 30 Sept 2025 at 17:53, tangtao1634 <tangtao1634@phytium.com.cn> wrote: >> The smmu-testdev is intentionally not a conformant, general-purpose PCIe >> or platform device. It is a purpose-built, highly simplified "DMA engine." >> I've designed it to be analogous to a minimal PCIe Root Complex that >> bypasses the full, realistic topology (Host Bridges, Switches, Endpoints) >> to provide a direct, programmable path for a DMA request to reach the SMMU. >> Its sole purpose is to trigger a DMA transaction when its registers are >> written to, making it perfectly suited for direct control from a test >> environment like qtest. > This makes sense to me. But looking at the code it looks like the > device itself has a lot of code for setting up IOMMU page tables in > guest memory when the test code writes to its registers. That > surprised me, as I was expecting the test device to essentially > be "do DMA on command". Is there a reason why we can't have the > test code do the setting up of the IOMMU page tables itself > using the qtest functions for writing guest memory? (Obviously > you'd abstract this out into functions for the purpose in > libqos/ somewhere.) > > If we did it that way, we could use the same test device as > part of non-SMMUv3 iommu emulation tests too -- the qtest > test case code would just set up the different IOMMU in > the way that IOMMU expects before triggering DMA. > > thanks > -- PMM And Alex's guidance in another mail [2](https://lore.kernel.org/qemu-devel/87jz0gxw01.fsf@draig.linaro.org/): > Yes - generally I think having a single test device that can be used to > test multiple models will be useful. I guess each qtest will be very > tied to the SMMU it is modelling as it needs to program both sides but > if we take care to encapsulate the programming of the test device and > verification of the results we should be able to ensure good code > re-use. > >> I'll admit this is an area I haven't looked into. I'm very >> open to ideas—do you or others have suggestions on how this >> test-device pattern could be generalized or what would be needed to >> make it useful across different architectures? > My only initial thought is the device might be better called > iommu-testdev (as in a device to test IOMMUs, of which the SMMU is one). > You nailed the core problem. I hadn't properly thought through the separation of concerns, leading to a device that was doing work it shouldn't. As Alex pointed out in [2], this architectural refactoring allows us to build a generic iommu-testdev to ensure good code re-use. This elevates the work from a single-purpose tool into a framework that can benefit all IOMMU implementations, which I understand is a far more valuable contribution. Furthermore, Pierrick's comment in mail [3](https://lore.kernel.org/qemu-devel/792a06cd-302c-46a5-997c-026cb67f8f2e@linaro.org/): > We have to start somewhere, so something simple and not trying to > solve all use cases is the right approach. It can even just be > read/write config/registers before trying to add any DMA scenario. Also as Alex said in mail [4](https://lore.kernel.org/qemu-devel/87ecqoxohg.fsf@draig.linaro.org/): > We should be thinking of targeted unit tests. The difference between > this and a full OS is we don't need to manage multiple shifting memory > maps over time. Setup a page (or two) with the permissions you expect > and check that works. The goal is to avoid "accidentally rewriting a driver." Instead, we should start simple and provide "targeted unit tests." The idea of setting up a simple, static state ("a page or two") to verify atomic features and edge cases that are hard to trigger in a dynamic OS is exactly the right philosophy for this framework. Based on this, my plan for V3 maybe now much clearer: - Refactor the device: It will be renamed iommu-testdev and become a "dumb," generic DMA engine. All architecture-specific logic, including the construction of page tables and other structures, will be moved into the qtest. - Abstract for reuse: Following Peter's and Alex's advice, the table-building logic will be abstracted into reusable helper functions within the libqos/ library. - Limit the initial scope: As you all suggested, the first set of tests will be simple unit tests, focusing on the core paths like different security states and translation stages One final question to manage the scope of this large refactoring: my plan is to implement the generic iommu-testdev framework in V3, but provide only the SMMUv3-specific qtest helpers and tests for now. We can leave the implementation for other architectures (like VT-d) to future work. Does this seem like a reasonable approach? Thanks again for helping to shape this work. Best regards, Tao [1] (https://lore.kernel.org/qemu-devel/CAFEAcA92dTDn+Zf-GZVv9zQ3_mwJHZY5hrkdgrRyE7XUio4Sjw@mail.gmail.com/) [2] (https://lore.kernel.org/qemu-devel/87jz0gxw01.fsf@draig.linaro.org/) [3] (https://lore.kernel.org/qemu-devel/792a06cd-302c-46a5-997c-026cb67f8f2e@linaro.org/) [4] (https://lore.kernel.org/qemu-devel/87ecqoxohg.fsf@draig.linaro.org/) ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2025-10-28 14:07 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-09-30 16:53 [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework tangtao1634 2025-09-30 16:53 ` [RFC v2 1/2] hw/misc/smmu-testdev: introduce minimal SMMUv3 test device tangtao1634 2025-10-23 10:31 ` Alex Bennée 2025-10-27 14:54 ` Tao Tang 2025-09-30 16:53 ` [RFC v2 2/2] tests/qtest: add SMMUv3 smoke test using smmu-testdev DMA source tangtao1634 2025-10-23 11:02 ` Alex Bennée 2025-10-27 15:26 ` Tao Tang 2025-10-27 15:52 ` Pierrick Bouvier 2025-10-27 17:07 ` Alex Bennée 2025-10-23 9:04 ` [RFC v2 0/2] hw/misc: Introduce a new SMMUv3 test framework Tao Tang 2025-10-23 10:06 ` Alex Bennée 2025-10-23 10:11 ` Peter Maydell 2025-10-27 13:32 ` Tao Tang 2025-10-27 14:25 ` Alex Bennée 2025-10-27 13:58 ` Peter Maydell 2025-10-28 14:05 ` Tao Tang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).