Re: [RFC PATCH 00/14] KVM: ITS hardening for pKVM

Mostafa Saleh
2026-03-13T15:18:17Z

Hi Seb,

On Tue, Mar 10, 2026 at 12:49:19PM +0000, Sebastian Ene wrote:
> This series introduces the necessary machinery to perform trap & emulate
> on device access in pKVM. Furthermore, it hardens the GIC/ITS controller to
> prevent an attacker from tampering with the hypervisor protected memory
> through this device.
>
> In pKVM, the host kernel is initially trusted to manage the boot process but
> its permissions are revoked once KVM initializes. The GIC/ITS device is
> configured before the kernel deprivileges itself. Once the hypervisor
> becomes available, sanitize the accesses to the ITS controller by
> trapping and emulating certain registers and by shadowing some memory
> structures used by the ITS.
>
> This is required because the ITS can issue transactions on the memory
> bus *directly*, without having an SMMU in front of it, which makes it
> an interesting target for crossing the hypervisor-established privilege
> boundary.
>
>
> Patch overview
> ==============
>
> The first patch is re-used from Mostafa's series[1] which brings SMMU-v3
> support to pKVM.
>
> [1] https://lore.kernel.org/linux-iommu/20251117184815.1027271-1-smostafa@google.com/#r
>
> Some of the infrastructure built in that series might intersect and we
> agreed to converge on some changes. The patches [1 - 3] allow unmapping
> devices from the host address space and installing a handler to trap
> accesses from the host. While executing in the handler, enough context
> has to be given from mem-abort to perform the emulation of the device
> such as: the offset, the access size, direction of the write and private
> related data specific to the device.
> The unmapping of the device from the host address space is performed
> after the host deprivilege (during _kvm_host_prot_finalize call).
>
> The 4th patch looks up the ITS node from the device tree and adds it to
> an array of unmapped devices. It install a handler that forwards all the
> MMIO request to mediate the host access inside the emulation layer and
> to prevent breaking ITS functionality.
>
> The 5th patch changes the GIC/ITS driver to exposes two new methods
> which will be called from the KVM layer to setup the shadow state and
> to take the appropriate locks. This one is the most intrusive as it
> changes the current GIC/ITS driver. I tried to avoid creating a
> dependency with KVM to keep the GIC driver agnostic of the virtualization
> layer but I am happy to explore other options as well.
> To avoid re-programming the ITS device with new shadow structures after
> pKVM is ready, I exposed two functions to change the
> pointers inside the driver for the following structures:
> - the command queue points to a newly allocated queue
> - the GITS_BASER<n> tables configured with an indirect layout have the
> first layer shadowed and they point to a new memory region
>
> Patch 6 adds the entry point into the emulation setup and sets up the
> shadow command queue. It adds some helper macros to define the offset
> register and the associate action that we want to execute in the
> emulation. It also unmaps the state passed from the host kernel
> to prevent it from playing nasty games later on. The patch
> traps accesses to CWRITER register and copies the commands from the
> host command queue to the shadow command queue.
>
> Patch 7 prevents the host from directly accessing the first layer of the
> indirect tables held in GITS_BASER<n>. It also prevents the host from
> directly accesssing the last layer of the Device Table (since the entries
> in this table hold the address of the ITT table) and of the vPE Table
> (since the vPE table entries hold the address of the virtual LPI pending
> table.
>
> Patches [8-10] sanitize the commands sent to the ITS and their
> arguments.
>
> Patches [11-13] restrict the access of the host to certain registers
> and prevent undefined behaviour. Prevent the host from re-programming
> the tables held in the GITS_BASER register.
>
> The last patch introduces an hvc to setup the ITS emulation and calls
> into the ITS driver to setup the shadow state.
>
>
> Design
> ======
>
>
> 1. Command queue shadowing
>
> The ITS hardware supports a command queue which is programmed by the driver
> in the GITS_CBASER register. To inform the hardware that a new command
> has been added, the driver updates an index into the GITS_CWRITER
> register. The driver then reads the GITS_CREADR register to see if the
> command was processed or if the queue is stalled.
>
> To create a new command, the emulation layer mirrors the behavior
> as following:
> (i) The host ITS driver creates a command in the shadow queue:
> its_allocate_entry() -> builder()
> (ii) Notifies the hardware that a new command is available:
> its_post_commands()
> (iii) Hypervisor traps the write to GITS_CWRITER:
> handle_host_mem_abort() -> handle_host_mmio_trap() ->
> pkvm_handle_gic_emulation()
> (iv) Hypervisor copies the command from the host command queue
> to the original queue which is not accessible to the host.
> It parses the command and updates the hardware write.
>
> The driver allocates space for the original command queue and programs
> the hardware (GITS_CWRITER). When pKVM becomes available, the driver
> allocates a new (shadow) queue and replaces its original pointer to
> the queue with this new one. This is to prevent a malicious host from
> tampering with the commands sent to the ITS hardware.
>
> The entry point of our emulation shares the memory of the newly
> allocated queue with the hypervisor and donates the memory of the
> original queue to make it inaccesible to the host.
>
>
> 2. Indirect tables first level shadowing
>
> The ITS hardware supports indirection to minimize the space required to
> accommodate large tables (eg. deviceId space used to index the Device Table
> is quite sparse). This is a 2-level indirection, with entries from the
> first table pointing to a second table.
>
> An attacker in control of the host can insert an address that points to
> the hypervisor protected memory in the first level table and then use
> subsequent ITS commands to write to this memory (MAPD).
>
> To shadow this tables, we rely on the driver to allocate space for it
> and we copy the original content from the table into the copy. When
> pKVM becomes available we switch the pointers that hold the orginal
> tables to point to the copy.
> To keep the tables from the hypervisor in sync with what the host
> has, we update the tables when commands are sent to the ITS.
>
>
> 3. Hiding the last layer of the Device Table and vPE Table from the host
>
> An attacker in control of the host kernel can alter the content of these
> tables directly (the Arm IHI 0069H.b spec says that is undefined behavior
> if entries are created by software). Normally these entries are created in
> response of commands sent to the ITS.
>
> A Device Table entry that has the following structure:
>
> type DeviceTableEntry is (
> boolean Valid,
> Address ITT_base,
> bits(5) ITT_size
> )
>
> This can be maliciously created by an attacker and the ITT_base can be
> pointed to hypervisor protected memory. The MAPTI command can then be
> used to write over the ITT_base with an ITE entry.
>
> Similarly a vCPU Table entry has the following structure:
>
> type VCPUTableEntry is (
> boolean Valid,
> bits(32) RDbase,
> Address VPT_base,
> bits(5) VPT_size
> )
>
> VPT_base can be pointed to hypervisor protected memory and then a
> command can be used to raise interrupts and set the corresponding
> bit. This would give a 1-bit write primitive so is not "as generous"
> as the others.
>
>
> Notes
> =====
>
>
> Performance impact is expected with this as the emulation dance is not
> cost free.
> I haven't implemented any ITS quirks in the emulation and I don't know
> whether we will need it ? (some hardware needs explicit dcache flushing
> ITS_FLAGS_CMDQ_NEEDS_FLUSHING).
>
> Please note that Redistributors trapping hasn't been addressed at all in
> this series and the solution is not sufficient but this can be extended
> afterwards.
> The current series has been tested with Qemu (-machine
> virt,virtualization=true,gic-version=4) and with Pixel 10.
>
>
> Thanks,
> Sebastian E.
>
> Mostafa Saleh (1):
> KVM: arm64: Donate MMIO to the hypervisor
>
> Sebastian Ene (13):
> KVM: arm64: Track host-unmapped MMIO regions in a static array
> KVM: arm64: Support host MMIO trap handlers for unmapped devices
> KVM: arm64: Mediate host access to GIC/ITS MMIO via unmapping
> irqchip/gic-v3-its: Prepare shadow structures for KVM host deprivilege
> KVM: arm64: Add infrastructure for ITS emulation setup
> KVM: arm64: Restrict host access to the ITS tables
> KVM: arm64: Trap & emulate the ITS MAPD command
> KVM: arm64: Trap & emulate the ITS VMAPP command
> KVM: arm64: Trap & emulate the ITS MAPC command
> KVM: arm64: Restrict host updates to GITS_CTLR
> KVM: arm64: Restrict host updates to GITS_CBASER
> KVM: arm64 Restrict host updates to GITS_BASER
> KVM: arm64: Implement HVC interface for ITS emulation setup

I tested the patches on Lenovo ideacenter Mini X Gen 10 Snapdragon,
and the kernel hangs at boot for me with messags the following log:

[ 2.735838] ITS queue timeout (1056 1024)
[ 2.739969] ITS cmd its_build_mapd_cmd failed
[ 4.776344] ITS queue timeout (1120 1024)
[ 4.780472] ITS cmd its_build_mapti_cmd failed
[ 6.816677] ITS queue timeout (1184 1024)
[ 6.820806] ITS cmd its_build_mapti_cmd failed
[ 8.857009] ITS queue timeout (1248 1024)
[ 8.861129] ITS cmd its_build_mapti_cmd failed

I am happy to do more debugging, let me know if I can try anything.

Thanks,
Mostafa

>
> arch/arm64/include/asm/kvm_arm.h | 3 +
> arch/arm64/include/asm/kvm_asm.h | 1 +
> arch/arm64/include/asm/kvm_pkvm.h | 20 +
> arch/arm64/kvm/hyp/include/nvhe/its_emulate.h | 17 +
> arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 2 +
> arch/arm64/kvm/hyp/nvhe/Makefile | 3 +-
> arch/arm64/kvm/hyp/nvhe/hyp-main.c | 14 +
> arch/arm64/kvm/hyp/nvhe/its_emulate.c | 653 ++++++++++++++++++
> arch/arm64/kvm/hyp/nvhe/mem_protect.c | 134 ++++
> arch/arm64/kvm/hyp/nvhe/setup.c | 28 +
> arch/arm64/kvm/hyp/pgtable.c | 9 +-
> arch/arm64/kvm/pkvm.c | 60 ++
> drivers/irqchip/irq-gic-v3-its.c | 177 ++++-
> include/linux/irqchip/arm-gic-v3.h | 36 +
> 14 files changed, 1126 insertions(+), 31 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/include/nvhe/its_emulate.h
> create mode 100644 arch/arm64/kvm/hyp/nvhe/its_emulate.c
>
> --
> 2.53.0.473.g4a7958ca14-goog
>