[PATCH v2 00/39] [RISC-V] virtual interrupt controller (vAPLIC/vIMSIC) support

[Oleksii Kurochko <oleksii.kurochko@gmail.com>]

Hi all,

This series adds the initial virtual interrupt controller (vINTC) support
for RISC-V guests in Xen, based on the Advanced Interrupt Architecture
(AIA): a virtual APLIC (vAPLIC) in MSI mode backed by a virtual IMSIC
(vIMSIC) using hardware guest interrupt files.

Rather than emulating APLIC in direct-delivery mode (which requires
trap-and-emulate for every interrupt and is costly), the series targets
IMSIC from the start. AIA lets a hart implement several "guest interrupt
files" (up to GEILEN), so external interrupts can be delivered to a vCPU
directly by hardware via the VGEIN field of hstatus, without a hypervisor
round-trip. Xen only has to emulate the APLIC MMIO programming interface
and route the guest's intent onto the physical MSI topology; interrupt
delivery itself stays in hardware.

The work breaks down into a few logical blocks:

Preparatory fixes and cleanups (patches 1-6)
  - Drop the unused pregs field of struct cpu_user_regs and bug.h's
    duplicate instruction length helpers.
  - Program hstatus.VSXL explicitly, as decoding a trapped instruction
    depends on the effective XLEN of the guest.
  - csr_read64() as the counterpart of csr_write64(), used for CSR_TIME so
    that get_cycles() no longer truncates the time counter on RV32, and
    UINT64_MAX rather than ULONG_MAX to disable the VS-timer.
  - Request a G-stage flush on vmenter where VMIDs are unavailable, so a
    domain cannot run on the translations left behind by the one which ran
    on that hart before it.

APLIC groundwork and vAPLIC MMIO emulation (patches 7-11)
  - Add the missing APLIC register offsets/masks needed by both the
    physical and virtual APLIC code, rearranging asm/aplic.h in the style
    of x86's asm/msr-index.h (no functional change).
  - A per-domain MMIO handler table modelled on Arm's framework, so
    emulated devices self-register their GPA ranges and the fault path
    stays agnostic via a single try_handle_mmio() entry point.
  - vAPLIC MMIO read/write emulation. Writes are gated by the domain's
    authorised-IRQ bitmap so a guest cannot touch interrupts it does not
    own, and TARGET writes are translated from virtual to physical
    hart/guest-file indices. Delegation (SOURCECFG.D) is not yet
    supported.
  - Build the physical APLIC's target hart index with aplic_hart_field()
    as well, dropping the last in-tree duplicate of the AIA hart index
    formula, and add a helper to test for APLIC MSI mode.

vCPU context switching (patches 12-15)
  - context_switch() and the helpers it needs: save/restore of H/VS CSRs,
    the virtual timer and P2M context, and __context_switch() in assembly.
    The VMID is claimed in p2m_ctxt_switch_to() rather than at the next
    guest entry, which leaves p2m_handle_vmenter() with nothing to do.
  - Save and restore the AIA CSRs a guest can change (vsiselect and
    hviprio{1,2}), gated by hstateen0 where Smstateen is implemented.
  - vintc_ctxt_switch_{from,to}() wrappers over new ctxt_switch_{from,to}
    hooks in struct vintc_ops, called from the context switch path, plus
    the IMSIC implementation of those hooks: it records which pCPU owns a
    vCPU's guest interrupt file, as the pCPU id is part of the MSI
    address.

Trap and instruction emulation infrastructure (patches 16-26)
  - Extend the exception-table format with type/data fields and add
    EX_TYPE_TRAP_INFO so fixups can capture sepc/scause/stval, and look
    the table up for any trap taken in Xen context rather than for illegal
    instructions only, so that the hlv/hlvx sequences reach their fixup.
  - A guest page-fault handler, and trap_redirect() to forward a
    synchronous trap back into the guest's VS-mode handler for the faults
    which can never become an emulated access.
  - Resolve the faulting guest physical address from htval and stval,
    which first needs Shtvala to be detected, and define all four
    INSN_PSEUDO_VS_* values independently of the hypervisor's XLEN, so
    that a fault taken on an implicit VS-stage access is recognized as one
    rather than mistaken for an MMIO trap.
  - riscv_read_guest() (HLV/HLVX) to read guest memory and instructions
    safely, the decoding helpers shared by both access types, and the load
    and store emulation which dispatches the access through
    try_handle_mmio().

vCPU migration between pCPUs (patches 27-34) (introduced here for better context
of VGEIN fumctions usage)
  - arch_move_irqs(), dispatching through a new move_irqs hook in
    struct vintc_ops down to imsic_migrate_vcpu(), and the case where a
    vCPU has no guest interrupt file to move yet.
  - The move of a vCPU's IMSIC guest interrupt file itself, following the
    sequence the AIA spec prescribes: quiesce and save eidelivery/
    eithreshold of the old file, zero the new one, G-stage remap it,
    retarget the domain's APLIC interrupts at it
    (aplic_reconfigure_target()) and fence off straggler MSIs with a
    genmsi barrier, dump the old file's eip/eie arrays to memory, then
    restore that state into the new file and update the vCPU's
    hstatus.VGEIN.

VGEIN allocation and vCPU bring-up (patches 35-39)
  - Per-pCPU VGEIN (guest interrupt file) allocator: a bitmap of the files
    a hart implements (up to GEILEN) with helpers to assign and release
    one, and an owners[] map so a file reported pending in HGEIP can be
    traced back to the vCPU it belongs to.
  - Watch a descheduled vCPU's guest interrupt file through HGEIE, so that
    a guest blocked on an external interrupt is woken up instead of
    waiting for an unrelated event to schedule it again.
  - Stage-2 map a vCPU's physical guest interrupt file to the fixed
    per-vCPU GPA page the guest expects at offset 0.
  - continue_new_vcpu(): switch to the idle vCPU's own stack for the idle
    vCPU, and enter the guest through the new return_to_new_vcpu() path in
    entry.S for a guest one.
  - imsic_vsfile_attach(), called once the pCPU a vCPU will run on is
    known: it assigns a VGEIN, maps the guest interrupt file and records
    the IMSIC state as a consistent unit.

CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2796779115

The series depends on [1].

[1] https://lore.kernel.org/xen-devel/cover.1787836900.git.oleksii.kurochko@gmail.com/T/#t

---
Changes in v2:
 - The series has grown from 17 to 39 patches. vCPU context switching, vCPU
   migration between pCPUs and the vCPU bring-up path (continue_new_vcpu(),
   attaching an IMSIC h/w interrupt file) are now part of it to have better
   context of how things are using, together with the trap-side pieces the
   MMIO emulation depends on (faulting GPA resolution, Shtvala detection,
   instruction decoding).
 - vintc_state_{save,restore}() became vintc_ctxt_switch_{from,to}() and
   vcpu_aia_init() became imsic_vsfile_attach(); "xen/riscv: manage
   IRQ_DISABLED flag in APLIC irq enable/disable callbacks" is no longer part
   of this series. The remaining changes are described in the per-patch
   changelogs.
---

Oleksii Kurochko (39):
  xen/riscv: drop pregs from struct cpu_user_regs
  xen/riscv: drop bug.h's duplicate instruction length helpers
  xen/riscv: set the guest's XLEN explicitly in hstatus.VSXL
  xen/riscv: introduce csr_read64()
  xen/riscv: request a G-stage flush on vmenter when VMIDs are disabled
  xen/riscv: use UINT64_MAX to disable the VS-timer
  xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h
  xen/riscv: introduce device-agnostic MMIO emulation dispatch
  xen/riscv: implement virtual APLIC MMIO emulation
  xen/riscv: build the target hart index via aplic_hart_field()
  xen/riscv: add helper to check APLIC MSI mode
  xen/riscv: implement vCPU context switching
  xen/riscv: save and restore AIA state on vCPU context switch
  xen/riscv: introduce vintc_ctxt_switch_{from,to}()
  xen/riscv: add IMSIC vCPU context switch handlers
  xen/riscv: extend exception tables with type and data fields
  xen/riscv: decouple INSN_PSEUDO_VS_* from the hypervisor's XLEN
  xen/riscv: add guest page fault handling stub
  xen/riscv: implement trap redirection to a guest
  xen/riscv: detect Shtvala
  xen/riscv: resolve the faulting guest physical address
  xen/riscv: add guest memory read helper
  xen/riscv: look up the exception table for any trap taken in Xen
    context
  xen/riscv: add helpers for decoding a trapped load or store
  xen/riscv: add guest load emulation for trapped MMIO accesses
  xen/riscv: add guest store emulation for trapped MMIO accesses
  xen/riscv: introduce arch_move_irqs()
  xen/riscv: handle the case when no vCPU migration is needed
  xen/riscv: introduce aplic_reconfigure_target()
  xen/riscv: prepare new IMSIC VS-file
  xen/riscv: implement APLIC-hart sync barrier for vCPU migration
  xen/riscv: remap interrupts to new IMSIC VS-file
  xen/riscv: dump old interrupt file to memory
  xen/riscv: restore register state in the new IMSIC VS-file
  xen/riscv: add basic VGEIN management for AIA guests
  xen/riscv: wake up a descheduled vCPU on a guest external interrupt
  xen/riscv: map IMSIC interrupt file for vCPUs
  xen/riscv: implement continue_new_vcpu()
  xen/riscv: introduce IMSIC h/w interrupt file attaching to vcpu

 xen/arch/riscv/Makefile                     |   2 +
 xen/arch/riscv/aia.c                        | 193 ++++++
 xen/arch/riscv/aplic-priv.h                 |   3 +
 xen/arch/riscv/aplic.c                      | 235 ++++++-
 xen/arch/riscv/cpufeature.c                 |   1 +
 xen/arch/riscv/domain.c                     | 271 +++++++-
 xen/arch/riscv/emulate.c                    | 578 ++++++++++++++++
 xen/arch/riscv/entry.S                      |  67 ++
 xen/arch/riscv/extable.c                    |  70 +-
 xen/arch/riscv/guestcopy.c                  |  87 +++
 xen/arch/riscv/imsic.c                      | 691 ++++++++++++++++++++
 xen/arch/riscv/include/asm/aia.h            |   7 +
 xen/arch/riscv/include/asm/aplic.h          | 130 +++-
 xen/arch/riscv/include/asm/bug.h            |  19 -
 xen/arch/riscv/include/asm/cpufeature.h     |   1 +
 xen/arch/riscv/include/asm/csr.h            |  24 +
 xen/arch/riscv/include/asm/current.h        |   4 +
 xen/arch/riscv/include/asm/domain.h         |  23 +-
 xen/arch/riscv/include/asm/emulate.h        |  10 +
 xen/arch/riscv/include/asm/extable.h        |  64 +-
 xen/arch/riscv/include/asm/gpr-num.h        |  37 ++
 xen/arch/riscv/include/asm/guest_access.h   |   4 +
 xen/arch/riscv/include/asm/imsic.h          |  31 +
 xen/arch/riscv/include/asm/intc.h           |  12 +
 xen/arch/riscv/include/asm/irq.h            |   5 +-
 xen/arch/riscv/include/asm/mmio.h           |  63 ++
 xen/arch/riscv/include/asm/p2m.h            |   1 -
 xen/arch/riscv/include/asm/processor.h      |  16 +-
 xen/arch/riscv/include/asm/riscv_encoding.h |  34 +-
 xen/arch/riscv/include/asm/system.h         |   4 +
 xen/arch/riscv/include/asm/time.h           |   4 +-
 xen/arch/riscv/include/asm/traps.h          |   9 +
 xen/arch/riscv/include/asm/vaplic.h         |   5 +
 xen/arch/riscv/intc.c                       |  22 +
 xen/arch/riscv/mmio.c                       | 176 +++++
 xen/arch/riscv/p2m.c                        |  55 +-
 xen/arch/riscv/riscv64/asm-offsets.c        |  20 +-
 xen/arch/riscv/stubs.c                      |   5 -
 xen/arch/riscv/time.c                       |   4 +-
 xen/arch/riscv/traps.c                      | 110 +++-
 xen/arch/riscv/vaplic.c                     | 358 +++++++++-
 xen/arch/riscv/vmid.c                       |   4 +-
 xen/include/xen/config.h                    |   1 +
 43 files changed, 3281 insertions(+), 179 deletions(-)
 create mode 100644 xen/arch/riscv/emulate.c
 create mode 100644 xen/arch/riscv/include/asm/emulate.h
 create mode 100644 xen/arch/riscv/include/asm/gpr-num.h
 create mode 100644 xen/arch/riscv/include/asm/mmio.h
 create mode 100644 xen/arch/riscv/mmio.c

-- 
2.55.0