[PATCH v9 00/16] KVM/ARM Implementation

[Christoffer Dall <c.dall@virtualopensystems.com>]

The following series implements KVM support for ARM processors,
specifically on the Cortex A-15 platform.  Work is done in
collaboration between Columbia University, Virtual Open Systems and
ARM/Linaro.

The patch series applies to kvm/next, specifically commit:
ae7a2a3fb6f8b784c2752863f4f1f20c656f76fb

This is Version 9 of the patch series, but the first two versions
were reviewed outside of the KVM mailing list. Changes can also be
pulled from:
 git://github.com/virtualopensystems/linux-kvm-arm.git kvm-a15-v9

A non-flattened edition of the patch series can be found at:
 git://github.com/virtualopensystems/linux-kvm-arm.git kvm-a15-v9-stage

The implementation is broken up into a logical set of patches, the first
five are preparatory patches:
 1. ARM: Add mem_type prot_pte accessor
 2. ARM: ARM_VIRT_EXT config option
 3. ARM: Section based HYP idmaps
 4. KVM: Move KVM_IRQ_LINE to arch-generic code
 5. KVM: Guard code with CONFIG_MMU_NOTIFIER (repost)

KVM guys, please consider pulling the KVM generic patches as early as
possible. Thanks.

The main implementation is broken up into separate patches, the first
containing a skeleton of files, makefile changes, the basic user space
interface and KVM architecture specific stubs.  Subsequent patches
implement parts of the system as listed:
 1.  Skeleton
 2.  Reset values for the Cortex-A15 type processor
 3.  Hypervisor initialization
 4.  Hypervisor module unloading
 5.  Memory virtualization setup (hyp mode mappings and 2nd stage)
 6.  Inject IRQs and FIQs from userspace
 7.  World-switch implementation and Hyp exception vectors
 8.  Emulation framework and CP15 emulation
 9.  Handle guest user memory aborts
 10. Handle guest MMIO aborts
 11. Support guest wait-for-interrupt instructions

Testing:
Limited testing, but have run GCC inside guest, which compiled a small
hello-world program, which was successfully run. For v9 both ARM/Thumb-2
kernels were tested as both host/guest and both a compiled-in version
and a kernel module version of KVM was tested. Hardware still
unavailable to me, so all testing has been done on ARM Fast Models.

For a guide on how to set up a testing environment and try out these
patches, see:
 http://www.virtualopensystems.com/media/pdf/kvm-arm-guide.pdf

There is an issue list available using the issue tracker on:
https://github.com/virtualopensystems/linux-kvm-arm

Additionally a few major milestones are coming up shortly:
 - Support Thumb MMIO emulation and test MMIO emulation code (under way)
 - Merge Marc Zyngier's patch series for VGIC and timers (review in
   progress)
 - Change from SMC based install to relying on booting the kernel in Hyp
   mode. This requires some larger changes, but will allow a guest
   kernel to boot with KVM configured.

Changes since v8:
 - Support cache maintenance on SMP through set/way
 - Hyp mode idmaps are now section based and happen at kernel init
 - Handle aborts in Hyp mode
 - Inject undefined exceptions into the guest on error
 - Kernel-side reset of all crucial registers
 - Specifically state which target CPU is being virtualized
 - Exit statistics in debugfs
 - Some L2CTLR cp15 emulation cleanups
 - Support spte_hva for MMU notifiers and take write faults
 - FIX: Race condition in VMID generation
 - BUG: Run exit handling code with disabled preemption
 - Save/Restore abort fault register during world switch

Changes since v7:
 - Traps accesses to ACTLR
 - Do not trap WFE execution
 - Upgrade barriers and TLB operations to inner-shareable domain
 - Restrucure hyp_pgd related code to be more opaque
 - Random SMP fixes
 - Random BUG fixes
 - Improve commenting
 - Support module loading/unloading of KVM/ARM
 - Thumb-2 support for host kernel and KVM
 - Unaligned cross-page wide guest Thumb instruction fetching
 - Support ITSTATE fields in CPSR for Thumb guests
 - Document HCR settings

Changes since v6:
 - Support for MMU notifiers to not pin user pages in memory
 - Suport build with log debugging
 - Bugfix: v6 clobbered r7 in init code
 - Simplify hyp code mapping
 - Cleanup of register access code
 - Table-based CP15 emulation from Rusty Russell
 - Various other bug fixes and cleanups

Changes since v5:
 - General bugfixes and nit fixes from reviews
 - Implemented re-use of VMIDs
 - Cleaned up the Hyp-mapping code to be readable by non-mm hackers
   (including myself)
 - Integrated preliminary SMP support in base patches
 - Lock-less interrupt injection and WFI support
 - Fixed signal-handling in while in guest (increases overall stability)

Changes since v4:
 - Addressed reviewer comments from v4
    * cleanup debug and trace code
    * remove printks
    * fixup kvm_arch_vcpu_ioctl_run
    * add trace details to mmio emulation
 - Fix from Marc Zyngier: Move kvm_guest_enter/exit into non-preemptible
   section (squashed into world-switch patch)
 - Cleanup create_hyp_mappings/remove_hyp_mappings from Marc Zyngier
   (squashed into hypervisor initialization patch)
 - Removed the remove_hyp_mappings feature. Removing hypervisor mappings
   could potentially unmap other important data shared in the same page.
 - Removed the arm_ prefix from the arch-specific files.
 - Initial SMP host/guest support

Changes since v3:
 - v4 actually works, fully boots a guest
 - Support compiling as a module
 - Use static inlines instead of macros for vcpu_reg and friends
 - Optimize kvm_vcpu_reg function
 - Use Ftrace for trace capabilities
 - Updated documentation and commenting
 - Use KVM_IRQ_LINE instead of KVM_INTERRUPT
 - Emulates load/store instructions not supported through HSR
  syndrome information.
 - Frees 2nd stage translation tables on VM teardown
 - Handles IRQ/FIQ instructions
 - Handles more CP15 accesses
 - Support guest WFI calls
 - Uses debugfs instead of /proc
 - Support compiling in Thumb mode

Changes since v2:
 - Performs world-switch code
 - Maps guest memory using 2nd stage translation
 - Emulates co-processor 15 instructions
 - Forwards I/O faults to QEMU.

---

Christoffer Dall (14):
      ARM: Add config option ARM_VIRT_EXT
      ARM: Section based HYP idmap
      KVM: Move KVM_IRQ_LINE to arch-generic code
      ARM: KVM: Initial skeleton to compile KVM support
      ARM: KVM: Support Cortex-A15 VCPUs reset
      ARM: KVM: Hypervisor inititalization
      ARM: KVM: Module unloading support
      ARM: KVM: Memory virtualization setup
      ARM: KVM: Inject IRQs and FIQs from userspace
      ARM: KVM: World-switch implementation
      ARM: KVM: Emulation framework and CP15 emulation
      ARM: KVM: Handle guest faults in KVM
      ARM: KVM: Handle I/O aborts
      ARM: KVM: Guest wait-for-interrupts (WFI) support

Marc Zyngier (2):
      ARM: add mem_type prot_pte accessor
      KVM: Guard mmu_notifier specific code with CONFIG_MMU_NOTIFIER


 Documentation/virtual/kvm/api.txt           |   12 
 arch/arm/Kconfig                            |    2 
 arch/arm/Makefile                           |    1 
 arch/arm/include/asm/idmap.h                |    7 
 arch/arm/include/asm/kvm.h                  |   89 +++
 arch/arm/include/asm/kvm_arm.h              |  184 ++++++
 arch/arm/include/asm/kvm_asm.h              |   58 ++
 arch/arm/include/asm/kvm_emulate.h          |  120 ++++
 arch/arm/include/asm/kvm_host.h             |  159 +++++
 arch/arm/include/asm/kvm_mmu.h              |   41 +
 arch/arm/include/asm/mach/map.h             |    1 
 arch/arm/include/asm/pgtable-3level-hwdef.h |    5 
 arch/arm/include/asm/pgtable-3level.h       |   13 
 arch/arm/include/asm/pgtable.h              |    5 
 arch/arm/include/asm/unified.h              |   12 
 arch/arm/kernel/armksyms.c                  |    7 
 arch/arm/kernel/asm-offsets.c               |   43 +
 arch/arm/kernel/entry-armv.S                |    1 
 arch/arm/kernel/vmlinux.lds.S               |    6 
 arch/arm/kvm/Kconfig                        |   45 +
 arch/arm/kvm/Makefile                       |   17 +
 arch/arm/kvm/arm.c                          |  881 +++++++++++++++++++++++++++
 arch/arm/kvm/emulate.c                      |  873 +++++++++++++++++++++++++++
 arch/arm/kvm/exports.c                      |   38 +
 arch/arm/kvm/guest.c                        |  165 +++++
 arch/arm/kvm/init.S                         |  149 +++++
 arch/arm/kvm/interrupts.S                   |  698 +++++++++++++++++++++
 arch/arm/kvm/mmu.c                          |  644 ++++++++++++++++++++
 arch/arm/kvm/reset.c                        |  133 ++++
 arch/arm/kvm/trace.h                        |  117 ++++
 arch/arm/mm/Kconfig                         |   10 
 arch/arm/mm/idmap.c                         |   88 ++-
 arch/arm/mm/mmu.c                           |    9 
 arch/ia64/kvm/kvm-ia64.c                    |   33 -
 arch/x86/kvm/x86.c                          |   33 -
 include/linux/kvm.h                         |    1 
 include/linux/kvm_host.h                    |    5 
 mm/memory.c                                 |    2 
 virt/kvm/kvm_main.c                         |   19 +
 39 files changed, 4661 insertions(+), 65 deletions(-)
 create mode 100644 arch/arm/include/asm/kvm.h
 create mode 100644 arch/arm/include/asm/kvm_arm.h
 create mode 100644 arch/arm/include/asm/kvm_asm.h
 create mode 100644 arch/arm/include/asm/kvm_emulate.h
 create mode 100644 arch/arm/include/asm/kvm_host.h
 create mode 100644 arch/arm/include/asm/kvm_mmu.h
 create mode 100644 arch/arm/kvm/Kconfig
 create mode 100644 arch/arm/kvm/Makefile
 create mode 100644 arch/arm/kvm/arm.c
 create mode 100644 arch/arm/kvm/emulate.c
 create mode 100644 arch/arm/kvm/exports.c
 create mode 100644 arch/arm/kvm/guest.c
 create mode 100644 arch/arm/kvm/init.S
 create mode 100644 arch/arm/kvm/interrupts.S
 create mode 100644 arch/arm/kvm/mmu.c
 create mode 100644 arch/arm/kvm/reset.c
 create mode 100644 arch/arm/kvm/trace.h

--