Re: [Qemu-devel] [PATCH v1 0/6] Multi-Arch Phase 2

[Paolo Bonzini <pbonzini@redhat.com>]

On 26/10/2015 16:27, Peter Crosthwaite wrote:
> This is the second set of patches needed to enable Multi-arch system
> emulation. For full context refer to RFCv3:
> 
> [PATCH v3 00/35] Multi Architecture System Emulation
> https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg03929.html
> 
> Phase 1 was mostly merged:
> 
> [PATCH v1 00/15] Multi-Arch Phase 1
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03054.html
> 
> This pack contains no functional diff, but contains some of the higher
> diff refactorings that prepare MA support. It makes a start on the
> arch-specific changes needed touching Microblaze and ARM and giving an
> idea of what an Arches hw/<arch> and target-<arch> are going to look
> like after conversion.
> 
> Original cover, as well as overall series state below for further
> information.
> 
> Regards,
> Peter
> 
> Original Multi-arch arch patch series cover:
> 
> ***
> 
> This is target-multi, a system-mode build that can support multiple
> cpu-types.
> 
> Two architectures are initially converted. Microblaze and ARM. Step
> by step conversion in done for each. A microblaze is added to
> Xilinx Zynq platform as a test case. This will be elaborted more in
> future spins. This use case is valid, as Microblazes can be added (any
> number of them!) in Zynq FPGA programmable logic configuration.
> 
> The general approach (radically different to approach in V1 RFC) is to build
> and prelink an object (arch-obj.o) per-arch containing:
> 
> 1: target-foo/*
> 2: All uses of env internals and CPU_GET_ENV
>     * cputlb, translate-all, cpu-exec
>     * TCG backend
> 
> This means cputlb and friends are compiled multiple times fo each arch. The
> symbols for each of these pre-links are then localised to avoid link time name
> collisions. This is based on Paolo's suggestion to templatify cputlb and
> friends. Just the net of what to multi-compile is widened to include the TCG
> stuff as well now.
> 
> Despite being some "major surgery" this approach actually solves many of big
> the problems raised in V1. Big problems sovled:
> 
> 1: With the multi-compile TCG backends there are now multiple tcg_ctx's for
> each architecture. This solves the issue PMM raised WRT false positives on TB
> hashing as archs no longer share translation context.
> 
> 2: There is no longer a need to reorder the CPU_COMMON within the ENV or the ENV
> within the CPU. This was flagged as a performance issue by multiple people in
> V1.
> All users of the env internals as well as ENV_GET_CPU are now in multi-compile
> code and so multi-arch does not need to define a generic ENV nor does in need to
> def the problematic ENV_GET_CPU.
> 
> 3: With the prelink symbol localisation, link time namespace collision of
> helpers from multiple arches is no longer an issue. No need to bloat all the
> function names with arch specific prefixes.
> 
> 4: The architecture specifics used/defined by cpu-defs can now vary from arch to
> arch (incl. target_ulong) greatly reducing coversion effort needed. The list
> of restrictions for multi-arch capability is much reduced since V1. No
> target_long issues anymore.
> 
> include/exec/*.h and some of the common code needs some refactoring to setup
> this single vs multi compile split. Mostly code movements.
> 
> Some functions (like tcg_enabled) need to be listified for each of the
> now-multiple TCG engines.
> 
> The interface between the multi compile and single compiled files needs to be
> virtualised using QOM cpu functions. But this is now a very low footprint
> change as most of the virtualised hooks are now in mutli-compiled code (they
> only exist as text once). There are more new hooks than before, but the per
> target change pattern is reduced.
> 
> For the implementation of the series, the trickiest part is (still) cpu.h
> inclusion management. There are now more than one cpu.h's and different
> parts of the tree need a different include scheme. target-multi defines
> it's own cpu.h which is bare minimum defs as needed by core code only.
> target-foo/cpu.h are mostly the same but refactored to avoid collisions
> with other cpu.h's. Inclusion scheme goes something like
> this (for the multi-arch build):
> 
> *: Core code includes only target-multi/cpu.h
> *: target-foo/ implementation code includes target-foo/cpu.h locally
> *: System level code (e.g. mach models) can use multiple target-foo/cpu.h's
> 
> The hardest unasnwered Q is (still) what to do about bootloading. Currently
> each arch has it's own architecture specific bootloading which may assume a
> single architecture. I have applied some hacks to at least get this
> RFC testable using a -kernel -firmware split but going forward being
> able to associate an elf/image with a cpu explictitly needs to be
> solved.
> 
> No support for KVM, im not sure if a mix of TCG and KVM is supported even for
> a single arch? (which would be prerequisite to MA KVM).
> 
> ***
> 
> Current review state of full multi-arch work in progress branch:
> 
> target-*: Don't redefine cpu_exec()
> target-*: cpu.h: Undefine core code symbols
> arm: cpu: static inline cpu_arm_init()
> target-arm: Split cp helper API to new C file
> hw: arm: Explicitly include cpu.h for consumers
> hw: mb: Explicitly include cpu.h for consumers
> translate: Listify tcg_exec_init()                          R:rth@twiddle.net
> cpus: Listify cpu_list() function
> translate-common: Listify tcg_enabled()
> core: Convert tcg_enabled() uses to any/all variants
> exec-all: Move cpu_can_do_io() to qom/cpu.h                 R:rth@twiddle.net
> cpu-common: Define tb_page_addr_t for everyone
> include/exec: Split target_long def to new header
> cpu-defs: Allow multiple inclusions
> Makefile.target: Introduce arch-obj
> core: virtualise CPU interfaces completely
> core: Introduce multi-arch build
> arm: register cpu_list() function
> arm: enable multi-arch
> microblaze: enable multi-arch
> arm: boot: Don't assume all CPUs are ARM
> arm: xilinx_zynq: Add a Microblaze
> HACK: mb: boot: Assume using -firmware for mb software
> HACK: mb: boot: Disable dtb load in multi-arch
> 
> 
> Peter Crosthwaite (6):
>   target-*: Don't redefine cpu_exec()
>   target-*: cpu.h: Undefine core code symbols
>   arm: cpu: static inline cpu_arm_init()
>   target-arm: Split cp helper API to new C file
>   hw: arm: Explicitly include cpu.h for consumers
>   hw: mb: boot Explicitly include cpu.h for consumers
> 
>  bsd-user/main.c               |   4 +-
>  hw/arm/strongarm.h            |   2 +
>  hw/microblaze/boot.h          |   2 +
>  include/exec/cpu-all.h        |   2 +
>  include/exec/cpu-defs-clear.h |  33 +++++
>  include/hw/arm/arm.h          |   3 +
>  include/hw/arm/digic.h        |   2 +
>  include/hw/arm/exynos4210.h   |   2 +
>  include/hw/arm/omap.h         |   2 +
>  include/hw/arm/pxa.h          |   2 +
>  linux-user/main.c             |  32 ++--
>  target-alpha/cpu.h            |   3 +-
>  target-arm/Makefile.objs      |   1 +
>  target-arm/cp.c               | 328 +++++++++++++++++++++++++++++++++++++++++
>  target-arm/cpu.h              |   9 +-
>  target-arm/helper.c           | 329 ------------------------------------------
>  target-cris/cpu.h             |   3 +-
>  target-i386/cpu.h             |   3 +-
>  target-lm32/cpu.h             |   4 +-
>  target-m68k/cpu.h             |   4 +-
>  target-microblaze/cpu.h       |   3 +-
>  target-mips/cpu.h             |   4 +-
>  target-moxie/cpu.h            |   3 +-
>  target-openrisc/cpu.h         |   4 +-
>  target-ppc/cpu.h              |   3 +-
>  target-s390x/cpu.h            |   3 +-
>  target-sh4/cpu.h              |   3 +-
>  target-sparc/cpu.h            |   3 +-
>  target-tilegx/cpu.h           |   3 +-
>  target-tricore/cpu.h          |   3 +-
>  target-unicore32/cpu.h        |   3 +-
>  target-xtensa/cpu.h           |   4 +-
>  32 files changed, 426 insertions(+), 383 deletions(-)
>  create mode 100644 include/exec/cpu-defs-clear.h
>  create mode 100644 target-arm/cp.c
> 

I think we can merge all patches except 2 in 2.5, but it depends on
Peter as they would go through his tree (only patch 1 is generic,
everything else is ARM).  They definitely were posted before soft freeze.

Paolo