[PATCH v3 00/10] xen/arm: live migration support in arndale board

[Jaeyong Yoo <jaeyong.yoo@samsung.com>]

Hi all,
here goes the v3 patch series for live migration in arndale board.
This version applies the comments from v2 patch series, which are majorly:

  1) just use one timer struct for storing vtimer and ptimer
     for hvm context: patch 1

  2) for dirty page tracing, use virtual-linear page table for accessing
     guest p2m in xen: patch 6, 7, and 9

  3) Rather than using hard-coded guest memory map in xen, use the one from
     toolstack by implementing set/get_memory_map hypercall: patch 3 and 10

This patch series does not support SMP guest migration. We are expecting
to provide SMP-guests live migration in version 4 patch series.

We also have tested the stability of v3 patch series as follows:

  - setup two arndale boards with xen (let's say A and B)
  - launch 3 domUs at A
  - simulataneously migrate 3 domUs from A to B back and forth.
  - we say one round of migration if all 3 domUs migrate
    from A to B and migrate back from B to A.

When we perform the above tests without any load on domUs, the migration
goes to 80~100 rounds. After that, dom0 suddenly stops responding. When we
perform with network load (iperf on each domU), the migration goes to 2~3
rounds and dom0 stops responding.

After several repeated tests, we gather the PCs where dom0 stucks, and those are
  - _raw_spin_lock: (called by try_to_wake_up)
  - panic_smp_self_stop
  - cpu_v7_dcache_clean_area

I think those bugs are somehow related to live migration or maybe other parts
that may result in complicated cause-and-effect chain to live migration. In any
case, I would like to look into the detail to figure out the cause and possibly
fix the bugs.

In the meanwhile, I would appreciate your comments to this patch series :)


Best,
Jaeyong

lexey Sokolov, Elena Pyatunina, Evgeny Fedotov, and Nikolay Martyanov (1):
  xen/arm: Implement toolstack for xl restore/save and migrate

Alexey Sokolov (1):
  xen/arm: Implement modify_returncode

Evgeny Fedotov (2):
  xen/arm: Implement set_memory_map hypercall
  xen/arm: Implement get_maximum_gpfn hypercall for arm

Jaeyong Yoo and Evgeny Fedotov (1):
  xen/arm: Implement hvm save and restore

Jaeyong Yoo and Alexey Sokolov (1):
  xen/arm: Add more registers for saving and restoring vcpu  registers

Jaeyong Yoo and Elena Pyatunina (2)
  xen/arm: Add handling write fault for dirty-page tracing
  xen/arm: Implement hypercall for dirty page tracing (shadow op)

Jaeyong Yoo (2):
  xen/arm: Implement virtual-linear page table for guest      p2m
    mapping in live migration
  xen/arm: Fixing clear_guest_offset macro

 config/arm32.mk                          |   1 +
 tools/include/xen-foreign/reference.size |   2 +-
 tools/libxc/Makefile                     |   5 +
 tools/libxc/xc_arm_migrate.c             | 686 +++++++++++++++++++++++++++++++
 tools/libxc/xc_dom_arm.c                 |  12 +-
 tools/libxc/xc_domain.c                  |  44 ++
 tools/libxc/xc_resume.c                  |  25 ++
 tools/libxc/xenctrl.h                    |  23 ++
 tools/misc/Makefile                      |   4 +
 xen/arch/arm/Makefile                    |   2 +
 xen/arch/arm/domain.c                    |  44 ++
 xen/arch/arm/domctl.c                    | 137 +++++-
 xen/arch/arm/hvm.c                       | 124 ++++++
 xen/arch/arm/mm.c                        | 284 ++++++++++++-
 xen/arch/arm/p2m.c                       | 307 ++++++++++++++
 xen/arch/arm/save.c                      |  66 +++
 xen/arch/arm/setup.c                     |   3 +
 xen/arch/arm/traps.c                     |  16 +-
 xen/arch/arm/vlpt.c                      | 162 ++++++++
 xen/common/Makefile                      |   2 +
 xen/include/asm-arm/config.h             |   3 +
 xen/include/asm-arm/domain.h             |  13 +
 xen/include/asm-arm/guest_access.h       |   5 +-
 xen/include/asm-arm/hvm/support.h        |  29 ++
 xen/include/asm-arm/mm.h                 |   7 +
 xen/include/asm-arm/p2m.h                |   4 +
 xen/include/asm-arm/processor.h          |   2 +
 xen/include/asm-arm/vlpt.h               |  10 +
 xen/include/public/arch-arm.h            |  35 ++
 xen/include/public/arch-arm/hvm/save.h   |  41 ++
 xen/include/public/memory.h              |  15 +-
 xen/include/xsm/dummy.h                  |   5 +
 xen/include/xsm/xsm.h                    |   5 +
 33 files changed, 2113 insertions(+), 10 deletions(-)
 create mode 100644 tools/libxc/xc_arm_migrate.c
 create mode 100644 xen/arch/arm/save.c
 create mode 100644 xen/arch/arm/vlpt.c
 create mode 100644 xen/include/asm-arm/hvm/support.h
 create mode 100644 xen/include/asm-arm/vlpt.h

-- 
1.8.1.2