qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [RFC PATCH v6 00/32] Deterministic replay core
@ 2014-12-08  7:53 Pavel Dovgalyuk
  2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 01/32] i386: partial revert of interrupt poll fix Pavel Dovgalyuk
                   ` (31 more replies)
  0 siblings, 32 replies; 40+ messages in thread
From: Pavel Dovgalyuk @ 2014-12-08  7:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: peter.maydell, peter.crosthwaite, alex.bennee, mark.burton, real,
	batuzovk, maria.klimushenkova, pavel.dovgaluk, pbonzini, afaerber,
	fred.konrad

This set of patches is related to the reverse execution and deterministic 
replay of qemu execution  This implementation of deterministic replay can 
be used for deterministic debugging of guest code through gdb remote
interface.

These patches include only core function of the replay,
excluding the support for replaying serial, audio, network, and USB devices'
operations. Reverse debugging and monitor commands were also excluded to
be submitted later as separate patches.

Execution recording writes non-deterministic events log, which can be later 
used for replaying the execution anywhere and for unlimited number of times. 
It also supports checkpointing for faster rewinding during reverse debugging. 
Execution replaying reads the log and replays all non-deterministic events 
including external input, hardware clocks, and interrupts.

Deterministic replay has the following features:
 * Deterministically replays whole system execution and all contents of the memory,
   state of the hadrware devices, clocks, and screen of the VM.
 * Writes execution log into the file for latter replaying for multiple times 
   on different machines.
 * Supports i386, x86_64, and ARM hardware platforms.
 * Performs deterministic replay of all operations with keyboard and mouse
   input devices.
 * Supports auto-checkpointing for convenient reverse debugging.

Usage of the record/replay:
 * First, record the execution, by adding the following string to the command line:
   '-record fname=replay.bin -icount 7 -net none'. Block devices' images are not
   actually changed in the recording mode, because all of the changes are
   written to the temporary overlay file.
 * Then you can replay it for the multiple times by using another command
   line option: '-replay fname=replay.bin -icount 7 -net none'
 * '-net none' option should also be specified if network replay patches
   are not applied.

Paper with short description of deterministic replay implementation:
http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html

Modifications of qemu include:
 * wrappers for clock and time functions to save their return values in the log
 * saving different asynchronous events (e.g. system shutdown) into the log
 * synchronization of the bottom halves execution
 * synchronization of the threads from thread pool
 * recording/replaying user input (mouse and keyboard)
 * adding internal events for cpu and io synchronization

v6 changes:
 * Fixed replay stub return value (as suggested by Eric Blake)
 * Fixed icount warping.
 * Virtual rt clock now uses cpu_get_clock() (as suggested by Paolo Bonzini)
 * Replated get_clock_realtime and get_clock calls with qemu clock requests (as suggested by Paolo Bonzini)
 * Modified can_do_io logic to allow requesting icount from cpu_exec function (as suggested by Paolo Bonzini)
 * Removed applied patches.

v5 changes:
 * Minor changes.
 * Used fixed-width integer types for read/write functions (as suggested by Alex Bennee)
 * Moved savevm-related code out of the core.
 * Added new traced clock for deterministic virtual clock warping (as suggested by Paolo Bonzini)
 * Fixed exception_index reset for user mode (as suggested by Paolo Bonzini)
 * Adopted Paolo's icount patches
 * Fixed hardware interrupts replaying

v4 changes:
 * Updated block drivers to support new bdrv_open interface.
 * Moved migration patches into separate series (as suggested by Paolo Bonzini)
 * Fixed a bug in replay_break operation.
 * Fixed rtl8139 migration for replay.
 * Fixed 'period' parameter processing for record mode.
 * Fixed bug in 'reverse-stepi' implementation.
 * Fixed replay without making any snapshots (even the starting one).
 * Moved core replay patches into the separate series.
 * Fixed reverse step and reverse continue support.
 * Fixed several bugs in icount subsystem.
 * Reusing native qemu icount for replay instructions counting.
 * Separated core patches into their own series.

v3 changes:
 * Fixed bug with replay of the aio write operations.
 * Added virtual clock based on replay icount.
 * Removed duplicated saving of interrupt_request CPU field.
 * Fixed some coding style issues.
 * Renamed QMP commands for controlling reverse execution (as suggested by Eric Blake)
 * Replay mode and submode implemented as QAPI enumerations (as suggested by Eric Blake)
 * Added description and example for replay-info command (as suggested by Eric Blake)
 * Added information about the current breakpoint to the output of replay-info (as suggested by Eric Blake)
 * Updated version id for HPET vmstate (as suggested by Paolo Bonzini)
 * Removed static fields from parallel vmstate (as suggested by Paolo Bonzini)
 * New vmstate fields for mc146818rtc, pckbd, kvmapic, serial, fdc, rtl8139 moved to subsection (as suggested by Paolo Bonzini)
 * Disabled textmode cursor blinking, when virtual machine is stopped (as suggested by Paolo Bonzini)
 * Extracted saving of exception_index to separate patch (as suggested by Paolo Bonzini)

v2 changes:
 * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
 * Added QMP versions of replay commands (as suggested by Eric Blake)
 * Removed some optional features of replay to make patches cleaner
 * Minor changes and code cleanup were made

---

Paolo Bonzini (4):
      From 7abf2f72777958d395cfd01d97fe707cc06152b5 Mon Sep 17 00:00:00 2001
      From 185a3a47d08857a66332ae862b372a153ce92bb9 Mon Sep 17 00:00:00 2001
      From a0cb9e80ba0de409b5ad556109a1c71ce4d8ce19 Mon Sep 17 00:00:00 2001
      From 04bbd21134dd2c6b7309a7f5f2b780aae2757003 Mon Sep 17 00:00:00 2001

Pavel Dovgalyuk (28):
      i386: partial revert of interrupt poll fix
      cpu-exec: fix cpu_exec_nocache
      icount: set can_do_io outside TB execution
      replay: global variables and function stubs
      sysemu: system functions for replay
      replay: internal functions for replay log
      cpu-exec: reset exception_index correctly
      icount: implement icount requesting
      replay: introduce icount event
      i386: do not cross the pages boundaries in replay mode
      cpu-exec: allow temporary disabling icount
      cpu-exec: invalidate nocache translation if they are interrupted
      cpu: replay instructions sequence
      replay: interrupts and exceptions
      replay: asynchronous events infrastructure
      timer: introduce new QEMU_CLOCK_VIRTUAL_RT clock
      cpus: make icount warp deterministic in replay mode
      timer: fix usage of clock functions
      replay: recording and replaying clock ticks
      replay: recording and replaying different timers
      replay: shutdown event
      replay: checkpoints
      replay: bottom halves
      replay: replay aio requests
      replay: thread pool
      replay: initialization and deinitialization
      replay: command line options
      replay: recording of the user input


 Makefile.target                |    1 
 async.c                        |   46 +++++-
 block.c                        |   92 ++++++++++-
 block/accounting.c             |    5 -
 block/block-backend.c          |   30 ++++
 block/qcow2.c                  |    4 
 block/raw-posix.c              |   16 +-
 block/raw-win32.c              |    4 
 cpu-exec.c                     |   66 ++++++--
 cpus.c                         |   80 +++++++---
 dma-helpers.c                  |   10 +
 exec.c                         |    1 
 hw/block/virtio-blk.c          |   10 +
 hw/ide/ahci.c                  |    4 
 hw/ide/atapi.c                 |   10 +
 hw/ide/core.c                  |   18 +-
 hw/ppc/ppc.c                   |    4 
 hw/timer/arm_timer.c           |    2 
 hw/timer/mc146818rtc.c         |   10 +
 hw/timer/pl031.c               |   10 +
 hw/usb/hcd-uhci.c              |    2 
 include/block/aio.h            |   18 ++
 include/block/block.h          |   15 ++
 include/block/thread-pool.h    |    4 
 include/exec/exec-all.h        |    8 +
 include/exec/gen-icount.h      |    6 -
 include/qemu-common.h          |    3 
 include/qemu/main-loop.h       |    1 
 include/qemu/timer.h           |   24 +++
 include/qom/cpu.h              |   10 +
 include/sysemu/block-backend.h |   10 +
 include/sysemu/cpus.h          |    1 
 include/sysemu/sysemu.h        |    1 
 include/ui/input.h             |    2 
 main-loop.c                    |   10 +
 pc-bios/s390-ccw/virtio.c      |    2 
 qapi-schema.json               |   32 ++++
 qemu-io-cmds.c                 |    2 
 qemu-options.hx                |   27 +++
 qemu-timer.c                   |   53 +++++--
 replay/Makefile.objs           |    5 +
 replay/replay-events.c         |  292 ++++++++++++++++++++++++++++++++++++
 replay/replay-input.c          |  108 +++++++++++++
 replay/replay-internal.c       |  155 +++++++++++++++++++
 replay/replay-internal.h       |  132 ++++++++++++++++
 replay/replay-time.c           |  191 ++++++++++++++++++++++++
 replay/replay.c                |  321 ++++++++++++++++++++++++++++++++++++++++
 replay/replay.h                |  119 +++++++++++++++
 stubs/Makefile.objs            |    1 
 stubs/replay.c                 |   42 +++++
 target-alpha/translate.c       |   10 +
 target-arm/translate-a64.c     |    6 -
 target-arm/translate.c         |    6 -
 target-cris/translate.c        |    2 
 target-i386/cpu.c              |   10 -
 target-i386/cpu.h              |    3 
 target-i386/seg_helper.c       |    4 
 target-i386/translate.c        |   62 +++++---
 target-lm32/translate.c        |   10 +
 target-m68k/translate.c        |    2 
 target-microblaze/translate.c  |    2 
 target-mips/kvm.c              |    2 
 target-mips/translate.c        |   26 ++-
 target-moxie/translate.c       |    2 
 target-openrisc/translate.c    |    2 
 target-ppc/cpu.h               |   13 +-
 target-ppc/translate.c         |   12 +
 target-ppc/translate_init.c    |  271 +++++++++++++++++-----------------
 target-s390x/translate.c       |    2 
 target-sh4/translate.c         |    2 
 target-sparc/translate.c       |    2 
 target-tricore/translate.c     |    2 
 target-unicore32/translate.c   |    2 
 target-xtensa/translate.c      |    2 
 tests/test-thread-pool.c       |    7 -
 thread-pool.c                  |   49 ++++--
 trace-events                   |    2 
 translate-all.c                |   19 ++
 ui/input.c                     |   80 ++++++++--
 util/iov.c                     |    4 
 vl.c                           |  112 +++++++++++++-
 81 files changed, 2378 insertions(+), 372 deletions(-)
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay-events.c
 create mode 100755 replay/replay-input.c
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h
 create mode 100755 replay/replay-time.c
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c

-- 
Pavel Dovgalyuk

^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2014-12-17 13:02 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-12-08  7:53 [Qemu-devel] [RFC PATCH v6 00/32] Deterministic replay core Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 01/32] i386: partial revert of interrupt poll fix Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 02/32] cpu-exec: fix cpu_exec_nocache Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 03/32] icount: set can_do_io outside TB execution Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 04/32] replay: global variables and function stubs Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 05/32] sysemu: system functions for replay Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 06/32] replay: internal functions for replay log Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 07/32] cpu-exec: reset exception_index correctly Pavel Dovgalyuk
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 08/32] icount: implement icount requesting Pavel Dovgalyuk
2014-12-09 17:39   ` Paolo Bonzini
2014-12-10  6:35     ` Pavel Dovgalyuk
2014-12-10  7:41       ` Paolo Bonzini
2014-12-11  8:16         ` Pavel Dovgaluk
     [not found]         ` <5947.68890082396$1418285825@news.gmane.org>
2014-12-11 11:06           ` Paolo Bonzini
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 09/32] replay: introduce icount event Pavel Dovgalyuk
2014-12-08 12:47   ` Gonglei
2014-12-08  7:53 ` [Qemu-devel] [RFC PATCH v6 10/32] i386: do not cross the pages boundaries in replay mode Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 11/32] From 7abf2f72777958d395cfd01d97fe707cc06152b5 Mon Sep 17 00:00:00 2001 Pavel Dovgalyuk
2014-12-17 13:02   ` Alexander Graf
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 12/32] From 185a3a47d08857a66332ae862b372a153ce92bb9 " Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 13/32] From a0cb9e80ba0de409b5ad556109a1c71ce4d8ce19 " Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 14/32] From 04bbd21134dd2c6b7309a7f5f2b780aae2757003 " Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 15/32] cpu-exec: allow temporary disabling icount Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 16/32] cpu-exec: invalidate nocache translation if they are interrupted Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 17/32] cpu: replay instructions sequence Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 18/32] replay: interrupts and exceptions Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 19/32] replay: asynchronous events infrastructure Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 20/32] timer: introduce new QEMU_CLOCK_VIRTUAL_RT clock Pavel Dovgalyuk
2014-12-08  7:54 ` [Qemu-devel] [RFC PATCH v6 21/32] cpus: make icount warp deterministic in replay mode Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 22/32] timer: fix usage of clock functions Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 23/32] replay: recording and replaying clock ticks Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 24/32] replay: recording and replaying different timers Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 25/32] replay: shutdown event Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 26/32] replay: checkpoints Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 27/32] replay: bottom halves Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 28/32] replay: replay aio requests Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 29/32] replay: thread pool Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 30/32] replay: initialization and deinitialization Pavel Dovgalyuk
2014-12-08  7:55 ` [Qemu-devel] [RFC PATCH v6 31/32] replay: command line options Pavel Dovgalyuk
2014-12-08  7:56 ` [Qemu-devel] [RFC PATCH v6 32/32] replay: recording of the user input Pavel Dovgalyuk

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).