qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [RFC PATCH v2 00/49] Series short description
@ 2014-07-17 11:01 Pavel Dovgalyuk
  2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 01/49] acpi: accurate overflow check Pavel Dovgalyuk
                   ` (50 more replies)
  0 siblings, 51 replies; 83+ messages in thread
From: Pavel Dovgalyuk @ 2014-07-17 11:01 UTC (permalink / raw)
  To: qemu-devel
  Cc: peter.maydell, peter.crosthwaite, mark.burton, real, batuzovk,
	pavel.dovgaluk, pbonzini, fred.konrad

This set of patches is related to the reverse execution and deterministic 
replay of qemu execution  Our implementation of deterministic replay can 
be used for deterministic and reverse debugging of guest code through gdb 
remote interface.

Execution recording writes non-deterministic events log, which can be later 
used for replaying the execution anywhere and for unlimited number of times. 
It also supports checkpointing for faster rewinding during reverse debugging. 
Execution replaying reads the log and replays all non-deterministic events 
including external input, hardware clocks, and interrupts.

Reverse execution has the following features:
 * Deterministically replays whole system execution and all contents of the memory,
   state of the hadrware devices, clocks, and screen of the VM.
 * Writes execution log into the file for latter replaying for multiple times 
   on different machines.
 * Supports i386, x86_64, and ARM hardware platforms.
 * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
   audio devices, serial interfaces, and physical USB devices connected to the emulator.
 * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
 * Supports auto-checkpointing for convenient reverse debugging.

Usage of the record/replay:
 * First, record the execution, by adding '-record fname=replay.bin' to the
   command line.
 * Then you can replay it for the multiple times by using another command
   line option: '-replay fname=replay.bin'
 * Virtual machine should have at least one virtual disk, which is used to
   store checkpoints. If you want to enable automatic checkpointing, simply
   add ',period=XX' to record options, where XX is the checkpointing period
   in seconds.
 * Using of the network adapters in record/replay mode is possible with 
   the following command-line options:
   - '-net user' (or another host adapter) in record mode
   - '-net replay' in replay mode. Every host network adapter should be
     replaced by 'replay' when replaying the execution.
 * Reverse debugging can be used through gdb remote interface.
   reverse-stepi and reverse-continue commands are supported. Other reverse
   commands should also work, because they reuse these ones.
 * Monitor is extended by the following commands:
   - replay_info - prints information about replay mode and current step
     (number of instructions executed)
   - replay_break - sets "breakpoint" at the specified instructions count.
   - replay_seek - rewinds (using the checkpoints, if possible) to the
     specified step of replay log.

Paper with short description of deterministic replay implementation:
http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html

Modifications of qemu include:
 * adding missed fields of the virtual devices' states to the vmstate 
   structures to allow deterministic saving and restoring the VM state
 * adding virtual clock-based timers to vmstate structures, because virtual 
   clock is the part of the virtual machine state
 * modification of block layer to support automatic creation of the overlay
   files to store the changes and snapshots while recording
 * disabling of system reset while loading VM state to avoid generating of
   interrupts by reset handlers
 * adding warpers for clock and time functions to save their return
   values in the log
 * saving different asynchronous events (e.g. system shutdown) into the log
 * synchronization of the bottom halves execution
 * synchronization of the threads from thread pool
 * recording/replaying user input (mouse and keyboard), input from virtual
   serial ports, incoming network packets, input from connected USB devices
 * adding HMP/QMP commands to monitor for controlling replay execution

v2 changes:
 * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
 * Added QMP versions of replay commands (as suggested by Eric Blake)
 * Removed some optional features of replay to make patches cleaner
 * Minor changes and code cleanup were made

---

Pavel Dovgalyuk (49):
      acpi: accurate overflow check
      integratorcp: adding vmstate for save/restore
      pcspk: adding vmstate for save/restore
      fdc: adding vmstate for save/restore
      parallel: adding vmstate for save/restore
      serial: fixing vmstate for save/restore
      kvmapic: fixing loading vmstate
      hpet: fixing saving and loading process
      pckbd: adding new fields to vmstate
      rtl8139: adding new fields to vmstate
      piix: do not raise irq while loading vmstate
      mc146818rtc: add missed field to vmstate
      pl031: add missed field to vmstate
      ide pci: reset status field before loading the vmstate
      softmmu: fixing usage of cpu_st/ld* from helpers
      target: save cpu state fields
      target-i386: update fp status fix
      migration: add vmstate for int8 and char arrays
      replay: global variables and function stubs
      block: add suffix parameter to bdrv_open functions
      sysemu: system functions for replay
      replay: internal functions for replay log
      cpu: invent instruction count for accurate replay
      target-arm: instructions counting code for replay
      target-i386: instructions counting code for replay
      replay: interrupts and exceptions
      vga: do not use virtual clock for blinking cursor
      replay: asynchronous events infrastructure
      replay: recording and replaying clock ticks
      replay: recording and replaying different timers
      replay: shutdown event
      replay: checkpoints
      replay: bottom halves
      replay: replay aio requests
      replay: thread pool
      pl031: vmstate in replay mode
      replay: initialization and deinitialization
      replay: command line options
      replay: snapshotting the virtual machine
      replay: recording of the user input
      tap-win32: destroy the thread at exit
      replay: network packets record/replay
      replay: audio data record/replay
      replay: serial port
      replay: USB passthrough
      replay: replay_info command
      replay: replay_break command
      replay: replay_seek_step command
      gdbstub: reverse debugging


 Makefile.target                  |    1 
 arch_init.c                      |    8 
 async.c                          |   45 ++
 audio/audio.c                    |   14 +
 audio/audio_win_int.h            |    3 
 audio/winwaveaudio.c             |  167 +++++++--
 block.c                          |  134 ++++++-
 block/blkdebug.c                 |    2 
 block/blkverify.c                |    4 
 block/cow.c                      |    2 
 block/qcow.c                     |    2 
 block/qcow2.c                    |   10 -
 block/qed.c                      |    2 
 block/raw-posix.c                |    6 
 block/raw-win32.c                |    4 
 block/sheepdog.c                 |    4 
 block/vmdk.c                     |    8 
 block/vvfat.c                    |    2 
 blockdev.c                       |   11 -
 cpu-exec.c                       |   34 +-
 cpus.c                           |   81 ++++
 dma-helpers.c                    |   10 -
 exec.c                           |   12 +
 gdbstub.c                        |   79 +++-
 hmp-commands.hx                  |   41 ++
 hw/acpi/core.c                   |    7 
 hw/arm/integratorcp.c            |   38 ++
 hw/audio/pcspk.c                 |   19 +
 hw/block/fdc.c                   |   11 -
 hw/block/virtio-blk.c            |   10 -
 hw/char/parallel.c               |   22 +
 hw/char/serial.c                 |  115 ++++--
 hw/display/vga.c                 |    4 
 hw/i386/kvmvapic.c               |   22 +
 hw/ide/ahci.c                    |    4 
 hw/ide/atapi.c                   |    9 
 hw/ide/core.c                    |   18 +
 hw/ide/pci.c                     |   10 +
 hw/input/pckbd.c                 |    3 
 hw/intc/apic_common.c            |    5 
 hw/net/rtl8139.c                 |    5 
 hw/pci-host/piix.c               |   22 +
 hw/timer/arm_timer.c             |    2 
 hw/timer/hpet.c                  |   13 -
 hw/timer/mc146818rtc.c           |    5 
 hw/timer/pl031.c                 |   35 +-
 hw/usb/hcd-uhci.c                |    2 
 hw/usb/host-libusb.c             |  525 ++++++++++++++++++---------
 include/block/aio.h              |   17 +
 include/block/block.h            |   21 +
 include/block/thread-pool.h      |    4 
 include/exec/cpu-defs.h          |    1 
 include/exec/cpu_ldst_template.h |   28 +
 include/exec/exec-all.h          |   31 ++
 include/hw/host-libusb.h         |  105 +++++
 include/migration/vmstate.h      |   13 +
 include/qemu-common.h            |    3 
 include/qemu/main-loop.h         |    1 
 include/qemu/timer.h             |   48 ++
 include/qom/cpu.h                |    4 
 include/sysemu/char.h            |   25 +
 include/sysemu/cpus.h            |    1 
 include/sysemu/sysemu.h          |    2 
 include/ui/input.h               |    2 
 main-loop.c                      |    5 
 monitor.c                        |   49 +++
 net/Makefile.objs                |    1 
 net/clients.h                    |    3 
 net/dump.c                       |    6 
 net/hub.c                        |    1 
 net/net-replay.c                 |   66 +++
 net/net.c                        |    7 
 net/slirp.c                      |   14 +
 net/socket.c                     |   35 ++
 net/tap-win32.c                  |   25 +
 net/tap.c                        |   23 +
 net/vde.c                        |   14 +
 qapi-schema.json                 |   62 +++
 qemu-char.c                      |   55 +++
 qemu-img.c                       |    6 
 qemu-io-cmds.c                   |    2 
 qemu-io.c                        |    4 
 qemu-nbd.c                       |    2 
 qemu-options.hx                  |   28 +
 qemu-timer.c                     |   45 ++
 qmp-commands.hx                  |   60 +++
 replay/Makefile.objs             |   11 +
 replay/replay-audio.c            |  228 ++++++++++++
 replay/replay-char.c             |   99 +++++
 replay/replay-debug.c            |  148 ++++++++
 replay/replay-events.c           |  381 ++++++++++++++++++++
 replay/replay-input.c            |  107 ++++++
 replay/replay-internal.c         |  159 ++++++++
 replay/replay-internal.h         |  224 ++++++++++++
 replay/replay-net.c              |  190 ++++++++++
 replay/replay-qmp.c              |   57 +++
 replay/replay-time.c             |  181 +++++++++
 replay/replay-usb.c              |  188 ++++++++++
 replay/replay.c                  |  735 ++++++++++++++++++++++++++++++++++++++
 replay/replay.h                  |  218 +++++++++++
 savevm.c                         |   32 +-
 slirp/slirp.c                    |    9 
 softmmu_template.h               |   18 +
 stubs/Makefile.objs              |    1 
 stubs/replay.c                   |   42 ++
 target-arm/Makefile.objs         |    1 
 target-arm/helper.h              |    3 
 target-arm/machine.c             |    5 
 target-arm/replay_helper.c       |   38 ++
 target-arm/translate.c           |   62 +++
 target-i386/Makefile.objs        |    1 
 target-i386/cpu.c                |    1 
 target-i386/cpu.h                |    8 
 target-i386/fpu_helper.c         |    5 
 target-i386/helper.h             |    3 
 target-i386/machine.c            |    7 
 target-i386/replay_helper.c      |   38 ++
 target-i386/translate.c          |   91 ++++-
 tests/test-thread-pool.c         |    7 
 thread-pool.c                    |   53 ++-
 trace-events                     |    2 
 translate-all.c                  |   15 +
 ui/input.c                       |   79 +++-
 util/iov.c                       |    4 
 vl.c                             |  108 +++++-
 vmstate.c                        |    6 
 126 files changed, 5433 insertions(+), 543 deletions(-)
 create mode 100755 include/hw/host-libusb.h
 create mode 100755 net/net-replay.c
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay-audio.c
 create mode 100755 replay/replay-char.c
 create mode 100755 replay/replay-debug.c
 create mode 100755 replay/replay-events.c
 create mode 100755 replay/replay-input.c
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h
 create mode 100755 replay/replay-net.c
 create mode 100755 replay/replay-qmp.c
 create mode 100755 replay/replay-time.c
 create mode 100755 replay/replay-usb.c
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c
 create mode 100755 target-arm/replay_helper.c
 create mode 100755 target-i386/replay_helper.c

-- 
Pavel Dovgalyuk

^ permalink raw reply	[flat|nested] 83+ messages in thread

end of thread, other threads:[~2014-07-31 14:18 UTC | newest]

Thread overview: 83+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-07-17 11:01 [Qemu-devel] [RFC PATCH v2 00/49] Series short description Pavel Dovgalyuk
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 01/49] acpi: accurate overflow check Pavel Dovgalyuk
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 02/49] integratorcp: adding vmstate for save/restore Pavel Dovgalyuk
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 03/49] pcspk: " Pavel Dovgalyuk
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 04/49] fdc: " Pavel Dovgalyuk
2014-07-28  9:47   ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 05/49] parallel: " Pavel Dovgalyuk
2014-07-28 10:02   ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 06/49] serial: fixing " Pavel Dovgalyuk
2014-07-28  9:58   ` Paolo Bonzini
2014-07-30  7:01     ` Pavel Dovgaluk
     [not found]     ` <19697.8771281012$1406703748@news.gmane.org>
2014-07-30  9:19       ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 07/49] kvmapic: fixing loading vmstate Pavel Dovgalyuk
2014-07-28  8:49   ` Paolo Bonzini
2014-07-29 12:03     ` Pavel Dovgaluk
2014-07-29 12:16       ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 08/49] hpet: fixing saving and loading process Pavel Dovgalyuk
2014-07-28  8:33   ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 09/49] pckbd: adding new fields to vmstate Pavel Dovgalyuk
2014-07-28  9:36   ` Paolo Bonzini
2014-07-17 11:02 ` [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: " Pavel Dovgalyuk
2014-07-28  9:41   ` Paolo Bonzini
2014-07-28  9:54     ` Pavel Dovgaluk
     [not found]     ` <37740.9009532586$1406541296@news.gmane.org>
2014-07-28 10:12       ` Paolo Bonzini
2014-07-30  8:24         ` Pavel Dovgaluk
2014-07-30  9:26           ` Paolo Bonzini
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 11/49] piix: do not raise irq while loading vmstate Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 12/49] mc146818rtc: add missed field to vmstate Pavel Dovgalyuk
2014-07-28  9:42   ` Paolo Bonzini
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 13/49] pl031: " Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 14/49] ide pci: reset status field before loading the vmstate Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 15/49] softmmu: fixing usage of cpu_st/ld* from helpers Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 16/49] target: save cpu state fields Pavel Dovgalyuk
2014-07-31  6:48   ` Andreas Färber
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 17/49] target-i386: update fp status fix Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 18/49] migration: add vmstate for int8 and char arrays Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 19/49] replay: global variables and function stubs Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 20/49] block: add suffix parameter to bdrv_open functions Pavel Dovgalyuk
2014-07-17 11:03 ` [Qemu-devel] [RFC PATCH v2 21/49] sysemu: system functions for replay Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 22/49] replay: internal functions for replay log Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 23/49] cpu: invent instruction count for accurate replay Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 24/49] target-arm: instructions counting code for replay Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 25/49] target-i386: " Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 26/49] replay: interrupts and exceptions Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 27/49] vga: do not use virtual clock for blinking cursor Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 28/49] replay: asynchronous events infrastructure Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 29/49] replay: recording and replaying clock ticks Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 30/49] replay: recording and replaying different timers Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 31/49] replay: shutdown event Pavel Dovgalyuk
2014-07-17 11:04 ` [Qemu-devel] [RFC PATCH v2 32/49] replay: checkpoints Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 33/49] replay: bottom halves Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 34/49] replay: replay aio requests Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 35/49] replay: thread pool Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 36/49] pl031: vmstate in replay mode Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 37/49] replay: initialization and deinitialization Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 38/49] replay: command line options Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 39/49] replay: snapshotting the virtual machine Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 40/49] replay: recording of the user input Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 41/49] tap-win32: destroy the thread at exit Pavel Dovgalyuk
2014-07-17 11:05 ` [Qemu-devel] [RFC PATCH v2 42/49] replay: network packets record/replay Pavel Dovgalyuk
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 43/49] replay: audio data record/replay Pavel Dovgalyuk
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 44/49] replay: serial port Pavel Dovgalyuk
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 45/49] replay: USB passthrough Pavel Dovgalyuk
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 46/49] replay: replay_info command Pavel Dovgalyuk
2014-07-18 15:55   ` Eric Blake
2014-07-18 15:56   ` Eric Blake
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 47/49] replay: replay_break command Pavel Dovgalyuk
2014-07-18 15:58   ` Eric Blake
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 48/49] replay: replay_seek_step command Pavel Dovgalyuk
2014-07-18 15:59   ` Eric Blake
2014-07-17 11:06 ` [Qemu-devel] [RFC PATCH v2 49/49] gdbstub: reverse debugging Pavel Dovgalyuk
2014-07-18  8:10 ` [Qemu-devel] [RFC PATCH v2 00/49] Series short description Frederic Konrad
2014-07-24 17:48 ` Paolo Bonzini
2014-07-28  7:50   ` Pavel Dovgaluk
     [not found]   ` <2596.37912172384$1406533875@news.gmane.org>
2014-07-28 10:12     ` Paolo Bonzini
2014-07-30  7:44       ` Pavel Dovgaluk
2014-07-30  9:25         ` Paolo Bonzini
2014-07-30 13:19           ` Frederic Konrad
2014-07-30 13:35             ` Paolo Bonzini
2014-07-30 14:51               ` Frederic Konrad
2014-07-31 13:05                 ` Frederic Konrad
2014-07-31 14:18                   ` Paolo Bonzini
2014-07-31  5:44           ` Pavel Dovgaluk

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).