[Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Pavel Dovgalyuk]

This set of patches is related to the reverse execution and deterministic 
replay of qemu execution  Our implementation of deterministic replay can 
be used for deterministic and reverse debugging of guest code through gdb 
remote interface.

Execution recording writes non-deterministic events log, which can be later 
used for replaying the execution anywhere and for unlimited number of times. 
It also supports checkpointing for faster rewinding during reverse debugging. 
Execution replaying reads the log and replays all non-deterministic events 
including external input, hardware clocks, and interrupts.

Reverse execution has the following features:
 * Deterministically replays whole system execution and all contents of the memory,
   state of the hadrware devices, clocks, and screen of the VM.
 * Writes execution log into the file for latter replaying for multiple times 
   on different machines.
 * Supports i386, x86_64, and ARM hardware platforms.
 * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
   audio devices, serial interfaces, and physical USB devices connected to the emulator.
 * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
 * Supports auto-checkpointing for convenient reverse debugging.

Usage of the record/replay:
 * First, record the execution, by adding '-record fname=replay.bin' to the
   command line.
 * Then you can replay it for the multiple times by using another command
   line option: '-replay fname=replay.bin'
 * Virtual machine should have at least one virtual disk, which is used to
   store checkpoints. If you want to enable automatic checkpointing, simply
   add ',period=XX' to record options, where XX is the checkpointing period
   in seconds.
 * Using of the network adapters in record/replay mode is possible with 
   the following command-line options:
   - '-net user' (or another host adapter) in record mode
   - '-net replay' in replay mode. Every host network adapter should be
     replaced by 'replay' when replaying the execution.
 * Reverse debugging can be used through gdb remote interface.
   reverse-stepi and reverse-continue commands are supported. Other reverse
   commands should also work, because they reuse these ones.
 * Monitor is extended by the following commands:
   - replay_info - prints information about replay mode and current step
     (number of instructions executed)
   - replay_break - sets "breakpoint" at the specified instructions count.
   - replay_seek - rewinds (using the checkpoints, if possible) to the
     specified step of replay log.

Paper with short description of deterministic replay implementation:
http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html

Modifications of qemu include:
 * adding missed fields of the virtual devices' states to the vmstate 
   structures to allow deterministic saving and restoring the VM state
 * adding virtual clock-based timers to vmstate structures, because virtual 
   clock is the part of the virtual machine state
 * modification of block layer to support automatic creation of the overlay
   files to store the changes and snapshots while recording
 * disabling of system reset while loading VM state to avoid generating of
   interrupts by reset handlers
 * adding warpers for clock and time functions to save their return
   values in the log
 * saving different asynchronous events (e.g. system shutdown) into the log
 * synchronization of the bottom halves execution
 * synchronization of the threads from thread pool
 * recording/replaying user input (mouse and keyboard), input from virtual
   serial ports, incoming network packets, input from connected USB devices
 * adding HMP/QMP commands to monitor for controlling replay execution

v2 changes:
 * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
 * Added QMP versions of replay commands (as suggested by Eric Blake)
 * Removed some optional features of replay to make patches cleaner
 * Minor changes and code cleanup were made

---

Pavel Dovgalyuk (49):
      acpi: accurate overflow check
      integratorcp: adding vmstate for save/restore
      pcspk: adding vmstate for save/restore
      fdc: adding vmstate for save/restore
      parallel: adding vmstate for save/restore
      serial: fixing vmstate for save/restore
      kvmapic: fixing loading vmstate
      hpet: fixing saving and loading process
      pckbd: adding new fields to vmstate
      rtl8139: adding new fields to vmstate
      piix: do not raise irq while loading vmstate
      mc146818rtc: add missed field to vmstate
      pl031: add missed field to vmstate
      ide pci: reset status field before loading the vmstate
      softmmu: fixing usage of cpu_st/ld* from helpers
      target: save cpu state fields
      target-i386: update fp status fix
      migration: add vmstate for int8 and char arrays
      replay: global variables and function stubs
      block: add suffix parameter to bdrv_open functions
      sysemu: system functions for replay
      replay: internal functions for replay log
      cpu: invent instruction count for accurate replay
      target-arm: instructions counting code for replay
      target-i386: instructions counting code for replay
      replay: interrupts and exceptions
      vga: do not use virtual clock for blinking cursor
      replay: asynchronous events infrastructure
      replay: recording and replaying clock ticks
      replay: recording and replaying different timers
      replay: shutdown event
      replay: checkpoints
      replay: bottom halves
      replay: replay aio requests
      replay: thread pool
      pl031: vmstate in replay mode
      replay: initialization and deinitialization
      replay: command line options
      replay: snapshotting the virtual machine
      replay: recording of the user input
      tap-win32: destroy the thread at exit
      replay: network packets record/replay
      replay: audio data record/replay
      replay: serial port
      replay: USB passthrough
      replay: replay_info command
      replay: replay_break command
      replay: replay_seek_step command
      gdbstub: reverse debugging


 Makefile.target                  |    1 
 arch_init.c                      |    8 
 async.c                          |   45 ++
 audio/audio.c                    |   14 +
 audio/audio_win_int.h            |    3 
 audio/winwaveaudio.c             |  167 +++++++--
 block.c                          |  134 ++++++-
 block/blkdebug.c                 |    2 
 block/blkverify.c                |    4 
 block/cow.c                      |    2 
 block/qcow.c                     |    2 
 block/qcow2.c                    |   10 -
 block/qed.c                      |    2 
 block/raw-posix.c                |    6 
 block/raw-win32.c                |    4 
 block/sheepdog.c                 |    4 
 block/vmdk.c                     |    8 
 block/vvfat.c                    |    2 
 blockdev.c                       |   11 -
 cpu-exec.c                       |   34 +-
 cpus.c                           |   81 ++++
 dma-helpers.c                    |   10 -
 exec.c                           |   12 +
 gdbstub.c                        |   79 +++-
 hmp-commands.hx                  |   41 ++
 hw/acpi/core.c                   |    7 
 hw/arm/integratorcp.c            |   38 ++
 hw/audio/pcspk.c                 |   19 +
 hw/block/fdc.c                   |   11 -
 hw/block/virtio-blk.c            |   10 -
 hw/char/parallel.c               |   22 +
 hw/char/serial.c                 |  115 ++++--
 hw/display/vga.c                 |    4 
 hw/i386/kvmvapic.c               |   22 +
 hw/ide/ahci.c                    |    4 
 hw/ide/atapi.c                   |    9 
 hw/ide/core.c                    |   18 +
 hw/ide/pci.c                     |   10 +
 hw/input/pckbd.c                 |    3 
 hw/intc/apic_common.c            |    5 
 hw/net/rtl8139.c                 |    5 
 hw/pci-host/piix.c               |   22 +
 hw/timer/arm_timer.c             |    2 
 hw/timer/hpet.c                  |   13 -
 hw/timer/mc146818rtc.c           |    5 
 hw/timer/pl031.c                 |   35 +-
 hw/usb/hcd-uhci.c                |    2 
 hw/usb/host-libusb.c             |  525 ++++++++++++++++++---------
 include/block/aio.h              |   17 +
 include/block/block.h            |   21 +
 include/block/thread-pool.h      |    4 
 include/exec/cpu-defs.h          |    1 
 include/exec/cpu_ldst_template.h |   28 +
 include/exec/exec-all.h          |   31 ++
 include/hw/host-libusb.h         |  105 +++++
 include/migration/vmstate.h      |   13 +
 include/qemu-common.h            |    3 
 include/qemu/main-loop.h         |    1 
 include/qemu/timer.h             |   48 ++
 include/qom/cpu.h                |    4 
 include/sysemu/char.h            |   25 +
 include/sysemu/cpus.h            |    1 
 include/sysemu/sysemu.h          |    2 
 include/ui/input.h               |    2 
 main-loop.c                      |    5 
 monitor.c                        |   49 +++
 net/Makefile.objs                |    1 
 net/clients.h                    |    3 
 net/dump.c                       |    6 
 net/hub.c                        |    1 
 net/net-replay.c                 |   66 +++
 net/net.c                        |    7 
 net/slirp.c                      |   14 +
 net/socket.c                     |   35 ++
 net/tap-win32.c                  |   25 +
 net/tap.c                        |   23 +
 net/vde.c                        |   14 +
 qapi-schema.json                 |   62 +++
 qemu-char.c                      |   55 +++
 qemu-img.c                       |    6 
 qemu-io-cmds.c                   |    2 
 qemu-io.c                        |    4 
 qemu-nbd.c                       |    2 
 qemu-options.hx                  |   28 +
 qemu-timer.c                     |   45 ++
 qmp-commands.hx                  |   60 +++
 replay/Makefile.objs             |   11 +
 replay/replay-audio.c            |  228 ++++++++++++
 replay/replay-char.c             |   99 +++++
 replay/replay-debug.c            |  148 ++++++++
 replay/replay-events.c           |  381 ++++++++++++++++++++
 replay/replay-input.c            |  107 ++++++
 replay/replay-internal.c         |  159 ++++++++
 replay/replay-internal.h         |  224 ++++++++++++
 replay/replay-net.c              |  190 ++++++++++
 replay/replay-qmp.c              |   57 +++
 replay/replay-time.c             |  181 +++++++++
 replay/replay-usb.c              |  188 ++++++++++
 replay/replay.c                  |  735 ++++++++++++++++++++++++++++++++++++++
 replay/replay.h                  |  218 +++++++++++
 savevm.c                         |   32 +-
 slirp/slirp.c                    |    9 
 softmmu_template.h               |   18 +
 stubs/Makefile.objs              |    1 
 stubs/replay.c                   |   42 ++
 target-arm/Makefile.objs         |    1 
 target-arm/helper.h              |    3 
 target-arm/machine.c             |    5 
 target-arm/replay_helper.c       |   38 ++
 target-arm/translate.c           |   62 +++
 target-i386/Makefile.objs        |    1 
 target-i386/cpu.c                |    1 
 target-i386/cpu.h                |    8 
 target-i386/fpu_helper.c         |    5 
 target-i386/helper.h             |    3 
 target-i386/machine.c            |    7 
 target-i386/replay_helper.c      |   38 ++
 target-i386/translate.c          |   91 ++++-
 tests/test-thread-pool.c         |    7 
 thread-pool.c                    |   53 ++-
 trace-events                     |    2 
 translate-all.c                  |   15 +
 ui/input.c                       |   79 +++-
 util/iov.c                       |    4 
 vl.c                             |  108 +++++-
 vmstate.c                        |    6 
 126 files changed, 5433 insertions(+), 543 deletions(-)
 create mode 100755 include/hw/host-libusb.h
 create mode 100755 net/net-replay.c
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay-audio.c
 create mode 100755 replay/replay-char.c
 create mode 100755 replay/replay-debug.c
 create mode 100755 replay/replay-events.c
 create mode 100755 replay/replay-input.c
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h
 create mode 100755 replay/replay-net.c
 create mode 100755 replay/replay-qmp.c
 create mode 100755 replay/replay-time.c
 create mode 100755 replay/replay-usb.c
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c
 create mode 100755 target-arm/replay_helper.c
 create mode 100755 target-i386/replay_helper.c

-- 
Pavel Dovgalyuk

[Qemu-devel] [RFC PATCH v2 01/49] acpi: accurate overflow check

[Pavel Dovgalyuk]

Compare clock in ns, because acpi_pm_tmr_update uses rounded
to ns value instead of ticks.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/acpi/core.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/core.c b/hw/acpi/core.c
index a7368fb..f7272b4 100644
--- a/hw/acpi/core.c
+++ b/hw/acpi/core.c
@@ -376,8 +376,11 @@ static void acpi_notify_wakeup(Notifier *notifier, void *data)
 /* ACPI PM1a EVT */
 uint16_t acpi_pm1_evt_get_sts(ACPIREGS *ar)
 {
-    int64_t d = acpi_pm_tmr_get_clock();
-    if (d >= ar->tmr.overflow_time) {
+    /* Compare ns-clock, not PM timer ticks, because 
+       acpi_pm_tmr_update function uses ns for setting the timer. */
+    int64_t d = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+    if (d >= muldiv64(ar->tmr.overflow_time, 
+                      get_ticks_per_sec(), PM_TIMER_FREQUENCY)) {
         ar->pm1.evt.sts |= ACPI_BITMASK_TIMER_STATUS;
     }
     return ar->pm1.evt.sts;

[Qemu-devel] [RFC PATCH v2 02/49] integratorcp: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the integratorcp device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/arm/integratorcp.c |   38 +++++++++++++++++++++++++++++++++++++-
 1 files changed, 37 insertions(+), 1 deletions(-)

diff --git a/hw/arm/integratorcp.c b/hw/arm/integratorcp.c
index 0e476c3..496f84e 100644
--- a/hw/arm/integratorcp.c
+++ b/hw/arm/integratorcp.c
@@ -42,6 +42,27 @@ typedef struct IntegratorCMState {
     uint32_t fiq_enabled;
 } IntegratorCMState;
 
+static const VMStateDescription vmstate_integratorcm = {
+    .name = "integratorcm",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_UINT32(cm_osc, IntegratorCMState),
+        VMSTATE_UINT32(cm_ctrl, IntegratorCMState),
+        VMSTATE_UINT32(cm_lock, IntegratorCMState),
+        VMSTATE_UINT32(cm_auxosc, IntegratorCMState),
+        VMSTATE_UINT32(cm_sdram, IntegratorCMState),
+        VMSTATE_UINT32(cm_init, IntegratorCMState),
+        VMSTATE_UINT32(cm_flags, IntegratorCMState),
+        VMSTATE_UINT32(cm_nvflags, IntegratorCMState),
+        VMSTATE_UINT32(int_level, IntegratorCMState),
+        VMSTATE_UINT32(irq_enabled, IntegratorCMState),
+        VMSTATE_UINT32(fiq_enabled, IntegratorCMState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static uint8_t integrator_spd[128] = {
    128, 8, 4, 11, 9, 1, 64, 0,  2, 0xa0, 0xa0, 0, 0, 8, 0, 1,
    0xe, 4, 0x1c, 1, 2, 0x20, 0xc0, 0, 0, 0, 0, 0x30, 0x28, 0x30, 0x28, 0x40
@@ -272,7 +293,7 @@ static int integratorcm_init(SysBusDevice *dev)
     sysbus_init_mmio(dev, &s->iomem);
 
     integratorcm_do_remap(s);
-    /* ??? Save/restore.  */
+    vmstate_register(NULL, -1, &vmstate_integratorcm, s);
     return 0;
 }
 
@@ -296,6 +317,20 @@ typedef struct icp_pic_state {
     qemu_irq parent_fiq;
 } icp_pic_state;
 
+
+static const VMStateDescription vmstate_icp_pic = {
+    .name = "icp_pic_state",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_UINT32(level, icp_pic_state),
+        VMSTATE_UINT32(irq_enabled, icp_pic_state),
+        VMSTATE_UINT32(fiq_enabled, icp_pic_state),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static void icp_pic_update(icp_pic_state *s)
 {
     uint32_t flags;
@@ -399,6 +434,7 @@ static int icp_pic_init(SysBusDevice *sbd)
     memory_region_init_io(&s->iomem, OBJECT(s), &icp_pic_ops, s,
                           "icp-pic", 0x00800000);
     sysbus_init_mmio(sbd, &s->iomem);
+    vmstate_register(NULL, -1, &vmstate_icp_pic, s);
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v2 03/49] pcspk: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the PC speaker device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/audio/pcspk.c |   19 +++++++++++++++++--
 1 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/hw/audio/pcspk.c b/hw/audio/pcspk.c
index 1d81bbe..2afcffb 100644
--- a/hw/audio/pcspk.c
+++ b/hw/audio/pcspk.c
@@ -50,8 +50,8 @@ typedef struct {
     unsigned int pit_count;
     unsigned int samples;
     unsigned int play_pos;
-    int data_on;
-    int dummy_refresh_clock;
+    uint8_t data_on;
+    uint8_t dummy_refresh_clock;
 } PCSpkState;
 
 static const char *s_spk = "pcspk";
@@ -163,6 +163,19 @@ static const MemoryRegionOps pcspk_io_ops = {
     },
 };
 
+static const VMStateDescription vmstate_spk = {
+    .name = "pcspk",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        //VMSTATE_UINT32(play_pos, PCSpkState),
+        VMSTATE_UINT8(data_on, PCSpkState),
+        VMSTATE_UINT8(dummy_refresh_clock, PCSpkState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static void pcspk_initfn(Object *obj)
 {
     PCSpkState *s = PC_SPEAKER(obj);
@@ -175,6 +188,8 @@ static void pcspk_realizefn(DeviceState *dev, Error **errp)
     ISADevice *isadev = ISA_DEVICE(dev);
     PCSpkState *s = PC_SPEAKER(dev);
 
+    vmstate_register(NULL, 0, &vmstate_spk, s);
+
     isa_register_ioport(isadev, &s->ioport, s->iobase);
 
     pcspk_state = s;

[Qemu-devel] [RFC PATCH v2 04/49] fdc: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the FDC device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/block/fdc.c |   11 +++++++++--
 1 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 490d127..132310a 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -697,12 +697,17 @@ static const VMStateDescription vmstate_fdrive_media_rate = {
 
 static const VMStateDescription vmstate_fdrive = {
     .name = "fdrive",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(head, FDrive),
         VMSTATE_UINT8(track, FDrive),
         VMSTATE_UINT8(sect, FDrive),
+        VMSTATE_UINT8_V(last_sect, FDrive, 2),
+        VMSTATE_UINT8_V(max_track, FDrive, 2),
+        VMSTATE_UINT16_V(bps, FDrive, 2),
+        VMSTATE_UINT8_V(ro, FDrive, 2),
+        VMSTATE_UINT8_V(perpendicular, FDrive, 2),
         VMSTATE_END_OF_LIST()
     },
     .subsections = (VMStateSubsection[]) {
@@ -736,7 +741,7 @@ static int fdc_post_load(void *opaque, int version_id)
 
 static const VMStateDescription vmstate_fdc = {
     .name = "fdc",
-    .version_id = 2,
+    .version_id = 3,
     .minimum_version_id = 2,
     .pre_save = fdc_pre_save,
     .post_load = fdc_post_load,
@@ -769,6 +774,8 @@ static const VMStateDescription vmstate_fdc = {
         VMSTATE_UINT8_EQUAL(num_floppies, FDCtrl),
         VMSTATE_STRUCT_ARRAY(drives, FDCtrl, MAX_FD, 1,
                              vmstate_fdrive, FDrive),
+        VMSTATE_INT32_V(reset_sensei, FDCtrl, 3),
+        VMSTATE_TIMER_V(result_timer, FDCtrl, 3),
         VMSTATE_END_OF_LIST()
     }
 };

[Qemu-devel] [RFC PATCH v2 05/49] parallel: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the parallel port controller state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/char/parallel.c |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/hw/char/parallel.c b/hw/char/parallel.c
index 7ac90a5..bc34e55 100644
--- a/hw/char/parallel.c
+++ b/hw/char/parallel.c
@@ -477,6 +477,26 @@ static const MemoryRegionPortio isa_parallel_portio_sw_list[] = {
     PORTIO_END_OF_LIST(),
 };
 
+
+static const VMStateDescription vmstate_parallel_isa = {
+    .name = "parallel_isa",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_UINT8(state.dataw, ISAParallelState),
+        VMSTATE_UINT8(state.datar, ISAParallelState),
+        VMSTATE_UINT8(state.status, ISAParallelState),
+        VMSTATE_UINT8(state.control, ISAParallelState),
+        VMSTATE_INT32(state.irq_pending, ISAParallelState),
+        VMSTATE_INT32(state.hw_driver, ISAParallelState),
+        VMSTATE_INT32(state.epp_timeout, ISAParallelState),
+        VMSTATE_INT32(state.it_shift, ISAParallelState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 static void parallel_isa_realizefn(DeviceState *dev, Error **errp)
 {
     static int index;
@@ -518,6 +538,8 @@ static void parallel_isa_realizefn(DeviceState *dev, Error **errp)
                               ? &isa_parallel_portio_hw_list[0]
                               : &isa_parallel_portio_sw_list[0]),
                              s, "parallel");
+
+    vmstate_register(NULL, -1, &vmstate_parallel_isa, isa);
 }
 
 /* Memory mapped interface */

[Qemu-devel] [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore

[Pavel Dovgalyuk]

Some fields were added to VMState by this patch to preserve correct
loading of the serial port controller state.
Updating FCR value while loading was also modified to disable generating
an interrupt by loadvm.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/char/serial.c |  115 ++++++++++++++++++++++++++++++++----------------------
 1 files changed, 69 insertions(+), 46 deletions(-)

diff --git a/hw/char/serial.c b/hw/char/serial.c
index 54180a9..1969723 100644
--- a/hw/char/serial.c
+++ b/hw/char/serial.c
@@ -267,6 +267,61 @@ static gboolean serial_xmit(GIOChannel *chan, GIOCondition cond, void *opaque)
 }
 
 
+/* Setter for FCR.
+   is_load flag means, that value is set while loading VM state
+   and interrupt should not be invoked */
+static void serial_write_fcr(void *opaque, uint32_t val, int is_load)
+{
+    SerialState *s = opaque;
+    val = val & 0xFF;
+
+    if (s->fcr == val)
+        return;
+
+    /* Did the enable/disable flag change? If so, make sure FIFOs get flushed */
+    if ((val ^ s->fcr) & UART_FCR_FE)
+        val |= UART_FCR_XFR | UART_FCR_RFR;
+
+    /* FIFO clear */
+
+    if (val & UART_FCR_RFR) {
+        timer_del(s->fifo_timeout_timer);
+        s->timeout_ipending=0;
+        fifo8_reset(&s->recv_fifo);
+    }
+
+    if (val & UART_FCR_XFR) {
+        fifo8_reset(&s->xmit_fifo);
+    }
+
+    if (val & UART_FCR_FE) {
+        s->iir |= UART_IIR_FE;
+        /* Set recv_fifo trigger Level */
+        switch (val & 0xC0) {
+        case UART_FCR_ITL_1:
+            s->recv_fifo_itl = 1;
+            break;
+        case UART_FCR_ITL_2:
+            s->recv_fifo_itl = 4;
+            break;
+        case UART_FCR_ITL_3:
+            s->recv_fifo_itl = 8;
+            break;
+        case UART_FCR_ITL_4:
+            s->recv_fifo_itl = 14;
+            break;
+        }
+    } else
+        s->iir &= ~UART_IIR_FE;
+
+    /* Set fcr - or at least the bits in it that are supposed to "stick" */
+    s->fcr = val & 0xC9;
+
+    if (!is_load) {
+        serial_update_irq(s);
+    }
+}
+
 static void serial_ioport_write(void *opaque, hwaddr addr, uint64_t val,
                                 unsigned size)
 {
@@ -320,50 +375,7 @@ static void serial_ioport_write(void *opaque, hwaddr addr, uint64_t val,
         }
         break;
     case 2:
-        val = val & 0xFF;
-
-        if (s->fcr == val)
-            break;
-
-        /* Did the enable/disable flag change? If so, make sure FIFOs get flushed */
-        if ((val ^ s->fcr) & UART_FCR_FE)
-            val |= UART_FCR_XFR | UART_FCR_RFR;
-
-        /* FIFO clear */
-
-        if (val & UART_FCR_RFR) {
-            timer_del(s->fifo_timeout_timer);
-            s->timeout_ipending=0;
-            fifo8_reset(&s->recv_fifo);
-        }
-
-        if (val & UART_FCR_XFR) {
-            fifo8_reset(&s->xmit_fifo);
-        }
-
-        if (val & UART_FCR_FE) {
-            s->iir |= UART_IIR_FE;
-            /* Set recv_fifo trigger Level */
-            switch (val & 0xC0) {
-            case UART_FCR_ITL_1:
-                s->recv_fifo_itl = 1;
-                break;
-            case UART_FCR_ITL_2:
-                s->recv_fifo_itl = 4;
-                break;
-            case UART_FCR_ITL_3:
-                s->recv_fifo_itl = 8;
-                break;
-            case UART_FCR_ITL_4:
-                s->recv_fifo_itl = 14;
-                break;
-            }
-        } else
-            s->iir &= ~UART_IIR_FE;
-
-        /* Set fcr - or at least the bits in it that are supposed to "stick" */
-        s->fcr = val & 0xC9;
-        serial_update_irq(s);
+        serial_write_fcr(s, val, 0);
         break;
     case 3:
         {
@@ -591,20 +603,22 @@ static int serial_post_load(void *opaque, int version_id)
         s->fcr_vmstate = 0;
     }
     /* Initialize fcr via setter to perform essential side-effects */
-    serial_ioport_write(s, 0x02, s->fcr_vmstate, 1);
+    serial_write_fcr(s, s->fcr_vmstate, 1);
     serial_update_parameters(s);
     return 0;
 }
 
 const VMStateDescription vmstate_serial = {
     .name = "serial",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 2,
     .pre_save = serial_pre_save,
     .post_load = serial_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT16_V(divider, SerialState, 2),
         VMSTATE_UINT8(rbr, SerialState),
+        VMSTATE_UINT8_V(thr, SerialState, 4),
+        VMSTATE_UINT8_V(tsr, SerialState, 4),
         VMSTATE_UINT8(ier, SerialState),
         VMSTATE_UINT8(iir, SerialState),
         VMSTATE_UINT8(lcr, SerialState),
@@ -613,6 +627,15 @@ const VMStateDescription vmstate_serial = {
         VMSTATE_UINT8(msr, SerialState),
         VMSTATE_UINT8(scr, SerialState),
         VMSTATE_UINT8_V(fcr_vmstate, SerialState, 3),
+        VMSTATE_INT32_V(thr_ipending, SerialState, 4),
+        VMSTATE_INT32_V(last_break_enable, SerialState, 4),
+        VMSTATE_INT32_V(tsr_retry, SerialState, 4),
+        VMSTATE_STRUCT(recv_fifo, SerialState, 4, vmstate_fifo8, Fifo8),
+        VMSTATE_STRUCT(xmit_fifo, SerialState, 4, vmstate_fifo8, Fifo8),
+        VMSTATE_TIMER_V(fifo_timeout_timer, SerialState, 4),
+        VMSTATE_INT32_V(timeout_ipending, SerialState, 4),
+        VMSTATE_INT32_V(poll_msl, SerialState, 4),
+        VMSTATE_TIMER_V(modem_status_poll, SerialState, 4),
         VMSTATE_END_OF_LIST()
     }
 };

[Qemu-devel] [RFC PATCH v2 07/49] kvmapic: fixing loading vmstate

[Pavel Dovgalyuk]

vapic state should not be synchronized with APIC while loading,
because APIC state could be not loaded yet at that moment.
We just save vapic_paddr in APIC VMState instead of synchronization.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/i386/kvmvapic.c    |   22 +++++++++++++++++++++-
 hw/intc/apic_common.c |    5 ++++-
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index cb855c7..417ab6a 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -351,6 +351,24 @@ static int get_kpcr_number(X86CPU *cpu)
     return kpcr.number;
 }
 
+static int vapic_enable_post_load(VAPICROMState *s, X86CPU *cpu)
+{
+    int cpu_number = get_kpcr_number(cpu);
+    hwaddr vapic_paddr;
+    static const uint8_t enabled = 1;
+
+    if (cpu_number < 0) {
+        return -1;
+    }
+    vapic_paddr = s->vapic_paddr +
+        (((hwaddr)cpu_number) << VAPIC_CPU_SHIFT);
+    cpu_physical_memory_rw(vapic_paddr + offsetof(VAPICState, enabled),
+                           (void *)&enabled, sizeof(enabled), 1);
+    s->state = VAPIC_ACTIVE;
+
+    return 0;
+}
+
 static int vapic_enable(VAPICROMState *s, X86CPU *cpu)
 {
     int cpu_number = get_kpcr_number(cpu);
@@ -731,7 +749,9 @@ static void do_vapic_enable(void *data)
     VAPICROMState *s = data;
     X86CPU *cpu = X86_CPU(first_cpu);
 
-    vapic_enable(s, cpu);
+    /* Do not synchronize with APIC, because it was not loaded yet.
+       Just call the enable function which does not have synchronization. */
+    vapic_enable_post_load(s, cpu);
 }
 
 static int vapic_post_load(void *opaque, int version_id)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index ce3d903..9d75ee0 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -347,7 +347,7 @@ static int apic_dispatch_post_load(void *opaque, int version_id)
 
 static const VMStateDescription vmstate_apic_common = {
     .name = "apic",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = apic_load_old,
@@ -374,6 +374,9 @@ static const VMStateDescription vmstate_apic_common = {
         VMSTATE_INT64(next_time, APICCommonState),
         VMSTATE_INT64(timer_expiry,
                       APICCommonState), /* open-coded timer state */
+        VMSTATE_INT32_V(sipi_vector, APICCommonState, 4),
+        VMSTATE_INT32_V(wait_for_sipi, APICCommonState, 4),
+        VMSTATE_UINT64_V(vapic_paddr, APICCommonState, 4),
         VMSTATE_END_OF_LIST()
     }
 };

[Qemu-devel] [RFC PATCH v2 08/49] hpet: fixing saving and loading process

[Pavel Dovgalyuk]

VM clock does not run while saving, so there is no need for saving the ticks
in HPET. Also added saving of hpet_offset field.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/hpet.c |   13 +------------
 1 files changed, 1 insertions(+), 12 deletions(-)

diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c
index e160e8f..73401b9 100644
--- a/hw/timer/hpet.c
+++ b/hw/timer/hpet.c
@@ -222,14 +222,6 @@ static void update_irq(struct HPETTimer *timer, int set)
     }
 }
 
-static void hpet_pre_save(void *opaque)
-{
-    HPETState *s = opaque;
-
-    /* save current counter value */
-    s->hpet_counter = hpet_get_ticks(s);
-}
-
 static int hpet_pre_load(void *opaque)
 {
     HPETState *s = opaque;
@@ -255,9 +247,6 @@ static int hpet_post_load(void *opaque, int version_id)
 {
     HPETState *s = opaque;
 
-    /* Recalculate the offset between the main counter and guest time */
-    s->hpet_offset = ticks_to_ns(s->hpet_counter) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-
     /* Push number of timers into capability returned via HPET_ID */
     s->capability &= ~HPET_ID_NUM_TIM_MASK;
     s->capability |= (s->num_timers - 1) << HPET_ID_NUM_TIM_SHIFT;
@@ -308,13 +297,13 @@ static const VMStateDescription vmstate_hpet = {
     .name = "hpet",
     .version_id = 2,
     .minimum_version_id = 1,
-    .pre_save = hpet_pre_save,
     .pre_load = hpet_pre_load,
     .post_load = hpet_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT64(config, HPETState),
         VMSTATE_UINT64(isr, HPETState),
         VMSTATE_UINT64(hpet_counter, HPETState),
+        VMSTATE_UINT64(hpet_offset, HPETState),
         VMSTATE_UINT8_V(num_timers, HPETState, 2),
         VMSTATE_VALIDATE("num_timers in range", hpet_validate_num_timers),
         VMSTATE_STRUCT_VARRAY_UINT8(timer, HPETState, num_timers, 0,

[Qemu-devel] [RFC PATCH v2 09/49] pckbd: adding new fields to vmstate

[Pavel Dovgalyuk]

This patch adds outport to VMState to allow correct saving and restoring
the state of PC keyboard controller.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/input/pckbd.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/hw/input/pckbd.c b/hw/input/pckbd.c
index ca1cffc..19f6658 100644
--- a/hw/input/pckbd.c
+++ b/hw/input/pckbd.c
@@ -371,13 +371,14 @@ static void kbd_reset(void *opaque)
 
 static const VMStateDescription vmstate_kbd = {
     .name = "pckbd",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(write_cmd, KBDState),
         VMSTATE_UINT8(status, KBDState),
         VMSTATE_UINT8(mode, KBDState),
         VMSTATE_UINT8(pending, KBDState),
+        VMSTATE_UINT8_V(outport, KBDState, 4),
         VMSTATE_END_OF_LIST()
     }
 };

[Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Pavel Dovgalyuk]

This patch adds virtual clock-dependent timers to VMState to allow correct
saving and restoring the state of RTL8139 network controller.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/net/rtl8139.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c
index 90bc5ec..992caf0 100644
--- a/hw/net/rtl8139.c
+++ b/hw/net/rtl8139.c
@@ -3289,7 +3289,7 @@ static void rtl8139_pre_save(void *opaque)
 
 static const VMStateDescription vmstate_rtl8139 = {
     .name = "rtl8139",
-    .version_id = 4,
+    .version_id = 5,
     .minimum_version_id = 3,
     .post_load = rtl8139_post_load,
     .pre_save  = rtl8139_pre_save,
@@ -3363,6 +3363,9 @@ static const VMStateDescription vmstate_rtl8139 = {
         VMSTATE_STRUCT(tally_counters, RTL8139State, 0,
                        vmstate_tally_counters, RTL8139TallyCounters),
 
+        VMSTATE_TIMER_V(timer, RTL8139State, 5),
+        VMSTATE_INT64_V(TimerExpire, RTL8139State, 5),
+
         VMSTATE_UINT32_V(cplus_enabled, RTL8139State, 4),
         VMSTATE_END_OF_LIST()
     },

[Qemu-devel] [RFC PATCH v2 11/49] piix: do not raise irq while loading vmstate

[Pavel Dovgalyuk]

This patch disables raising an irq while loading the state of PCI bridge.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/pci-host/piix.c |   22 ++++++++++++++++++++--
 1 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
index e0e0946..86d6d20 100644
--- a/hw/pci-host/piix.c
+++ b/hw/pci-host/piix.c
@@ -409,7 +409,7 @@ static void piix3_set_irq_pic(PIIX3State *piix3, int pic_irq)
                      (pic_irq * PIIX_NUM_PIRQS))));
 }
 
-static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
+static void piix3_set_irq_level_internal(PIIX3State *piix3, int pirq, int level)
 {
     int pic_irq;
     uint64_t mask;
@@ -422,6 +422,18 @@ static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
     mask = 1ULL << ((pic_irq * PIIX_NUM_PIRQS) + pirq);
     piix3->pic_levels &= ~mask;
     piix3->pic_levels |= mask * !!level;
+}
+
+static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
+{
+    int pic_irq;
+
+    pic_irq = piix3->dev.config[PIIX_PIRQC + pirq];
+    if (pic_irq >= PIIX_NUM_PIC_IRQS) {
+        return;
+    }
+
+    piix3_set_irq_level_internal(piix3, pirq, level);
 
     piix3_set_irq_pic(piix3, pic_irq);
 }
@@ -527,7 +539,13 @@ static void piix3_reset(void *opaque)
 static int piix3_post_load(void *opaque, int version_id)
 {
     PIIX3State *piix3 = opaque;
-    piix3_update_irq_levels(piix3);
+    int pirq;
+
+    piix3->pic_levels = 0;
+    for (pirq = 0; pirq < PIIX_NUM_PIRQS; pirq++) {
+        piix3_set_irq_level_internal(piix3, pirq,
+                            pci_bus_get_irq_level(piix3->dev.bus, pirq));
+    }
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v2 12/49] mc146818rtc: add missed field to vmstate

[Pavel Dovgalyuk]

This patch adds irq_reinject_on_ack_count field to VMState to allow correct
saving/loading the state of MC146818 RTC.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/mc146818rtc.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index 9d817ca..c204abb 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -735,7 +735,7 @@ static int rtc_post_load(void *opaque, int version_id)
 
 static const VMStateDescription vmstate_rtc = {
     .name = "mc146818rtc",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 1,
     .post_load = rtc_post_load,
     .fields = (VMStateField[]) {
@@ -752,6 +752,7 @@ static const VMStateDescription vmstate_rtc = {
         VMSTATE_INT64_V(offset, RTCState, 3),
         VMSTATE_TIMER_V(update_timer, RTCState, 3),
         VMSTATE_UINT64_V(next_alarm_time, RTCState, 3),
+        VMSTATE_UINT16_V(irq_reinject_on_ack_count, RTCState, 4),
         VMSTATE_END_OF_LIST()
     }
 };

[Qemu-devel] [RFC PATCH v2 13/49] pl031: add missed field to vmstate

[Pavel Dovgalyuk]

This patch adds timer which uses virtual clock to the VMState.
Such timers are required for saving because virtual clock is the part
of the virtual machine state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/pl031.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index 34d9b44..f8e5abc 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -230,12 +230,13 @@ static int pl031_post_load(void *opaque, int version_id)
 
 static const VMStateDescription vmstate_pl031 = {
     .name = "pl031",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
     .pre_save = pl031_pre_save,
     .post_load = pl031_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT32(tick_offset_vmstate, PL031State),
+        VMSTATE_TIMER_V(timer, PL031State, 2),
         VMSTATE_UINT32(mr, PL031State),
         VMSTATE_UINT32(lr, PL031State),
         VMSTATE_UINT32(cr, PL031State),

[Qemu-devel] [RFC PATCH v2 14/49] ide pci: reset status field before loading the vmstate

[Pavel Dovgalyuk]

This patch resets status field before loading the state of IDE BMDMA device.
Resetting status is needed for replay, because it does not reset whole virtual
machine before loading state of the VM.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/ide/pci.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/hw/ide/pci.c b/hw/ide/pci.c
index 6257a21..47ab24d 100644
--- a/hw/ide/pci.c
+++ b/hw/ide/pci.c
@@ -382,6 +382,15 @@ static bool ide_bmdma_status_needed(void *opaque)
     return ((bm->status & abused_bits) != 0);
 }
 
+static int ide_bmdma_pre_load(void *opaque)
+{
+    BMDMAState *bm = opaque;
+    /* Reset is not performed in replay mode,
+       so reset status manually to allow ide_bmdma_post_load function initialize it. */
+    bm->status = 0;
+    return 0;
+}
+
 static void ide_bmdma_pre_save(void *opaque)
 {
     BMDMAState *bm = opaque;
@@ -434,6 +443,7 @@ static const VMStateDescription vmstate_bmdma = {
     .name = "ide bmdma",
     .version_id = 3,
     .minimum_version_id = 0,
+    .pre_load  = ide_bmdma_pre_load,
     .pre_save  = ide_bmdma_pre_save,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(cmd, BMDMAState),

[Qemu-devel] [RFC PATCH v2 15/49] softmmu: fixing usage of cpu_st/ld* from helpers

[Pavel Dovgalyuk]

MMU helper functions are called from generated code and other helper
functions. In both cases they try to get function's return address for
using it while restoring virtual CPU state.

When MMU helper is called from some other helper function
(like helper_maskmov_xmm) through cpu_st* function, the return address
will point to that helper. That is why CPU state cannot be restored in
the case of MMU fault.

This patch introduces several inline helpers to load return address
which points to the right place.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/exec/cpu_ldst_template.h |   28 ++++++++++++++++++++++++----
 include/exec/exec-all.h          |   27 +++++++++++++++++++++++++++
 softmmu_template.h               |   18 ++++++++++++++++++
 3 files changed, 69 insertions(+), 4 deletions(-)

diff --git a/include/exec/cpu_ldst_template.h b/include/exec/cpu_ldst_template.h
index 006093a..10d9f8f 100644
--- a/include/exec/cpu_ldst_template.h
+++ b/include/exec/cpu_ldst_template.h
@@ -61,6 +61,16 @@
 #define MMUSUFFIX _mmu
 #endif
 
+/* inline helper ld function */
+
+static inline DATA_TYPE
+glue(glue(helper_inline_ld, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong addr,
+                                         int mmu_idx)
+{
+    return glue(glue(helper_call_ld, SUFFIX), MMUSUFFIX)(env, addr, mmu_idx,
+                                                         GETRA());
+}
+
 /* generic load/store macros */
 
 static inline RES_TYPE
@@ -76,7 +86,7 @@ glue(glue(cpu_ld, USUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        res = glue(glue(helper_ld, SUFFIX), MMUSUFFIX)(env, addr, mmu_idx);
+        res = glue(glue(helper_inline_ld, SUFFIX), MEMSUFFIX)(env, addr, mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         res = glue(glue(ld, USUFFIX), _raw)(hostaddr);
@@ -97,8 +107,8 @@ glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        res = (DATA_STYPE)glue(glue(helper_ld, SUFFIX),
-                               MMUSUFFIX)(env, addr, mmu_idx);
+        res = (DATA_STYPE)glue(glue(helper_inline_ld, SUFFIX),
+                               MEMSUFFIX)(env, addr, mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         res = glue(glue(lds, SUFFIX), _raw)(hostaddr);
@@ -109,6 +119,16 @@ glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
 
 #ifndef SOFTMMU_CODE_ACCESS
 
+/* inline helper st function */
+
+static inline void
+glue(glue(helper_inline_st, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong addr,
+                                         DATA_TYPE val, int mmu_idx)
+{
+    glue(glue(helper_call_st, SUFFIX), MMUSUFFIX)(env, addr, val, mmu_idx,
+                                                  GETRA());
+}
+
 /* generic store macro */
 
 static inline void
@@ -124,7 +144,7 @@ glue(glue(cpu_st, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr,
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].addr_write !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        glue(glue(helper_st, SUFFIX), MMUSUFFIX)(env, addr, v, mmu_idx);
+        glue(glue(helper_inline_st, SUFFIX), MEMSUFFIX)(env, addr, v, mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         glue(glue(st, SUFFIX), _raw)(hostaddr, v);
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 5e5d86e..528928f 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -344,6 +344,33 @@ bool io_mem_write(struct MemoryRegion *mr, hwaddr addr,
 void tlb_fill(CPUState *cpu, target_ulong addr, int is_write, int mmu_idx,
               uintptr_t retaddr);
 
+uint8_t helper_call_ldb_cmmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint16_t helper_call_ldw_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+uint32_t helper_call_ldl_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+uint64_t helper_call_ldq_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+
+uint8_t helper_call_ldb_mmu(CPUArchState *env, target_ulong addr,
+                            int mmu_idx, uintptr_t retaddr);
+uint16_t helper_call_ldw_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint32_t helper_call_ldl_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint64_t helper_call_ldq_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+
+void helper_call_stb_mmu(CPUArchState *env, target_ulong addr,
+                         uint8_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stw_mmu(CPUArchState *env, target_ulong addr,
+                         uint16_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stl_mmu(CPUArchState *env, target_ulong addr,
+                         uint32_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stq_mmu(CPUArchState *env, target_ulong addr,
+                         uint64_t val, int mmu_idx, uintptr_t retaddr);
+
 #endif
 
 #if defined(CONFIG_USER_ONLY)
diff --git a/softmmu_template.h b/softmmu_template.h
index 5a07f99..1053cf3 100644
--- a/softmmu_template.h
+++ b/softmmu_template.h
@@ -311,6 +311,15 @@ glue(glue(helper_ld, SUFFIX), MMUSUFFIX)(CPUArchState *env, target_ulong addr,
     return helper_te_ld_name (env, addr, mmu_idx, GETRA());
 }
 
+DATA_TYPE
+glue(glue(helper_call_ld, SUFFIX), MMUSUFFIX)(CPUArchState *env,
+                                              target_ulong addr,
+                                              int mmu_idx,
+                                              uintptr_t retaddr)
+{
+    return helper_te_ld_name(env, addr, mmu_idx, retaddr);
+}
+
 #ifndef SOFTMMU_CODE_ACCESS
 
 /* Provide signed versions of the load routines as well.  We can of course
@@ -505,6 +514,15 @@ glue(glue(helper_st, SUFFIX), MMUSUFFIX)(CPUArchState *env, target_ulong addr,
     helper_te_st_name(env, addr, val, mmu_idx, GETRA());
 }
 
+void
+glue(glue(helper_call_st, SUFFIX), MMUSUFFIX)(CPUArchState *env,
+                                              target_ulong addr,
+                                              DATA_TYPE val, int mmu_idx,
+                                              uintptr_t retaddr)
+{
+    helper_te_st_name(env, addr, val, mmu_idx, retaddr);
+}
+
 #endif /* !defined(SOFTMMU_CODE_ACCESS) */
 
 #undef READ_ACCESS_TYPE

[Qemu-devel] [RFC PATCH v2 16/49] target: save cpu state fields

[Pavel Dovgalyuk]

This patch adds interrupt fields to VMState for correct saving the CPU state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-arm/machine.c  |    5 ++++-
 target-i386/machine.c |    5 ++++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/target-arm/machine.c b/target-arm/machine.c
index 3bcc7cc..29bfc10 100644
--- a/target-arm/machine.c
+++ b/target-arm/machine.c
@@ -218,7 +218,7 @@ static int cpu_post_load(void *opaque, int version_id)
 
 const VMStateDescription vmstate_arm_cpu = {
     .name = "cpu",
-    .version_id = 20,
+    .version_id = 21,
     .minimum_version_id = 20,
     .pre_save = cpu_pre_save,
     .post_load = cpu_post_load,
@@ -259,6 +259,9 @@ const VMStateDescription vmstate_arm_cpu = {
         VMSTATE_UINT64(env.exception.vaddress, ARMCPU),
         VMSTATE_TIMER(gt_timer[GTIMER_PHYS], ARMCPU),
         VMSTATE_TIMER(gt_timer[GTIMER_VIRT], ARMCPU),
+        /* Fields required by replay */
+        VMSTATE_UINT32_V(parent_obj.interrupt_request, ARMCPU, 21),
+        VMSTATE_INT32_V(parent_obj.exception_index, ARMCPU, 21),
         VMSTATE_END_OF_LIST()
     },
     .subsections = (VMStateSubsection[]) {
diff --git a/target-i386/machine.c b/target-i386/machine.c
index 16d2f6a..9dfac33 100644
--- a/target-i386/machine.c
+++ b/target-i386/machine.c
@@ -605,7 +605,7 @@ static const VMStateDescription vmstate_msr_hyperv_time = {
 
 VMStateDescription vmstate_x86_cpu = {
     .name = "cpu",
-    .version_id = 12,
+    .version_id = 13,
     .minimum_version_id = 3,
     .pre_save = cpu_pre_save,
     .post_load = cpu_post_load,
@@ -702,6 +702,9 @@ VMStateDescription vmstate_x86_cpu = {
         VMSTATE_UINT64_V(env.xcr0, X86CPU, 12),
         VMSTATE_UINT64_V(env.xstate_bv, X86CPU, 12),
         VMSTATE_YMMH_REGS_VARS(env.ymmh_regs, X86CPU, CPU_NB_REGS, 12),
+        /* Fields required by replay */
+        VMSTATE_UINT32_V(parent_obj.interrupt_request, X86CPU, 13),
+        VMSTATE_INT32_V(parent_obj.exception_index, X86CPU, 13),
         VMSTATE_END_OF_LIST()
         /* The above list is not sorted /wrt version numbers, watch out! */
     },

[Qemu-devel] [RFC PATCH v2 17/49] target-i386: update fp status fix

[Pavel Dovgalyuk]

This patch adds calls to update_fp_status() function from several
places where FPU state is changed.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-i386/cpu.c        |    1 +
 target-i386/cpu.h        |    1 +
 target-i386/fpu_helper.c |    5 ++++-
 target-i386/machine.c    |    2 +-
 4 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 45c662d..27269ad 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2573,6 +2573,7 @@ static void x86_cpu_reset(CPUState *s)
         env->fptags[i] = 1;
     }
     env->fpuc = 0x37f;
+    update_fp_status(env);
 
     env->mxcsr = 0x1f80;
     env->xstate_bv = XSTATE_FP | XSTATE_SSE;
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index e634d83..42bda46 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1249,6 +1249,7 @@ void QEMU_NORETURN raise_interrupt(CPUX86State *nenv, int intno, int is_int,
 /* cc_helper.c */
 extern const uint8_t parity_table[256];
 uint32_t cpu_cc_compute_all(CPUX86State *env1, int op);
+void update_fp_status(CPUX86State *env);
 
 static inline uint32_t cpu_compute_eflags(CPUX86State *env)
 {
diff --git a/target-i386/fpu_helper.c b/target-i386/fpu_helper.c
index 1b2900d..a8ffba9 100644
--- a/target-i386/fpu_helper.c
+++ b/target-i386/fpu_helper.c
@@ -537,7 +537,7 @@ uint32_t helper_fnstcw(CPUX86State *env)
     return env->fpuc;
 }
 
-static void update_fp_status(CPUX86State *env)
+void update_fp_status(CPUX86State *env)
 {
     int rnd_type;
 
@@ -1006,6 +1006,7 @@ void helper_fstenv(CPUX86State *env, target_ulong ptr, int data32)
         cpu_stw_data(env, ptr + 10, 0);
         cpu_stw_data(env, ptr + 12, 0);
     }
+    update_fp_status(env);
 }
 
 void helper_fldenv(CPUX86State *env, target_ulong ptr, int data32)
@@ -1055,6 +1056,7 @@ void helper_fsave(CPUX86State *env, target_ulong ptr, int data32)
     env->fptags[5] = 1;
     env->fptags[6] = 1;
     env->fptags[7] = 1;
+    update_fp_status(env);
 }
 
 void helper_frstor(CPUX86State *env, target_ulong ptr, int data32)
@@ -1158,6 +1160,7 @@ void helper_fxrstor(CPUX86State *env, target_ulong ptr, int data64)
     }
 
     env->fpuc = cpu_lduw_data(env, ptr);
+    update_fp_status(env);
     fpus = cpu_lduw_data(env, ptr + 2);
     fptag = cpu_lduw_data(env, ptr + 4);
     env->fpstt = (fpus >> 11) & 7;
diff --git a/target-i386/machine.c b/target-i386/machine.c
index 9dfac33..b8f3467 100644
--- a/target-i386/machine.c
+++ b/target-i386/machine.c
@@ -315,13 +315,13 @@ static int cpu_post_load(void *opaque, int version_id)
     env->hflags &= ~HF_CPL_MASK;
     env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK;
 
-    /* XXX: restore FPU round state */
     env->fpstt = (env->fpus_vmstate >> 11) & 7;
     env->fpus = env->fpus_vmstate & ~0x3800;
     env->fptag_vmstate ^= 0xff;
     for(i = 0; i < 8; i++) {
         env->fptags[i] = (env->fptag_vmstate >> i) & 1;
     }
+    update_fp_status(env);
 
     cpu_breakpoint_remove_all(cs, BP_CPU);
     cpu_watchpoint_remove_all(cs, BP_CPU);

[Qemu-devel] [RFC PATCH v2 18/49] migration: add vmstate for int8 and char arrays

[Pavel Dovgalyuk]

This patch adds macros for vmstate int8 and char arrays.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/migration/vmstate.h |   13 +++++++++++++
 vmstate.c                   |    6 ++++++
 2 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 9a001bd..2f7207b 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -143,6 +143,7 @@ extern const VMStateDescription vmstate_dummy;
 #endif
 
 extern const VMStateInfo vmstate_info_bool;
+extern const VMStateInfo vmstate_info_char;
 
 extern const VMStateInfo vmstate_info_int8;
 extern const VMStateInfo vmstate_info_int16;
@@ -694,6 +695,18 @@ extern const VMStateInfo vmstate_info_bitmap;
 #define VMSTATE_UINT64_ARRAY(_f, _s, _n)                              \
     VMSTATE_UINT64_ARRAY_V(_f, _s, _n, 0)
 
+#define VMSTATE_INT8_ARRAY_V(_f, _s, _n, _v)                          \
+    VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_int8, int8_t)
+
+#define VMSTATE_INT8_ARRAY(_f, _s, _n)                                \
+    VMSTATE_INT8_ARRAY_V(_f, _s, _n, 0)
+
+#define VMSTATE_CHAR_ARRAY_V(_f, _s, _n, _v)                          \
+    VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_char, char)
+
+#define VMSTATE_CHAR_ARRAY(_f, _s, _n)                                \
+    VMSTATE_CHAR_ARRAY_V(_f, _s, _n, 0)
+
 #define VMSTATE_INT16_ARRAY_V(_f, _s, _n, _v)                         \
     VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_int16, int16_t)
 
diff --git a/vmstate.c b/vmstate.c
index ef2f87b..6a7f7c2 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -286,6 +286,12 @@ const VMStateInfo vmstate_info_int8 = {
     .put  = put_int8,
 };
 
+const VMStateInfo vmstate_info_char = {
+    .name = "char",
+    .get  = get_int8,
+    .put  = put_int8,
+};
+
 /* 16 bit int */
 
 static int get_int16(QEMUFile *f, void *pv, size_t size)

[Qemu-devel] [RFC PATCH v2 19/49] replay: global variables and function stubs

[Pavel Dovgalyuk]

This patch adds global variables, defines, functions declarations,
and function stubs for deterministic VM replay used by external modules.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 Makefile.target      |    1 +
 replay/Makefile.objs |    1 +
 replay/replay.c      |   19 +++++++++++++++++++
 replay/replay.h      |   32 ++++++++++++++++++++++++++++++++
 stubs/Makefile.objs  |    1 +
 stubs/replay.c       |    9 +++++++++
 6 files changed, 63 insertions(+), 0 deletions(-)
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c

diff --git a/Makefile.target b/Makefile.target
index 137d0b0..9830313 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -76,6 +76,7 @@ all: $(PROGS) stap
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
+obj-y += replay/
 obj-y += tcg/tcg.o tcg/optimize.o
 obj-$(CONFIG_TCG_INTERPRETER) += tci.o
 obj-$(CONFIG_TCG_INTERPRETER) += disas/tci.o
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
new file mode 100755
index 0000000..7ea860f
--- /dev/null
+++ b/replay/Makefile.objs
@@ -0,0 +1 @@
+obj-y += replay.o
diff --git a/replay/replay.c b/replay/replay.c
new file mode 100755
index 0000000..5a69465
--- /dev/null
+++ b/replay/replay.c
@@ -0,0 +1,19 @@
+/*
+ * replay.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+
+int replay_mode = REPLAY_NONE;
+/*! Stores current submode for PLAY mode */
+int play_submode = REPLAY_PLAY_UNKNOWN;
+
+/* Suffix for the disk images filenames */
+char *replay_image_suffix;
diff --git a/replay/replay.h b/replay/replay.h
new file mode 100755
index 0000000..a883aa4
--- /dev/null
+++ b/replay/replay.h
@@ -0,0 +1,32 @@
+#ifndef REPLAY_H
+#define REPLAY_H
+
+/*
+ * replay.h
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+/* replay modes */
+#define REPLAY_NONE 0
+#define REPLAY_SAVE 1
+#define REPLAY_PLAY 2
+/* replay submodes */
+#define REPLAY_PLAY_UNKNOWN 0
+/* Normal log replaying */
+#define REPLAY_PLAY_NORMAL  1
+
+extern int replay_mode;
+extern char *replay_image_suffix;
+/*! Shift value for icount based on replay or zero, if it is disabled. */
+extern int replay_icount;
+
+/*! Returns replay play submode */
+int replay_get_play_submode(void);
+
+#endif
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 528e161..b90c60a 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -26,6 +26,7 @@ stub-obj-y += notify-event.o
 stub-obj-y += pci-drive-hot-add.o
 stub-obj-$(CONFIG_SPICE) += qemu-chr-open-spice.o
 stub-obj-y += qtest.o
+stub-obj-y += replay.o
 stub-obj-y += reset.o
 stub-obj-y += runstate-check.o
 stub-obj-y += set-fd-handler.o
diff --git a/stubs/replay.c b/stubs/replay.c
new file mode 100755
index 0000000..1e99ca4
--- /dev/null
+++ b/stubs/replay.c
@@ -0,0 +1,9 @@
+#include "replay/replay.h"
+
+int replay_mode;
+int replay_icount;
+
+int replay_get_play_submode(void)
+{
+    return 0;
+}

[Qemu-devel] [RFC PATCH v2 20/49] block: add suffix parameter to bdrv_open functions

[Pavel Dovgalyuk]

This patch adds parameter with suffix for overlay filenames. This parameter
will be used by replay to automatically create overlay files based on
drive images supplied to VM.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c               |   44 ++++++++++++++++++++++++++++++--------------
 block/blkdebug.c      |    2 +-
 block/blkverify.c     |    4 ++--
 block/cow.c           |    2 +-
 block/qcow.c          |    2 +-
 block/qcow2.c         |    6 +++---
 block/qed.c           |    2 +-
 block/sheepdog.c      |    4 ++--
 block/vmdk.c          |    8 ++++----
 block/vvfat.c         |    2 +-
 blockdev.c            |   11 ++++++-----
 include/block/block.h |    6 +++---
 qemu-img.c            |    6 +++---
 qemu-io.c             |    4 ++--
 qemu-nbd.c            |    2 +-
 15 files changed, 61 insertions(+), 44 deletions(-)

diff --git a/block.c b/block.c
index 8800a6b..3ff1052 100644
--- a/block.c
+++ b/block.c
@@ -35,6 +35,7 @@
 #include "qmp-commands.h"
 #include "qemu/timer.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #ifdef CONFIG_BSD
 #include <sys/types.h>
@@ -1209,7 +1210,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     assert(bs->backing_hd == NULL);
     ret = bdrv_open(&backing_hd,
                     *backing_filename ? backing_filename : NULL, NULL, options,
-                    bdrv_backing_flags(bs->open_flags), back_drv, &local_err);
+                    bdrv_backing_flags(bs->open_flags), back_drv, &local_err, NULL);
     if (ret < 0) {
         bdrv_unref(backing_hd);
         backing_hd = NULL;
@@ -1244,7 +1245,7 @@ free_exit:
  */
 int bdrv_open_image(BlockDriverState **pbs, const char *filename,
                     QDict *options, const char *bdref_key, int flags,
-                    bool allow_none, Error **errp)
+                    bool allow_none, Error **errp, const char *fname_suffix)
 {
     QDict *image_options;
     int ret;
@@ -1271,14 +1272,14 @@ int bdrv_open_image(BlockDriverState **pbs, const char *filename,
         goto done;
     }
 
-    ret = bdrv_open(pbs, filename, reference, image_options, flags, NULL, errp);
+    ret = bdrv_open(pbs, filename, reference, image_options, flags, NULL, errp, NULL);
 
 done:
     qdict_del(options, bdref_key);
     return ret;
 }
 
-int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
+int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp, const char *fname_suffix)
 {
     /* TODO: extra byte is a hack to ensure MAX_PATH space on Windows. */
     char *tmp_filename = g_malloc0(PATH_MAX + 1);
@@ -1303,17 +1304,29 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
     total_size &= BDRV_SECTOR_MASK;
 
     /* Create the temporary image */
-    ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
-    if (ret < 0) {
-        error_setg_errno(errp, -ret, "Could not get temporary filename");
-        goto out;
+    /* We will not delete created file in record/replay mode */
+    if (replay_mode != REPLAY_NONE && fname_suffix) {
+        ret = snprintf(tmp_filename, PATH_MAX + 1, "%s.%s", bs->filename, fname_suffix);
+        if (ret < 0) {
+            goto out;
+        }
+    } else {
+        ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Could not get temporary filename");
+            goto out;
+        }
     }
 
     bdrv_qcow2 = bdrv_find_format("qcow2");
     opts = qemu_opts_create(bdrv_qcow2->create_opts, NULL, 0,
                             &error_abort);
     qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size);
-    ret = bdrv_create(bdrv_qcow2, tmp_filename, opts, &local_err);
+    if (replay_mode == REPLAY_PLAY) {
+        ret = 0;
+    } else {
+        ret = bdrv_create(bdrv_qcow2, tmp_filename, opts, &local_err);
+    }
     qemu_opts_del(opts);
     if (ret < 0) {
         error_setg_errno(errp, -ret, "Could not create temporary overlay "
@@ -1333,7 +1346,7 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
     bs_snapshot = bdrv_new("", &error_abort);
 
     ret = bdrv_open(&bs_snapshot, NULL, NULL, snapshot_options,
-                    flags, bdrv_qcow2, &local_err);
+                    flags, bdrv_qcow2, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
@@ -1363,7 +1376,7 @@ out:
  */
 int bdrv_open(BlockDriverState **pbs, const char *filename,
               const char *reference, QDict *options, int flags,
-              BlockDriver *drv, Error **errp)
+              BlockDriver *drv, Error **errp, const char *fname_suffix)
 {
     int ret;
     BlockDriverState *file = NULL, *bs;
@@ -1444,13 +1457,16 @@ int bdrv_open(BlockDriverState **pbs, const char *filename,
         }
         if (flags & BDRV_O_SNAPSHOT) {
             snapshot_flags = bdrv_temp_snapshot_flags(flags);
+            if (replay_mode != REPLAY_NONE) {
+                snapshot_flags &= ~BDRV_O_TEMPORARY;
+            }
             flags = bdrv_backing_flags(flags);
         }
 
         assert(file == NULL);
         ret = bdrv_open_image(&file, filename, options, "file",
                               bdrv_inherited_flags(flags),
-                              true, &local_err);
+                              true, &local_err, NULL);
         if (ret < 0) {
             goto fail;
         }
@@ -1493,7 +1509,7 @@ int bdrv_open(BlockDriverState **pbs, const char *filename,
     /* For snapshot=on, create a temporary qcow2 overlay. bs points to the
      * temporary snapshot afterwards. */
     if (snapshot_flags) {
-        ret = bdrv_append_temp_snapshot(bs, snapshot_flags, &local_err);
+        ret = bdrv_append_temp_snapshot(bs, snapshot_flags, &local_err, fname_suffix);
         if (local_err) {
             goto close_and_fail;
         }
@@ -5573,7 +5589,7 @@ void bdrv_img_create(const char *filename, const char *fmt,
 
             bs = NULL;
             ret = bdrv_open(&bs, backing_file, NULL, NULL, back_flags,
-                            backing_drv, &local_err);
+                            backing_drv, &local_err, NULL);
             if (ret < 0) {
                 error_setg_errno(errp, -ret, "Could not open '%s': %s",
                                  backing_file,
diff --git a/block/blkdebug.c b/block/blkdebug.c
index f51407d..151538a 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -412,7 +412,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the backing file */
     assert(bs->file == NULL);
     ret = bdrv_open_image(&bs->file, qemu_opt_get(opts, "x-image"), options, "image",
-                          flags | BDRV_O_PROTOCOL, false, &local_err);
+                          flags | BDRV_O_PROTOCOL, false, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
diff --git a/block/blkverify.c b/block/blkverify.c
index 621b785..94d7db7 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -138,7 +138,7 @@ static int blkverify_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the raw file */
     assert(bs->file == NULL);
     ret = bdrv_open_image(&bs->file, qemu_opt_get(opts, "x-raw"), options,
-                          "raw", flags | BDRV_O_PROTOCOL, false, &local_err);
+                          "raw", flags | BDRV_O_PROTOCOL, false, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto fail;
@@ -147,7 +147,7 @@ static int blkverify_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the test file */
     assert(s->test_file == NULL);
     ret = bdrv_open_image(&s->test_file, qemu_opt_get(opts, "x-image"), options,
-                          "test", flags, false, &local_err);
+                          "test", flags, false, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         s->test_file = NULL;
diff --git a/block/cow.c b/block/cow.c
index 6ee4833..d99c4d7 100644
--- a/block/cow.c
+++ b/block/cow.c
@@ -345,7 +345,7 @@ static int cow_create(const char *filename, QemuOpts *opts, Error **errp)
     }
 
     ret = bdrv_open(&cow_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
diff --git a/block/qcow.c b/block/qcow.c
index a874056..6e21cea 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -720,7 +720,7 @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp)
 
     qcow_bs = NULL;
     ret = bdrv_open(&qcow_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto cleanup;
diff --git a/block/qcow2.c b/block/qcow2.c
index 67e55c9..d9da238 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1635,7 +1635,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
 
     bs = NULL;
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, &local_err);
+                    NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return ret;
@@ -1697,7 +1697,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     BlockDriver* drv = bdrv_find_format("qcow2");
     assert(drv != NULL);
     ret = bdrv_open(&bs, filename, NULL, NULL,
-        BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH, drv, &local_err);
+        BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
@@ -1749,7 +1749,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     /* Reopen the image without BDRV_O_NO_FLUSH to flush it before returning */
     ret = bdrv_open(&bs, filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING,
-                    drv, &local_err);
+                    drv, &local_err, NULL);
     if (local_err) {
         error_propagate(errp, local_err);
         goto out;
diff --git a/block/qed.c b/block/qed.c
index b69374b..3417187 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -595,7 +595,7 @@ static int qed_create(const char *filename, uint32_t cluster_size,
     bs = NULL;
     ret = bdrv_open(&bs, filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_PROTOCOL, NULL,
-                    &local_err);
+                    &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return ret;
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 8d9350c..bb3dfef 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1576,7 +1576,7 @@ static int sd_prealloc(const char *filename, Error **errp)
     int ret;
 
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, errp);
+                    NULL, errp, NULL);
     if (ret < 0) {
         goto out_with_err_set;
     }
@@ -1739,7 +1739,7 @@ static int sd_create(const char *filename, QemuOpts *opts,
 
         bs = NULL;
         ret = bdrv_open(&bs, backing_file, NULL, NULL, BDRV_O_PROTOCOL, NULL,
-                        errp);
+                        errp, NULL);
         if (ret < 0) {
             goto out;
         }
diff --git a/block/vmdk.c b/block/vmdk.c
index 27a78da..5bafe31 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -809,7 +809,7 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                 desc_file_path, fname);
         extent_file = NULL;
         ret = bdrv_open(&extent_file, extent_path, NULL, NULL,
-                        bs->open_flags | BDRV_O_PROTOCOL, NULL, errp);
+                        bs->open_flags | BDRV_O_PROTOCOL, NULL, errp, NULL);
         if (ret) {
             return ret;
         }
@@ -1547,7 +1547,7 @@ static int vmdk_create_extent(const char *filename, int64_t filesize,
 
     assert(bs == NULL);
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, &local_err);
+                    NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
@@ -1805,7 +1805,7 @@ static int vmdk_create(const char *filename, QemuOpts *opts, Error **errp)
     if (backing_file) {
         BlockDriverState *bs = NULL;
         ret = bdrv_open(&bs, backing_file, NULL, NULL, BDRV_O_NO_BACKING, NULL,
-                        errp);
+                        errp, NULL);
         if (ret != 0) {
             goto exit;
         }
@@ -1881,7 +1881,7 @@ static int vmdk_create(const char *filename, QemuOpts *opts, Error **errp)
     }
     assert(new_bs == NULL);
     ret = bdrv_open(&new_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
diff --git a/block/vvfat.c b/block/vvfat.c
index 70176b1..f86f890 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -2939,7 +2939,7 @@ static int enable_write_target(BDRVVVFATState *s, Error **errp)
     s->qcow = NULL;
     ret = bdrv_open(&s->qcow, s->qcow_filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH,
-                    bdrv_qcow, errp);
+                    bdrv_qcow, errp, NULL);
     if (ret < 0) {
         goto err;
     }
diff --git a/blockdev.c b/blockdev.c
index 48bd9a3..e5d4b4d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -44,6 +44,7 @@
 #include "qmp-commands.h"
 #include "trace.h"
 #include "sysemu/arch_init.h"
+#include "replay/replay.h"
 
 static QTAILQ_HEAD(drivelist, DriveInfo) drives = QTAILQ_HEAD_INITIALIZER(drives);
 
@@ -520,7 +521,7 @@ static DriveInfo *blockdev_init(const char *file, QDict *bs_opts,
     bdrv_flags |= ro ? 0 : BDRV_O_RDWR;
 
     QINCREF(bs_opts);
-    ret = bdrv_open(&dinfo->bdrv, file, NULL, bs_opts, bdrv_flags, drv, &error);
+    ret = bdrv_open(&dinfo->bdrv, file, NULL, bs_opts, bdrv_flags, drv, &error, replay_image_suffix);
 
     if (ret < 0) {
         error_setg(errp, "could not open disk image %s: %s",
@@ -1367,7 +1368,7 @@ static void external_snapshot_prepare(BlkTransactionState *common,
      * extended QMP command? */
     assert(state->new_bs == NULL);
     ret = bdrv_open(&state->new_bs, new_image_file, NULL, options,
-                    flags | BDRV_O_NO_BACKING, drv, &local_err);
+                    flags | BDRV_O_NO_BACKING, drv, &local_err, NULL);
     /* We will manually add the backing_hd field to the bs later */
     if (ret != 0) {
         error_propagate(errp, local_err);
@@ -1616,7 +1617,7 @@ static void qmp_bdrv_open_encrypted(BlockDriverState *bs, const char *filename,
     Error *local_err = NULL;
     int ret;
 
-    ret = bdrv_open(&bs, filename, NULL, NULL, bdrv_flags, drv, &local_err);
+    ret = bdrv_open(&bs, filename, NULL, NULL, bdrv_flags, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
@@ -2108,7 +2109,7 @@ void qmp_drive_backup(const char *device, const char *target,
     }
 
     target_bs = NULL;
-    ret = bdrv_open(&target_bs, target, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&target_bs, target, NULL, NULL, flags, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
@@ -2282,7 +2283,7 @@ void qmp_drive_mirror(const char *device, const char *target,
      */
     target_bs = NULL;
     ret = bdrv_open(&target_bs, target, NULL, options,
-                    flags | BDRV_O_NO_BACKING, drv, &local_err);
+                    flags | BDRV_O_NO_BACKING, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
diff --git a/include/block/block.h b/include/block/block.h
index 32d3676..dd05458 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -211,13 +211,13 @@ int bdrv_parse_cache_flags(const char *mode, int *flags);
 int bdrv_parse_discard_flags(const char *mode, int *flags);
 int bdrv_open_image(BlockDriverState **pbs, const char *filename,
                     QDict *options, const char *bdref_key, int flags,
-                    bool allow_none, Error **errp);
+                    bool allow_none, Error **errp, const char *fname_suffix);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd);
 int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp);
-int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp);
+int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp, const char *fname_suffix);
 int bdrv_open(BlockDriverState **pbs, const char *filename,
               const char *reference, QDict *options, int flags,
-              BlockDriver *drv, Error **errp);
+              BlockDriver *drv, Error **errp, const char *fname_suffix);
 BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue *bs_queue,
                                     BlockDriverState *bs, int flags);
 int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp);
diff --git a/qemu-img.c b/qemu-img.c
index c98896b..d38c398 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -321,7 +321,7 @@ static BlockDriverState *bdrv_new_open(const char *id,
         drv = NULL;
     }
 
-    ret = bdrv_open(&bs, filename, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&bs, filename, NULL, NULL, flags, drv, &local_err, NULL);
     if (ret < 0) {
         error_report("Could not open '%s': %s", filename,
                      error_get_pretty(local_err));
@@ -2384,7 +2384,7 @@ static int img_rebase(int argc, char **argv)
         bs_old_backing = bdrv_new("old_backing", &error_abort);
         bdrv_get_backing_filename(bs, backing_name, sizeof(backing_name));
         ret = bdrv_open(&bs_old_backing, backing_name, NULL, NULL, BDRV_O_FLAGS,
-                        old_backing_drv, &local_err);
+                        old_backing_drv, &local_err, NULL);
         if (ret) {
             error_report("Could not open old backing file '%s': %s",
                          backing_name, error_get_pretty(local_err));
@@ -2394,7 +2394,7 @@ static int img_rebase(int argc, char **argv)
         if (out_baseimg[0]) {
             bs_new_backing = bdrv_new("new_backing", &error_abort);
             ret = bdrv_open(&bs_new_backing, out_baseimg, NULL, NULL,
-                            BDRV_O_FLAGS, new_backing_drv, &local_err);
+                            BDRV_O_FLAGS, new_backing_drv, &local_err, NULL);
             if (ret) {
                 error_report("Could not open new backing file '%s': %s",
                              out_baseimg, error_get_pretty(local_err));
diff --git a/qemu-io.c b/qemu-io.c
index b55a550..849067d 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -60,7 +60,7 @@ static int openfile(char *name, int flags, int growable, QDict *opts)
 
     if (growable) {
         if (bdrv_open(&qemuio_bs, name, NULL, opts, flags | BDRV_O_PROTOCOL,
-                      NULL, &local_err))
+                      NULL, &local_err, NULL))
         {
             fprintf(stderr, "%s: can't open%s%s: %s\n", progname,
                     name ? " device " : "", name ?: "",
@@ -71,7 +71,7 @@ static int openfile(char *name, int flags, int growable, QDict *opts)
     } else {
         qemuio_bs = bdrv_new("hda", &error_abort);
 
-        if (bdrv_open(&qemuio_bs, name, NULL, opts, flags, NULL, &local_err)
+        if (bdrv_open(&qemuio_bs, name, NULL, opts, flags, NULL, &local_err, NULL)
             < 0)
         {
             fprintf(stderr, "%s: can't open%s%s: %s\n", progname,
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 626e584..e780d1c 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -663,7 +663,7 @@ int main(int argc, char **argv)
     bs = bdrv_new("hda", &error_abort);
 
     srcpath = argv[optind];
-    ret = bdrv_open(&bs, srcpath, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&bs, srcpath, NULL, NULL, flags, drv, &local_err, NULL);
     if (ret < 0) {
         errno = -ret;
         err(EXIT_FAILURE, "Failed to bdrv_open '%s': %s", argv[optind],

[Qemu-devel] [RFC PATCH v2 21/49] sysemu: system functions for replay

[Pavel Dovgalyuk]

This patch removes "static" specifier from several qemu function to make
them visible to the replay module. It also invents several system functions
that will be used by replay.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 arch_init.c             |    8 ++++++++
 cpu-exec.c              |    6 ++++--
 cpus.c                  |    4 ++--
 include/exec/exec-all.h |    4 +++-
 include/sysemu/cpus.h   |    1 +
 include/sysemu/sysemu.h |    1 +
 savevm.c                |   32 ++++++++++++++++++++++++++------
 translate-all.c         |    8 ++++++++
 8 files changed, 53 insertions(+), 11 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index 8ddaf35..5ee33d9 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -52,6 +52,7 @@
 #include "exec/ram_addr.h"
 #include "hw/acpi/acpi.h"
 #include "qemu/host-utils.h"
+#include "replay/replay.h"
 
 #ifdef DEBUG_ARCH_INIT
 #define DPRINTF(fmt, ...) \
@@ -1092,6 +1093,13 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
 
                 total_ram_bytes -= length;
             }
+            if (replay_mode == REPLAY_PLAY) {
+                RAMBlock *block;
+                /* Clear the blocks' memory instead of resetting the machine */
+                QTAILQ_FOREACH(block, &ram_list.blocks, next) {
+                    memset(block->host, 0, block->length);
+                }
+            }
         } else if (flags & RAM_SAVE_FLAG_COMPRESS) {
             void *host;
             uint8_t ch;
diff --git a/cpu-exec.c b/cpu-exec.c
index 38e5f02..66a693c 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -189,12 +189,14 @@ static inline TranslationBlock *tb_find_fast(CPUArchState *env)
 
 static CPUDebugExcpHandler *debug_excp_handler;
 
-void cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler)
+CPUDebugExcpHandler *cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler)
 {
+    CPUDebugExcpHandler *old = debug_excp_handler;
     debug_excp_handler = handler;
+    return old;
 }
 
-static void cpu_handle_debug_exception(CPUArchState *env)
+void cpu_handle_debug_exception(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
     CPUWatchpoint *wp;
diff --git a/cpus.c b/cpus.c
index 5e7f2cf..bbad529 100644
--- a/cpus.c
+++ b/cpus.c
@@ -85,7 +85,7 @@ static bool cpu_thread_is_idle(CPUState *cpu)
     return true;
 }
 
-static bool all_cpu_threads_idle(void)
+bool all_cpu_threads_idle(void)
 {
     CPUState *cpu;
 
@@ -1033,7 +1033,7 @@ bool qemu_cpu_is_self(CPUState *cpu)
     return qemu_thread_is_self(cpu->thread);
 }
 
-static bool qemu_in_vcpu_thread(void)
+bool qemu_in_vcpu_thread(void)
 {
     return current_cpu && qemu_cpu_is_self(current_cpu);
 }
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 528928f..4b7a241 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -212,6 +212,7 @@ static inline unsigned int tb_phys_hash_func(tb_page_addr_t pc)
 
 void tb_free(TranslationBlock *tb);
 void tb_flush(CPUArchState *env);
+void tb_flush_all(void);
 void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr);
 
 #if defined(USE_DIRECT_JUMP)
@@ -385,7 +386,8 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr);
 
 typedef void (CPUDebugExcpHandler)(CPUArchState *env);
 
-void cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler);
+CPUDebugExcpHandler *cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler);
+void cpu_handle_debug_exception(CPUArchState *env);
 
 /* vl.c */
 extern int singlestep;
diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h
index 4f79081..8455e0e 100644
--- a/include/sysemu/cpus.h
+++ b/include/sysemu/cpus.h
@@ -6,6 +6,7 @@ void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+bool all_cpu_threads_idle(void);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index d8539fd..1ebfef9 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -76,6 +76,7 @@ void qemu_add_machine_init_done_notifier(Notifier *notify);
 
 void do_savevm(Monitor *mon, const QDict *qdict);
 int load_vmstate(const char *name);
+int save_vmstate(Monitor *mon, const char *name);
 void do_delvm(Monitor *mon, const QDict *qdict);
 void do_info_snapshots(Monitor *mon, const QDict *qdict);
 
diff --git a/savevm.c b/savevm.c
index e19ae0a..9136778 100644
--- a/savevm.c
+++ b/savevm.c
@@ -42,7 +42,7 @@
 #include "qemu/iov.h"
 #include "block/snapshot.h"
 #include "block/qapi.h"
-
+#include "replay/replay.h"
 
 #ifndef ETH_P_RARP
 #define ETH_P_RARP 0x8035
@@ -1039,7 +1039,7 @@ static int del_existing_snapshots(Monitor *mon, const char *name)
     return 0;
 }
 
-void do_savevm(Monitor *mon, const QDict *qdict)
+int save_vmstate(Monitor *mon, const char *name)
 {
     BlockDriverState *bs, *bs1;
     QEMUSnapshotInfo sn1, *sn = &sn1, old_sn1, *old_sn = &old_sn1;
@@ -1049,7 +1049,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
     uint64_t vm_state_size;
     qemu_timeval tv;
     struct tm tm;
-    const char *name = qdict_get_try_str(qdict, "name");
+    int success = 0;
 
     /* Verify if there is a device that doesn't support snapshots and is writable */
     bs = NULL;
@@ -1062,14 +1062,18 @@ void do_savevm(Monitor *mon, const QDict *qdict)
         if (!bdrv_can_snapshot(bs)) {
             monitor_printf(mon, "Device '%s' is writable but does not support snapshots.\n",
                                bdrv_get_device_name(bs));
-            return;
+            return success;
         }
     }
 
     bs = find_vmstate_bs();
     if (!bs) {
         monitor_printf(mon, "No block device can accept snapshots\n");
-        return;
+        if (replay_mode != REPLAY_NONE) {
+            fprintf(stderr, "At least one hdd should be attached to QEMU for replay\n");
+            exit(1);
+        }
+        return success;
     }
 
     saved_vm_running = runstate_is_running();
@@ -1118,6 +1122,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
 
     /* create the snapshots */
 
+    success = 1;
     bs1 = NULL;
     while ((bs1 = bdrv_next(bs1))) {
         if (bdrv_can_snapshot(bs1)) {
@@ -1127,6 +1132,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
             if (ret < 0) {
                 monitor_printf(mon, "Error while creating snapshot on '%s'\n",
                                bdrv_get_device_name(bs1));
+                success = 0;
             }
         }
     }
@@ -1135,6 +1141,14 @@ void do_savevm(Monitor *mon, const QDict *qdict)
     if (saved_vm_running) {
         vm_start();
     }
+
+    return success;
+}
+
+void do_savevm(Monitor *mon, const QDict *qdict)
+{
+    const char *name = qdict_get_try_str(qdict, "name");
+    save_vmstate(mon, name);
 }
 
 void qmp_xen_save_devices_state(const char *filename, Error **errp)
@@ -1231,7 +1245,13 @@ int load_vmstate(const char *name)
         return -EINVAL;
     }
 
-    qemu_system_reset(VMRESET_SILENT);
+    /* Do not reset in replay mode.
+       1. Reset will alter the behavior in play mode compared to save one
+       2. Timers read by reset handlers are not correct, because
+          replay reads them from the unknown part of the log */
+    if (replay_mode == REPLAY_NONE) {
+        qemu_system_reset(VMRESET_SILENT);
+    }
     ret = qemu_loadvm_state(f);
 
     qemu_fclose(f);
diff --git a/translate-all.c b/translate-all.c
index 8f7e11b..fcb0697 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -803,6 +803,14 @@ void tb_flush(CPUArchState *env1)
     tcg_ctx.tb_ctx.tb_flush_count++;
 }
 
+void tb_flush_all(void)
+{
+    CPUState *cpu;
+    for (cpu = first_cpu ; cpu != NULL ; cpu = CPU_NEXT(cpu)) {
+        tb_flush(cpu->env_ptr);
+    }
+}
+
 #ifdef DEBUG_TB_CHECK
 
 static void tb_invalidate_check(target_ulong address)

[Qemu-devel] [RFC PATCH v2 22/49] replay: internal functions for replay log

[Pavel Dovgalyuk]

This patch adds functions to perform read and write operations
with replay log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 replay/Makefile.objs     |    1 
 replay/replay-internal.c |  141 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   50 ++++++++++++++++
 replay/replay.c          |    6 ++
 4 files changed, 198 insertions(+), 0 deletions(-)
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h

diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 7ea860f..1148f45 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1 +1,2 @@
 obj-y += replay.o
+obj-y += replay-internal.o
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
new file mode 100755
index 0000000..706b46b
--- /dev/null
+++ b/replay/replay-internal.c
@@ -0,0 +1,141 @@
+/*
+ * replay-internal.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay-internal.h"
+
+volatile unsigned int replay_data_kind = -1;
+volatile unsigned int replay_has_unread_data;
+
+/* File for replay writing */
+FILE *replay_file;
+
+void replay_put_byte(unsigned char byte)
+{
+    if (replay_file) {
+        fwrite(&byte, sizeof(byte), 1, replay_file);
+    }
+}
+
+void replay_put_event(unsigned char event)
+{
+    replay_put_byte(event);
+}
+
+
+void replay_put_word(uint16_t word)
+{
+    if (replay_file) {
+        fwrite(&word, sizeof(word), 1, replay_file);
+    }
+}
+
+void replay_put_dword(unsigned int dword)
+{
+    if (replay_file) {
+        fwrite(&dword, sizeof(dword), 1, replay_file);
+    }
+}
+
+void replay_put_qword(int64_t qword)
+{
+    if (replay_file) {
+        fwrite(&qword, sizeof(qword), 1, replay_file);
+    }
+}
+
+void replay_put_array(const uint8_t *buf, size_t size)
+{
+    if (replay_file) {
+        fwrite(&size, sizeof(size), 1, replay_file);
+        fwrite(buf, 1, size, replay_file);
+    }
+}
+
+unsigned char replay_get_byte(void)
+{
+    unsigned char byte;
+    if (replay_file) {
+        fread(&byte, sizeof(byte), 1, replay_file);
+    }
+    return byte;
+}
+
+uint16_t replay_get_word(void)
+{
+    uint16_t word;
+    if (replay_file) {
+        fread(&word, sizeof(word), 1, replay_file);
+    }
+
+    return word;
+}
+
+unsigned int replay_get_dword(void)
+{
+    unsigned int dword;
+    if (replay_file) {
+        fread(&dword, sizeof(dword), 1, replay_file);
+    }
+
+    return dword;
+}
+
+int64_t replay_get_qword(void)
+{
+    int64_t qword;
+    if (replay_file) {
+        fread(&qword, sizeof(qword), 1, replay_file);
+    }
+
+    return qword;
+}
+
+void replay_get_array(uint8_t *buf, size_t *size)
+{
+    if (replay_file) {
+        fread(size, sizeof(*size), 1, replay_file);
+        fread(buf, 1, *size, replay_file);
+    }
+}
+
+void replay_get_array_alloc(uint8_t **buf, size_t *size)
+{
+    if (replay_file) {
+        fread(size, sizeof(*size), 1, replay_file);
+        *buf = g_malloc(*size);
+        fread(*buf, 1, *size, replay_file);
+    }
+}
+
+void replay_check_error(void)
+{
+    if (replay_file) {
+        if (feof(replay_file)) {
+            fprintf(stderr, "replay file is over\n");
+            exit(1);
+        } else if (ferror(replay_file)) {
+            fprintf(stderr, "replay file is over or something goes wrong\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_fetch_data_kind(void)
+{
+    if (replay_file) {
+        if (!replay_has_unread_data) {
+            replay_data_kind = replay_get_byte();
+            replay_check_error();
+            replay_has_unread_data = 1;
+        }
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
new file mode 100755
index 0000000..b959bca
--- /dev/null
+++ b/replay/replay-internal.h
@@ -0,0 +1,50 @@
+#ifndef REPLAY_INTERNAL_H
+#define REPLAY_INTERNAL_H
+
+/*
+ * replay-internal.h
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+
+extern volatile unsigned int replay_data_kind;
+extern volatile unsigned int replay_has_unread_data;
+
+/* File for replay writing */
+extern FILE *replay_file;
+
+void replay_put_byte(unsigned char byte);
+void replay_put_event(unsigned char event);
+void replay_put_word(uint16_t word);
+void replay_put_dword(unsigned int dword);
+void replay_put_qword(int64_t qword);
+void replay_put_array(const uint8_t *buf, size_t size);
+
+unsigned char replay_get_byte(void);
+uint16_t replay_get_word(void);
+unsigned int replay_get_dword(void);
+int64_t replay_get_qword(void);
+void replay_get_array(uint8_t *buf, size_t *size);
+void replay_get_array_alloc(uint8_t **buf, size_t *size);
+
+/*! Checks error status of the file. */
+void replay_check_error(void);
+
+/*! Reads data type from the file and stores it in the
+    replay_data_kind variable. */
+void replay_fetch_data_kind(void);
+
+/*! Saves queued events (like instructions and sound). */
+void replay_save_instructions(void);
+/*! Checks that the next data is corresponding to the desired kind.
+    Terminates the program in case of error. */
+void validate_data_kind(int kind);
+
+#endif
diff --git a/replay/replay.c b/replay/replay.c
index 5a69465..2961e17 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -17,3 +17,9 @@ int play_submode = REPLAY_PLAY_UNKNOWN;
 
 /* Suffix for the disk images filenames */
 char *replay_image_suffix;
+
+
+int replay_get_play_submode(void)
+{
+    return play_submode;
+}

[Qemu-devel] [RFC PATCH v2 23/49] cpu: invent instruction count for accurate replay

[Pavel Dovgalyuk]

This patch adds instructions count fields to cpu structure and
invents several functions for increasing this counter while executing
translation blocks.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpu-exec.c               |   14 ++++-
 cpus.c                   |    5 ++
 exec.c                   |    4 +
 include/exec/cpu-defs.h  |    1 
 include/qom/cpu.h        |    4 +
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   26 +++++++++
 replay/replay-internal.c |   14 +++++
 replay/replay-internal.h |   37 ++++++++++++-
 replay/replay.c          |  136 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   16 +++++
 translate-all.c          |    7 ++
 12 files changed, 260 insertions(+), 5 deletions(-)
 create mode 100755 replay/replay-events.c

diff --git a/cpu-exec.c b/cpu-exec.c
index 66a693c..13c0ec6 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -22,6 +22,7 @@
 #include "tcg.h"
 #include "qemu/atomic.h"
 #include "sysemu/qtest.h"
+#include "qemu/main-loop.h"
 
 void cpu_loop_exit(CPUState *cpu)
 {
@@ -283,19 +284,24 @@ int cpu_exec(CPUArchState *env)
 #else
 #error unsupported target CPU
 #endif
-    cpu->exception_index = -1;
-
     /* prepare setjmp context for exception handling */
     for(;;) {
         if (sigsetjmp(cpu->jmp_env, 0) == 0) {
             /* if an exception is pending, we execute it here */
             if (cpu->exception_index >= 0) {
+                if (cpu->exception_index == EXCP_REPLAY) {
+                    ret = cpu->exception_index;
+                    cpu->exception_index = -1;
+                    qemu_notify_event();
+                    break;
+                }
                 if (cpu->exception_index >= EXCP_INTERRUPT) {
                     /* exit request from the cpu execution loop */
                     ret = cpu->exception_index;
                     if (ret == EXCP_DEBUG) {
                         cpu_handle_debug_exception(env);
                     }
+                    cpu->exception_index = -1;
                     break;
                 } else {
 #if defined(CONFIG_USER_ONLY)
@@ -601,6 +607,10 @@ int cpu_exec(CPUArchState *env)
                         next_tb = 0;
                     }
                 }
+                if (cpu->exception_index == EXCP_REPLAY) {
+                    /* go to exception_index checking */
+                    break;
+                }
                 if (unlikely(cpu->exit_request)) {
                     cpu->exit_request = 0;
                     cpu->exception_index = EXCP_INTERRUPT;
diff --git a/cpus.c b/cpus.c
index bbad529..eb2a795 100644
--- a/cpus.c
+++ b/cpus.c
@@ -937,6 +937,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     CPU_FOREACH(cpu) {
         cpu->thread_id = qemu_get_thread_id();
         cpu->created = true;
+        /* init exception index here */
+        cpu->exception_index = -1;
     }
     qemu_cond_signal(&qemu_cpu_cond);
 
@@ -1186,6 +1188,7 @@ void qemu_init_vcpu(CPUState *cpu)
     cpu->nr_cores = smp_cores;
     cpu->nr_threads = smp_threads;
     cpu->stopped = true;
+    cpu->instructions_count = 0;
     if (kvm_enabled()) {
         qemu_kvm_start_vcpu(cpu);
     } else if (tcg_enabled()) {
@@ -1307,6 +1310,8 @@ static void tcg_exec_all(void)
             if (r == EXCP_DEBUG) {
                 cpu_handle_guest_debug(cpu);
                 break;
+            } else if (r == EXCP_REPLAY) {
+                break;
             }
         } else if (cpu->stop || cpu->stopped) {
             break;
diff --git a/exec.c b/exec.c
index 5a2a25e..36adb62 100644
--- a/exec.c
+++ b/exec.c
@@ -52,6 +52,7 @@
 #include "exec/ram_addr.h"
 
 #include "qemu/range.h"
+#include "replay/replay.h"
 
 //#define DEBUG_SUBPAGE
 
@@ -1615,6 +1616,9 @@ static void check_watchpoint(int offset, int len_mask, int flags)
             if (!cpu->watchpoint_hit) {
                 cpu->watchpoint_hit = wp;
                 tb_check_watchpoint(cpu);
+                /* Current instruction is already processed by replay.
+                   Set flags that allow skpping this events */
+                replay_undo_last_instruction();
                 if (wp->flags & BP_STOP_BEFORE_ACCESS) {
                     cpu->exception_index = EXCP_DEBUG;
                     cpu_loop_exit(cpu);
diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index 2dd6206..db9ae1a 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -59,6 +59,7 @@ typedef uint64_t target_ulong;
 #define EXCP_DEBUG      0x10002 /* cpu stopped after a breakpoint or singlestep */
 #define EXCP_HALTED     0x10003 /* cpu is halted (waiting for external event) */
 #define EXCP_YIELD      0x10004 /* cpu wants to yield timeslice to another */
+#define EXCP_REPLAY     0x10005 /* for breaking execution loop to make correct order of events */
 
 /* Only the bottom TB_JMP_PAGE_BITS of the jump cache hash bits vary for
    addresses on the same page.  The top bits are the same.  This allows
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 1aafbf5..9c7b5c8 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -287,6 +287,7 @@ struct CPUState {
        (absolute value) offset as small as possible.  This reduces code
        size, especially for hosts without large memory offsets.  */
     volatile sig_atomic_t tcg_exit_req;
+    uint32_t instructions_count;
 };
 
 QTAILQ_HEAD(CPUTailQ, CPUState);
@@ -466,6 +467,9 @@ static inline bool cpu_has_work(CPUState *cpu)
  */
 bool qemu_cpu_is_self(CPUState *cpu);
 
+//need here for replay work
+bool qemu_in_vcpu_thread(void);
+
 /**
  * qemu_cpu_kick:
  * @cpu: The vCPU to kick.
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 1148f45..56da09c 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1,2 +1,3 @@
 obj-y += replay.o
 obj-y += replay-internal.o
+obj-y += replay-events.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
new file mode 100755
index 0000000..eaffc48
--- /dev/null
+++ b/replay/replay-events.c
@@ -0,0 +1,26 @@
+/*
+ * replay-events.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+#include "replay-internal.h"
+
+bool replay_has_events(void)
+{
+    return false;
+}
+
+void replay_save_events(int opt)
+{
+}
+
+void replay_read_events(int opt)
+{
+}
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
index 706b46b..f416317 100755
--- a/replay/replay-internal.c
+++ b/replay/replay-internal.c
@@ -10,6 +10,7 @@
  */
 
 #include "qemu-common.h"
+#include "replay.h"
 #include "replay-internal.h"
 
 volatile unsigned int replay_data_kind = -1;
@@ -139,3 +140,16 @@ void replay_fetch_data_kind(void)
         }
     }
 }
+
+/*! Saves cached instructions. */
+void replay_save_instructions(void)
+{
+    if (replay_file && replay_mode == REPLAY_SAVE) {
+        if (first_cpu != NULL && first_cpu->instructions_count > 0) {
+            replay_put_event(EVENT_INSTRUCTION);
+            replay_put_dword(first_cpu->instructions_count);
+            replay_state.current_step += first_cpu->instructions_count;
+            first_cpu->instructions_count = 0;
+        }
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index b959bca..9ebfa83 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -13,6 +13,21 @@
  */
 
 #include <stdio.h>
+#include "sysemu/sysemu.h"
+
+/* for async events */
+#define EVENT_ASYNC                 24
+/* for instruction event */
+#define EVENT_INSTRUCTION           32
+
+typedef struct ReplayState {
+    /*! Nonzero, when next instruction is repeated one and was already
+        processed. */
+    int skipping_instruction;
+    /*! Current step - number of processed instructions and timer events. */
+    uint64_t current_step;
+} ReplayState;
+extern ReplayState replay_state;
 
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
@@ -40,11 +55,27 @@ void replay_check_error(void);
 /*! Reads data type from the file and stores it in the
     replay_data_kind variable. */
 void replay_fetch_data_kind(void);
-
-/*! Saves queued events (like instructions and sound). */
-void replay_save_instructions(void);
 /*! Checks that the next data is corresponding to the desired kind.
     Terminates the program in case of error. */
 void validate_data_kind(int kind);
 
+/*! Saves queued events (like instructions and sound). */
+void replay_save_instructions(void);
+
+/*! Skips async events until some sync event will be found. */
+bool skip_async_events(int stop_event);
+/*! Skips async events invocations from the input,
+    until required data kind is found. If the requested data is not found
+    reports an error and stops the execution. */
+void skip_async_events_until(unsigned int kind);
+
+/* Asynchronous events queue */
+
+/*! Returns true if there are any unsaved events in the queue */
+bool replay_has_events(void);
+/*! Saves events from queue into the file */
+void replay_save_events(int opt);
+/*! Read events from the file into the input queue */
+void replay_read_events(int opt);
+
 #endif
diff --git a/replay/replay.c b/replay/replay.c
index 2961e17..8a9826e 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -9,7 +9,9 @@
  *
  */
 
+#include "qemu-common.h"
 #include "replay.h"
+#include "replay-internal.h"
 
 int replay_mode = REPLAY_NONE;
 /*! Stores current submode for PLAY mode */
@@ -18,8 +20,142 @@ int play_submode = REPLAY_PLAY_UNKNOWN;
 /* Suffix for the disk images filenames */
 char *replay_image_suffix;
 
+ReplayState replay_state;
+
 
 int replay_get_play_submode(void)
 {
     return play_submode;
 }
+
+bool skip_async_events(int stop_event)
+{
+    /* nothing to skip - not all instructions used */
+    if (first_cpu != NULL && first_cpu->instructions_count != 0 
+        && replay_has_unread_data) {
+        return stop_event == EVENT_INSTRUCTION;
+    }
+
+    bool res = false;
+    while (true) {
+        replay_fetch_data_kind();
+        if (stop_event == replay_data_kind) {
+            res = true;
+        }
+        switch (replay_data_kind) {
+        case EVENT_INSTRUCTION:
+            first_cpu->instructions_count = replay_get_dword();
+            return res;
+        default:
+            /* clock, time_t, checkpoint and other events */
+            return res;
+        }
+    }
+
+    return res;
+}
+
+void skip_async_events_until(unsigned int kind)
+{
+    if (!skip_async_events(kind)) {
+        if (replay_data_kind == EVENT_ASYNC && kind == EVENT_INSTRUCTION) {
+            return;
+        }
+
+        fprintf(stderr, "%"PRId64": Read data kind %d instead of expected %d\n",
+            replay_get_current_step(), replay_data_kind, kind);
+        exit(1);
+    }
+}
+
+void replay_instruction(int process_events)
+{
+    if (replay_state.skipping_instruction) {
+        replay_state.skipping_instruction = 0;
+        return;
+    }
+
+    if (replay_file) {
+        if (replay_mode == REPLAY_SAVE) {
+            if (process_events && replay_has_events()) {
+                replay_save_instructions();
+                /* events will be after the last instruction */
+                replay_save_events(-1);
+            } else {
+                /* instruction - increase the step counter */
+                ++first_cpu->instructions_count;
+            }
+        } else if (replay_mode == REPLAY_PLAY) {
+            skip_async_events_until(EVENT_INSTRUCTION);
+            if (first_cpu->instructions_count >= 1) {
+                ++replay_state.current_step;
+                --first_cpu->instructions_count;
+                if (first_cpu->instructions_count == 0) {
+                    replay_has_unread_data = 0;
+                }
+            } else {
+                replay_read_events(-1);
+            }
+        }
+    }
+}
+
+void replay_undo_last_instruction(void)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        first_cpu->instructions_count--;
+    } else {
+        replay_state.skipping_instruction = 1;
+    }
+}
+
+bool replay_has_async_request(void)
+{
+    if (replay_state.skipping_instruction) {
+        return false;
+    }
+
+    if (replay_mode == REPLAY_PLAY) {
+        if (skip_async_events(EVENT_ASYNC)) {
+            return true;
+        }
+
+        return false;
+    }
+    else if (replay_mode == REPLAY_SAVE) {
+        if (replay_has_events()) {
+            return true;
+        }
+    }
+
+    return false;
+}
+
+bool replay_has_instruction(void)
+{
+    if (replay_state.skipping_instruction) {
+        return true;
+    }
+
+    if (replay_mode == REPLAY_PLAY) {
+        skip_async_events(EVENT_INSTRUCTION);
+        if (replay_data_kind != EVENT_INSTRUCTION
+            && replay_data_kind != EVENT_ASYNC) {
+            return false;
+        }
+    }
+    return true;
+}
+
+uint64_t replay_get_current_step(void)
+{
+    if (first_cpu == NULL) {
+        return 0;
+    }
+    if (replay_file) {
+        if (replay_mode == REPLAY_SAVE) {
+            return replay_state.current_step + first_cpu->instructions_count;
+        }
+    }
+    return replay_state.current_step;
+}
diff --git a/replay/replay.h b/replay/replay.h
index a883aa4..1ce7f78 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -12,6 +12,9 @@
  *
  */
 
+#include <stdbool.h>
+#include <stdint.h>
+
 /* replay modes */
 #define REPLAY_NONE 0
 #define REPLAY_SAVE 1
@@ -29,4 +32,17 @@ extern int replay_icount;
 /*! Returns replay play submode */
 int replay_get_play_submode(void);
 
+/* Processing the instructions */
+
+/*! Returns number of executed instructions. */
+uint64_t replay_get_current_step(void);
+/*! Called before instruction execution */
+void replay_instruction(int process_events);
+/*! Undo last instruction count, when exception occurs */
+void replay_undo_last_instruction(void);
+/*! Returns true if asynchronous event is pending */
+bool replay_has_async_request(void);
+/*! Returns non-zero if next event is instruction. */
+bool replay_has_instruction(void);
+
 #endif
diff --git a/translate-all.c b/translate-all.c
index fcb0697..37582a2 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -59,6 +59,7 @@
 #include "exec/cputlb.h"
 #include "translate-all.h"
 #include "qemu/timer.h"
+#include "replay/replay.h"
 
 //#define DEBUG_TB_INVALIDATE
 //#define DEBUG_FLUSH
@@ -1171,6 +1172,9 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,
                 cpu_restore_state_from_tb(cpu, current_tb, cpu->mem_io_pc);
                 cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                      &current_flags);
+                /* Current instruction is already processed by replay.
+                   Set flags that allow skpping this event */
+                replay_undo_last_instruction();
             }
 #endif /* TARGET_HAS_PRECISE_SMC */
             /* we need to do that to handle the case where a signal
@@ -1290,6 +1294,9 @@ static void tb_invalidate_phys_page(tb_page_addr_t addr,
             cpu_restore_state_from_tb(cpu, current_tb, pc);
             cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                  &current_flags);
+            /* Current instruction is already processed by replay.
+               Set flags that allow skpping this event */
+            replay_undo_last_instruction();
         }
 #endif /* TARGET_HAS_PRECISE_SMC */
         tb_phys_invalidate(tb, addr);

[Qemu-devel] [RFC PATCH v2 24/49] target-arm: instructions counting code for replay

[Pavel Dovgalyuk]

This patch adds instructions counting into the target-specific part
of arm simulator. In record/replay mode it inserts replay functions
calls and instructions counter increment into the translated code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-arm/Makefile.objs   |    1 +
 target-arm/helper.h        |    2 ++
 target-arm/replay_helper.c |   33 ++++++++++++++++++++++++++++++
 target-arm/translate.c     |   48 +++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 81 insertions(+), 3 deletions(-)
 create mode 100755 target-arm/replay_helper.c

diff --git a/target-arm/Makefile.objs b/target-arm/Makefile.objs
index dcd167e..b8f61df 100644
--- a/target-arm/Makefile.objs
+++ b/target-arm/Makefile.objs
@@ -9,3 +9,4 @@ obj-y += neon_helper.o iwmmxt_helper.o
 obj-y += gdbstub.o
 obj-$(TARGET_AARCH64) += cpu64.o translate-a64.o helper-a64.o gdbstub64.o
 obj-y += crypto_helper.o
+obj-y += replay_helper.o
diff --git a/target-arm/helper.h b/target-arm/helper.h
index facfcd2..0233b64 100644
--- a/target-arm/helper.h
+++ b/target-arm/helper.h
@@ -529,3 +529,5 @@ DEF_HELPER_FLAGS_2(neon_pmull_64_hi, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 #ifdef TARGET_AARCH64
 #include "helper-a64.h"
 #endif
+
+DEF_HELPER_1(replay_instruction, i32, env)
diff --git a/target-arm/replay_helper.c b/target-arm/replay_helper.c
new file mode 100755
index 0000000..736cd51
--- /dev/null
+++ b/target-arm/replay_helper.c
@@ -0,0 +1,33 @@
+/*
+ * replay_helper.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "cpu.h"
+#include "exec/helper-proto.h"
+#include "replay/replay.h"
+
+uint32_t helper_replay_instruction(CPUARMState *env)
+{
+    CPUState *cpu = ENV_GET_CPU(env);
+    if (replay_mode == REPLAY_PLAY
+        && !replay_has_instruction()) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    if (cpu->exit_request) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    int timer = replay_has_async_request();
+    replay_instruction(timer);
+    return timer;
+}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index cf4e767..dd40998 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -31,6 +31,7 @@
 #include "qemu/log.h"
 #include "qemu/bitops.h"
 #include "arm_ldst.h"
+#include "replay/replay.h"
 
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
@@ -10858,6 +10859,32 @@ undef:
     gen_exception_insn(s, 2, EXCP_UDEF, syn_uncategorized());
 }
 
+static void gen_instructions_count(void)
+{
+    TCGv_i32 cpu_tmp_i32 = tcg_temp_new();
+    tcg_gen_ld_i32(cpu_tmp_i32, cpu_env, 
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_gen_addi_i32(cpu_tmp_i32, cpu_tmp_i32, 1);
+    tcg_gen_st_i32(cpu_tmp_i32, cpu_env, 
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_temp_free(cpu_tmp_i32);
+}
+
+static void gen_instr_replay(DisasContext *s, target_ulong pc_ptr)
+{
+    int l1 = gen_new_label();
+    TCGv_i32 cpu_tmp2_i32 = tcg_temp_new();
+
+    gen_helper_replay_instruction(cpu_tmp2_i32, cpu_env);
+    tcg_gen_brcondi_i32(TCG_COND_EQ, cpu_tmp2_i32, 0, l1);
+
+    gen_set_condexec(s);
+    gen_set_pc_im(s, pc_ptr);
+    tcg_gen_exit_tb(0);
+    gen_set_label(l1);
+    tcg_temp_free(cpu_tmp2_i32);
+}
+
 /* generate intermediate code in gen_opc_buf and gen_opparam_buf for
    basic block 'tb'. If search_pc is TRUE, also generate PC
    information for each intermediate instruction. */
@@ -11015,13 +11042,27 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
             tcg_ctx.gen_opc_icount[lj] = num_insns;
         }
 
-        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO))
+        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO)
+            && replay_mode == REPLAY_NONE) {
             gen_io_start();
+        }
 
         if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP | CPU_LOG_TB_OP_OPT))) {
             tcg_gen_debug_insn_start(dc->pc);
         }
 
+        if (replay_mode != REPLAY_NONE) {
+            /* In PLAY mode check events at every instruction, not only at
+               the beginning of the block. This is needed, when replaying
+               has changed the bounds of translation blocks.
+             */
+            if (dc->pc == pc_start || replay_mode == REPLAY_PLAY) {
+                gen_instr_replay(dc, dc->pc);
+            } else {
+                gen_instructions_count();
+            }
+        }
+
         if (dc->thumb) {
             disas_thumb_insn(env, dc);
             if (dc->condexec_mask) {
@@ -11054,10 +11095,11 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
     } while (!dc->is_jmp && tcg_ctx.gen_opc_ptr < gen_opc_end &&
              !cs->singlestep_enabled &&
              !singlestep &&
-             dc->pc < next_page_start &&
+             /* +3 is for unaligned Thumb2 instructions */
+             dc->pc + 3 < next_page_start &&
              num_insns < max_insns);
 
-    if (tb->cflags & CF_LAST_IO) {
+    if ((tb->cflags & CF_LAST_IO) && replay_mode == REPLAY_NONE) {
         if (dc->condjmp) {
             /* FIXME:  This can theoretically happen with self-modifying
                code.  */

[Qemu-devel] [RFC PATCH v2 25/49] target-i386: instructions counting code for replay

[Pavel Dovgalyuk]

This patch adds instructions counting into the target-specific part
of i386/x86_64 simulator. In record/replay mode it inserts replay functions
calls and instructions counter increment into the translated code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-i386/Makefile.objs   |    1 +
 target-i386/cpu.h           |    7 ++++
 target-i386/helper.h        |    2 +
 target-i386/replay_helper.c |   33 ++++++++++++++++++
 target-i386/translate.c     |   79 +++++++++++++++++++++++++++++++++++++++++--
 5 files changed, 118 insertions(+), 4 deletions(-)
 create mode 100755 target-i386/replay_helper.c

diff --git a/target-i386/Makefile.objs b/target-i386/Makefile.objs
index 027b94e..09a6e8a 100644
--- a/target-i386/Makefile.objs
+++ b/target-i386/Makefile.objs
@@ -2,6 +2,7 @@ obj-y += translate.o helper.o cpu.o
 obj-y += excp_helper.o fpu_helper.o cc_helper.o int_helper.o svm_helper.o
 obj-y += smm_helper.o misc_helper.o mem_helper.o seg_helper.o
 obj-y += gdbstub.o
+obj-y += replay_helper.o
 obj-$(CONFIG_SOFTMMU) += machine.o arch_memory_mapping.o arch_dump.o
 obj-$(CONFIG_KVM) += kvm.o
 obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 42bda46..c9b92af 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -28,6 +28,13 @@
 #define TARGET_LONG_BITS 32
 #endif
 
+/* Maximum instruction code size */
+#ifdef TARGET_X86_64
+#define TARGET_MAX_INSN_SIZE 16
+#else
+#define TARGET_MAX_INSN_SIZE 16
+#endif
+
 /* target supports implicit self modifying code */
 #define TARGET_HAS_SMC
 /* support for self modifying code even if the modified instruction is
diff --git a/target-i386/helper.h b/target-i386/helper.h
index 8eb0145..058302b 100644
--- a/target-i386/helper.h
+++ b/target-i386/helper.h
@@ -217,3 +217,5 @@ DEF_HELPER_3(rcrl, tl, env, tl, tl)
 DEF_HELPER_3(rclq, tl, env, tl, tl)
 DEF_HELPER_3(rcrq, tl, env, tl, tl)
 #endif
+
+DEF_HELPER_1(replay_instruction, i32, env)
diff --git a/target-i386/replay_helper.c b/target-i386/replay_helper.c
new file mode 100755
index 0000000..8b82eee
--- /dev/null
+++ b/target-i386/replay_helper.c
@@ -0,0 +1,33 @@
+/*
+ * replay_helper.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "cpu.h"
+#include "exec/helper-proto.h"
+#include "replay/replay.h"
+
+uint32_t helper_replay_instruction(CPUX86State *env)
+{
+    CPUState *cpu = ENV_GET_CPU(env);
+    if (replay_mode == REPLAY_PLAY
+        && !replay_has_instruction()) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    if (cpu->exit_request) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    int timer = replay_has_async_request();
+    replay_instruction(timer);
+    return timer;
+}
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 6fcd824..6ae3fc3 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -28,6 +28,7 @@
 #include "disas/disas.h"
 #include "tcg-op.h"
 #include "exec/cpu_ldst.h"
+#include "replay/replay.h"
 
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
@@ -112,6 +113,7 @@ typedef struct DisasContext {
     int tf;     /* TF cpu flag */
     int singlestep_enabled; /* "hardware" single step enabled */
     int jmp_opt; /* use direct block chaining for direct jumps */
+    int repz_opt; /* optimize jumps within repz instructions */
     int mem_index; /* select memory access functions */
     uint64_t flags; /* all execution flags */
     struct TranslationBlock *tb;
@@ -1212,7 +1214,7 @@ static inline void gen_repz_ ## op(DisasContext *s, TCGMemOp ot,              \
     gen_op_add_reg_im(s->aflag, R_ECX, -1);                                   \
     /* a loop would cause two single step exceptions if ECX = 1               \
        before rep string_insn */                                              \
-    if (!s->jmp_opt)                                                          \
+    if (!s->repz_opt)                                                         \
         gen_op_jz_ecx(s->aflag, l2);                                          \
     gen_jmp(s, cur_eip);                                                      \
 }
@@ -1230,7 +1232,7 @@ static inline void gen_repz_ ## op(DisasContext *s, TCGMemOp ot,              \
     gen_op_add_reg_im(s->aflag, R_ECX, -1);                                   \
     gen_update_cc_op(s);                                                      \
     gen_jcc1(s, (JCC_Z << 1) | (nz ^ 1), l2);                                 \
-    if (!s->jmp_opt)                                                          \
+    if (!s->repz_opt)                                                         \
         gen_op_jz_ecx(s->aflag, l2);                                          \
     gen_jmp(s, cur_eip);                                                      \
 }
@@ -7887,6 +7889,32 @@ void optimize_flags_init(void)
     }
 }
 
+static void gen_instr_replay(DisasContext *s, target_ulong pc_ptr)
+{
+    int l1 = gen_new_label();
+
+    gen_helper_replay_instruction(cpu_tmp2_i32, cpu_env);
+    tcg_gen_brcondi_i32(TCG_COND_EQ, cpu_tmp2_i32, 0, l1);
+
+    /* Don't reset dirty flag */
+    if (s->cc_op_dirty) {
+        tcg_gen_movi_i32(cpu_cc_op, s->cc_op);
+    }
+    gen_jmp_im(pc_ptr - s->cs_base);
+    tcg_gen_exit_tb(0);
+
+    gen_set_label(l1);
+}
+
+static void gen_instructions_count(void)
+{
+    tcg_gen_ld_i32(cpu_tmp2_i32, cpu_env, 
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_gen_addi_i32(cpu_tmp2_i32, cpu_tmp2_i32, 1);
+    tcg_gen_st_i32(cpu_tmp2_i32, cpu_env, 
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+}
+
 /* generate intermediate code in gen_opc_buf and gen_opparam_buf for
    basic block 'tb'. If search_pc is TRUE, also generate PC
    information for each intermediate instruction. */
@@ -7948,6 +7976,19 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
                     || (flags & HF_SOFTMMU_MASK)
 #endif
                     );
+    dc->repz_opt = dc->jmp_opt
+                    /* Do not optimize repz jumps at all in replay mode, because
+                       rep movsS instructions are execured with different paths
+                       in repz_opt and !repz_opt modes. The first one was used
+                       always except single step mode. And this setting
+                       disables jumps optimization and control paths become
+                       equivalent in run and single step modes.
+                       Now there will be no jump optimization for repz in
+                       trace and replay modes and there will always be an
+                       additional step for ecx=0.
+                     */
+                    || replay_mode != REPLAY_NONE
+                    ;
 #if 0
     /* check addseg logic */
     if (!dc->addseg && (dc->vm86 || !dc->pe || !dc->code32))
@@ -8000,8 +8041,22 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             tcg_ctx.gen_opc_instr_start[lj] = 1;
             tcg_ctx.gen_opc_icount[lj] = num_insns;
         }
-        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO))
+        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO)
+            && replay_mode == REPLAY_NONE) {
             gen_io_start();
+        }
+        /* generate instruction counter code for replay */
+        if (replay_mode != REPLAY_NONE) {
+            /* In PLAY mode check timer event at every instruction,
+               not only at the beginning of the block. This is needed,
+               when replaying has changed the bounds of translation blocks.
+             */
+            if (pc_ptr == pc_start || replay_mode == REPLAY_PLAY) {
+                gen_instr_replay(dc, pc_ptr);
+            } else {
+                gen_instructions_count();
+            }
+        }
 
         pc_ptr = disas_insn(env, dc, pc_ptr);
         num_insns++;
@@ -8019,6 +8074,20 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             gen_eob(dc);
             break;
         }
+        /* In replay mode do not cross the boundary of the pages,
+           it can cause an exception. Do it only when boundary is 
+           crossed by the first instruction in the block.
+           If current instruction already crossed the bound - it's ok,
+           because an exception hasn't stopped this code.
+         */
+        if (replay_mode != REPLAY_NONE
+            && ((pc_ptr & TARGET_PAGE_MASK) 
+                != ((pc_ptr + TARGET_MAX_INSN_SIZE - 1) & TARGET_PAGE_MASK)
+                || (pc_ptr & ~TARGET_PAGE_MASK) == 0)) {
+            gen_jmp_im(pc_ptr - dc->cs_base);
+            gen_eob(dc);
+            break;
+        }
         /* if too long translation, stop generation too */
         if (tcg_ctx.gen_opc_ptr >= gen_opc_end ||
             (pc_ptr - pc_start) >= (TARGET_PAGE_SIZE - 32) ||
@@ -8033,8 +8102,10 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             break;
         }
     }
-    if (tb->cflags & CF_LAST_IO)
+    if ((tb->cflags & CF_LAST_IO)
+        && replay_mode == REPLAY_NONE) {
         gen_io_end();
+    }
     gen_tb_end(tb, num_insns);
     *tcg_ctx.gen_opc_ptr = INDEX_op_end;
     /* we don't forget to fill the last values */

[Qemu-devel] [RFC PATCH v2 26/49] replay: interrupts and exceptions

[Pavel Dovgalyuk]

This patch includes modifications of common cpu files. All interrupts and
exceptions occured during recording are written into the replay log.
These events allow correct replaying the execution by kicking cpu thread
when one of these events is found in the log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpu-exec.c               |   14 +++++++++++---
 replay/replay-internal.h |    4 ++++
 replay/replay.c          |   42 ++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   14 ++++++++++++++
 4 files changed, 71 insertions(+), 3 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index 13c0ec6..fcde7ce 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -23,6 +23,7 @@
 #include "qemu/atomic.h"
 #include "sysemu/qtest.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 void cpu_loop_exit(CPUState *cpu)
 {
@@ -314,8 +315,14 @@ int cpu_exec(CPUArchState *env)
                     ret = cpu->exception_index;
                     break;
 #else
-                    cc->do_interrupt(cpu);
-                    cpu->exception_index = -1;
+                    if (replay_exception()) {
+                        cc->do_interrupt(cpu);
+                        cpu->exception_index = -1;
+                    } else if (!replay_has_interrupt()) {
+                        /* give a chance to iothread in replay mode */
+                        ret = EXCP_REPLAY;
+                        break;
+                    }
 #endif
                 }
             }
@@ -323,7 +330,8 @@ int cpu_exec(CPUArchState *env)
             next_tb = 0; /* force lookup of first TB */
             for(;;) {
                 interrupt_request = cpu->interrupt_request;
-                if (unlikely(interrupt_request)) {
+                if (unlikely((interrupt_request & CPU_INTERRUPT_DEBUG)
+                             || (interrupt_request && replay_interrupt()))) {
                     if (unlikely(cpu->singlestep_enabled & SSTEP_NOIRQ)) {
                         /* Mask out external interrupts for this step. */
                         interrupt_request &= ~CPU_INTERRUPT_SSTEP_MASK;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 9ebfa83..019ae20 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,6 +15,10 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+/* for software interrupt */
+#define EVENT_INTERRUPT             15
+/* for emulated exceptions */
+#define EVENT_EXCEPTION             23
 /* for async events */
 #define EVENT_ASYNC                 24
 /* for instruction event */
diff --git a/replay/replay.c b/replay/replay.c
index 8a9826e..41a27a9 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -159,3 +159,45 @@ uint64_t replay_get_current_step(void)
     }
     return replay_state.current_step;
 }
+
+bool replay_exception(void)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_save_instructions();
+        replay_put_event(EVENT_EXCEPTION);
+        return true;
+    } else if (replay_mode == REPLAY_PLAY) {
+        if (skip_async_events(EVENT_EXCEPTION)) {
+            replay_has_unread_data = 0;
+            return true;
+        }
+        return false;
+    }
+
+    return true;
+}
+
+bool replay_interrupt(void)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_save_instructions();
+        replay_put_event(EVENT_INTERRUPT);
+        return true;
+    } else if (replay_mode == REPLAY_PLAY) {
+        if (skip_async_events(EVENT_INTERRUPT)) {
+            replay_has_unread_data = 0;
+            return true;
+        }
+        return false;
+    }
+
+    return true;
+}
+
+bool replay_has_interrupt(void)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        return skip_async_events(EVENT_INTERRUPT);
+    }
+    return false;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 1ce7f78..10e6681 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -45,4 +45,18 @@ bool replay_has_async_request(void);
 /*! Returns non-zero if next event is instruction. */
 bool replay_has_instruction(void);
 
+/* Interrupts and exceptions */
+
+/*! Called by exception handler to write or read
+    exception processing events. */
+bool replay_exception(void);
+/*! Called by interrupt handlers to write or read
+    interrupt processing events.
+    \return true if interrupt should be processed */
+bool replay_interrupt(void);
+/*! Tries to read interrupt event from the file.
+    Returns true, when interrupt request is pending */
+bool replay_has_interrupt(void);
+
+
 #endif

[Qemu-devel] [RFC PATCH v2 27/49] vga: do not use virtual clock for blinking cursor

[Pavel Dovgalyuk]

This patch changes virtual clock to realtime one in vga_draw_text and
vga_update_display functions. Display update process depends on realtime
clock. If we read virtual clock there, virtual machine becomes
non-deterministic, because every read to virtual clock is written to the
replay log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/display/vga.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/display/vga.c b/hw/display/vga.c
index 4b089a3..02d3be3 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -1304,7 +1304,7 @@ static void vga_draw_text(VGACommonState *s, int full_update)
     uint32_t *ch_attr_ptr;
     vga_draw_glyph8_func *vga_draw_glyph8;
     vga_draw_glyph9_func *vga_draw_glyph9;
-    int64_t now = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL);
+    int64_t now = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
 
     /* compute font data address (in plane 2) */
     v = s->sr[VGA_SEQ_CHARACTER_MAP];
@@ -1906,7 +1906,7 @@ static void vga_update_display(void *opaque)
         }
         if (graphic_mode != s->graphic_mode) {
             s->graphic_mode = graphic_mode;
-            s->cursor_blink_time = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL);
+            s->cursor_blink_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
             full_update = 1;
         }
         switch(graphic_mode) {

[Qemu-devel] [RFC PATCH v2 28/49] replay: asynchronous events infrastructure

[Pavel Dovgalyuk]

This patch adds module for saving and replaying asynchronous events.
These events include network packets, keyboard and mouse input,
USB packets, thread pool and bottom halves callbacks.
All events are stored in the queue to be processed at synchronization points
such as beginning of TB execution, or checkpoint in the iothread.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 replay/replay-events.c   |  198 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   17 ++++
 replay/replay.h          |    4 +
 3 files changed, 218 insertions(+), 1 deletions(-)

diff --git a/replay/replay-events.c b/replay/replay-events.c
index eaffc48..eda574e 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -12,15 +12,211 @@
 #include "replay.h"
 #include "replay-internal.h"
 
+typedef struct Event {
+    int event_kind;
+    void *opaque;
+    void *opaque2;
+    uint64_t id;
+
+    QTAILQ_ENTRY(Event) events;
+} Event;
+
+static QTAILQ_HEAD(, Event) events_list = QTAILQ_HEAD_INITIALIZER(events_list);
+
+static QemuMutex lock;
+static unsigned int read_event_kind = -1;
+static uint64_t read_id = -1;
+static int read_opt = -1;
+
+static bool replay_events_enabled = false;
+
+/* Functions */
+
+static void replay_run_event(Event *event)
+{
+    switch (event->event_kind) {
+    default:
+        fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
+                event->event_kind);
+        exit(1);
+        break;
+    }
+}
+
+void replay_enable_events(void)
+{
+    replay_events_enabled = true;
+}
+
 bool replay_has_events(void)
 {
-    return false;
+    return !QTAILQ_EMPTY(&events_list);
+}
+
+void replay_flush_events(void)
+{
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        replay_run_event(event);
+        QTAILQ_REMOVE(&events_list, event, events);
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
+}
+
+void replay_disable_events(void)
+{
+    replay_events_enabled = false;
+    /* Flush events queue before waiting of completion */
+    replay_flush_events();
+}
+
+void replay_clear_events(void)
+{
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        QTAILQ_REMOVE(&events_list, event, events);
+
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
+}
+
+static void replay_add_event_internal(int event_kind, void *opaque, void *opaque2, uint64_t id)
+{
+    if (event_kind >= REPLAY_ASYNC_COUNT) {
+        fprintf(stderr, "Replay: invalid async event ID (%d)\n", event_kind);
+        exit(1);
+    }
+    if (!replay_file || replay_mode == REPLAY_NONE
+        || !replay_events_enabled) {
+        Event e;
+        e.event_kind = event_kind;
+        e.opaque = opaque;
+        e.opaque2 = opaque2;
+        e.id = id;
+        replay_run_event(&e);
+        return;
+    }
+
+    Event *event = g_malloc0(sizeof(Event));
+    event->event_kind = event_kind;
+    event->opaque = opaque;
+    event->opaque2 = opaque2;
+    event->id = id;
+
+    qemu_mutex_lock(&lock);
+    QTAILQ_INSERT_TAIL(&events_list, event, events);
+    qemu_mutex_unlock(&lock);
+}
+
+void replay_add_event(int event_kind, void *opaque)
+{
+    replay_add_event_internal(event_kind, opaque, NULL, 0);
 }
 
 void replay_save_events(int opt)
 {
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        if (replay_mode != REPLAY_PLAY) {
+            /* put the event into the file */
+            if (opt == -1) {
+                replay_put_event(EVENT_ASYNC);
+            } else {
+                replay_put_event(EVENT_ASYNC_OPT);
+                replay_put_byte(opt);
+            }
+            replay_put_byte(event->event_kind);
+
+            /* save event-specific data */
+            switch (event->event_kind) {
+            }
+        }
+
+        replay_run_event(event);
+        QTAILQ_REMOVE(&events_list, event, events);
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
 }
 
 void replay_read_events(int opt)
 {
+    replay_fetch_data_kind();
+    while ((opt == -1 && replay_data_kind == EVENT_ASYNC)
+        || (opt != -1 && replay_data_kind == EVENT_ASYNC_OPT)) {
+
+        if (read_event_kind == -1) {
+            if (opt != -1) {
+                read_opt = replay_get_byte();
+            }
+            read_event_kind = replay_get_byte();
+            read_id = -1;
+            replay_check_error();
+        }
+        
+        if (opt != read_opt)
+            break;
+        /* Execute some events without searching them in the queue */
+        switch (read_event_kind) {
+        default:
+            fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
+            exit(1);
+            break;
+        }
+
+        qemu_mutex_lock(&lock);
+
+        Event *event = NULL;
+        Event *curr = NULL;
+        QTAILQ_FOREACH(curr, &events_list, events) {
+            if (curr->event_kind == read_event_kind
+                && (read_id == -1 || read_id == curr->id)) {
+                event = curr;
+                break;
+            }
+        }
+
+        if (event) {
+            /* read event-specific reading data */
+
+            QTAILQ_REMOVE(&events_list, event, events);
+
+            qemu_mutex_unlock(&lock);
+
+            /* reset unread data and other parameters to allow
+               reading other data from the log while
+               running the event */
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_id = -1;
+            read_opt = -1;
+
+            replay_run_event(event);
+            g_free(event);
+
+            replay_fetch_data_kind();
+        } else {
+            qemu_mutex_unlock(&lock);
+            /* No such event found in the queue */
+            break;
+        }
+    }
+}
+
+void replay_init_events(void)
+{
+    read_event_kind = -1;
+    qemu_mutex_init(&lock);
+}
+
+void replay_finish_events(void)
+{
+    replay_events_enabled = false;
+    replay_clear_events();
+    qemu_mutex_destroy(&lock);
 }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 019ae20..cffc4e2 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -21,9 +21,14 @@
 #define EVENT_EXCEPTION             23
 /* for async events */
 #define EVENT_ASYNC                 24
+#define EVENT_ASYNC_OPT             25
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 
+/* Asynchronous events IDs */
+
+#define REPLAY_ASYNC_COUNT             0
+
 typedef struct ReplayState {
     /*! Nonzero, when next instruction is repeated one and was already
         processed. */
@@ -75,11 +80,23 @@ void skip_async_events_until(unsigned int kind);
 
 /* Asynchronous events queue */
 
+/*! Initializes events' processing internals */
+void replay_init_events(void);
+/*! Clears internal data structures for events handling */
+void replay_finish_events(void);
+/*! Enables storing events in the queue */
+void replay_enable_events(void);
+/*! Flushes events queue */
+void replay_flush_events(void);
+/*! Clears events list before loading new VM state */
+void replay_clear_events(void);
 /*! Returns true if there are any unsaved events in the queue */
 bool replay_has_events(void);
 /*! Saves events from queue into the file */
 void replay_save_events(int opt);
 /*! Read events from the file into the input queue */
 void replay_read_events(int opt);
+/*! Adds specified async event to the queue */
+void replay_add_event(int event_id, void *opaque);
 
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index 10e6681..7f461c5 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -58,5 +58,9 @@ bool replay_interrupt(void);
     Returns true, when interrupt request is pending */
 bool replay_has_interrupt(void);
 
+/* Asynchronous events queue */
+
+/*! Disables storing events in the queue */
+void replay_disable_events(void);
 
 #endif

[Qemu-devel] [RFC PATCH v2 29/49] replay: recording and replaying clock ticks

[Pavel Dovgalyuk]

Clock ticks are considered as the sources of non-deterministic data for
virtual machine. This patch implements saving the clock values when they
are acquired (virtual, host clock, rdtsc, and some other timers).
When replaying the execution corresponding values are read from log and
transfered to the module, which wants to read the values.
Such a design required the clock polling to be synchronized. Sometimes
it is not true - e.g. when timeouts for timer lists are checked. In this case
we use a cached value of the clock, passing it to the client code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c                   |    9 ++++++
 include/qemu/timer.h     |   42 +++++++++++++++++++++++++-
 replay/Makefile.objs     |    1 +
 replay/replay-internal.h |   11 +++++++
 replay/replay-time.c     |   75 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   17 ++++++++++
 stubs/replay.c           |    9 ++++++
 7 files changed, 162 insertions(+), 2 deletions(-)
 create mode 100755 replay/replay-time.c

diff --git a/cpus.c b/cpus.c
index eb2a795..15a2106 100644
--- a/cpus.c
+++ b/cpus.c
@@ -40,6 +40,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/seqlock.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #ifndef _WIN32
 #include "qemu/compatfd.h"
@@ -206,11 +207,19 @@ int64_t cpu_get_clock(void)
     int64_t ti;
     unsigned start;
 
+    if (replay_mode == REPLAY_PLAY) {
+        return replay_read_clock(REPLAY_CLOCK_VIRTUAL);
+    }
+
     do {
         start = seqlock_read_begin(&timers_state.vm_clock_seqlock);
         ti = cpu_get_clock_locked();
     } while (seqlock_read_retry(&timers_state.vm_clock_seqlock, start));
 
+    if (replay_mode == REPLAY_SAVE) {
+        replay_save_clock(REPLAY_CLOCK_VIRTUAL, ti);
+    }
+
     return ti;
 }
 
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 7f9a074..9eaed56 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -4,6 +4,7 @@
 #include "qemu/typedefs.h"
 #include "qemu-common.h"
 #include "qemu/notify.h"
+#include "replay/replay.h"
 
 /* timers */
 
@@ -699,8 +700,8 @@ static inline int64_t get_ticks_per_sec(void)
  * Low level clock functions
  */
 
-/* real time host monotonic timer */
-static inline int64_t get_clock_realtime(void)
+/* real time host monotonic timer implementation */
+static inline int64_t get_clock_realtime_impl(void)
 {
     struct timeval tv;
 
@@ -708,6 +709,23 @@ static inline int64_t get_clock_realtime(void)
     return tv.tv_sec * 1000000000LL + (tv.tv_usec * 1000);
 }
 
+/* real time host monotonic timer interface */
+static inline int64_t get_clock_realtime(void)
+{
+    int64_t result;
+
+    if (replay_mode == REPLAY_SAVE) {
+        result = get_clock_realtime_impl();
+        replay_save_clock(REPLAY_CLOCK_REALTIME, result);
+    } else if (replay_mode == REPLAY_PLAY) {
+        result = replay_read_clock(REPLAY_CLOCK_REALTIME);
+    } else {
+        result = get_clock_realtime_impl();
+    }
+
+    return result;
+}
+
 /* Warning: don't insert tracepoints into these functions, they are
    also used by simpletrace backend and tracepoints would cause
    an infinite recursion! */
@@ -749,6 +767,8 @@ int64_t cpu_get_clock(void);
 /*******************************************/
 /* host CPU ticks (if available) */
 
+#define cpu_get_real_ticks cpu_get_real_ticks_impl
+
 #if defined(_ARCH_PPC)
 
 static inline int64_t cpu_get_real_ticks(void)
@@ -902,6 +922,24 @@ static inline int64_t cpu_get_real_ticks (void)
 }
 #endif
 
+#undef cpu_get_real_ticks 
+
+static inline int64_t cpu_get_real_ticks(void)
+{
+    int64_t val;
+
+    if (replay_mode == REPLAY_SAVE) {
+        val = cpu_get_real_ticks_impl();
+        replay_save_clock(REPLAY_CLOCK_REAL_TICKS, val);
+    } else if (replay_mode == REPLAY_PLAY) {
+        val = replay_read_clock(REPLAY_CLOCK_REAL_TICKS);
+    } else {
+        val = cpu_get_real_ticks_impl();
+    }
+
+    return val;
+}
+
 #ifdef CONFIG_PROFILER
 static inline int64_t profile_getclock(void)
 {
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 56da09c..257c320 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1,3 +1,4 @@
 obj-y += replay.o
 obj-y += replay-internal.o
 obj-y += replay-events.o
+obj-y += replay-time.o
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index cffc4e2..a4fbc26 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -24,12 +24,17 @@
 #define EVENT_ASYNC_OPT             25
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
+/* for clock read/writes */
+#define EVENT_CLOCK                 64
+/* some of grteater codes are reserved for clocks */
 
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_COUNT             0
 
 typedef struct ReplayState {
+    /*! Cached clock values. */
+    int64_t cached_clock[REPLAY_CLOCK_COUNT];
     /*! Nonzero, when next instruction is repeated one and was already
         processed. */
     int skipping_instruction;
@@ -78,6 +83,12 @@ bool skip_async_events(int stop_event);
     reports an error and stops the execution. */
 void skip_async_events_until(unsigned int kind);
 
+/*! Reads next clock value from the file.
+    If clock kind read from the file is different from the parameter,
+    the value is not used.
+    If the parameter is -1, the clock value is read to the cache anyway. */
+void replay_read_next_clock(unsigned int kind);
+
 /* Asynchronous events queue */
 
 /*! Initializes events' processing internals */
diff --git a/replay/replay-time.c b/replay/replay-time.c
new file mode 100755
index 0000000..905d3e4
--- /dev/null
+++ b/replay/replay-time.c
@@ -0,0 +1,75 @@
+/*
+ * replay-time.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+
+
+void replay_save_clock(unsigned int kind, int64_t clock)
+{
+    replay_save_instructions();
+
+    if (kind >= REPLAY_CLOCK_COUNT) {
+        fprintf(stderr, "invalid clock ID %d for replay\n", kind);
+        exit(1);
+    }
+
+    if (replay_file) {
+        replay_put_event(EVENT_CLOCK + kind);
+        replay_put_qword(clock);
+    }
+}
+
+void replay_read_next_clock(unsigned int kind)
+{
+    replay_fetch_data_kind();
+    if (replay_file) {
+        unsigned int read_kind = replay_data_kind - EVENT_CLOCK;
+
+        if (kind != -1 && read_kind != kind) {
+            return;
+        }
+        if (read_kind >= REPLAY_CLOCK_COUNT) {
+            fprintf(stderr, 
+                    "invalid clock ID %d was read from replay\n", read_kind);
+            exit(1);
+        }
+
+        int64_t clock = replay_get_qword();
+
+        replay_check_error();
+        replay_has_unread_data = 0;
+
+        replay_state.cached_clock[read_kind] = clock;
+    }
+}
+
+/*! Reads next clock event from the input. */
+int64_t replay_read_clock(unsigned int kind)
+{
+    if (kind >= REPLAY_CLOCK_COUNT) {
+        fprintf(stderr, "invalid clock ID %d for replay\n", kind);
+        exit(1);
+    }
+
+    if (replay_file) {
+        if (skip_async_events(EVENT_CLOCK + kind)) {
+            replay_read_next_clock(kind);
+        }
+        int64_t ret = replay_state.cached_clock[kind];
+        
+        return ret;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
diff --git a/replay/replay.h b/replay/replay.h
index 7f461c5..3d09ec8 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -24,6 +24,16 @@
 /* Normal log replaying */
 #define REPLAY_PLAY_NORMAL  1
 
+/* replay clock kinds */
+/* rdtsc */
+#define REPLAY_CLOCK_REAL_TICKS 0
+/* host_clock */
+#define REPLAY_CLOCK_REALTIME   1
+/* vm_clock */
+#define REPLAY_CLOCK_VIRTUAL    2
+
+#define REPLAY_CLOCK_COUNT      3
+
 extern int replay_mode;
 extern char *replay_image_suffix;
 /*! Shift value for icount based on replay or zero, if it is disabled. */
@@ -58,6 +68,13 @@ bool replay_interrupt(void);
     Returns true, when interrupt request is pending */
 bool replay_has_interrupt(void);
 
+/* Processing clocks and other time sources */
+
+/*! Save the specified clock */
+void replay_save_clock(unsigned int kind, int64_t clock);
+/*! Read the specified clock from the log or return cached data */
+int64_t replay_read_clock(unsigned int kind);
+
 /* Asynchronous events queue */
 
 /*! Disables storing events in the queue */
diff --git a/stubs/replay.c b/stubs/replay.c
index 1e99ca4..3b88b1a 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -7,3 +7,12 @@ int replay_get_play_submode(void)
 {
     return 0;
 }
+
+void replay_save_clock(unsigned int kind, int64_t clock)
+{
+}
+
+int64_t replay_read_clock(unsigned int kind)
+{
+    return 0;
+}

[Qemu-devel] [RFC PATCH v2 30/49] replay: recording and replaying different timers

[Pavel Dovgalyuk]

This patch introduces functions for recording and replaying realtime sources,
that do not use qemu-clock interface. These include return value of time()
function in time_t and struct tm forms. Patch also adds warning to
get_timedate function to prevent its usage in recording mode, because it may
lead to non-determinism.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/mc146818rtc.c   |    2 -
 hw/timer/pl031.c         |   10 ++++
 include/qemu-common.h    |    1 
 replay/replay-internal.h |    7 +++
 replay/replay-time.c     |  106 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |    8 +++
 vl.c                     |   16 ++++++-
 7 files changed, 145 insertions(+), 5 deletions(-)

diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index c204abb..b7c842f 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -703,7 +703,7 @@ static void rtc_set_date_from_host(ISADevice *dev)
     RTCState *s = MC146818_RTC(dev);
     struct tm tm;
 
-    qemu_get_timedate(&tm, 0);
+    qemu_get_timedate_no_warning(&tm, 0);
 
     s->base_rtc = mktimegm(&tm);
     s->last_update = qemu_clock_get_ns(rtc_clock);
diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index f8e5abc..02c814f 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -14,6 +14,7 @@
 #include "hw/sysbus.h"
 #include "qemu/timer.h"
 #include "sysemu/sysemu.h"
+#include "replay/replay.h"
 
 //#define DEBUG_PL031
 
@@ -200,7 +201,14 @@ static int pl031_init(SysBusDevice *dev)
     sysbus_init_mmio(dev, &s->iomem);
 
     sysbus_init_irq(dev, &s->irq);
-    qemu_get_timedate(&tm, 0);
+    if (replay_mode == REPLAY_SAVE) {
+        qemu_get_timedate_no_warning(&tm, 0);
+        replay_save_tm(&tm);
+    } else if (replay_mode == REPLAY_PLAY) {
+        replay_read_tm(&tm);
+    } else {
+        qemu_get_timedate_no_warning(&tm, 0);
+    }
     s->tick_offset = mktimegm(&tm) -
         qemu_clock_get_ns(rtc_clock) / get_ticks_per_sec();
 
diff --git a/include/qemu-common.h b/include/qemu-common.h
index ae76197..48013ce 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -121,6 +121,7 @@ extern int use_icount;
 int qemu_main(int argc, char **argv, char **envp);
 #endif
 
+void qemu_get_timedate_no_warning(struct tm *tm, int offset);
 void qemu_get_timedate(struct tm *tm, int offset);
 int qemu_timedate_diff(struct tm *tm);
 
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index a4fbc26..79e23ee 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,10 +15,17 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+<<<<<<< current
 /* for software interrupt */
 #define EVENT_INTERRUPT             15
 /* for emulated exceptions */
 #define EVENT_EXCEPTION             23
+=======
+/* for time_t event */
+#define EVENT_TIME_T                1
+/* for tm event */
+#define EVENT_TM                    2
+>>>>>>> patched
 /* for async events */
 #define EVENT_ASYNC                 24
 #define EVENT_ASYNC_OPT             25
diff --git a/replay/replay-time.c b/replay/replay-time.c
index 905d3e4..0811d22 100755
--- a/replay/replay-time.c
+++ b/replay/replay-time.c
@@ -73,3 +73,109 @@ int64_t replay_read_clock(unsigned int kind)
     fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
     exit(1);
 }
+
+/*! Saves time_t value to the log */
+static void replay_save_time_t(time_t tm)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        replay_put_event(EVENT_TIME_T);
+        if (sizeof(tm) == 4) {
+            replay_put_dword(tm);
+        } else if (sizeof(tm) == 8) {
+            replay_put_qword(tm);
+        } else {
+            fprintf(stderr, "invalid time_t sizeof: %u\n", (unsigned)sizeof(tm));
+            exit(1);
+        }
+    }
+}
+
+/*! Reads time_t value from the log. Stops execution in case of error */
+static time_t replay_read_time_t(void)
+{
+    if (replay_file) {
+        time_t tm;
+
+        skip_async_events_until(EVENT_TIME_T);
+
+        if (sizeof(tm) == 4) {
+            tm = replay_get_dword();
+        } else if (sizeof(tm) == 8) {
+            tm = replay_get_qword();
+        } else {
+            fprintf(stderr, "invalid time_t sizeof: %u\n", (unsigned)sizeof(tm));
+            exit(1);
+        }
+
+        replay_check_error();
+
+        replay_has_unread_data = 0;
+
+        return tm;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
+
+void replay_save_tm(struct tm *tm)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        replay_put_event(EVENT_TM);
+
+        replay_put_dword(tm->tm_sec);
+        replay_put_dword(tm->tm_min);
+        replay_put_dword(tm->tm_hour);
+        replay_put_dword(tm->tm_mday);
+        replay_put_dword(tm->tm_mon);
+        replay_put_dword(tm->tm_year);
+        replay_put_dword(tm->tm_wday);
+        replay_put_dword(tm->tm_yday);
+        replay_put_dword(tm->tm_isdst);
+    }
+}
+
+void replay_read_tm(struct tm *tm)
+{
+    if (replay_file) {
+        skip_async_events_until(EVENT_TM);
+
+        tm->tm_sec = replay_get_dword();
+        tm->tm_min = replay_get_dword();
+        tm->tm_hour = replay_get_dword();
+        tm->tm_mday = replay_get_dword();
+        tm->tm_mon = replay_get_dword();
+        tm->tm_year = replay_get_dword();
+        tm->tm_wday = replay_get_dword();
+        tm->tm_yday = replay_get_dword();
+        tm->tm_isdst = replay_get_dword();
+
+        replay_check_error();
+        replay_has_unread_data = 0;
+
+        return;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
+
+time_t replay_time(void)
+{
+    time_t systime;
+
+    if (replay_mode == REPLAY_SAVE) {
+        systime = time(NULL);
+        replay_save_time_t(systime);
+    } else if (replay_mode == REPLAY_PLAY) {
+        systime = replay_read_time_t();
+    } else {
+        systime = time(NULL);
+    }
+
+    return systime;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 3d09ec8..16e368c 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -14,6 +14,7 @@
 
 #include <stdbool.h>
 #include <stdint.h>
+#include <time.h>
 
 /* replay modes */
 #define REPLAY_NONE 0
@@ -74,6 +75,13 @@ bool replay_has_interrupt(void);
 void replay_save_clock(unsigned int kind, int64_t clock);
 /*! Read the specified clock from the log or return cached data */
 int64_t replay_read_clock(unsigned int kind);
+/*! Returns result of time() function execution in normal and record modes.
+    In play mode returns value read from the log. */
+time_t replay_time(void);
+/*! Saves struct tm value to the log */
+void replay_save_tm(struct tm *tm);
+/*! Reads struct tm value from the log. Stops execution in case of error */
+void replay_read_tm(struct tm *tm);
 
 /* Asynchronous events queue */
 
diff --git a/vl.c b/vl.c
index 6e084c2..86aba43 100644
--- a/vl.c
+++ b/vl.c
@@ -118,6 +118,7 @@ int main(int argc, char **argv)
 #include "qapi/opts-visitor.h"
 #include "qom/object_interfaces.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #define DEFAULT_RAM_SIZE 128
 
@@ -757,7 +758,7 @@ void vm_start(void)
 
 /***********************************************************/
 /* host time/date access */
-void qemu_get_timedate(struct tm *tm, int offset)
+void qemu_get_timedate_no_warning(struct tm *tm, int offset)
 {
     time_t ti;
 
@@ -774,6 +775,15 @@ void qemu_get_timedate(struct tm *tm, int offset)
     }
 }
 
+/* host time/date access with replay warning */
+void qemu_get_timedate(struct tm *tm, int offset)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        fprintf(stderr, "REPLAY WARNING! qemu_get_timedate function may lead to non-determinism\n");
+    }
+    qemu_get_timedate_no_warning(tm, offset);
+}
+
 int qemu_timedate_diff(struct tm *tm)
 {
     time_t seconds;
@@ -789,7 +799,7 @@ int qemu_timedate_diff(struct tm *tm)
     else
         seconds = mktimegm(tm) + rtc_date_offset;
 
-    return seconds - time(NULL);
+    return seconds - replay_time();
 }
 
 static void configure_rtc_date_offset(const char *startdate, int legacy)
@@ -827,7 +837,7 @@ static void configure_rtc_date_offset(const char *startdate, int legacy)
                             "'2006-06-17T16:01:21' or '2006-06-17'\n");
             exit(1);
         }
-        rtc_date_offset = time(NULL) - rtc_start_date;
+        rtc_date_offset = replay_time() - rtc_start_date;
     }
 }
 

[Qemu-devel] [RFC PATCH v2 31/49] replay: shutdown event

[Pavel Dovgalyuk]

This patch records and replays simulator shutdown event.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/sysemu/sysemu.h  |    1 +
 replay/replay-internal.h |   13 ++++++-------
 replay/replay.c          |   11 +++++++++++
 replay/replay.h          |    5 +++++
 vl.c                     |    8 +++++++-
 5 files changed, 30 insertions(+), 8 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 1ebfef9..9000feb 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -57,6 +57,7 @@ void qemu_register_suspend_notifier(Notifier *notifier);
 void qemu_system_wakeup_request(WakeupReason reason);
 void qemu_system_wakeup_enable(WakeupReason reason, bool enabled);
 void qemu_register_wakeup_notifier(Notifier *notifier);
+void qemu_system_shutdown_request_impl(void);
 void qemu_system_shutdown_request(void);
 void qemu_system_powerdown_request(void);
 void qemu_register_powerdown_notifier(Notifier *notifier);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 79e23ee..033dbdc 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,17 +15,16 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
-<<<<<<< current
-/* for software interrupt */
-#define EVENT_INTERRUPT             15
-/* for emulated exceptions */
-#define EVENT_EXCEPTION             23
-=======
 /* for time_t event */
 #define EVENT_TIME_T                1
 /* for tm event */
 #define EVENT_TM                    2
->>>>>>> patched
+/* for software interrupt */
+#define EVENT_INTERRUPT             15
+/* for shutdown request */
+#define EVENT_SHUTDOWN              20
+/* for emulated exceptions */
+#define EVENT_EXCEPTION             23
 /* for async events */
 #define EVENT_ASYNC                 24
 #define EVENT_ASYNC_OPT             25
diff --git a/replay/replay.c b/replay/replay.c
index 41a27a9..2067d6b 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -43,6 +43,10 @@ bool skip_async_events(int stop_event)
             res = true;
         }
         switch (replay_data_kind) {
+        case EVENT_SHUTDOWN:
+            replay_has_unread_data = 0;
+            qemu_system_shutdown_request_impl();
+            break;
         case EVENT_INSTRUCTION:
             first_cpu->instructions_count = replay_get_dword();
             return res;
@@ -201,3 +205,10 @@ bool replay_has_interrupt(void)
     }
     return false;
 }
+
+void replay_shutdown_request(void)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_put_event(EVENT_SHUTDOWN);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index 16e368c..a84c57c 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -83,6 +83,11 @@ void replay_save_tm(struct tm *tm);
 /*! Reads struct tm value from the log. Stops execution in case of error */
 void replay_read_tm(struct tm *tm);
 
+/* Events */
+
+/*! Called when qemu shutdown is requested. */
+void replay_shutdown_request(void);
+
 /* Asynchronous events queue */
 
 /*! Disables storing events in the queue */
diff --git a/vl.c b/vl.c
index 86aba43..1a837e6 100644
--- a/vl.c
+++ b/vl.c
@@ -1929,13 +1929,19 @@ void qemu_system_killed(int signal, pid_t pid)
     qemu_system_shutdown_request();
 }
 
-void qemu_system_shutdown_request(void)
+void qemu_system_shutdown_request_impl(void)
 {
     trace_qemu_system_shutdown_request();
     shutdown_requested = 1;
     qemu_notify_event();
 }
 
+void qemu_system_shutdown_request(void)
+{
+    replay_shutdown_request();
+    qemu_system_shutdown_request_impl();
+}
+
 static void qemu_system_powerdown(void)
 {
     qapi_event_send_powerdown(&error_abort);

[Qemu-devel] [RFC PATCH v2 32/49] replay: checkpoints

[Pavel Dovgalyuk]

This patch introduces checkpoints that synchronize cpu thread and iothread.
When checkpoint is met in the code all asynchronous events from the queue
are executed.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c                  |   11 +++++++++++
 cpus.c                   |    2 +-
 include/qemu/timer.h     |    6 ++++--
 qemu-timer.c             |   41 +++++++++++++++++++++++++++++++++--------
 replay/replay-internal.h |    3 +++
 replay/replay.c          |   28 ++++++++++++++++++++++++++++
 replay/replay.h          |    6 ++++++
 stubs/replay.c           |   11 +++++++++++
 vl.c                     |    3 ++-
 9 files changed, 99 insertions(+), 12 deletions(-)

diff --git a/block.c b/block.c
index 3ff1052..fb6c849 100644
--- a/block.c
+++ b/block.c
@@ -1914,6 +1914,11 @@ void bdrv_drain_all(void)
     BlockDriverState *bs;
 
     while (busy) {
+        if (!replay_checkpoint(8)) {
+            /* Do not wait anymore, we stopped at some place in 
+               the middle of execution during replay */
+            return;
+        }
         busy = false;
 
         QTAILQ_FOREACH(bs, &bdrv_states, device_list) {
@@ -1930,6 +1935,12 @@ void bdrv_drain_all(void)
             busy |= bs_busy;
         }
     }
+    if (replay_mode == REPLAY_PLAY) {
+        /* Skip checkpoints from the log */
+        while (replay_checkpoint(8)) {
+            /* Nothing */
+        }
+    }
 }
 
 /* make a BlockDriverState anonymous by removing from bdrv_state and
diff --git a/cpus.c b/cpus.c
index 15a2106..4f517ee 100644
--- a/cpus.c
+++ b/cpus.c
@@ -363,7 +363,7 @@ void qtest_clock_warp(int64_t dest)
         qemu_icount_bias += warp;
         seqlock_write_unlock(&timers_state.vm_clock_seqlock);
 
-        qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
+        qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL, false);
         clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
     }
     qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 9eaed56..6b00341 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -232,13 +232,14 @@ void qemu_clock_unregister_reset_notifier(QEMUClockType type,
 /**
  * qemu_clock_run_timers:
  * @type: clock on which to operate
+ * @run_all: true, when called from qemu_clock_run_all_timers
  *
  * Run all the timers associated with the default timer list
  * of a clock.
  *
  * Returns: true if any timer ran.
  */
-bool qemu_clock_run_timers(QEMUClockType type);
+bool qemu_clock_run_timers(QEMUClockType type, bool run_all);
 
 /**
  * qemu_clock_run_all_timers:
@@ -329,12 +330,13 @@ QEMUClockType timerlist_get_clock(QEMUTimerList *timer_list);
 /**
  * timerlist_run_timers:
  * @timer_list: the timer list to use
+ * @run_all: true, when called from qemu_clock_run_all_timers
  *
  * Call all expired timers associated with the timer list.
  *
  * Returns: true if any timer expired
  */
-bool timerlist_run_timers(QEMUTimerList *timer_list);
+bool timerlist_run_timers(QEMUTimerList *timer_list, bool run_all);
 
 /**
  * timerlist_notify:
diff --git a/qemu-timer.c b/qemu-timer.c
index 00a5d35..dcf9b14 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -457,7 +457,7 @@ bool timer_expired(QEMUTimer *timer_head, int64_t current_time)
     return timer_expired_ns(timer_head, current_time * timer_head->scale);
 }
 
-bool timerlist_run_timers(QEMUTimerList *timer_list)
+bool timerlist_run_timers(QEMUTimerList *timer_list, bool run_all)
 {
     QEMUTimer *ts;
     int64_t current_time;
@@ -465,6 +465,24 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)
     QEMUTimerCB *cb;
     void *opaque;
 
+    switch (timer_list->clock->type) {
+    case QEMU_CLOCK_REALTIME:
+        break;
+    default:
+    case QEMU_CLOCK_VIRTUAL:
+        if ((replay_mode != REPLAY_NONE && !runstate_is_running()) 
+            || !replay_checkpoint(run_all ? 2 : 3)) {
+            return false;
+        }
+        break;
+    case QEMU_CLOCK_HOST:
+        if ((replay_mode != REPLAY_NONE && !runstate_is_running()) 
+            || !replay_checkpoint(run_all ? 5 : 6)) {
+            return false;
+        }
+        break;
+    }
+
     qemu_event_reset(&timer_list->timers_done_ev);
     if (!timer_list->clock->enabled) {
         goto out;
@@ -497,9 +515,9 @@ out:
     return progress;
 }
 
-bool qemu_clock_run_timers(QEMUClockType type)
+bool qemu_clock_run_timers(QEMUClockType type, bool run_all)
 {
-    return timerlist_run_timers(main_loop_tlg.tl[type]);
+    return timerlist_run_timers(main_loop_tlg.tl[type], run_all);
 }
 
 void timerlistgroup_init(QEMUTimerListGroup *tlg,
@@ -524,7 +542,7 @@ bool timerlistgroup_run_timers(QEMUTimerListGroup *tlg)
     QEMUClockType type;
     bool progress = false;
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
-        progress |= timerlist_run_timers(tlg->tl[type]);
+        progress |= timerlist_run_timers(tlg->tl[type], false);
     }
     return progress;
 }
@@ -533,11 +551,18 @@ int64_t timerlistgroup_deadline_ns(QEMUTimerListGroup *tlg)
 {
     int64_t deadline = -1;
     QEMUClockType type;
+    bool play = replay_mode == REPLAY_PLAY;
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
         if (qemu_clock_use_for_deadline(tlg->tl[type]->clock->type)) {
-            deadline = qemu_soonest_timeout(deadline,
-                                            timerlist_deadline_ns(
-                                                tlg->tl[type]));
+            if (!play || tlg->tl[type]->clock->type == QEMU_CLOCK_REALTIME) {
+                deadline = qemu_soonest_timeout(deadline,
+                                                timerlist_deadline_ns(
+                                                    tlg->tl[type]));
+            } else {
+                /* Read clock from the replay file and 
+                   do not calculate the deadline, based on virtual clock. */
+                qemu_clock_get_ns(tlg->tl[type]->clock->type);
+            }
         }
     }
     return deadline;
@@ -605,7 +630,7 @@ bool qemu_clock_run_all_timers(void)
     QEMUClockType type;
 
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
-        progress |= qemu_clock_run_timers(type);
+        progress |= qemu_clock_run_timers(type, true);
     }
 
     return progress;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 033dbdc..95cbde6 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -34,6 +34,9 @@
 #define EVENT_CLOCK                 64
 /* some of grteater codes are reserved for clocks */
 
+/* for checkpoint event */
+#define EVENT_CHECKPOINT            96
+
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_COUNT             0
diff --git a/replay/replay.c b/replay/replay.c
index 2067d6b..225d8ec 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -212,3 +212,31 @@ void replay_shutdown_request(void)
         replay_put_event(EVENT_SHUTDOWN);
     }
 }
+
+/* Used checkpoints: 2 3 5 6 7 8 9 */
+int replay_checkpoint(unsigned int checkpoint)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        if (replay_mode == REPLAY_PLAY) {
+            if (!skip_async_events(EVENT_CHECKPOINT + checkpoint)) {
+                if (replay_data_kind == EVENT_ASYNC_OPT) {
+                    replay_read_events(checkpoint);
+                    replay_fetch_data_kind();
+                    return replay_data_kind != EVENT_ASYNC_OPT;
+                }
+                return 0;
+            }
+            replay_has_unread_data = 0;
+            replay_read_events(checkpoint);
+            replay_fetch_data_kind();
+            return replay_data_kind != EVENT_ASYNC_OPT;
+        } else if (replay_mode == REPLAY_SAVE) {
+            replay_put_event(EVENT_CHECKPOINT + checkpoint);
+            replay_save_events(checkpoint);
+        }
+    }
+
+    return 1;
+}
diff --git a/replay/replay.h b/replay/replay.h
index a84c57c..69a20af 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -87,6 +87,12 @@ void replay_read_tm(struct tm *tm);
 
 /*! Called when qemu shutdown is requested. */
 void replay_shutdown_request(void);
+/*! Should be called at check points in the execution.
+    These check points are skipped, if they were not met.
+    Saves checkpoint in the SAVE mode and validates in the PLAY mode.
+    Returns 0 in PLAY mode if checkpoint was not found.
+    Returns 1 in all other cases. */
+int replay_checkpoint(unsigned int checkpoint);
 
 /* Asynchronous events queue */
 
diff --git a/stubs/replay.c b/stubs/replay.c
index 3b88b1a..1b0b78f 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -1,4 +1,5 @@
 #include "replay/replay.h"
+#include "sysemu/sysemu.h"
 
 int replay_mode;
 int replay_icount;
@@ -16,3 +17,13 @@ int64_t replay_read_clock(unsigned int kind)
 {
     return 0;
 }
+
+int replay_checkpoint(unsigned int checkpoint)
+{
+    return 0;
+}
+
+int runstate_is_running(void)
+{
+    return 0;
+}
diff --git a/vl.c b/vl.c
index 1a837e6..053b2ba 100644
--- a/vl.c
+++ b/vl.c
@@ -1984,7 +1984,8 @@ static bool main_loop_should_exit(void)
             return true;
         }
     }
-    if (qemu_reset_requested()) {
+    if (qemu_reset_requested_get() && replay_checkpoint(7)) {
+        qemu_reset_requested();
         pause_all_vcpus();
         cpu_synchronize_all_states();
         qemu_system_reset(VMRESET_REPORT);

[Qemu-devel] [RFC PATCH v2 33/49] replay: bottom halves

[Pavel Dovgalyuk]

This patch introduces bottom half event for replay queue. It saves the events
into the queue and process them at the checkpoints and instructions execution.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 async.c                  |   45 +++++++++++++++++++++++++++++++++++++++------
 dma-helpers.c            |    4 +++-
 hw/ide/ahci.c            |    4 +++-
 hw/ide/core.c            |    4 +++-
 hw/timer/arm_timer.c     |    2 +-
 hw/usb/hcd-uhci.c        |    2 +-
 include/block/aio.h      |   17 +++++++++++++++++
 include/qemu/main-loop.h |    1 +
 main-loop.c              |    5 +++++
 replay/replay-events.c   |   16 ++++++++++++++++
 replay/replay-internal.h |    3 ++-
 replay/replay.h          |    2 ++
 stubs/replay.c           |    4 ++++
 13 files changed, 97 insertions(+), 12 deletions(-)

diff --git a/async.c b/async.c
index 5b6fe6b..687cdb2 100644
--- a/async.c
+++ b/async.c
@@ -26,6 +26,7 @@
 #include "block/aio.h"
 #include "block/thread-pool.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 /***********************************************************/
 /* bottom halves (can be seen as timers which expire ASAP) */
@@ -38,24 +39,52 @@ struct QEMUBH {
     bool scheduled;
     bool idle;
     bool deleted;
+    bool replay;
+    uint64_t id;
 };
 
 QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
 {
-    QEMUBH *bh;
+    QEMUBH *bh, **last;
     bh = g_malloc0(sizeof(QEMUBH));
     bh->ctx = ctx;
     bh->cb = cb;
     bh->opaque = opaque;
     qemu_mutex_lock(&ctx->bh_lock);
-    bh->next = ctx->first_bh;
-    /* Make sure that the members are ready before putting bh into list */
-    smp_wmb();
-    ctx->first_bh = bh;
+    if (replay_mode != REPLAY_NONE) {
+        /* Slower way, but this is a queue and not a stack.
+           Replay will process the BH in the same order they 
+           came into the queue. */
+        last = &ctx->first_bh;
+        while (*last) {
+            last = &(*last)->next;
+        }
+        smp_wmb();
+        *last = bh;
+    } else {
+        bh->next = ctx->first_bh;
+        /* Make sure that the members are ready before putting bh into list */
+        smp_wmb();
+        ctx->first_bh = bh;
+    }
     qemu_mutex_unlock(&ctx->bh_lock);
     return bh;
 }
 
+QEMUBH *aio_bh_new_replay(AioContext *ctx, QEMUBHFunc *cb, void *opaque, uint64_t id)
+{
+    QEMUBH *bh = aio_bh_new(ctx, cb, opaque);
+    bh->replay = true;
+    bh->id = id;
+    return bh;
+}
+
+void aio_bh_call(void *opaque)
+{
+    QEMUBH *bh = (QEMUBH *)opaque;
+    bh->cb(bh->opaque);
+}
+
 /* Multiple occurrences of aio_bh_poll cannot be called concurrently */
 int aio_bh_poll(AioContext *ctx)
 {
@@ -78,7 +107,11 @@ int aio_bh_poll(AioContext *ctx)
             if (!bh->idle)
                 ret = 1;
             bh->idle = 0;
-            bh->cb(bh->opaque);
+            if (!bh->replay) {
+                aio_bh_call(bh);
+            } else {
+                replay_add_bh_event(bh, bh->id);
+            }
         }
     }
 
diff --git a/dma-helpers.c b/dma-helpers.c
index 53cbe92..05aadea 100644
--- a/dma-helpers.c
+++ b/dma-helpers.c
@@ -12,6 +12,7 @@
 #include "qemu/range.h"
 #include "qemu/thread.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 /* #define DEBUG_IOMMU */
 
@@ -96,7 +97,8 @@ static void continue_after_map_failure(void *opaque)
 {
     DMAAIOCB *dbs = (DMAAIOCB *)opaque;
 
-    dbs->bh = qemu_bh_new(reschedule_dma, dbs);
+    dbs->bh = qemu_bh_new_replay(reschedule_dma, dbs,
+                                 replay_get_current_step());
     qemu_bh_schedule(dbs->bh);
 }
 
diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c
index 604152a..0762743 100644
--- a/hw/ide/ahci.c
+++ b/hw/ide/ahci.c
@@ -32,6 +32,7 @@
 #include "internal.h"
 #include <hw/ide/pci.h>
 #include <hw/ide/ahci.h>
+#include "replay/replay.h"
 
 /* #define DEBUG_AHCI */
 
@@ -1132,7 +1133,8 @@ static int ahci_async_cmd_done(IDEDMA *dma)
 
     if (!ad->check_bh) {
         /* maybe we still have something to process, check later */
-        ad->check_bh = qemu_bh_new(ahci_check_cmd_bh, ad);
+        ad->check_bh = qemu_bh_new_replay(ahci_check_cmd_bh, ad,
+                                          replay_get_current_step());
         qemu_bh_schedule(ad->check_bh);
     }
 
diff --git a/hw/ide/core.c b/hw/ide/core.c
index 3a38f1e..3df26f0 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -32,6 +32,7 @@
 #include "sysemu/dma.h"
 #include "hw/block/block.h"
 #include "sysemu/blockdev.h"
+#include "replay/replay.h"
 
 #include <hw/ide/internal.h>
 
@@ -409,7 +410,8 @@ BlockDriverAIOCB *ide_issue_trim(BlockDriverState *bs,
     TrimAIOCB *iocb;
 
     iocb = qemu_aio_get(&trim_aiocb_info, bs, cb, opaque);
-    iocb->bh = qemu_bh_new(ide_trim_bh_cb, iocb);
+    iocb->bh = qemu_bh_new_replay(ide_trim_bh_cb, iocb,
+                                  replay_get_current_step());
     iocb->ret = 0;
     iocb->qiov = qiov;
     iocb->i = -1;
diff --git a/hw/timer/arm_timer.c b/hw/timer/arm_timer.c
index 1452910..97784a0 100644
--- a/hw/timer/arm_timer.c
+++ b/hw/timer/arm_timer.c
@@ -168,7 +168,7 @@ static arm_timer_state *arm_timer_init(uint32_t freq)
     s->freq = freq;
     s->control = TIMER_CTRL_IE;
 
-    bh = qemu_bh_new(arm_timer_tick, s);
+    bh = qemu_bh_new_replay(arm_timer_tick, s, 0);
     s->timer = ptimer_init(bh);
     vmstate_register(NULL, -1, &vmstate_arm_timer, s);
     return s;
diff --git a/hw/usb/hcd-uhci.c b/hw/usb/hcd-uhci.c
index c3bf72c..3aaf506 100644
--- a/hw/usb/hcd-uhci.c
+++ b/hw/usb/hcd-uhci.c
@@ -1224,7 +1224,7 @@ static int usb_uhci_common_initfn(PCIDevice *dev)
                               USB_SPEED_MASK_LOW | USB_SPEED_MASK_FULL);
         }
     }
-    s->bh = qemu_bh_new(uhci_bh, s);
+    s->bh = qemu_bh_new_replay(uhci_bh, s, 0);
     s->frame_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, uhci_frame_timer, s);
     s->num_ports_vmstate = NB_PORTS;
     QTAILQ_INIT(&s->queues);
diff --git a/include/block/aio.h b/include/block/aio.h
index a92511b..3268ffa 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -34,6 +34,8 @@ struct BlockDriverAIOCB {
     const AIOCBInfo *aiocb_info;
     BlockDriverState *bs;
     BlockDriverCompletionFunc *cb;
+    bool replay;
+    uint64_t replay_step;
     void *opaque;
 };
 
@@ -132,6 +134,16 @@ void aio_context_release(AioContext *ctx);
 QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
 
 /**
+ * aio_bh_new_replay: Allocate a new bottom half structure for replay.
+ *
+ * This function calls aio_bh_new function and also fills replay parameters
+ * of the BH structure. BH created with this function in record/replay mode
+ * are executed through the replay queue only at checkpoints and instructions
+ * executions.
+ */
+QEMUBH *aio_bh_new_replay(AioContext *ctx, QEMUBHFunc *cb, void *opaque, uint64_t id);
+
+/**
  * aio_notify: Force processing of pending events.
  *
  * Similar to signaling a condition variable, aio_notify forces
@@ -147,6 +159,11 @@ QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
 void aio_notify(AioContext *ctx);
 
 /**
+ * aio_bh_call: Executes callback function of the specified BH.
+ */
+void aio_bh_call(void *opaque);
+
+/**
  * aio_bh_poll: Poll bottom halves for an AioContext.
  *
  * These are internal functions used by the QEMU main loop.
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 6f0200a..c221ddd 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -306,6 +306,7 @@ void qemu_iohandler_fill(GArray *pollfds);
 void qemu_iohandler_poll(GArray *pollfds, int rc);
 
 QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque);
+QEMUBH *qemu_bh_new_replay(QEMUBHFunc *cb, void *opaque, uint64_t id);
 void qemu_bh_schedule_idle(QEMUBH *bh);
 
 #endif
diff --git a/main-loop.c b/main-loop.c
index 8a85493..4e1af51 100644
--- a/main-loop.c
+++ b/main-loop.c
@@ -499,6 +499,11 @@ QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque)
     return aio_bh_new(qemu_aio_context, cb, opaque);
 }
 
+QEMUBH *qemu_bh_new_replay(QEMUBHFunc *cb, void *opaque, uint64_t id)
+{
+    return aio_bh_new_replay(qemu_aio_context, cb, opaque, id);
+}
+
 bool qemu_aio_wait(void)
 {
     return aio_poll(qemu_aio_context, true);
diff --git a/replay/replay-events.c b/replay/replay-events.c
index eda574e..81a5a6b 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -35,6 +35,9 @@ static bool replay_events_enabled = false;
 static void replay_run_event(Event *event)
 {
     switch (event->event_kind) {
+    case REPLAY_ASYNC_EVENT_BH:
+        aio_bh_call(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -117,6 +120,11 @@ void replay_add_event(int event_kind, void *opaque)
     replay_add_event_internal(event_kind, opaque, NULL, 0);
 }
 
+void replay_add_bh_event(void *bh, uint64_t id)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_BH, bh, NULL, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -134,6 +142,9 @@ void replay_save_events(int opt)
 
             /* save event-specific data */
             switch (event->event_kind) {
+            case REPLAY_ASYNC_EVENT_BH:
+                replay_put_qword(event->id);
+                break;
             }
         }
 
@@ -163,6 +174,11 @@ void replay_read_events(int opt)
             break;
         /* Execute some events without searching them in the queue */
         switch (read_event_kind) {
+        case REPLAY_ASYNC_EVENT_BH:
+            if (read_id == -1) {
+                read_id = replay_get_qword();
+            }
+            break;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 95cbde6..bbb117e 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -39,7 +39,8 @@
 
 /* Asynchronous events IDs */
 
-#define REPLAY_ASYNC_COUNT             0
+#define REPLAY_ASYNC_EVENT_BH          0
+#define REPLAY_ASYNC_COUNT             1
 
 typedef struct ReplayState {
     /*! Cached clock values. */
diff --git a/replay/replay.h b/replay/replay.h
index 69a20af..3040b6b 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -98,5 +98,7 @@ int replay_checkpoint(unsigned int checkpoint);
 
 /*! Disables storing events in the queue */
 void replay_disable_events(void);
+/*! Adds BH event to the queue */
+void replay_add_bh_event(void *bh, uint64_t id);
 
 #endif
diff --git a/stubs/replay.c b/stubs/replay.c
index 1b0b78f..8a4d69e 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -27,3 +27,7 @@ int runstate_is_running(void)
 {
     return 0;
 }
+
+void replay_add_bh_event(void *bh, uint64_t id)
+{
+}

[Qemu-devel] [RFC PATCH v2 34/49] replay: replay aio requests

[Pavel Dovgalyuk]

This patch adds identifier to aio requests. ID is used for creating bottom
halves and identifying them while replaying.
The patch also introduces several functions that make possible replaying
of the aio requests.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c               |   79 +++++++++++++++++++++++++++++++++++++++++++------
 block/qcow2.c         |    4 ++
 dma-helpers.c         |    6 ++--
 hw/block/virtio-blk.c |   10 +++---
 hw/ide/atapi.c        |    9 ++++--
 hw/ide/core.c         |   14 ++++++---
 include/block/block.h |   15 +++++++++
 include/qemu-common.h |    2 +
 qemu-io-cmds.c        |    2 +
 stubs/replay.c        |    5 +++
 trace-events          |    2 +
 util/iov.c            |    4 ++
 12 files changed, 126 insertions(+), 26 deletions(-)

diff --git a/block.c b/block.c
index fb6c849..849b75a 100644
--- a/block.c
+++ b/block.c
@@ -84,7 +84,8 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
                                                BdrvRequestFlags flags,
                                                BlockDriverCompletionFunc *cb,
                                                void *opaque,
-                                               bool is_write);
+                                               bool is_write,
+                                               bool aio_replay);
 static void coroutine_fn bdrv_co_do_rw(void *opaque);
 static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
     int64_t sector_num, int nb_sectors, BdrvRequestFlags flags);
@@ -4370,7 +4371,19 @@ BlockDriverAIOCB *bdrv_aio_readv(BlockDriverState *bs, int64_t sector_num,
     trace_bdrv_aio_readv(bs, sector_num, nb_sectors, opaque);
 
     return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
-                                 cb, opaque, false);
+                                 cb, opaque, false, false);
+}
+
+BlockDriverAIOCB *bdrv_aio_readv_replay(BlockDriverState *bs,
+                                        int64_t sector_num,
+                                        QEMUIOVector *qiov, int nb_sectors,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque)
+{
+    trace_bdrv_aio_readv_replay(bs, sector_num, nb_sectors, opaque);
+
+    return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
+                                 cb, opaque, false, true);
 }
 
 BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
@@ -4380,7 +4393,19 @@ BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
     trace_bdrv_aio_writev(bs, sector_num, nb_sectors, opaque);
 
     return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
-                                 cb, opaque, true);
+                                 cb, opaque, true, false);
+}
+
+BlockDriverAIOCB *bdrv_aio_writev_replay(BlockDriverState *bs,
+                                         int64_t sector_num,
+                                         QEMUIOVector *qiov, int nb_sectors,
+                                         BlockDriverCompletionFunc *cb,
+                                         void *opaque)
+{
+    trace_bdrv_aio_writev_replay(bs, sector_num, nb_sectors, opaque);
+
+    return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
+                                 cb, opaque, true, false);
 }
 
 BlockDriverAIOCB *bdrv_aio_write_zeroes(BlockDriverState *bs,
@@ -4391,7 +4416,7 @@ BlockDriverAIOCB *bdrv_aio_write_zeroes(BlockDriverState *bs,
 
     return bdrv_co_aio_rw_vector(bs, sector_num, NULL, nb_sectors,
                                  BDRV_REQ_ZERO_WRITE | flags,
-                                 cb, opaque, true);
+                                 cb, opaque, true, true);
 }
 
 
@@ -4527,7 +4552,8 @@ static int multiwrite_merge(BlockDriverState *bs, BlockRequest *reqs,
  * requests. However, the fields opaque and error are left unmodified as they
  * are used to signal failure for a single request to the caller.
  */
-int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs)
+int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs,
+                        bool replay)
 {
     MultiwriteCB *mcb;
     int i;
@@ -4565,7 +4591,7 @@ int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs)
         bdrv_co_aio_rw_vector(bs, reqs[i].sector, reqs[i].qiov,
                               reqs[i].nb_sectors, reqs[i].flags,
                               multiwrite_cb, mcb,
-                              true);
+                              true, replay);
     }
 
     return 0;
@@ -4714,7 +4740,11 @@ static void coroutine_fn bdrv_co_do_rw(void *opaque)
             acb->req.nb_sectors, acb->req.qiov, acb->req.flags);
     }
 
-    acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    if (acb->common.replay) {
+        acb->bh = aio_bh_new_replay(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb, acb->common.replay_step);
+    } else {
+        acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    }
     qemu_bh_schedule(acb->bh);
 }
 
@@ -4725,7 +4755,8 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
                                                BdrvRequestFlags flags,
                                                BlockDriverCompletionFunc *cb,
                                                void *opaque,
-                                               bool is_write)
+                                               bool is_write,
+                                               bool aio_replay)
 {
     Coroutine *co;
     BlockDriverAIOCBCoroutine *acb;
@@ -4737,6 +4768,10 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
     acb->req.flags = flags;
     acb->is_write = is_write;
     acb->done = NULL;
+    acb->common.replay = aio_replay;
+    if (aio_replay) {
+        acb->common.replay_step = replay_get_current_step();
+    }
 
     co = qemu_coroutine_create(bdrv_co_do_rw);
     qemu_coroutine_enter(co, acb);
@@ -4750,7 +4785,12 @@ static void coroutine_fn bdrv_aio_flush_co_entry(void *opaque)
     BlockDriverState *bs = acb->common.bs;
 
     acb->req.error = bdrv_co_flush(bs);
-    acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    if (acb->common.replay) {
+        acb->bh = aio_bh_new_replay(bdrv_get_aio_context(bs), bdrv_co_em_bh,
+                                    acb, acb->common.replay_step);
+    } else {
+        acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    }
     qemu_bh_schedule(acb->bh);
 }
 
@@ -4771,6 +4811,25 @@ BlockDriverAIOCB *bdrv_aio_flush(BlockDriverState *bs,
     return &acb->common;
 }
 
+BlockDriverAIOCB *bdrv_aio_flush_replay(BlockDriverState *bs,
+        BlockDriverCompletionFunc *cb, void *opaque)
+{
+    trace_bdrv_aio_flush(bs, opaque);
+
+    Coroutine *co;
+    BlockDriverAIOCBCoroutine *acb;
+
+    acb = qemu_aio_get(&bdrv_em_co_aiocb_info, bs, cb, opaque);
+    acb->done = NULL;
+    acb->common.replay = true;
+    acb->common.replay_step = replay_get_current_step();
+
+    co = qemu_coroutine_create(bdrv_aio_flush_co_entry);
+    qemu_coroutine_enter(co, acb);
+
+    return &acb->common;
+}
+
 static void coroutine_fn bdrv_aio_discard_co_entry(void *opaque)
 {
     BlockDriverAIOCBCoroutine *acb = opaque;
@@ -4821,6 +4880,8 @@ void *qemu_aio_get(const AIOCBInfo *aiocb_info, BlockDriverState *bs,
     acb->bs = bs;
     acb->cb = cb;
     acb->opaque = opaque;
+    acb->replay_step = 0;
+    acb->replay = false;
     return acb;
 }
 
diff --git a/block/qcow2.c b/block/qcow2.c
index d9da238..358a144 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -988,6 +988,8 @@ static coroutine_fn int qcow2_co_readv(BlockDriverState *bs, int64_t sector_num,
     uint8_t *cluster_data = NULL;
 
     qemu_iovec_init(&hd_qiov, qiov->niov);
+    hd_qiov.replay = qiov->replay;
+    hd_qiov.replay_step = qiov->replay_step;
 
     qemu_co_mutex_lock(&s->lock);
 
@@ -1131,6 +1133,8 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState *bs,
                                  remaining_sectors);
 
     qemu_iovec_init(&hd_qiov, qiov->niov);
+    hd_qiov.replay = qiov->replay;
+    hd_qiov.replay_step = qiov->replay_step;
 
     s->cluster_cache_offset = -1; /* disable compressed cache */
 
diff --git a/dma-helpers.c b/dma-helpers.c
index 05aadea..1606a19 100644
--- a/dma-helpers.c
+++ b/dma-helpers.c
@@ -219,6 +219,8 @@ BlockDriverAIOCB *dma_bdrv_io(
     dbs->io_func = io_func;
     dbs->bh = NULL;
     qemu_iovec_init(&dbs->iov, sg->nsg);
+    dbs->iov.replay = true;
+    dbs->iov.replay_step = replay_get_current_step();
     dma_bdrv_cb(dbs, 0);
     return &dbs->common;
 }
@@ -228,7 +230,7 @@ BlockDriverAIOCB *dma_bdrv_read(BlockDriverState *bs,
                                 QEMUSGList *sg, uint64_t sector,
                                 void (*cb)(void *opaque, int ret), void *opaque)
 {
-    return dma_bdrv_io(bs, sg, sector, bdrv_aio_readv, cb, opaque,
+    return dma_bdrv_io(bs, sg, sector, bdrv_aio_readv_replay, cb, opaque,
                        DMA_DIRECTION_FROM_DEVICE);
 }
 
@@ -236,7 +238,7 @@ BlockDriverAIOCB *dma_bdrv_write(BlockDriverState *bs,
                                  QEMUSGList *sg, uint64_t sector,
                                  void (*cb)(void *opaque, int ret), void *opaque)
 {
-    return dma_bdrv_io(bs, sg, sector, bdrv_aio_writev, cb, opaque,
+    return dma_bdrv_io(bs, sg, sector, bdrv_aio_writev_replay, cb, opaque,
                        DMA_DIRECTION_TO_DEVICE);
 }
 
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index aec3146..adf8dd2 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -265,7 +265,7 @@ void virtio_submit_multiwrite(BlockDriverState *bs, MultiReqBuffer *mrb)
         return;
     }
 
-    ret = bdrv_aio_multiwrite(bs, mrb->blkreq, mrb->num_writes);
+    ret = bdrv_aio_multiwrite(bs, mrb->blkreq, mrb->num_writes, true);
     if (ret != 0) {
         for (i = 0; i < mrb->num_writes; i++) {
             if (mrb->blkreq[i].error) {
@@ -285,7 +285,7 @@ static void virtio_blk_handle_flush(VirtIOBlockReq *req, MultiReqBuffer *mrb)
      * Make sure all outstanding writes are posted to the backing device.
      */
     virtio_submit_multiwrite(req->dev->bs, mrb);
-    bdrv_aio_flush(req->dev->bs, virtio_blk_flush_complete, req);
+    bdrv_aio_flush_replay(req->dev->bs, virtio_blk_flush_complete, req);
 }
 
 static void virtio_blk_handle_write(VirtIOBlockReq *req, MultiReqBuffer *mrb)
@@ -341,9 +341,9 @@ static void virtio_blk_handle_read(VirtIOBlockReq *req)
         virtio_blk_rw_complete(req, -EIO);
         return;
     }
-    bdrv_aio_readv(req->dev->bs, sector, &req->qiov,
-                   req->qiov.size / BDRV_SECTOR_SIZE,
-                   virtio_blk_rw_complete, req);
+    bdrv_aio_readv_replay(req->dev->bs, sector, &req->qiov,
+                          req->qiov.size / BDRV_SECTOR_SIZE,
+                          virtio_blk_rw_complete, req);
 }
 
 void virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index f7d2009..e94cd93 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -25,6 +25,7 @@
 
 #include "hw/ide/internal.h"
 #include "hw/scsi/scsi.h"
+#include "replay/replay.h"
 
 static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret);
 
@@ -348,10 +349,12 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret)
     s->bus->dma->iov.iov_base = (void *)(s->io_buffer + data_offset);
     s->bus->dma->iov.iov_len = n * 4 * 512;
     qemu_iovec_init_external(&s->bus->dma->qiov, &s->bus->dma->iov, 1);
+    s->bus->dma->qiov.replay = true;
+    s->bus->dma->qiov.replay_step = replay_get_current_step();
 
-    s->bus->dma->aiocb = bdrv_aio_readv(s->bs, (int64_t)s->lba << 2,
-                                       &s->bus->dma->qiov, n * 4,
-                                       ide_atapi_cmd_read_dma_cb, s);
+    s->bus->dma->aiocb = bdrv_aio_readv_replay(s->bs, (int64_t)s->lba << 2,
+                                               &s->bus->dma->qiov, n * 4,
+                                               ide_atapi_cmd_read_dma_cb, s);
     return;
 
 eot:
diff --git a/hw/ide/core.c b/hw/ide/core.c
index 3df26f0..8c3cfe2 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -559,10 +559,12 @@ void ide_sector_read(IDEState *s)
     s->iov.iov_base = s->io_buffer;
     s->iov.iov_len  = n * BDRV_SECTOR_SIZE;
     qemu_iovec_init_external(&s->qiov, &s->iov, 1);
+    s->qiov.replay = true;
+    s->qiov.replay_step = replay_get_current_step();
 
     bdrv_acct_start(s->bs, &s->acct, n * BDRV_SECTOR_SIZE, BDRV_ACCT_READ);
-    s->pio_aiocb = bdrv_aio_readv(s->bs, sector_num, &s->qiov, n,
-                                  ide_sector_read_cb, s);
+    s->pio_aiocb = bdrv_aio_readv_replay(s->bs, sector_num, &s->qiov, n,
+                                         ide_sector_read_cb, s);
 }
 
 static void dma_buf_commit(IDEState *s)
@@ -795,10 +797,12 @@ void ide_sector_write(IDEState *s)
     s->iov.iov_base = s->io_buffer;
     s->iov.iov_len  = n * BDRV_SECTOR_SIZE;
     qemu_iovec_init_external(&s->qiov, &s->iov, 1);
+    s->qiov.replay = true;
+    s->qiov.replay_step = replay_get_current_step();
 
     bdrv_acct_start(s->bs, &s->acct, n * BDRV_SECTOR_SIZE, BDRV_ACCT_READ);
-    s->pio_aiocb = bdrv_aio_writev(s->bs, sector_num, &s->qiov, n,
-                                   ide_sector_write_cb, s);
+    s->pio_aiocb = bdrv_aio_writev_replay(s->bs, sector_num, &s->qiov, n,
+                                          ide_sector_write_cb, s);
 }
 
 static void ide_flush_cb(void *opaque, int ret)
@@ -827,7 +831,7 @@ void ide_flush_cache(IDEState *s)
 
     s->status |= BUSY_STAT;
     bdrv_acct_start(s->bs, &s->acct, 0, BDRV_ACCT_FLUSH);
-    bdrv_aio_flush(s->bs, ide_flush_cb, s);
+    bdrv_aio_flush_replay(s->bs, ide_flush_cb, s);
 }
 
 static void ide_cfata_metadata_inquiry(IDEState *s)
diff --git a/include/block/block.h b/include/block/block.h
index dd05458..636589a 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -325,11 +325,24 @@ typedef void BlockDriverDirtyHandler(BlockDriverState *bs, int64_t sector,
 BlockDriverAIOCB *bdrv_aio_readv(BlockDriverState *bs, int64_t sector_num,
                                  QEMUIOVector *iov, int nb_sectors,
                                  BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_readv_replay(BlockDriverState *bs,
+                                        int64_t sector_num,
+                                        QEMUIOVector *iov, int nb_sectors,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque);
 BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
                                   QEMUIOVector *iov, int nb_sectors,
                                   BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_writev_replay(BlockDriverState *bs,
+                                         int64_t sector_num,
+                                         QEMUIOVector *iov, int nb_sectors,
+                                         BlockDriverCompletionFunc *cb,
+                                         void *opaque);
 BlockDriverAIOCB *bdrv_aio_flush(BlockDriverState *bs,
                                  BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_flush_replay(BlockDriverState *bs,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque);
 BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
                                    int64_t sector_num, int nb_sectors,
                                    BlockDriverCompletionFunc *cb, void *opaque);
@@ -349,7 +362,7 @@ typedef struct BlockRequest {
 } BlockRequest;
 
 int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs,
-    int num_reqs);
+                        int num_reqs, bool replay);
 
 /* sg packet commands */
 int bdrv_ioctl(BlockDriverState *bs, unsigned long int req, void *buf);
diff --git a/include/qemu-common.h b/include/qemu-common.h
index 48013ce..50ab79c 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -309,6 +309,8 @@ typedef struct QEMUIOVector {
     int niov;
     int nalloc;
     size_t size;
+    bool replay;
+    uint64_t replay_step;
 } QEMUIOVector;
 
 void qemu_iovec_init(QEMUIOVector *qiov, int alloc_hint);
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index 60c1ceb..f8a809a 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -592,7 +592,7 @@ static int do_aio_multiwrite(BlockDriverState *bs, BlockRequest* reqs,
         *total += reqs[i].qiov->size;
     }
 
-    ret = bdrv_aio_multiwrite(bs, reqs, num_reqs);
+    ret = bdrv_aio_multiwrite(bs, reqs, num_reqs, false);
     if (ret < 0) {
         return ret;
     }
diff --git a/stubs/replay.c b/stubs/replay.c
index 8a4d69e..1dea1f6 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -31,3 +31,8 @@ int runstate_is_running(void)
 void replay_add_bh_event(void *bh, uint64_t id)
 {
 }
+
+uint64_t replay_get_current_step(void)
+{
+    return 0;
+}
diff --git a/trace-events b/trace-events
index 11a17a8..8382348 100644
--- a/trace-events
+++ b/trace-events
@@ -59,7 +59,9 @@ bdrv_aio_multiwrite(void *mcb, int num_callbacks, int num_reqs) "mcb %p num_call
 bdrv_aio_discard(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_flush(void *bs, void *opaque) "bs %p opaque %p"
 bdrv_aio_readv(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
+bdrv_aio_readv_replay(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_writev(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
+bdrv_aio_writev_replay(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_write_zeroes(void *bs, int64_t sector_num, int nb_sectors, int flags, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d flags %#x opaque %p"
 bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d"
 bdrv_co_readv(void *bs, int64_t sector_num, int nb_sector) "bs %p sector_num %"PRId64" nb_sectors %d"
diff --git a/util/iov.c b/util/iov.c
index 2b4f46d..3bf7092 100644
--- a/util/iov.c
+++ b/util/iov.c
@@ -257,6 +257,8 @@ void qemu_iovec_init(QEMUIOVector *qiov, int alloc_hint)
     qiov->niov = 0;
     qiov->nalloc = alloc_hint;
     qiov->size = 0;
+    qiov->replay = false;
+    qiov->replay_step = 0;
 }
 
 void qemu_iovec_init_external(QEMUIOVector *qiov, struct iovec *iov, int niov)
@@ -267,6 +269,8 @@ void qemu_iovec_init_external(QEMUIOVector *qiov, struct iovec *iov, int niov)
     qiov->niov = niov;
     qiov->nalloc = -1;
     qiov->size = 0;
+    qiov->replay = false;
+    qiov->replay_step = 0;
     for (i = 0; i < niov; i++)
         qiov->size += iov[i].iov_len;
 }

[Qemu-devel] [RFC PATCH v2 35/49] replay: thread pool

[Pavel Dovgalyuk]

This patch modifies thread pool to allow replaying asynchronous thread tasks
synchronously in replay mode.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block/raw-posix.c           |    6 +++--
 block/raw-win32.c           |    4 ++-
 include/block/thread-pool.h |    4 ++-
 replay/replay-events.c      |   11 +++++++++
 replay/replay-internal.h    |    3 ++
 replay/replay.h             |    2 ++
 stubs/replay.c              |    4 +++
 tests/test-thread-pool.c    |    7 +++---
 thread-pool.c               |   53 +++++++++++++++++++++++++++++--------------
 9 files changed, 69 insertions(+), 25 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index a857def..f600736 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -1025,7 +1025,9 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs, int fd,
 
     trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque,
+                                  qiov ? qiov->replay : false,
+                                  qiov ? qiov->replay_step : 0);
 }
 
 static BlockDriverAIOCB *raw_aio_submit(BlockDriverState *bs,
@@ -1853,7 +1855,7 @@ static BlockDriverAIOCB *hdev_aio_ioctl(BlockDriverState *bs,
     acb->aio_ioctl_buf = buf;
     acb->aio_ioctl_cmd = req;
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque, false, 0);
 }
 
 #elif defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 902eab6..212307c 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -158,7 +158,9 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs, HANDLE hfile,
 
     trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque,
+                                  qiov ? qiov->replay : false,
+                                  qiov ? qiov->replay_step : 0);
 }
 
 int qemu_ftruncate64(int fd, int64_t length)
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 32afcdd..df74f8d 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -33,9 +33,11 @@ void thread_pool_free(ThreadPool *pool);
 
 BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg,
-        BlockDriverCompletionFunc *cb, void *opaque);
+        BlockDriverCompletionFunc *cb, void *opaque,
+        bool replay, uint64_t replay_step);
 int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg);
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+void thread_pool_work(ThreadPool *pool, void *r);
 
 #endif
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 81a5a6b..f39889d 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -11,6 +11,7 @@
 
 #include "replay.h"
 #include "replay-internal.h"
+#include "block/thread-pool.h"
 
 typedef struct Event {
     int event_kind;
@@ -38,6 +39,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_BH:
         aio_bh_call(event->opaque);
         break;
+    case REPLAY_ASYNC_EVENT_THREAD:
+        thread_pool_work((ThreadPool *)event->opaque, event->opaque2);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -125,6 +129,11 @@ void replay_add_bh_event(void *bh, uint64_t id)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_BH, bh, NULL, id);
 }
 
+void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_THREAD, opaque, opaque2, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -143,6 +152,7 @@ void replay_save_events(int opt)
             /* save event-specific data */
             switch (event->event_kind) {
             case REPLAY_ASYNC_EVENT_BH:
+            case REPLAY_ASYNC_EVENT_THREAD:
                 replay_put_qword(event->id);
                 break;
             }
@@ -175,6 +185,7 @@ void replay_read_events(int opt)
         /* Execute some events without searching them in the queue */
         switch (read_event_kind) {
         case REPLAY_ASYNC_EVENT_BH:
+        case REPLAY_ASYNC_EVENT_THREAD:
             if (read_id == -1) {
                 read_id = replay_get_qword();
             }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index bbb117e..3f97fd7 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -40,7 +40,8 @@
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_EVENT_BH          0
-#define REPLAY_ASYNC_COUNT             1
+#define REPLAY_ASYNC_EVENT_THREAD      1
+#define REPLAY_ASYNC_COUNT             2
 
 typedef struct ReplayState {
     /*! Cached clock values. */
diff --git a/replay/replay.h b/replay/replay.h
index 3040b6b..e96c74f 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -100,5 +100,7 @@ int replay_checkpoint(unsigned int checkpoint);
 void replay_disable_events(void);
 /*! Adds BH event to the queue */
 void replay_add_bh_event(void *bh, uint64_t id);
+/*! Adds thread event to the queue */
+void replay_add_thread_event(void *pool, void *req, uint64_t id);
 
 #endif
diff --git a/stubs/replay.c b/stubs/replay.c
index 1dea1f6..c269b59 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -36,3 +36,7 @@ uint64_t replay_get_current_step(void)
 {
     return 0;
 }
+
+void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
+{
+}
diff --git a/tests/test-thread-pool.c b/tests/test-thread-pool.c
index aa156bc..6dfbb55 100644
--- a/tests/test-thread-pool.c
+++ b/tests/test-thread-pool.c
@@ -55,7 +55,7 @@ static void test_submit_aio(void)
 {
     WorkerTestData data = { .n = 0, .ret = -EINPROGRESS };
     data.aiocb = thread_pool_submit_aio(pool, worker_cb, &data,
-                                        done_cb, &data);
+                                        done_cb, &data, false, 0);
 
     /* The callbacks are not called until after the first wait.  */
     active = 1;
@@ -119,7 +119,8 @@ static void test_submit_many(void)
     for (i = 0; i < 100; i++) {
         data[i].n = 0;
         data[i].ret = -EINPROGRESS;
-        thread_pool_submit_aio(pool, worker_cb, &data[i], done_cb, &data[i]);
+        thread_pool_submit_aio(pool, worker_cb, &data[i], done_cb, &data[i],
+                               false, 0);
     }
 
     active = 100;
@@ -148,7 +149,7 @@ static void test_cancel(void)
         data[i].n = 0;
         data[i].ret = -EINPROGRESS;
         data[i].aiocb = thread_pool_submit_aio(pool, long_cb, &data[i],
-                                               done_cb, &data[i]);
+                                               done_cb, &data[i], false, 0);
     }
 
     /* Starting the threads may be left to a bottom half.  Let it
diff --git a/thread-pool.c b/thread-pool.c
index dfb699d..25cff25 100644
--- a/thread-pool.c
+++ b/thread-pool.c
@@ -24,6 +24,7 @@
 #include "qemu/event_notifier.h"
 #include "block/thread-pool.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 static void do_spawn_thread(ThreadPool *pool);
 
@@ -79,6 +80,30 @@ struct ThreadPool {
     bool stopping;
 };
 
+void thread_pool_work(ThreadPool *pool, void *r)
+{
+    ThreadPoolElement *req = (ThreadPoolElement *)r;
+    int ret;
+    if (replay_mode == REPLAY_NONE) {
+        qemu_mutex_unlock(&pool->lock);
+    }
+
+    ret = req->func(req->arg);
+    req->ret = ret;
+    /* Write ret before state.  */
+    smp_wmb();
+    req->state = THREAD_DONE;
+
+    if (replay_mode == REPLAY_NONE) {
+        qemu_mutex_lock(&pool->lock);
+    }
+    if (pool->pending_cancellations) {
+        qemu_cond_broadcast(&pool->check_cancel);
+    }
+
+    event_notifier_set(&pool->notifier);
+}
+
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -105,21 +130,12 @@ static void *worker_thread(void *opaque)
         req = QTAILQ_FIRST(&pool->request_list);
         QTAILQ_REMOVE(&pool->request_list, req, reqs);
         req->state = THREAD_ACTIVE;
-        qemu_mutex_unlock(&pool->lock);
-
-        ret = req->func(req->arg);
 
-        req->ret = ret;
-        /* Write ret before state.  */
-        smp_wmb();
-        req->state = THREAD_DONE;
-
-        qemu_mutex_lock(&pool->lock);
-        if (pool->pending_cancellations) {
-            qemu_cond_broadcast(&pool->check_cancel);
+        if (replay_mode != REPLAY_NONE && req->common.replay) {
+            replay_add_thread_event(pool, req, req->common.replay_step);
+        } else {
+            thread_pool_work(pool, req);
         }
-
-        event_notifier_set(&pool->notifier);
     }
 
     pool->cur_threads--;
@@ -234,7 +250,8 @@ static const AIOCBInfo thread_pool_aiocb_info = {
 
 BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg,
-        BlockDriverCompletionFunc *cb, void *opaque)
+        BlockDriverCompletionFunc *cb, void *opaque,
+        bool replay, uint64_t replay_step)
 {
     ThreadPoolElement *req;
 
@@ -243,6 +260,8 @@ BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
     req->arg = arg;
     req->state = THREAD_QUEUED;
     req->pool = pool;
+    req->common.replay = replay;
+    req->common.replay_step = replay_step;
 
     QLIST_INSERT_HEAD(&pool->head, req, all);
 
@@ -253,8 +272,8 @@ BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         spawn_thread(pool);
     }
     QTAILQ_INSERT_TAIL(&pool->request_list, req, reqs);
-    qemu_mutex_unlock(&pool->lock);
     qemu_sem_post(&pool->sem);
+    qemu_mutex_unlock(&pool->lock);
     return &req->common;
 }
 
@@ -276,14 +295,14 @@ int coroutine_fn thread_pool_submit_co(ThreadPool *pool, ThreadPoolFunc *func,
 {
     ThreadPoolCo tpc = { .co = qemu_coroutine_self(), .ret = -EINPROGRESS };
     assert(qemu_in_coroutine());
-    thread_pool_submit_aio(pool, func, arg, thread_pool_co_cb, &tpc);
+    thread_pool_submit_aio(pool, func, arg, thread_pool_co_cb, &tpc, false, 0);
     qemu_coroutine_yield();
     return tpc.ret;
 }
 
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
 {
-    thread_pool_submit_aio(pool, func, arg, NULL, NULL);
+    thread_pool_submit_aio(pool, func, arg, NULL, NULL, false, 0);
 }
 
 static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)

[Qemu-devel] [RFC PATCH v2 36/49] pl031: vmstate in replay mode

[Pavel Dovgalyuk]

This patch modifies vmstate for PL031 RTC. It removes querying of the rtc
and virtual clocks while saving and restoring VM state, because in replay
mode these clocks are stopped while saving. And reading the clock while
restoring the VM state may lead to read of the incorrect values, because
clocks cache in replay module could not be loaded yet.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/pl031.c |   22 +++++++++++++++-------
 1 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index 02c814f..017b1ce 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -220,19 +220,27 @@ static void pl031_pre_save(void *opaque)
 {
     PL031State *s = opaque;
 
-    /* tick_offset is base_time - rtc_clock base time.  Instead, we want to
-     * store the base time relative to the QEMU_CLOCK_VIRTUAL for backwards-compatibility.  */
-    int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-    s->tick_offset_vmstate = s->tick_offset + delta / get_ticks_per_sec();
+    if (replay_mode == REPLAY_NONE) {
+        /* tick_offset is base_time - rtc_clock base time.  Instead, we want to
+         * store the base time relative to the QEMU_CLOCK_VIRTUAL for backwards-compatibility.  */
+        int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        s->tick_offset_vmstate = s->tick_offset + delta / get_ticks_per_sec();
+    } else {
+        s->tick_offset_vmstate = s->tick_offset;
+    }
 }
 
 static int pl031_post_load(void *opaque, int version_id)
 {
     PL031State *s = opaque;
 
-    int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-    s->tick_offset = s->tick_offset_vmstate - delta / get_ticks_per_sec();
-    pl031_set_alarm(s);
+    if (replay_mode == REPLAY_NONE) {
+        int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        s->tick_offset = s->tick_offset_vmstate - delta / get_ticks_per_sec();
+        pl031_set_alarm(s);
+    } else {
+        s->tick_offset = s->tick_offset_vmstate;
+    }
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v2 37/49] replay: initialization and deinitialization

[Pavel Dovgalyuk]

This patch introduces the functions for enabling the record/replay and for
freeing the resources when simulator closes.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 exec.c                   |    1 
 replay/replay-internal.h |    2 +
 replay/replay.c          |  127 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   11 ++++
 vl.c                     |    5 ++
 5 files changed, 146 insertions(+), 0 deletions(-)

diff --git a/exec.c b/exec.c
index 36adb62..050e0a8 100644
--- a/exec.c
+++ b/exec.c
@@ -732,6 +732,7 @@ void cpu_abort(CPUState *cpu, const char *fmt, ...)
     }
     va_end(ap2);
     va_end(ap);
+    replay_finish();
 #if defined(CONFIG_USER_ONLY)
     {
         struct sigaction act;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 3f97fd7..7e5b3af 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -36,6 +36,8 @@
 
 /* for checkpoint event */
 #define EVENT_CHECKPOINT            96
+/* end of log event */
+#define EVENT_END                   127
 
 /* Asynchronous events IDs */
 
diff --git a/replay/replay.c b/replay/replay.c
index 225d8ec..c583bb4 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -13,10 +13,18 @@
 #include "replay.h"
 #include "replay-internal.h"
 
+/* Current version of the replay mechanism.
+   Increase it when file format changes. */
+#define REPLAY_VERSION              0xe02001
+/* Size of replay log header */
+#define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
+
 int replay_mode = REPLAY_NONE;
 /*! Stores current submode for PLAY mode */
 int play_submode = REPLAY_PLAY_UNKNOWN;
 
+/* Name of replay file  */
+static char *replay_filename;
 /* Suffix for the disk images filenames */
 char *replay_image_suffix;
 
@@ -240,3 +248,122 @@ int replay_checkpoint(unsigned int checkpoint)
 
     return 1;
 }
+
+static void replay_enable(const char *fname, int mode)
+{
+    const char *fmode = NULL;
+    if (replay_file) {
+        fprintf(stderr, "Replay: some record/replay operation is already started\n");
+        return;
+    }
+
+    switch (mode) {
+    case REPLAY_SAVE:
+        fmode = "wb";
+        break;
+    case REPLAY_PLAY:
+        fmode = "rb";
+        play_submode = REPLAY_PLAY_NORMAL;
+        break;
+    default:
+        fprintf(stderr, "Replay: internal error: invalid replay mode\n");
+        exit(1);
+    }
+
+    atexit(replay_finish);
+
+    replay_file = fopen(fname, fmode);
+    if (replay_file == NULL) {
+        fprintf(stderr, "Replay: open %s: %s\n", fname, strerror(errno));
+        exit(1);
+    }
+
+    replay_filename = g_strdup(fname);
+
+    replay_mode = mode;
+    replay_has_unread_data = 0;
+    replay_data_kind = -1;
+    replay_state.skipping_instruction = 0;
+    replay_state.current_step = 0;
+
+    /* skip file header for SAVE and check it for PLAY */
+    if (replay_mode == REPLAY_SAVE) {
+        fseek(replay_file, HEADER_SIZE, SEEK_SET);
+    } else if (replay_mode == REPLAY_PLAY) {
+        unsigned int version = replay_get_dword();
+        uint64_t offset = replay_get_qword();
+        if (version != REPLAY_VERSION) {
+            fprintf(stderr, "Replay: invalid input log file version\n");
+            exit(1);
+        }
+        /* go to the beginning */
+        fseek(replay_file, 12, SEEK_SET);
+    }
+
+    replay_init_events();
+}
+
+void replay_configure(QemuOpts *opts, int mode)
+{
+    const char *fname;
+    
+    fname = qemu_opt_get(opts, "fname");
+    if (!fname) {
+        fprintf(stderr, "File name not specified for replay\n");
+        exit(1);
+    }
+
+    const char *suffix = qemu_opt_get(opts, "suffix");
+    if (suffix) {
+        replay_image_suffix = g_strdup(suffix);
+    } else {
+        replay_image_suffix = g_strdup("replay_qcow");
+    }
+
+    replay_enable(fname, mode);
+}
+
+void replay_init_timer(void)
+{
+    if (replay_mode == REPLAY_NONE) {
+        return;
+    }
+
+    replay_enable_events();
+}
+
+void replay_finish(void)
+{
+    if (replay_mode == REPLAY_NONE) {
+        return;
+    }
+
+    replay_save_instructions();
+
+    /* finalize the file */
+    if (replay_file) {
+        if (replay_mode == REPLAY_SAVE) {
+            uint64_t offset = 0;
+            /* write end event */
+            replay_put_event(EVENT_END);
+
+            /* write header */
+            fseek(replay_file, 0, SEEK_SET);
+            replay_put_dword(REPLAY_VERSION);
+            replay_put_qword(offset);
+        }
+
+        fclose(replay_file);
+        replay_file = NULL;
+    }
+    if (replay_filename) {
+        g_free(replay_filename);
+        replay_filename = NULL;
+    }
+    if (replay_image_suffix) {
+        g_free(replay_image_suffix);
+        replay_image_suffix = NULL;
+    }
+
+    replay_finish_events();
+}
diff --git a/replay/replay.h b/replay/replay.h
index e96c74f..e066cb2 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -16,6 +16,8 @@
 #include <stdint.h>
 #include <time.h>
 
+struct QemuOpts;
+
 /* replay modes */
 #define REPLAY_NONE 0
 #define REPLAY_SAVE 1
@@ -43,6 +45,15 @@ extern int replay_icount;
 /*! Returns replay play submode */
 int replay_get_play_submode(void);
 
+/* Replay process control functions */
+
+/*! Enables recording or saving event log with specified parameters */
+void replay_configure(struct QemuOpts *opts, int mode);
+/*! Initializes timers used for snapshotting and enables events recording */
+void replay_init_timer(void);
+/*! Closes replay log file and frees other resources. */
+void replay_finish(void);
+
 /* Processing the instructions */
 
 /*! Returns number of executed instructions. */
diff --git a/vl.c b/vl.c
index 053b2ba..6050faf 100644
--- a/vl.c
+++ b/vl.c
@@ -4544,7 +4544,12 @@ int main(int argc, char **argv, char **envp)
         }
     }
 
+    replay_init_timer();
+
     main_loop();
+    if (replay_mode != REPLAY_NONE) {
+        replay_disable_events();
+    }
     bdrv_close_all();
     pause_all_vcpus();
     res_free();

[Qemu-devel] [RFC PATCH v2 38/49] replay: command line options

[Pavel Dovgalyuk]

This patch introduces command line options for enabling recording or replaying
virtual machine behavior. "-record" option starts recording of the execution
and saves it into the log, specified with "fname" parameter. "-replay" option
is intended for replaying previously saved log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c          |   15 +++++++++--
 qemu-options.hx |   27 +++++++++++++++++++++
 vl.c            |   72 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 109 insertions(+), 5 deletions(-)

diff --git a/cpus.c b/cpus.c
index 4f517ee..e16d8f8 100644
--- a/cpus.c
+++ b/cpus.c
@@ -829,12 +829,21 @@ static void qemu_wait_io_event_common(CPUState *cpu)
 static void qemu_tcg_wait_io_event(void)
 {
     CPUState *cpu;
+    GMainContext *context = g_main_context_default();
 
-    while (all_cpu_threads_idle()) {
-       /* Start accounting real time to the virtual clock if the CPUs
-          are idle.  */
+    if (replay_mode == REPLAY_PLAY
+        && all_cpu_threads_idle() && first_cpu->halted) {
+        /* wakeup iothread when there is no code to execute in replay mode */
         qemu_clock_warp(QEMU_CLOCK_VIRTUAL);
+        g_main_context_wakeup(context);
         qemu_cond_wait(tcg_halt_cond, &qemu_global_mutex);
+    } else {
+        while (all_cpu_threads_idle()) {
+           /* Start accounting real time to the virtual clock if the CPUs
+              are idle.  */
+            qemu_clock_warp(QEMU_CLOCK_VIRTUAL);
+            qemu_cond_wait(tcg_halt_cond, &qemu_global_mutex); 
+        }
     }
 
     while (iothread_requesting_mutex) {
diff --git a/qemu-options.hx b/qemu-options.hx
index 9e54686..7548f87 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3348,6 +3348,33 @@ Dump json-encoded vmstate information for current machine type to file
 in @var{file}
 ETEXI
 
+DEF("record", HAS_ARG, QEMU_OPTION_record,
+    "-record fname=<filename>[,suffix=<suffix>,snapshot=<on/off>]\n"
+    "                writes replay file for latter replaying\n",
+    QEMU_ARCH_ALL)
+STEXI
+@item -record fname=@var{file}[,suffix=@var{suffix},snapshot=@var{snapshot}]
+Writes compact execution trace into @var{file}.
+Changes for disk images are written
+into separate files with @var{suffix} added. If no @var{suffix} is
+specified, "replay_qcow" is used as suffix.
+If @var{snapshot} parameter is set as off, then original disk image will be
+modified. Default value is on.
+ETEXI
+
+DEF("replay", HAS_ARG, QEMU_OPTION_replay,
+    "-replay fname=<filename>[,suffix=<suffix>,snapshot=<on/off>]\n"
+    "                plays saved replay file\n", QEMU_ARCH_ALL)
+STEXI
+@item -replay fname=@var{filename}[,suffix=@var{suffix},snapshot=@var{snapshot}]
+Plays compact execution trace from @var{filename}.
+Changes for disk images and VM states are read
+from separate files with @var{suffix} added. If no @var{suffix} is
+specified, "replay_qcow" is used as suffix.
+If @var{snapshot} parameter is set as off, then original disk image will be
+modified. Default value is on.
+ETEXI
+
 HXCOMM This is the last statement. Insert new options before this line!
 STEXI
 @end table
diff --git a/vl.c b/vl.c
index 6050faf..f8c1e9e 100644
--- a/vl.c
+++ b/vl.c
@@ -538,6 +538,42 @@ static QemuOptsList qemu_mem_opts = {
     },
 };
 
+static QemuOptsList qemu_record_opts = {
+    .name = "record",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_record_opts.head),
+    .desc = {
+        {
+            .name = "fname",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "suffix",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "snapshot",
+            .type = QEMU_OPT_BOOL,
+        },
+        { /* end of list */ }
+    },
+};
+
+static QemuOptsList qemu_replay_opts = {
+    .name = "replay",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_replay_opts.head),
+    .desc = {
+        {
+            .name = "fname",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "suffix",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "snapshot",
+            .type = QEMU_OPT_BOOL,
+        },
+        { /* end of list */ }
+    },
+};
+
 /**
  * Get machine options
  *
@@ -2914,7 +2950,8 @@ out:
 int main(int argc, char **argv, char **envp)
 {
     int i;
-    int snapshot, linux_boot;
+    int snapshot, linux_boot, replay_snapshot;
+    int not_compatible_replay_param = 0;
     const char *icount_option = NULL;
     const char *initrd_filename;
     const char *kernel_filename, *kernel_cmdline;
@@ -2986,6 +3023,8 @@ int main(int argc, char **argv, char **envp)
     qemu_add_opts(&qemu_msg_opts);
     qemu_add_opts(&qemu_name_opts);
     qemu_add_opts(&qemu_numa_opts);
+    qemu_add_opts(&qemu_replay_opts);
+    qemu_add_opts(&qemu_record_opts);
 
     runstate_init();
 
@@ -2999,6 +3038,7 @@ int main(int argc, char **argv, char **envp)
     cpu_model = NULL;
     ram_size = default_ram_size;
     snapshot = 0;
+    replay_snapshot = 1;
     cyls = heads = secs = 0;
     translation = BIOS_ATA_TRANSLATION_AUTO;
 
@@ -3116,6 +3156,7 @@ int main(int argc, char **argv, char **envp)
                 break;
             case QEMU_OPTION_pflash:
                 drive_add(IF_PFLASH, -1, optarg, PFLASH_OPTS);
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_snapshot:
                 snapshot = 1;
@@ -3272,6 +3313,7 @@ int main(int argc, char **argv, char **envp)
 #endif
             case QEMU_OPTION_bt:
                 add_device_config(DEV_BT, optarg);
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_audio_help:
                 AUD_help ();
@@ -3485,6 +3527,7 @@ int main(int argc, char **argv, char **envp)
                 if (!opts) {
                     exit(1);
                 }
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_fsdev:
                 olist = qemu_find_opts("fsdev");
@@ -3613,6 +3656,7 @@ int main(int argc, char **argv, char **envp)
                 if (strncmp(optarg, "mon:", 4) == 0) {
                     default_monitor = 0;
                 }
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_debugcon:
                 add_device_config(DEV_DEBUGCON, optarg);
@@ -3733,6 +3777,7 @@ int main(int argc, char **argv, char **envp)
                 if (!qemu_opts_parse(qemu_find_opts("smp-opts"), optarg, 1)) {
                     exit(1);
                 }
+                not_compatible_replay_param++;
                 break;
 	    case QEMU_OPTION_vnc:
 #ifdef CONFIG_VNC
@@ -3967,6 +4012,24 @@ int main(int argc, char **argv, char **envp)
                     exit(1);
                 }
                 break;
+            case QEMU_OPTION_record:
+                opts = qemu_opts_parse(qemu_find_opts("record"), optarg, 0);
+                if (!opts) {
+                    fprintf(stderr, "Invalid record options: %s\n", optarg);
+                    exit(1);
+                }
+                replay_configure(opts, REPLAY_SAVE);
+                replay_snapshot = qemu_opt_get_bool(opts, "snapshot", 1);
+                break;
+            case QEMU_OPTION_replay:
+                opts = qemu_opts_parse(qemu_find_opts("replay"), optarg, 0);
+                if (!opts) {
+                    fprintf(stderr, "Invalid replay options: %s\n", optarg);
+                    exit(1);
+                }
+                replay_configure(opts, REPLAY_PLAY);
+                replay_snapshot = qemu_opt_get_bool(opts, "snapshot", 1);
+                break;
             default:
                 os_parse_cmd_args(popt->index, optarg);
             }
@@ -3981,6 +4044,11 @@ int main(int argc, char **argv, char **envp)
         exit(1);
     }
 
+    if (not_compatible_replay_param && (replay_mode != REPLAY_NONE)) {
+        fprintf(stderr, "options -smp, -pflash, -chardev, -bt, -parallel are not compatible with record/replay\n");
+        exit(1);
+    }
+
     if (qemu_opts_foreach(qemu_find_opts("sandbox"), parse_sandbox, NULL, 0)) {
         exit(1);
     }
@@ -4347,7 +4415,7 @@ int main(int argc, char **argv, char **envp)
     ram_mig_init();
 
     /* open the virtual block devices */
-    if (snapshot)
+    if (snapshot || (replay_mode != REPLAY_NONE && replay_snapshot))
         qemu_opts_foreach(qemu_find_opts("drive"), drive_enable_snapshot, NULL, 0);
     if (qemu_opts_foreach(qemu_find_opts("drive"), drive_init_func,
                           &machine_class->block_default_type, 1) != 0) {

[Qemu-devel] [RFC PATCH v2 39/49] replay: snapshotting the virtual machine

[Pavel Dovgalyuk]

This patch adds 'period' parameter to the 'record' command line option. This
parameters turns on periodic snapshotting of the VM which could be used by
replay to move forward and backward in time.
If 'period' parameter is not specified, only one snapshot is made at the start
of the virtual machine.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c                   |   46 ++++++++-
 qemu-options.hx          |    3 -
 qemu-timer.c             |    4 +
 replay/replay-internal.h |   18 ++++
 replay/replay.c          |  223 ++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 287 insertions(+), 7 deletions(-)

diff --git a/cpus.c b/cpus.c
index e16d8f8..019cc0a 100644
--- a/cpus.c
+++ b/cpus.c
@@ -228,11 +228,22 @@ int64_t cpu_get_clock(void)
  */
 void cpu_enable_ticks(void)
 {
+    int64_t ti;
     /* Here, the really thing protected by seqlock is cpu_clock_offset. */
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     if (!timers_state.cpu_ticks_enabled) {
         timers_state.cpu_ticks_offset -= cpu_get_real_ticks();
-        timers_state.cpu_clock_offset -= get_clock();
+
+        if (replay_mode == REPLAY_SAVE) {
+            ti = get_clock();
+            replay_save_clock(REPLAY_CLOCK_VIRTUAL, ti);
+        } else if (replay_mode == REPLAY_PLAY) {
+            ti = replay_read_clock(REPLAY_CLOCK_VIRTUAL);
+        } else {
+            ti = get_clock();
+        }
+
+        timers_state.cpu_clock_offset -= ti;
         timers_state.cpu_ticks_enabled = 1;
     }
     seqlock_write_unlock(&timers_state.vm_clock_seqlock);
@@ -437,6 +448,22 @@ void qemu_clock_warp(QEMUClockType type)
     }
 }
 
+static bool is_replay_enabled(void *opaque)
+{
+    return replay_mode != REPLAY_NONE;
+}
+
+static const VMStateDescription vmstate_timers_for_replay = {
+    .name = "timer for replay",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_INT64(cpu_ticks_prev, TimersState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static const VMStateDescription vmstate_timers = {
     .name = "timer",
     .version_id = 2,
@@ -446,6 +473,14 @@ static const VMStateDescription vmstate_timers = {
         VMSTATE_INT64(dummy, TimersState),
         VMSTATE_INT64_V(cpu_clock_offset, TimersState, 2),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection []) {
+        {
+            .vmsd = &vmstate_timers_for_replay,
+            .needed = is_replay_enabled,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -537,9 +572,11 @@ static int do_vm_stop(RunState state)
     int ret = 0;
 
     if (runstate_is_running()) {
+        runstate_set(state);
+        /* Disable ticks can cause recursive call of vm_stop.
+           Stopping before calling functions prevents infinite recursion. */
         cpu_disable_ticks();
         pause_all_vcpus();
-        runstate_set(state);
         vm_state_notify(0, state);
         qapi_event_send_stop(&error_abort);
     }
@@ -1320,10 +1357,9 @@ static void tcg_exec_all(void)
         CPUState *cpu = next_cpu;
         CPUArchState *env = cpu->env_ptr;
 
-        qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
-                          (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
-
         if (cpu_can_run(cpu)) {
+            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
+                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
             r = tcg_cpu_exec(env);
             if (r == EXCP_DEBUG) {
                 cpu_handle_guest_debug(cpu);
diff --git a/qemu-options.hx b/qemu-options.hx
index 7548f87..fcfc8d1 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3353,8 +3353,9 @@ DEF("record", HAS_ARG, QEMU_OPTION_record,
     "                writes replay file for latter replaying\n",
     QEMU_ARCH_ALL)
 STEXI
-@item -record fname=@var{file}[,suffix=@var{suffix},snapshot=@var{snapshot}]
+@item -record fname=@var{file}[,period=@var{period},suffix=@var{suffix},snapshot=@var{snapshot}]
 Writes compact execution trace into @var{file}.
+VM state is auto saved every @var{period} second, if this parameter is specified.
 Changes for disk images are written
 into separate files with @var{suffix} added. If no @var{suffix} is
 specified, "replay_qcow" is used as suffix.
diff --git a/qemu-timer.c b/qemu-timer.c
index dcf9b14..0dcba4c 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -103,7 +103,9 @@ QEMUTimerList *timerlist_new(QEMUClockType type,
     QEMUClock *clock = qemu_clock_ptr(type);
 
     timer_list = g_malloc0(sizeof(QEMUTimerList));
-    qemu_event_init(&timer_list->timers_done_ev, false);
+    /* Create signaled event, because they should be signaled 
+       outside the timerlist_run_timers function */
+    qemu_event_init(&timer_list->timers_done_ev, true);
     timer_list->clock = clock;
     timer_list->notify_cb = cb;
     timer_list->notify_opaque = opaque;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 7e5b3af..03a9cec 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,6 +15,8 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+/* internal data for savevm */
+#define EVENT_END_STARTUP           0
 /* for time_t event */
 #define EVENT_TIME_T                1
 /* for tm event */
@@ -23,6 +25,10 @@
 #define EVENT_INTERRUPT             15
 /* for shutdown request */
 #define EVENT_SHUTDOWN              20
+/* for save VM event */
+#define EVENT_SAVE_VM_BEGIN         21
+/* for save VM event */
+#define EVENT_SAVE_VM_END           22
 /* for emulated exceptions */
 #define EVENT_EXCEPTION             23
 /* for async events */
@@ -53,9 +59,21 @@ typedef struct ReplayState {
     int skipping_instruction;
     /*! Current step - number of processed instructions and timer events. */
     uint64_t current_step;
+    /*! Temporary data for saving/loading replay file position. */
+    uint64_t file_offset;
 } ReplayState;
 extern ReplayState replay_state;
 
+/*! Information about saved VM state */
+struct SavedStateInfo {
+    /* Offset in the replay log file where state is saved. */
+    uint64_t file_offset;
+    /* Step number, corresponding to the saved state. */
+    uint64_t step;
+};
+/*! Reference to the saved state */
+typedef struct SavedStateInfo SavedStateInfo;
+
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
 
diff --git a/replay/replay.c b/replay/replay.c
index c583bb4..67b32e0 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -12,6 +12,8 @@
 #include "qemu-common.h"
 #include "replay.h"
 #include "replay-internal.h"
+#include "migration/vmstate.h"
+#include "monitor/monitor.h"
 
 /* Current version of the replay mechanism.
    Increase it when file format changes. */
@@ -30,12 +32,177 @@ char *replay_image_suffix;
 
 ReplayState replay_state;
 
+/* 
+    Auto-saving for VM states data 
+*/
+
+/* Minimum capacity of saved states information array */
+#define SAVED_STATES_MIN_CAPACITY   128
+/* Format of the name for the saved state */
+#define SAVED_STATE_NAME_FORMAT     "replay_%" PRId64
+
+/* Timer for auto-save VM states */
+static QEMUTimer *save_timer;
+/* Save state period in seconds */
+static uint64_t save_state_period;
+/* List of the saved states information */
+SavedStateInfo *saved_states;
+/* Number of saved states */
+static size_t saved_states_count;
+/* Capacity of the buffer for saved states */
+static size_t saved_states_capacity;
+/* Number of last loaded/saved state */
+static uint64_t current_saved_state;
+
+/*
+   Replay functions
+ */
 
 int replay_get_play_submode(void)
 {
     return play_submode;
 }
 
+static void replay_pre_save(void *opaque)
+{
+    ReplayState *state = opaque;
+    state->file_offset = ftello64(replay_file);
+}
+
+static int replay_post_load(void *opaque, int version_id)
+{
+    first_cpu->instructions_count = 0;
+
+    ReplayState *state = opaque;
+    fseeko64(replay_file, state->file_offset, SEEK_SET);
+    replay_has_unread_data = 0;
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_replay = {
+    .name = "replay",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .pre_save = replay_pre_save,
+    .post_load = replay_post_load,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT64_ARRAY(cached_clock, ReplayState, REPLAY_CLOCK_COUNT),
+        VMSTATE_INT32(skipping_instruction, ReplayState),
+        VMSTATE_UINT64(current_step, ReplayState),
+        VMSTATE_UINT64(file_offset, ReplayState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void replay_savevm(void *opaque)
+{
+    char name[128];
+    uint64_t offset;
+
+    offset = ftello64(replay_file);
+
+    replay_save_instructions();
+
+    replay_put_event(EVENT_SAVE_VM_BEGIN);
+
+    vm_stop(RUN_STATE_SAVE_VM);
+
+    /* save VM state */
+    sprintf(name, SAVED_STATE_NAME_FORMAT, current_saved_state);
+    if (save_vmstate(default_mon, name) > 0) {
+        /* if period is 0, save only once */
+        if (save_state_period != 0) {
+            timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME)
+                                  + save_state_period);
+        }
+
+        /* add more memory to buffer */
+        if (saved_states_count >= saved_states_capacity) {
+            saved_states_capacity += SAVED_STATES_MIN_CAPACITY;
+            saved_states = g_realloc(saved_states, saved_states_capacity
+                                                   * sizeof(SavedStateInfo));
+            if (!saved_states) {
+                saved_states_count = 0;
+                fprintf(stderr, "Replay: Saved states memory reallocation failed.\n");
+                exit(1);
+            }
+        }
+        /* save state ID into the buffer */
+        saved_states[saved_states_count].file_offset = offset;
+        saved_states[saved_states_count].step = replay_get_current_step();
+        ++saved_states_count;
+        ++current_saved_state;
+    } else {
+        fprintf(stderr, "Cannot save simulator states for replay.\n");
+    }
+
+    replay_put_event(EVENT_SAVE_VM_END);
+
+    tb_flush_all();
+
+    vm_start();
+}
+
+/*! Checks SAVEVM event while reading event log. */
+static void check_savevm(void)
+{
+    replay_fetch_data_kind();
+    if (replay_data_kind != EVENT_SAVE_VM_BEGIN
+        && replay_data_kind != EVENT_SAVE_VM_END) {
+        fprintf(stderr, "Replay: read wrong data kind %d within savevm\n",
+                replay_data_kind);
+        exit(1);
+    }
+    replay_has_unread_data = 0;
+}
+
+/*! Loads specified VM state. */
+static void replay_loadvm(int64_t state)
+{
+    char name[128];
+    bool running = runstate_is_running();
+    if (running && !qemu_in_vcpu_thread()) {
+        vm_stop(RUN_STATE_RESTORE_VM);
+    } else {
+        cpu_disable_ticks();
+    }
+    
+    replay_clear_events();
+
+    sprintf(name, SAVED_STATE_NAME_FORMAT, state);
+    if (load_vmstate(name) < 0) {
+        fprintf(stderr, "Replay: cannot load VM state\n");
+        exit(1);
+    }
+    /* check end event */
+    check_savevm();
+
+    tb_flush_all();
+
+    current_saved_state = state;
+
+    cpu_enable_ticks();
+    if (running && !qemu_in_vcpu_thread()) {
+        vm_start();
+    }
+
+    replay_fetch_data_kind();
+    while (replay_data_kind >= EVENT_CLOCK
+           && replay_data_kind < EVENT_CLOCK + REPLAY_CLOCK_COUNT) {
+        replay_read_next_clock(-1);
+        replay_fetch_data_kind();
+    }
+}
+
+/*! Skips clock events saved to file while saving the VM state. */
+static void replay_skip_savevm(void)
+{
+    replay_has_unread_data = 0;
+    replay_loadvm(current_saved_state + 1);
+}
+
 bool skip_async_events(int stop_event)
 {
     /* nothing to skip - not all instructions used */
@@ -55,6 +222,13 @@ bool skip_async_events(int stop_event)
             replay_has_unread_data = 0;
             qemu_system_shutdown_request_impl();
             break;
+        case EVENT_SAVE_VM_BEGIN:
+            /* cannot correctly load VM while in CPU thread */
+            if (qemu_in_vcpu_thread()) {
+                return res;
+            }
+            replay_skip_savevm();
+            break;
         case EVENT_INSTRUCTION:
             first_cpu->instructions_count = replay_get_dword();
             return res;
@@ -285,6 +459,7 @@ static void replay_enable(const char *fname, int mode)
     replay_data_kind = -1;
     replay_state.skipping_instruction = 0;
     replay_state.current_step = 0;
+    current_saved_state = 0;
 
     /* skip file header for SAVE and check it for PLAY */
     if (replay_mode == REPLAY_SAVE) {
@@ -296,11 +471,21 @@ static void replay_enable(const char *fname, int mode)
             fprintf(stderr, "Replay: invalid input log file version\n");
             exit(1);
         }
+        /* read states table */
+        fseeko64(replay_file, offset, SEEK_SET);
+        saved_states_count = replay_get_qword();
+        saved_states_capacity = saved_states_count;
+        if (saved_states_count) {
+            saved_states = g_malloc(sizeof(SavedStateInfo) * saved_states_count);
+            fread(saved_states, sizeof(SavedStateInfo), saved_states_count, replay_file);
+        }
         /* go to the beginning */
         fseek(replay_file, 12, SEEK_SET);
     }
 
     replay_init_events();
+
+    vmstate_register(NULL, 0, &vmstate_replay, &replay_state);
 }
 
 void replay_configure(QemuOpts *opts, int mode)
@@ -320,6 +505,8 @@ void replay_configure(QemuOpts *opts, int mode)
         replay_image_suffix = g_strdup("replay_qcow");
     }
 
+    save_state_period = 1000LL * qemu_opt_get_number(opts, "period", 0);
+
     replay_enable(fname, mode);
 }
 
@@ -330,6 +517,25 @@ void replay_init_timer(void)
     }
 
     replay_enable_events();
+
+    /* create timer for states auto-saving */
+    if (replay_mode == REPLAY_SAVE) {
+        saved_states_count = 0;
+        if (!saved_states) {
+            saved_states = g_malloc(sizeof(SavedStateInfo) * SAVED_STATES_MIN_CAPACITY);
+            saved_states_capacity = SAVED_STATES_MIN_CAPACITY;
+        }
+        if (save_state_period) {
+            save_timer = timer_new_ms(QEMU_CLOCK_REALTIME, replay_savevm, NULL);
+            timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME));
+        }
+        replay_put_event(EVENT_END_STARTUP);
+        /* Save it right now without waiting for timer */
+        replay_savevm(NULL);
+    } else if (replay_mode == REPLAY_PLAY) {
+        /* load starting VM state */
+        replay_loadvm(0);
+    }
 }
 
 void replay_finish(void)
@@ -347,6 +553,14 @@ void replay_finish(void)
             /* write end event */
             replay_put_event(EVENT_END);
 
+            /* write states table */
+            offset = ftello64(replay_file);
+            replay_put_qword(saved_states_count);
+            if (saved_states && saved_states_count) {
+                fwrite(saved_states, sizeof(SavedStateInfo),
+                       saved_states_count, replay_file);
+            }
+
             /* write header */
             fseek(replay_file, 0, SEEK_SET);
             replay_put_dword(REPLAY_VERSION);
@@ -356,6 +570,15 @@ void replay_finish(void)
         fclose(replay_file);
         replay_file = NULL;
     }
+    if (save_timer) {
+        timer_del(save_timer);
+        timer_free(save_timer);
+        save_timer = NULL;
+    }
+    if (saved_states) {
+        g_free(saved_states);
+        saved_states = NULL;
+    }
     if (replay_filename) {
         g_free(replay_filename);
         replay_filename = NULL;

[Qemu-devel] [RFC PATCH v2 40/49] replay: recording of the user input

[Pavel Dovgalyuk]

This records user input (keyboard and mouse events) in record mode and replays
these input events in replay mode.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/ui/input.h       |    2 +
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   48 +++++++++++++++++++++
 replay/replay-input.c    |  107 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   11 ++++-
 replay/replay.h          |    5 ++
 ui/input.c               |   79 ++++++++++++++++++++++++++--------
 7 files changed, 233 insertions(+), 20 deletions(-)
 create mode 100755 replay/replay-input.c

diff --git a/include/ui/input.h b/include/ui/input.h
index 5d5ac00..d06a12d 100644
--- a/include/ui/input.h
+++ b/include/ui/input.h
@@ -33,7 +33,9 @@ void qemu_input_handler_bind(QemuInputHandlerState *s,
                              const char *device_id, int head,
                              Error **errp);
 void qemu_input_event_send(QemuConsole *src, InputEvent *evt);
+void qemu_input_event_send_impl(QemuConsole *src, InputEvent *evt);
 void qemu_input_event_sync(void);
+void qemu_input_event_sync_impl(void);
 
 InputEvent *qemu_input_event_new_key(KeyValue *key, bool down);
 void qemu_input_event_send_key(QemuConsole *src, KeyValue *key, bool down);
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 257c320..3936296 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -2,3 +2,4 @@ obj-y += replay.o
 obj-y += replay-internal.o
 obj-y += replay-events.o
 obj-y += replay-time.o
+obj-y += replay-input.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index f39889d..d7ed0b2 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -12,6 +12,7 @@
 #include "replay.h"
 #include "replay-internal.h"
 #include "block/thread-pool.h"
+#include "ui/input.h"
 
 typedef struct Event {
     int event_kind;
@@ -42,6 +43,16 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_THREAD:
         thread_pool_work((ThreadPool *)event->opaque, event->opaque2);
         break;
+    case REPLAY_ASYNC_EVENT_INPUT:
+        qemu_input_event_send_impl(NULL, (InputEvent*)event->opaque);
+        /* Using local variables, when replaying. Do not free them. */
+        if (replay_mode == REPLAY_SAVE) {
+            qapi_free_InputEvent((InputEvent*)event->opaque);
+        }
+        break;
+    case REPLAY_ASYNC_EVENT_INPUT_SYNC:
+        qemu_input_event_sync_impl();
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -134,6 +145,16 @@ void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_THREAD, opaque, opaque2, id);
 }
 
+void replay_add_input_event(struct InputEvent *event)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT, event, NULL, 0);
+}
+
+void replay_add_input_sync_event(void)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT_SYNC, NULL, NULL, 0);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -155,6 +176,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_THREAD:
                 replay_put_qword(event->id);
                 break;
+            case REPLAY_ASYNC_EVENT_INPUT:
+                replay_save_input_event(event->opaque);
+                break;
             }
         }
 
@@ -183,6 +207,7 @@ void replay_read_events(int opt)
         if (opt != read_opt)
             break;
         /* Execute some events without searching them in the queue */
+        Event e;
         switch (read_event_kind) {
         case REPLAY_ASYNC_EVENT_BH:
         case REPLAY_ASYNC_EVENT_THREAD:
@@ -190,6 +215,29 @@ void replay_read_events(int opt)
                 read_id = replay_get_qword();
             }
             break;
+        case REPLAY_ASYNC_EVENT_INPUT:
+            e.event_kind = read_event_kind;
+            e.opaque = replay_read_input_event();
+
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
+        case REPLAY_ASYNC_EVENT_INPUT_SYNC:
+            e.event_kind = read_event_kind;
+            e.opaque = 0;
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-input.c b/replay/replay-input.c
new file mode 100755
index 0000000..b754caf
--- /dev/null
+++ b/replay/replay-input.c
@@ -0,0 +1,107 @@
+/*
+ * replay-input.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+#include "replay-internal.h"
+#include "ui/input.h"
+
+void replay_save_input_event(InputEvent *evt)
+{
+    replay_put_dword(evt->kind);
+
+    switch (evt->kind) {
+    case INPUT_EVENT_KIND_KEY:
+        replay_put_dword(evt->key->key->kind);
+
+        switch (evt->key->key->kind) {
+        case KEY_VALUE_KIND_NUMBER:
+            replay_put_qword(evt->key->key->number);
+            replay_put_byte(evt->key->down);
+            break;
+        case KEY_VALUE_KIND_QCODE:
+            replay_put_dword(evt->key->key->qcode);
+            replay_put_byte(evt->key->down);
+            break;
+        case KEY_VALUE_KIND_MAX:
+            /* keep gcc happy */
+            break;
+        }
+        break;
+    case INPUT_EVENT_KIND_BTN:
+        replay_put_dword(evt->btn->button);
+        replay_put_byte(evt->btn->down);
+        break;
+    case INPUT_EVENT_KIND_REL:
+        replay_put_dword(evt->rel->axis);
+        replay_put_qword(evt->rel->value);
+        break;
+    case INPUT_EVENT_KIND_ABS:
+        replay_put_dword(evt->abs->axis);
+        replay_put_qword(evt->abs->value);
+        break;
+    case INPUT_EVENT_KIND_MAX:
+        /* keep gcc happy */
+        break;
+    }
+}
+
+InputEvent *replay_read_input_event(void)
+{
+    static InputEvent evt;
+    static KeyValue keyValue;
+    static InputKeyEvent key;
+    key.key = &keyValue;
+    static InputBtnEvent btn;
+    static InputMoveEvent rel;
+    static InputMoveEvent abs;
+
+    evt.kind = replay_get_dword();
+    switch (evt.kind) {
+    case INPUT_EVENT_KIND_KEY:
+        evt.key = &key;
+        evt.key->key->kind = replay_get_dword();
+
+        switch (evt.key->key->kind) {
+        case KEY_VALUE_KIND_NUMBER:
+            evt.key->key->number = replay_get_qword();
+            evt.key->down = replay_get_byte();
+            break;
+        case KEY_VALUE_KIND_QCODE:
+            evt.key->key->qcode = (QKeyCode)replay_get_dword();
+            evt.key->down = replay_get_byte();
+            break;
+        case KEY_VALUE_KIND_MAX:
+            /* keep gcc happy */
+            break;
+        }
+        break;
+    case INPUT_EVENT_KIND_BTN:
+        evt.btn = &btn;
+        evt.btn->button = (InputButton)replay_get_dword();
+        evt.btn->down = replay_get_byte();
+        break;
+    case INPUT_EVENT_KIND_REL:
+        evt.rel = &rel;
+        evt.rel->axis = (InputAxis)replay_get_dword();
+        evt.rel->value = replay_get_qword();
+        break;
+    case INPUT_EVENT_KIND_ABS:
+        evt.abs = &abs;
+        evt.abs->axis = (InputAxis)replay_get_dword();
+        evt.abs->value = replay_get_qword();
+        break;
+    case INPUT_EVENT_KIND_MAX:
+        /* keep gcc happy */
+        break;
+    }
+
+    return &evt;
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 03a9cec..6aea0f0 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -49,7 +49,9 @@
 
 #define REPLAY_ASYNC_EVENT_BH          0
 #define REPLAY_ASYNC_EVENT_THREAD      1
-#define REPLAY_ASYNC_COUNT             2
+#define REPLAY_ASYNC_EVENT_INPUT       2
+#define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
+#define REPLAY_ASYNC_COUNT             4
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -141,4 +143,11 @@ void replay_read_events(int opt);
 /*! Adds specified async event to the queue */
 void replay_add_event(int event_id, void *opaque);
 
+/* Input events */
+
+/*! Saves input event to the log */
+void replay_save_input_event(InputEvent *evt);
+/*! Reads input event from the log */
+InputEvent *replay_read_input_event(void);
+
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index e066cb2..cc0b4f4 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -17,6 +17,7 @@
 #include <time.h>
 
 struct QemuOpts;
+struct InputEvent;
 
 /* replay modes */
 #define REPLAY_NONE 0
@@ -113,5 +114,9 @@ void replay_disable_events(void);
 void replay_add_bh_event(void *bh, uint64_t id);
 /*! Adds thread event to the queue */
 void replay_add_thread_event(void *pool, void *req, uint64_t id);
+/*! Adds input event to the queue */
+void replay_add_input_event(struct InputEvent *event);
+/*! Adds input sync event to the queue */
+void replay_add_input_sync_event(void);
 
 #endif
diff --git a/ui/input.c b/ui/input.c
index 89d9db7..b690a52 100644
--- a/ui/input.c
+++ b/ui/input.c
@@ -5,6 +5,7 @@
 #include "trace.h"
 #include "ui/input.h"
 #include "ui/console.h"
+#include "replay/replay.h"
 
 struct QemuInputHandlerState {
     DeviceState       *dev;
@@ -258,14 +259,10 @@ static void qemu_input_queue_sync(struct QemuInputEventQueueHead *queue)
     QTAILQ_INSERT_TAIL(queue, item, node);
 }
 
-void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
+void qemu_input_event_send_impl(QemuConsole *src, InputEvent *evt)
 {
     QemuInputHandlerState *s;
 
-    if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
-        return;
-    }
-
     qemu_input_event_trace(src, evt);
 
     /* pre processing */
@@ -282,14 +279,25 @@ void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
     s->events++;
 }
 
-void qemu_input_event_sync(void)
+void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
 {
-    QemuInputHandlerState *s;
-
     if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
         return;
     }
 
+    if (replay_mode == REPLAY_PLAY) {
+        /* Nothing */
+    } else if (replay_mode == REPLAY_SAVE) {
+        replay_add_input_event(evt);
+    } else {
+        qemu_input_event_send_impl(src, evt);
+    }
+}
+
+void qemu_input_event_sync_impl(void)
+{
+    QemuInputHandlerState *s;
+
     trace_input_event_sync();
 
     QTAILQ_FOREACH(s, &handlers, node) {
@@ -303,6 +311,21 @@ void qemu_input_event_sync(void)
     }
 }
 
+void qemu_input_event_sync(void)
+{
+    if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
+        return;
+    }
+
+    if (replay_mode == REPLAY_PLAY) {
+        /* Nothing */
+    } else if (replay_mode == REPLAY_SAVE) {
+        replay_add_input_sync_event();
+    } else {
+        qemu_input_event_sync_impl();
+    }
+}
+
 InputEvent *qemu_input_event_new_key(KeyValue *key, bool down)
 {
     InputEvent *evt = g_new0(InputEvent, 1);
@@ -316,14 +339,22 @@ InputEvent *qemu_input_event_new_key(KeyValue *key, bool down)
 void qemu_input_event_send_key(QemuConsole *src, KeyValue *key, bool down)
 {
     InputEvent *evt;
-    evt = qemu_input_event_new_key(key, down);
-    if (QTAILQ_EMPTY(&kbd_queue)) {
-        qemu_input_event_send(src, evt);
-        qemu_input_event_sync();
-        qapi_free_InputEvent(evt);
-    } else {
-        qemu_input_queue_event(&kbd_queue, src, evt);
-        qemu_input_queue_sync(&kbd_queue);
+    if (replay_mode != REPLAY_PLAY) {
+        evt = qemu_input_event_new_key(key, down);
+        if (QTAILQ_EMPTY(&kbd_queue)) {
+            qemu_input_event_send(src, evt);
+            qemu_input_event_sync();
+            if (replay_mode != REPLAY_SAVE) {
+                qapi_free_InputEvent(evt);
+            }
+        } else {
+            if (replay_mode != REPLAY_NONE) {
+                fprintf(stderr, "Input queue is not supported in record/replay mode\n");
+                exit(1);
+            }
+            qemu_input_queue_event(&kbd_queue, src, evt);
+            qemu_input_queue_sync(&kbd_queue);
+        }
     }
 }
 
@@ -349,6 +380,10 @@ void qemu_input_event_send_key_delay(uint32_t delay_ms)
         kbd_timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, qemu_input_queue_process,
                                  &kbd_queue);
     }
+    if (replay_mode != REPLAY_NONE) {
+        fprintf(stderr, "Input queue is not supported in record/replay mode\n");
+        exit(1);
+    }
     qemu_input_queue_delay(&kbd_queue, kbd_timer,
                            delay_ms ? delay_ms : kbd_default_delay_ms);
 }
@@ -368,7 +403,9 @@ void qemu_input_queue_btn(QemuConsole *src, InputButton btn, bool down)
     InputEvent *evt;
     evt = qemu_input_event_new_btn(btn, down);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_SAVE) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_update_buttons(QemuConsole *src, uint32_t *button_map,
@@ -421,7 +458,9 @@ void qemu_input_queue_rel(QemuConsole *src, InputAxis axis, int value)
     InputEvent *evt;
     evt = qemu_input_event_new_move(INPUT_EVENT_KIND_REL, axis, value);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_SAVE) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_queue_abs(QemuConsole *src, InputAxis axis, int value, int size)
@@ -430,7 +469,9 @@ void qemu_input_queue_abs(QemuConsole *src, InputAxis axis, int value, int size)
     int scaled = qemu_input_scale_axis(value, size, INPUT_EVENT_ABS_SIZE);
     evt = qemu_input_event_new_move(INPUT_EVENT_KIND_ABS, axis, scaled);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_SAVE) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_check_mode_change(void)

[Qemu-devel] [RFC PATCH v2 41/49] tap-win32: destroy the thread at exit

[Pavel Dovgalyuk]

This patch fixes resource leak caused by created thread which is not destroyed
at exit.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 net/tap-win32.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/net/tap-win32.c b/net/tap-win32.c
index 8aee611..efd1c75 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -99,6 +99,7 @@ typedef struct tap_win32_overlapped {
     HANDLE output_queue_semaphore;
     HANDLE free_list_semaphore;
     HANDLE tap_semaphore;
+    HANDLE hThread;
     CRITICAL_SECTION output_queue_cs;
     CRITICAL_SECTION free_list_cs;
     OVERLAPPED read_overlapped;
@@ -625,8 +626,9 @@ static int tap_win32_open(tap_win32_overlapped_t **phandle,
 
     *phandle = &tap_overlapped;
 
-    CreateThread(NULL, 0, tap_win32_thread_entry,
-                 (LPVOID)&tap_overlapped, 0, &idThread);
+    tap_overlapped.hThread = CreateThread(NULL, 0, tap_win32_thread_entry,
+                                          (LPVOID)&tap_overlapped,
+                                          0, &idThread);
     return 0;
 }
 
@@ -643,9 +645,8 @@ static void tap_cleanup(NetClientState *nc)
 
     qemu_del_wait_object(s->handle->tap_semaphore, NULL, NULL);
 
-    /* FIXME: need to kill thread and close file handle:
-       tap_win32_close(s);
-    */
+    TerminateThread(s->handle->hThread, 0);
+    CloseHandle(s->handle->handle);
 }
 
 static ssize_t tap_receive(NetClientState *nc, const uint8_t *buf, size_t size)

[Qemu-devel] [RFC PATCH v2 42/49] replay: network packets record/replay

[Pavel Dovgalyuk]

This patch implements passing network packets to replay module in
record mode. New virtual network adapter is implemented to replay the
packets when they are read from the log file.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 net/Makefile.objs        |    1 
 net/clients.h            |    3 +
 net/dump.c               |    6 +
 net/hub.c                |    1 
 net/net-replay.c         |   66 ++++++++++++++++
 net/net.c                |    7 +-
 net/slirp.c              |   14 +++
 net/socket.c             |   35 ++++++++
 net/tap-win32.c          |   14 +++
 net/tap.c                |   23 +++++-
 net/vde.c                |   14 +++
 qapi-schema.json         |   13 +++
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   17 ++++
 replay/replay-internal.h |   21 +++++
 replay/replay-net.c      |  190 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.c          |    5 +
 replay/replay.h          |   10 ++
 slirp/slirp.c            |    9 ++
 19 files changed, 436 insertions(+), 14 deletions(-)
 create mode 100755 net/net-replay.c
 create mode 100755 replay/replay-net.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index ec19cb3..def44e3 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -2,6 +2,7 @@ common-obj-y = net.o queue.o checksum.o util.o hub.o
 common-obj-y += socket.o
 common-obj-y += dump.o
 common-obj-y += eth.o
+common-obj-y += net-replay.o
 common-obj-$(CONFIG_L2TPV3) += l2tpv3.o
 common-obj-$(CONFIG_POSIX) += tap.o vhost-user.o
 common-obj-$(CONFIG_LINUX) += tap-linux.o
diff --git a/net/clients.h b/net/clients.h
index 2e8feda..f75f43b 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -62,4 +62,7 @@ int net_init_netmap(const NetClientOptions *opts, const char *name,
 int net_init_vhost_user(const NetClientOptions *opts, const char *name,
                         NetClientState *peer);
 
+int net_init_replay(const NetClientOptions *opts, const char *name,
+                    NetClientState *peer);
+
 #endif /* QEMU_NET_CLIENTS_H */
diff --git a/net/dump.c b/net/dump.c
index 9d3a09e..962f2cd 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -28,6 +28,7 @@
 #include "qemu/log.h"
 #include "qemu/timer.h"
 #include "hub.h"
+#include "replay/replay.h"
 
 typedef struct DumpState {
     NetClientState nc;
@@ -158,6 +159,11 @@ int net_init_dump(const NetClientOptions *opts, const char *name,
 
     assert(peer);
 
+    if (replay_mode == REPLAY_SAVE) {
+        fprintf(stderr, "-net dump is not permitted in replay mode\n");
+        exit(1);
+    }
+
     if (dump->has_file) {
         file = dump->file;
     } else {
diff --git a/net/hub.c b/net/hub.c
index 7e0f2d6..91fdfb6 100644
--- a/net/hub.c
+++ b/net/hub.c
@@ -323,6 +323,7 @@ void net_hub_check_clients(void)
             case NET_CLIENT_OPTIONS_KIND_SOCKET:
             case NET_CLIENT_OPTIONS_KIND_VDE:
             case NET_CLIENT_OPTIONS_KIND_VHOST_USER:
+            case NET_CLIENT_OPTIONS_KIND_REPLAY:
                 has_host_dev = 1;
                 break;
             default:
diff --git a/net/net-replay.c b/net/net-replay.c
new file mode 100755
index 0000000..b184b22
--- /dev/null
+++ b/net/net-replay.c
@@ -0,0 +1,66 @@
+/*
+ * net-replay.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "net/net.h"
+#include "clients.h"
+#include "qemu-common.h"
+#include "sysemu/sysemu.h"
+#include "replay/replay.h"
+
+typedef struct NetReplayState {
+    NetClientState nc;
+} NetReplayState;
+
+static ssize_t net_replay_receive(NetClientState *nc, const uint8_t *buf, size_t size)
+{
+    return size;
+}
+
+static void net_replay_cleanup(NetClientState *nc)
+{
+}
+
+static NetClientInfo net_replay_info = {
+    .type = NET_CLIENT_OPTIONS_KIND_REPLAY,
+    .size = sizeof(NetReplayState),
+    .receive = net_replay_receive,
+    .cleanup = net_replay_cleanup,
+};
+
+static int net_replay_init(NetClientState *vlan, const char *device,
+                         const char *name)
+{
+    NetClientState *nc;
+
+    nc = qemu_new_net_client(&net_replay_info, vlan, device, name);
+
+    snprintf(nc->info_str, sizeof(nc->info_str), "replayer");
+
+    if (replay_mode == REPLAY_SAVE) {
+        fprintf(stderr, "-net replay is not permitted in record mode\n");
+        exit(1);
+    } else if (replay_mode == REPLAY_PLAY) {
+        replay_add_network_client(nc);
+    } else {
+        fprintf(stderr, "-net replay is not permitted without replay\n");
+        exit(1);
+    }
+    
+    return 0;
+}
+
+int net_init_replay(const NetClientOptions *opts, const char *name, NetClientState *peer)
+{
+    assert(peer);
+    assert(opts->kind == NET_CLIENT_OPTIONS_KIND_REPLAY);
+
+    return net_replay_init(peer, "replay", name);
+}
diff --git a/net/net.c b/net/net.c
index 6d930ea..62067f5 100644
--- a/net/net.c
+++ b/net/net.c
@@ -809,6 +809,7 @@ static int (* const net_client_init_fun[NET_CLIENT_OPTIONS_KIND_MAX])(
 #ifdef CONFIG_L2TPV3
         [NET_CLIENT_OPTIONS_KIND_L2TPV3]    = net_init_l2tpv3,
 #endif
+        [NET_CLIENT_OPTIONS_KIND_REPLAY]    = net_init_replay,
 };
 
 
@@ -1264,7 +1265,11 @@ int net_init_clients(void)
         /* if no clients, we use a default config */
         qemu_opts_set(net, NULL, "type", "nic");
 #ifdef CONFIG_SLIRP
-        qemu_opts_set(net, NULL, "type", "user");
+        if (replay_mode != REPLAY_PLAY) {
+            qemu_opts_set(net, NULL, "type", "user");
+        } else {
+            qemu_opts_set(net, NULL, "type", "replay");
+        }
 #endif
     }
 
diff --git a/net/slirp.c b/net/slirp.c
index 8fddc03..2ac3b6b 100644
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -36,6 +36,7 @@
 #include "qemu/sockets.h"
 #include "slirp/libslirp.h"
 #include "sysemu/char.h"
+#include "replay/replay.h"
 
 static int get_str_sep(char *buf, int buf_size, const char **pp, int sep)
 {
@@ -103,7 +104,11 @@ void slirp_output(void *opaque, const uint8_t *pkt, int pkt_len)
 {
     SlirpState *s = opaque;
 
-    qemu_send_packet(&s->nc, pkt, pkt_len);
+    if (replay_mode == REPLAY_SAVE) {
+        replay_save_net_packet(&s->nc, pkt, pkt_len);
+    } else {
+        qemu_send_packet(&s->nc, pkt, pkt_len);
+    }
 }
 
 static ssize_t net_slirp_receive(NetClientState *nc, const uint8_t *buf, size_t size)
@@ -267,6 +272,13 @@ static int net_slirp_init(NetClientState *peer, const char *model,
     }
 #endif
 
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net user is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 
 error:
diff --git a/net/socket.c b/net/socket.c
index fb21e20..bbca37b 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -32,6 +32,7 @@
 #include "qemu/sockets.h"
 #include "qemu/iov.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 typedef struct NetSocketState {
     NetClientState nc;
@@ -211,7 +212,11 @@ static void net_socket_send(void *opaque)
             buf += l;
             size -= l;
             if (s->index >= s->packet_len) {
-                qemu_send_packet(&s->nc, s->buf, s->packet_len);
+                if (replay_mode == REPLAY_SAVE) {
+                    replay_save_net_packet(&s->nc, s->buf, s->packet_len);
+                } else {
+                    qemu_send_packet(&s->nc, s->buf, s->packet_len);
+                }
                 s->index = 0;
                 s->state = 0;
             }
@@ -234,7 +239,11 @@ static void net_socket_send_dgram(void *opaque)
         net_socket_write_poll(s, false);
         return;
     }
-    qemu_send_packet(&s->nc, s->buf, size);
+    if (replay_mode == REPLAY_SAVE) {
+        replay_save_net_packet(&s->nc, s->buf, size);
+    } else {
+        qemu_send_packet(&s->nc, s->buf, size);
+    }
 }
 
 static int net_socket_mcast_create(struct sockaddr_in *mcastaddr, struct in_addr *localaddr)
@@ -406,6 +415,13 @@ static NetSocketState *net_socket_fd_init_dgram(NetClientState *peer,
         s->dgram_dst = saddr;
     }
 
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 
 err:
@@ -452,6 +468,14 @@ static NetSocketState *net_socket_fd_init_stream(NetClientState *peer,
     } else {
         qemu_set_fd_handler(s->fd, NULL, net_socket_connect, s);
     }
+
+    if (replay_mode == REPLAY_PLAY) {
+        printf("-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 }
 
@@ -548,6 +572,13 @@ static int net_socket_listen_init(NetClientState *peer,
     s->listen_fd = fd;
     s->nc.link_down = true;
 
+    if (replay_mode == REPLAY_PLAY) {
+        printf("-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     qemu_set_fd_handler(s->listen_fd, net_socket_accept, NULL, s);
     return 0;
 }
diff --git a/net/tap-win32.c b/net/tap-win32.c
index efd1c75..1f2a2f3 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -34,6 +34,7 @@
 #include "net/tap.h"            /* tap_has_ufo, ... */
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
+#include "replay/replay.h"
 #include <stdio.h>
 #include <windows.h>
 #include <winioctl.h>
@@ -665,7 +666,11 @@ static void tap_win32_send(void *opaque)
 
     size = tap_win32_read(s->handle, &buf, max_size);
     if (size > 0) {
-        qemu_send_packet(&s->nc, buf, size);
+        if (replay_mode == REPLAY_SAVE) {
+            replay_save_net_packet(&s->nc, buf, size);
+        } else {
+            qemu_send_packet(&s->nc, buf, size);
+        }
         tap_win32_free_buffer(s->handle, buf);
     }
 }
@@ -749,6 +754,13 @@ static int tap_win32_init(NetClientState *peer, const char *model,
 
     qemu_add_wait_object(s->handle->tap_semaphore, tap_win32_send, s);
 
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net tap is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 }
 
diff --git a/net/tap.c b/net/tap.c
index a40f7f0..8b2094e 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -39,6 +39,7 @@
 #include "sysemu/sysemu.h"
 #include "qemu-common.h"
 #include "qemu/error-report.h"
+#include "replay/replay.h"
 
 #include "net/tap.h"
 
@@ -203,11 +204,17 @@ static void tap_send(void *opaque)
             size -= s->host_vnet_hdr_len;
         }
 
-        size = qemu_send_packet_async(&s->nc, buf, size, tap_send_completed);
-        if (size == 0) {
-            tap_read_poll(s, false);
+        if (replay_mode == REPLAY_SAVE) {
+            replay_save_net_packet(&s->nc, buf, size);
             break;
-        } else if (size < 0) {
+        } else {
+            size = qemu_send_packet_async(&s->nc, buf, size, tap_send_completed);
+            if (size == 0) {
+                tap_read_poll(s, false);
+                break;
+            } else if (size < 0) {
+                break;
+            }
             break;
         }
     }
@@ -353,6 +360,14 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
     }
     tap_read_poll(s, true);
     s->vhost_net = NULL;
+
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net tap is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 }
 
diff --git a/net/vde.c b/net/vde.c
index 2a619fb..ca34c4b 100644
--- a/net/vde.c
+++ b/net/vde.c
@@ -30,6 +30,7 @@
 #include "qemu-common.h"
 #include "qemu/option.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 typedef struct VDEState {
     NetClientState nc;
@@ -44,7 +45,11 @@ static void vde_to_qemu(void *opaque)
 
     size = vde_recv(s->vde, (char *)buf, sizeof(buf), 0);
     if (size > 0) {
-        qemu_send_packet(&s->nc, buf, size);
+        if (replay_mode == REPLAY_SAVE) {
+            replay_save_net_packet(&s->nc, buf, size);
+        } else {
+            qemu_send_packet(&s->nc, buf, size);
+        }
     }
 }
 
@@ -106,6 +111,13 @@ static int net_vde_init(NetClientState *peer, const char *model,
 
     qemu_set_fd_handler(vde_datafd(s->vde), vde_to_qemu, NULL, s);
 
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net vde is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 }
 
diff --git a/qapi-schema.json b/qapi-schema.json
index b11aad2..fee541a 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2211,6 +2211,16 @@
     '*vhostforce':    'bool' } }
 
 ##
+# @NetdevReplayOptions
+#
+# Reads network traffic from the log in replay mode.
+#
+# Since: 2.2
+##
+{ 'type': 'NetdevReplayOptions',
+  'data': { } }
+
+##
 # @NetClientOptions
 #
 # A discriminated record of network device traits.
@@ -2233,7 +2243,8 @@
     'bridge':   'NetdevBridgeOptions',
     'hubport':  'NetdevHubPortOptions',
     'netmap':   'NetdevNetmapOptions',
-    'vhost-user': 'NetdevVhostUserOptions' } }
+    'vhost-user': 'NetdevVhostUserOptions',
+    'replay':   'NetdevReplayOptions' } }
 
 ##
 # @NetLegacy
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 3936296..43bafbb 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -3,3 +3,4 @@ obj-y += replay-internal.o
 obj-y += replay-events.o
 obj-y += replay-time.o
 obj-y += replay-input.o
+obj-y += replay-net.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index d7ed0b2..077b012 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -53,6 +53,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_INPUT_SYNC:
         qemu_input_event_sync_impl();
         break;
+    case REPLAY_ASYNC_EVENT_NETWORK:
+        replay_net_send_packet(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -179,6 +182,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_INPUT:
                 replay_save_input_event(event->opaque);
                 break;
+            case REPLAY_ASYNC_EVENT_NETWORK:
+                replay_net_save_packet(event->opaque);
+                break;
             }
         }
 
@@ -238,6 +244,17 @@ void replay_read_events(int opt)
             replay_fetch_data_kind();
             /* continue with the next event */
             continue;
+        case REPLAY_ASYNC_EVENT_NETWORK:
+            e.opaque = replay_net_read_packet();
+            e.event_kind = read_event_kind;
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 6aea0f0..254a386 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -51,7 +51,8 @@
 #define REPLAY_ASYNC_EVENT_THREAD      1
 #define REPLAY_ASYNC_EVENT_INPUT       2
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
-#define REPLAY_ASYNC_COUNT             4
+#define REPLAY_ASYNC_EVENT_NETWORK     4
+#define REPLAY_ASYNC_COUNT             5
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -150,4 +151,22 @@ void replay_save_input_event(InputEvent *evt);
 /*! Reads input event from the log */
 InputEvent *replay_read_input_event(void);
 
+/* Network events */
+
+/*! Initializes network data structures. */
+void replay_net_init(void);
+/*! Cleans up network data structures. */
+void replay_net_free(void);
+/*! Reads packets offsets array from the log. */
+void replay_net_read_packets_data(void);
+/*! Writes packets offsets array to the log. */
+void replay_net_write_packets_data(void);
+/*! Saves network packet into the log. */
+void replay_net_save_packet(void *opaque);
+/*! Reads network packet from the log. */
+void *replay_net_read_packet(void);
+/*! Called to send packet that was read or received from external input
+    to the net queue. */
+void replay_net_send_packet(void *opaque);
+
 #endif
diff --git a/replay/replay-net.c b/replay/replay-net.c
new file mode 100755
index 0000000..59134d5
--- /dev/null
+++ b/replay/replay-net.c
@@ -0,0 +1,190 @@
+/*
+ * replay-net.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "net/net.h"
+#include "replay.h"
+#include "replay-internal.h"
+
+/* limited by Ethernet frame size */
+#define MAX_NET_PACKET_SIZE             1560
+
+/* Network data */
+NetClientState **vlan_states = NULL;
+size_t vlan_states_count = 0;
+size_t vlan_states_capacity = 0;
+
+/* Structure for storing information about the network packet */
+typedef struct {
+    /* Offset in the replay log file where packet is saved. */
+    uint64_t file_offset;
+    /* Number of step when packet came. */
+    uint64_t step;
+} QEMU_PACKED NetPacketInfo;
+
+typedef struct NetPacketQueue {
+    /* ID of the packet */
+    uint64_t id;
+    /* ID of the network client */
+    int32_t nc_id;
+    size_t size;
+    uint8_t buf[MAX_NET_PACKET_SIZE];
+    uint64_t offset;
+} NetPacketQueue;
+
+/* Network packets count. */
+static uint64_t net_packets_count;
+/* Capacity of the array for packets parameters. */
+static uint64_t net_packets_capacity;
+/* Array for storing network packets parameters. */
+static NetPacketInfo *net_packets;
+
+
+void replay_net_init(void)
+{
+    net_packets_count = 0;
+}
+
+void replay_net_read_packets_data(void)
+{
+    net_packets_count = replay_get_qword();
+    net_packets_capacity = net_packets_count;
+    if (net_packets_count) {
+        net_packets = (NetPacketInfo*)g_malloc(sizeof(NetPacketInfo)
+                                               * net_packets_count);
+        if (fread(net_packets, sizeof(NetPacketInfo),
+                  net_packets_count, replay_file) != net_packets_count) {
+            fprintf(stderr, "Internal error in replay_net_read_packets_data\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_net_write_packets_data(void)
+{
+    replay_put_qword(net_packets_count);
+    if (net_packets && net_packets_count) {
+        fwrite(net_packets, sizeof(NetPacketInfo),
+               net_packets_count, replay_file);
+    }
+}
+
+void replay_add_network_client(NetClientState *nc)
+{
+    if (vlan_states_count == 0) {
+        vlan_states = (NetClientState **)g_malloc(sizeof(*vlan_states));
+        vlan_states_count = 0;
+        vlan_states_capacity = 1;
+    } else if (vlan_states_count == vlan_states_capacity) {
+        vlan_states_capacity *= 2;
+        vlan_states = (NetClientState **)g_realloc(vlan_states,
+                                                   sizeof(*vlan_states)
+                                                   * vlan_states_capacity);
+    }
+
+    vlan_states[vlan_states_count++] = nc;
+}
+
+void replay_net_free(void)
+{
+    if (vlan_states) {
+        g_free(vlan_states);
+        vlan_states = NULL;
+    }
+}
+
+void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
+                            size_t size)
+{
+    if (replay_file) {
+        if (net_packets_capacity == net_packets_count) {
+            if (net_packets_capacity == 0) {
+                net_packets_capacity = 1;
+            } else {
+                net_packets_capacity *= 2;
+            }
+            net_packets = (NetPacketInfo*)g_realloc(net_packets,
+                                                    net_packets_capacity 
+                                                    * sizeof(NetPacketInfo));
+        }
+
+        /* add packet processing event to the queue */
+        NetPacketQueue *p = (NetPacketQueue*)g_malloc0(sizeof(NetPacketQueue));
+        p->id = net_packets_count;
+        p->size = size;
+        if (net_hub_id_for_client(nc, &p->nc_id) < 0) {
+            fprintf(stderr, "Replay: Cannot determine net client id\n");
+            exit(1);
+        }
+        memcpy(p->buf, buf, size);
+        replay_add_event(REPLAY_ASYNC_EVENT_NETWORK, p);
+
+        ++net_packets_count;
+    }
+}
+
+static NetClientState *replay_net_find_vlan(NetPacketQueue *packet)
+{
+    int i;
+    for (i = 0 ; i < vlan_states_count ; ++i) {
+        int id = 0;
+        if (net_hub_id_for_client(vlan_states[i], &id) < 0) {
+            fprintf(stderr, "Replay: Cannot determine net client id\n");
+            exit(1);
+        }
+        if (id == packet->nc_id) {
+            return vlan_states[i];
+        }
+    }
+
+    fprintf(stderr, "Replay: please specify -net replay command-line option\n");
+    exit(1);
+
+    return NULL;
+}
+
+void replay_net_send_packet(void *opaque)
+{
+    NetPacketQueue *packet = (NetPacketQueue *)opaque;
+    NetClientState *vlan_state = replay_net_find_vlan(packet);
+
+    if (replay_mode == REPLAY_SAVE) {
+        net_packets[packet->id].file_offset = packet->offset;
+        net_packets[packet->id].step = replay_get_current_step();
+
+        qemu_send_packet(vlan_state, packet->buf, packet->size);
+    } else if (replay_mode == REPLAY_PLAY) {
+        qemu_send_packet(vlan_state, packet->buf, packet->size);
+    }
+
+    g_free(packet);
+}
+
+void replay_net_save_packet(void *opaque)
+{
+    NetPacketQueue *p = (NetPacketQueue*)opaque;
+    p->offset = ftello64(replay_file);
+    replay_put_qword(p->id);
+    replay_put_dword(p->nc_id);
+    replay_put_array(p->buf, p->size);
+}
+
+void *replay_net_read_packet(void)
+{
+    NetPacketQueue *p = g_malloc0(sizeof(NetPacketQueue));;
+    p->id = replay_get_qword();
+    p->nc_id = replay_get_dword();
+    replay_get_array(p->buf, &p->size);
+    replay_check_error();
+
+    return p;
+}
diff --git a/replay/replay.c b/replay/replay.c
index 67b32e0..47a5809 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -461,6 +461,8 @@ static void replay_enable(const char *fname, int mode)
     replay_state.current_step = 0;
     current_saved_state = 0;
 
+    replay_net_init();
+
     /* skip file header for SAVE and check it for PLAY */
     if (replay_mode == REPLAY_SAVE) {
         fseek(replay_file, HEADER_SIZE, SEEK_SET);
@@ -479,6 +481,7 @@ static void replay_enable(const char *fname, int mode)
             saved_states = g_malloc(sizeof(SavedStateInfo) * saved_states_count);
             fread(saved_states, sizeof(SavedStateInfo), saved_states_count, replay_file);
         }
+        replay_net_read_packets_data();
         /* go to the beginning */
         fseek(replay_file, 12, SEEK_SET);
     }
@@ -560,6 +563,7 @@ void replay_finish(void)
                 fwrite(saved_states, sizeof(SavedStateInfo),
                        saved_states_count, replay_file);
             }
+            replay_net_write_packets_data();
 
             /* write header */
             fseek(replay_file, 0, SEEK_SET);
@@ -588,5 +592,6 @@ void replay_finish(void)
         replay_image_suffix = NULL;
     }
 
+    replay_net_free();
     replay_finish_events();
 }
diff --git a/replay/replay.h b/replay/replay.h
index cc0b4f4..f654673 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -18,6 +18,7 @@
 
 struct QemuOpts;
 struct InputEvent;
+struct NetClientState;
 
 /* replay modes */
 #define REPLAY_NONE 0
@@ -119,4 +120,13 @@ void replay_add_input_event(struct InputEvent *event);
 /*! Adds input sync event to the queue */
 void replay_add_input_sync_event(void);
 
+/* Network */
+
+/*! Registers net client in the replay module. */
+void replay_add_network_client(struct NetClientState *nc);
+/*! Saves incoming network packet in the replay log. */
+void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
+                            size_t size);
+
+
 #endif
diff --git a/slirp/slirp.c b/slirp/slirp.c
index 35f819a..034c268 100644
--- a/slirp/slirp.c
+++ b/slirp/slirp.c
@@ -26,6 +26,7 @@
 #include "sysemu/char.h"
 #include "slirp.h"
 #include "hw/hw.h"
+#include "replay/replay.h"
 
 /* host loopback address */
 struct in_addr loopback_addr;
@@ -234,8 +235,12 @@ Slirp *slirp_init(int restricted, struct in_addr vnetwork,
 
     slirp->opaque = opaque;
 
-    register_savevm(NULL, "slirp", 0, 3,
-                    slirp_state_save, slirp_state_load, slirp);
+    /* Do not save slirp state in record mode, because slirp device
+       will not be created in replay mode */
+    if (replay_mode == REPLAY_NONE) {
+        register_savevm(NULL, "slirp", 0, 3,
+                        slirp_state_save, slirp_state_load, slirp);
+    }
 
     QTAILQ_INSERT_TAIL(&slirp_instances, slirp, entry);
 

[Qemu-devel] [RFC PATCH v2 43/49] replay: audio data record/replay

[Pavel Dovgalyuk]

This patch adds deterministic replay for audio adapter. Replay module saves
data from the microphone and "end-of-playback" events.
Support of audio record and replay is implemented only for Win32 hosts.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 audio/audio.c            |   14 ++-
 audio/audio_win_int.h    |    3 +
 audio/winwaveaudio.c     |  167 ++++++++++++++++++++++++++--------
 replay/Makefile.objs     |    1 
 replay/replay-audio.c    |  228 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.c |    4 +
 replay/replay-internal.h |   14 +++
 replay/replay.h          |   20 ++++
 8 files changed, 408 insertions(+), 43 deletions(-)
 create mode 100755 replay/replay-audio.c

diff --git a/audio/audio.c b/audio/audio.c
index 9d018e9..7b272b7 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -26,6 +26,7 @@
 #include "monitor/monitor.h"
 #include "qemu/timer.h"
 #include "sysemu/sysemu.h"
+#include "replay/replay.h"
 
 #define AUDIO_CAP "audio"
 #include "audio_int.h"
@@ -1201,7 +1202,9 @@ void AUD_set_active_out (SWVoiceOut *sw, int on)
             if (!hw->enabled) {
                 hw->enabled = 1;
                 if (s->vm_running) {
-                    hw->pcm_ops->ctl_out (hw, VOICE_ENABLE, conf.try_poll_out);
+                    hw->pcm_ops->ctl_out(hw, VOICE_ENABLE,
+                                         replay_mode == REPLAY_NONE
+                                             ? conf.try_poll_out : 0);
                     audio_reset_timer (s);
                 }
             }
@@ -1763,11 +1766,13 @@ static void audio_vm_change_state_handler (void *opaque, int running,
 
     s->vm_running = running;
     while ((hwo = audio_pcm_hw_find_any_enabled_out (hwo))) {
-        hwo->pcm_ops->ctl_out (hwo, op, conf.try_poll_out);
+        hwo->pcm_ops->ctl_out(hwo, op, replay_mode == REPLAY_NONE
+                                           ? conf.try_poll_out : 0);
     }
 
     while ((hwi = audio_pcm_hw_find_any_enabled_in (hwi))) {
-        hwi->pcm_ops->ctl_in (hwi, op, conf.try_poll_in);
+        hwi->pcm_ops->ctl_in(hwi, op, replay_mode == REPLAY_NONE
+                                          ? conf.try_poll_in : 0);
     }
     audio_reset_timer (s);
 }
@@ -1810,9 +1815,10 @@ static void audio_atexit (void)
 
 static const VMStateDescription vmstate_audio = {
     .name = "audio",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
     .fields = (VMStateField[]) {
+        VMSTATE_TIMER_V(ts, AudioState, 2),
         VMSTATE_END_OF_LIST()
     }
 };
diff --git a/audio/audio_win_int.h b/audio/audio_win_int.h
index fa5b3cb..0525ae6 100644
--- a/audio/audio_win_int.h
+++ b/audio/audio_win_int.h
@@ -7,4 +7,7 @@ int waveformat_from_audio_settings (WAVEFORMATEX *wfx,
 int waveformat_to_audio_settings (WAVEFORMATEX *wfx,
                                   struct audsettings *as);
 
+void winwave_callback_out_impl(void *dwInstance, WAVEHDR *h);
+void winwave_callback_in_impl(void *dwInstance, WAVEHDR *h);
+
 #endif /* AUDIO_WIN_INT_H */
diff --git a/audio/winwaveaudio.c b/audio/winwaveaudio.c
index 8dbd145..f7325a8 100644
--- a/audio/winwaveaudio.c
+++ b/audio/winwaveaudio.c
@@ -2,7 +2,9 @@
 
 #include "qemu-common.h"
 #include "sysemu/sysemu.h"
+#include "migration/vmstate.h"
 #include "audio.h"
+#include "replay/replay.h"
 
 #define AUDIO_CAP "winwave"
 #include "audio_int.h"
@@ -47,6 +49,7 @@ typedef struct {
     int paused;
     int rpos;
     int avail;
+    int non_added;
     CRITICAL_SECTION crit_sect;
 } WaveVoiceIn;
 
@@ -124,6 +127,24 @@ static void winwave_anal_close_out (WaveVoiceOut *wave)
     wave->hwo = NULL;
 }
 
+void winwave_callback_out_impl(void *dwInstance, WAVEHDR *h)
+{
+    WaveVoiceOut *wave = (WaveVoiceOut *)dwInstance;
+    if (!h->dwUser) {
+        h->dwUser = 1;
+        EnterCriticalSection(&wave->crit_sect);
+        {
+            wave->avail += conf.dac_samples;
+        }
+        LeaveCriticalSection(&wave->crit_sect);
+        if (wave->hw.poll_mode) {
+            if (!SetEvent(wave->event)) {
+                dolog("DAC SetEvent failed %lx\n", GetLastError());
+            }
+        }
+    }
+}
+
 static void CALLBACK winwave_callback_out (
     HWAVEOUT hwo,
     UINT msg,
@@ -132,24 +153,16 @@ static void CALLBACK winwave_callback_out (
     DWORD_PTR dwParam2
     )
 {
-    WaveVoiceOut *wave = (WaveVoiceOut *) dwInstance;
-
     switch (msg) {
     case WOM_DONE:
         {
-            WAVEHDR *h = (WAVEHDR *) dwParam1;
-            if (!h->dwUser) {
-                h->dwUser = 1;
-                EnterCriticalSection (&wave->crit_sect);
-                {
-                    wave->avail += conf.dac_samples;
-                }
-                LeaveCriticalSection (&wave->crit_sect);
-                if (wave->hw.poll_mode) {
-                    if (!SetEvent (wave->event)) {
-                        dolog ("DAC SetEvent failed %lx\n", GetLastError ());
-                    }
-                }
+            if (replay_mode == REPLAY_SAVE) {
+                replay_sound_out_event((WAVEHDR *)dwParam1);
+            } else if (replay_mode == REPLAY_PLAY) {
+                /* Do nothing */
+            } else {
+                winwave_callback_out_impl((void*)dwInstance,
+                                          (WAVEHDR *)dwParam1);
             }
         }
         break;
@@ -163,6 +176,21 @@ static void CALLBACK winwave_callback_out (
     }
 }
 
+static const VMStateDescription vmstate_audio_out = {
+    .name = "audio_out",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_INT32(enabled, HWVoiceOut),
+        VMSTATE_INT32(poll_mode, HWVoiceOut),
+        VMSTATE_INT32(pending_disable, HWVoiceOut),
+        VMSTATE_INT32(rpos, HWVoiceOut),
+        VMSTATE_UINT64(ts_helper, HWVoiceOut),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
 {
     int i;
@@ -173,6 +201,8 @@ static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
 
     wave = (WaveVoiceOut *) hw;
 
+    vmstate_register(NULL, 0, &vmstate_audio_out, hw);
+
     InitializeCriticalSection (&wave->crit_sect);
 
     err = waveformat_from_audio_settings (&wfx, as);
@@ -219,6 +249,10 @@ static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
         }
     }
 
+    if (replay_mode != REPLAY_NONE) {
+        replay_init_sound_out(wave, wave->hdrs, conf.dac_headers);
+    }
+
     return 0;
 
  err4:
@@ -262,10 +296,20 @@ static int winwave_run_out (HWVoiceOut *hw, int live)
         WAVEHDR *h = &wave->hdrs[wave->curhdr];
 
         h->dwUser = 0;
-        mr = waveOutWrite (wave->hwo, h, sizeof (*h));
-        if (mr != MMSYSERR_NOERROR) {
-            winwave_logerr (mr, "waveOutWrite(%d)", wave->curhdr);
-            break;
+        /* Only events will be simulated in REPLAY_PLAY mode,
+           no sounds will be emitted */
+        if (replay_mode != REPLAY_PLAY) {
+            mr = waveOutWrite (wave->hwo, h, sizeof (*h));
+            if (mr != MMSYSERR_NOERROR) {
+                winwave_logerr (mr, "waveOutWrite(%d)", wave->curhdr);
+                break;
+            }
+        } else {
+            /* This header will be passed to callback function, 
+               when event will be read in the log */
+            if (replay_sound_out_event(h)) {
+                break;
+            }
         }
 
         wave->pending -= conf.dac_samples;
@@ -382,6 +426,26 @@ static void winwave_anal_close_in (WaveVoiceIn *wave)
     wave->hwi = NULL;
 }
 
+void winwave_callback_in_impl(void *dwInstance, WAVEHDR *h)
+{
+    WaveVoiceIn *wave = (WaveVoiceIn *)dwInstance;
+
+    if (!h->dwUser) {
+        h->dwUser = 1;
+        EnterCriticalSection(&wave->crit_sect);
+        {
+            wave->avail += conf.adc_samples;
+        }
+        LeaveCriticalSection(&wave->crit_sect);
+
+        if (wave->hw.poll_mode) {
+            if (!SetEvent(wave->event)) {
+                dolog("ADC SetEvent failed %lx\n", GetLastError());
+            }
+        }
+    }
+}
+
 static void CALLBACK winwave_callback_in (
     HWAVEIN *hwi,
     UINT msg,
@@ -390,24 +454,16 @@ static void CALLBACK winwave_callback_in (
     DWORD_PTR dwParam2
     )
 {
-    WaveVoiceIn *wave = (WaveVoiceIn *) dwInstance;
-
     switch (msg) {
     case WIM_DATA:
         {
-            WAVEHDR *h = (WAVEHDR *) dwParam1;
-            if (!h->dwUser) {
-                h->dwUser = 1;
-                EnterCriticalSection (&wave->crit_sect);
-                {
-                    wave->avail += conf.adc_samples;
-                }
-                LeaveCriticalSection (&wave->crit_sect);
-                if (wave->hw.poll_mode) {
-                    if (!SetEvent (wave->event)) {
-                        dolog ("ADC SetEvent failed %lx\n", GetLastError ());
-                    }
-                }
+            if (replay_mode == REPLAY_SAVE) {
+                replay_sound_in_event((WAVEHDR *)dwParam1);
+            } else if (replay_mode == REPLAY_PLAY) {
+                /* Do nothing */
+            } else {
+                winwave_callback_in_impl((void*)dwInstance,
+                                         (WAVEHDR *)dwParam1);
             }
         }
         break;
@@ -421,9 +477,10 @@ static void CALLBACK winwave_callback_in (
     }
 }
 
-static void winwave_add_buffers (WaveVoiceIn *wave, int samples)
+static int winwave_add_buffers (WaveVoiceIn *wave, int samples)
 {
     int doreset;
+    int added = 0;
 
     doreset = wave->hw.poll_mode && (samples >= conf.adc_samples);
     if (doreset && !ResetEvent (wave->event)) {
@@ -435,15 +492,39 @@ static void winwave_add_buffers (WaveVoiceIn *wave, int samples)
         WAVEHDR *h = &wave->hdrs[wave->curhdr];
 
         h->dwUser = 0;
-        mr = waveInAddBuffer (wave->hwi, h, sizeof (*h));
-        if (mr != MMSYSERR_NOERROR) {
-            winwave_logerr (mr, "waveInAddBuffer(%d)", wave->curhdr);
+        /* Add buffer to replay queue in replay mode - it will be
+           loaded from log file. */
+        if (replay_mode != REPLAY_PLAY) {
+            mr = waveInAddBuffer(wave->hwi, h, sizeof (*h));
+            if (mr != MMSYSERR_NOERROR) {
+                winwave_logerr(mr, "waveInAddBuffer(%d)", wave->curhdr);
+            }
+        } else {
+            replay_sound_in_event(h);
         }
         wave->curhdr = (wave->curhdr + 1) % conf.adc_headers;
         samples -= conf.adc_samples;
+        added += conf.adc_samples;	
     }
+
+    return added;
 }
 
+static const VMStateDescription vmstate_audio_in = {
+    .name = "audio_in",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField []) {
+        VMSTATE_INT32(enabled, HWVoiceIn),
+        VMSTATE_INT32(poll_mode, HWVoiceIn),
+        VMSTATE_INT32(wpos, HWVoiceIn),
+        VMSTATE_INT32(total_samples_captured, HWVoiceIn),
+        VMSTATE_UINT64(ts_helper, HWVoiceIn),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
 {
     int i;
@@ -454,6 +535,8 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
 
     wave = (WaveVoiceIn *) hw;
 
+    vmstate_register(NULL, 0, &vmstate_audio_in, hw);
+
     InitializeCriticalSection (&wave->crit_sect);
 
     err = waveformat_from_audio_settings (&wfx, as);
@@ -478,6 +561,7 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
     audio_pcm_init_info (&hw->info, as);
     hw->samples = conf.adc_samples * conf.adc_headers;
     wave->avail = 0;
+    wave->non_added = 0;
 
     wave->pcm_buf = audio_calloc (AUDIO_FUNC, conf.adc_samples,
                                   conf.adc_headers << hw->info.shift);
@@ -500,6 +584,10 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
         }
     }
 
+    if (replay_mode != REPLAY_NONE) {
+        replay_init_sound_in(wave, wave->hdrs, conf.adc_headers);
+    }
+
     wave->paused = 1;
     winwave_add_buffers (wave, hw->samples);
     return 0;
@@ -570,6 +658,7 @@ static int winwave_run_in (HWVoiceIn *hw)
     LeaveCriticalSection (&wave->crit_sect);
 
     ret = decr;
+    wave->non_added += ret;
     while (decr) {
         int left = hw->samples - hw->wpos;
         int conv = audio_MIN (left, decr);
@@ -582,7 +671,7 @@ static int winwave_run_in (HWVoiceIn *hw)
         decr -= conv;
     }
 
-    winwave_add_buffers (wave, ret);
+    wave->non_added -= winwave_add_buffers(wave, wave->non_added);
     return ret;
 }
 
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 43bafbb..1d57e71 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -4,3 +4,4 @@ obj-y += replay-events.o
 obj-y += replay-time.o
 obj-y += replay-input.o
 obj-y += replay-net.o
+obj-y += replay-audio.o
diff --git a/replay/replay-audio.c b/replay/replay-audio.c
new file mode 100755
index 0000000..41e28c1
--- /dev/null
+++ b/replay/replay-audio.c
@@ -0,0 +1,228 @@
+/*
+ * replay-audio.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+#ifdef _WIN32
+struct audsettings;
+#include "audio/audio_win_int.h"
+#endif
+
+/* Sound card state */
+typedef struct {
+    void *instance;
+    const int event_id;
+#ifdef _WIN32
+    WAVEHDR *queue;
+#endif
+    /*! Maximum size of the queue */
+    int size;
+    /*! Current size of the queue */
+    sig_atomic_t cur_size;
+    unsigned int head, tail;
+} SoundQueue;
+
+
+static SoundQueue sound_in = {
+        .event_id = EVENT_SOUND_IN
+    },
+    sound_out = {
+        .event_id = EVENT_SOUND_OUT,
+    };
+
+#ifdef _WIN32
+/*! Spinlock for sound events processing. */
+static spinlock_t sound_lock = SPIN_LOCK_UNLOCKED;
+#endif
+
+/*****************************************************************************
+ *  Sound queue functions                                                    *
+ *****************************************************************************/
+
+/* callback functions */
+#ifdef _WIN32
+
+void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz)
+{
+    sound_in.instance = instance;
+    sound_in.queue = hdrs;
+    sound_in.size = sz;
+    sound_in.head = 0;
+    sound_in.tail = 0;
+    sound_in.cur_size = 0;
+}
+
+void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz)
+{
+    sound_out.instance = instance;
+    sound_out.queue = hdrs;
+    sound_out.size = sz;
+    sound_out.head = 0;
+    sound_out.tail = 0;
+    sound_out.cur_size = 0;
+}
+
+static int sound_queue_add(SoundQueue *q, WAVEHDR *hdr)
+{
+    if (q->queue + q->tail != hdr) {
+        /* state was loaded and we need to reset the queue */
+        if (q->cur_size == 0) {
+            q->head = q->tail = hdr - q->queue;
+        } else {
+            fprintf(stderr, "Replay: Sound queue error\n");
+            exit(1);
+        }
+    }
+
+    if (q->cur_size == q->size) {
+        if (replay_mode == REPLAY_PLAY) {
+            return 1;
+        }
+
+        fprintf(stderr, "Replay: Sound queue overflow\n");
+        exit(1);
+    }
+
+    q->tail = (q->tail + 1) % q->size;
+    ++q->cur_size;
+
+    return 0;
+}
+
+void replay_save_sound_out(void)
+{
+    spin_lock(&sound_lock);
+    while (sound_out.cur_size != 0) {
+        /* put the message ID */
+        replay_put_event(sound_out.event_id);
+        /* save the buffer size */
+        replay_put_dword(sound_out.queue[sound_out.head].dwBytesRecorded);
+        /* perform winwave-specific actions */
+        winwave_callback_out_impl(sound_out.instance,
+                                  &sound_out.queue[sound_out.head]);
+        /* goto the next buffer */
+        sound_out.head = (sound_out.head + 1) % sound_out.size;
+        --sound_out.cur_size;
+    }
+    spin_unlock(&sound_lock);
+}
+
+void replay_save_sound_in(void)
+{
+    spin_lock(&sound_lock);
+    while (sound_in.cur_size != 0) {
+        /* put the message ID */
+        replay_put_event(sound_in.event_id);
+        /* save the buffer */
+        replay_put_array((const uint8_t *)sound_in.queue[sound_in.head].lpData,
+                         sound_in.queue[sound_in.head].dwBytesRecorded);
+        /* perform winwave-specific actions */
+        winwave_callback_in_impl(sound_in.instance,
+                                 &sound_in.queue[sound_in.head]);
+        /* goto the next buffer */
+        sound_in.head = (sound_in.head + 1) % sound_in.size;
+        --sound_in.cur_size;
+    }
+    spin_unlock(&sound_lock);
+}
+
+void replay_read_sound_out(void)
+{
+    if (sound_out.cur_size == 0) {
+        fprintf(stderr, "Replay: Sound queue underflow\n");
+        exit(1);
+    }
+
+    /* get the buffer size */
+    sound_out.queue[sound_out.head].dwBytesRecorded = replay_get_dword();
+
+    replay_check_error();
+    replay_has_unread_data = 0;
+
+    /* perform winwave-specific actions */
+    winwave_callback_out_impl(sound_out.instance, &sound_out.queue[sound_out.head]);
+    sound_out.head = (sound_out.head + 1) % sound_out.size;
+    --sound_out.cur_size;
+}
+
+void replay_read_sound_in(void)
+{
+    if (sound_in.cur_size == 0) {
+        fprintf(stderr, "Replay: Sound queue underflow\n");
+        exit(1);
+    }
+
+    /* get the buffer size */
+    size_t size;
+    replay_get_array((uint8_t *)sound_in.queue[sound_in.head].lpData, &size);
+    sound_in.queue[sound_in.head].dwBytesRecorded = (unsigned int)size;
+
+    replay_check_error();
+    replay_has_unread_data = 0;
+
+    /* perform winwave-specific actions */
+    winwave_callback_in_impl(sound_in.instance, &sound_in.queue[sound_in.head]);
+    sound_in.head = (sound_in.head + 1) % sound_in.size;
+    --sound_in.cur_size;
+}
+
+void replay_sound_in_event(WAVEHDR *hdr)
+{
+    spin_lock(&sound_lock);
+    if (sound_queue_add(&sound_in, hdr)) {
+        fprintf(stderr, "Replay: Input sound buffer overflow\n");
+        exit(1);
+    }
+    spin_unlock(&sound_lock);
+}
+
+int replay_sound_out_event(WAVEHDR *hdr)
+{
+    spin_lock(&sound_lock);
+    int result = sound_queue_add(&sound_out, hdr);
+    spin_unlock(&sound_lock);
+
+    return result;
+}
+#endif
+
+bool replay_has_sound_events(void)
+{
+    return sound_in.cur_size || sound_out.cur_size;
+}
+
+void replay_sound_flush_queue(void)
+{
+#ifdef _WIN32
+    spin_lock(&sound_lock);
+    while (sound_out.cur_size != 0) {
+        /* perform winwave-specific actions */
+        winwave_callback_out_impl(sound_out.instance,
+                                  &sound_out.queue[sound_out.head]);
+        /* goto the next buffer */
+        sound_out.head = (sound_out.head + 1) % sound_out.size;
+        --sound_out.cur_size;
+    }
+    while (sound_in.cur_size != 0)
+    {
+        /* perform winwave-specific actions */
+        winwave_callback_in_impl(sound_in.instance,
+                                 &sound_in.queue[sound_in.head]);
+        /* goto the next buffer */
+        sound_in.head = (sound_in.head + 1) % sound_in.size;
+        --sound_in.cur_size;
+    }
+    spin_unlock(&sound_lock);
+#endif  
+}
+
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
index f416317..fab9c0c 100755
--- a/replay/replay-internal.c
+++ b/replay/replay-internal.c
@@ -151,5 +151,9 @@ void replay_save_instructions(void)
             replay_state.current_step += first_cpu->instructions_count;
             first_cpu->instructions_count = 0;
         }
+#ifdef _WIN32
+        replay_save_sound_in();
+        replay_save_sound_out();
+#endif
     }
 }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 254a386..0f001f4 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -21,6 +21,10 @@
 #define EVENT_TIME_T                1
 /* for tm event */
 #define EVENT_TM                    2
+/* for outgoing sound event */
+#define EVENT_SOUND_OUT             7
+/* for incoming sound event */
+#define EVENT_SOUND_IN              8
 /* for software interrupt */
 #define EVENT_INTERRUPT             15
 /* for shutdown request */
@@ -169,4 +173,14 @@ void *replay_net_read_packet(void);
     to the net queue. */
 void replay_net_send_packet(void *opaque);
 
+/* Sound events */
+
+/*! Returns true, when there are any pending sound events. */
+bool replay_has_sound_events(void);
+void replay_save_sound_out(void);
+void replay_save_sound_in(void);
+void replay_read_sound_out(void);
+void replay_read_sound_in(void);
+void replay_sound_flush_queue(void);
+
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index f654673..700b19a 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -15,6 +15,10 @@
 #include <stdbool.h>
 #include <stdint.h>
 #include <time.h>
+#ifdef _WIN32
+#include <windows.h>
+#include <mmsystem.h>
+#endif
 
 struct QemuOpts;
 struct InputEvent;
@@ -128,5 +132,21 @@ void replay_add_network_client(struct NetClientState *nc);
 void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
                             size_t size);
 
+/* Audio */
+
+#ifdef _WIN32
+/*! Microphone event. */
+void replay_sound_in_event(WAVEHDR *hdr);
+/*! Adds header to the queue.
+    In record mode this header is queued for saving into log.
+    In replay mode this header is queued for reading from log.
+    Returns 1 in replay mode when queue is full.
+    Otherwise returns 0. */
+int replay_sound_out_event(WAVEHDR *hdr);
+/*! Initializes queue for sound input. */
+void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz);
+/*! Initializes queue for sound output. */
+void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz);
+#endif
 
 #endif

[Qemu-devel] [RFC PATCH v2 44/49] replay: serial port

[Pavel Dovgalyuk]

This patch implements record and replay of serial ports.
In record mode serial port can be connected to the source of the data:
-serial tcp:127.0.0.1:23,nowait,server
In replay mode it should not be connected to any data source, but should be
initialized with null: -serial null

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/sysemu/char.h    |   25 ++++++++++++
 qemu-char.c              |   55 +++++++++++++++++++++++---
 replay/Makefile.objs     |    1 
 replay/replay-char.c     |   99 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-events.c   |   18 ++++++++
 replay/replay-internal.h |   14 ++++++-
 replay/replay.c          |   14 +++++++
 replay/replay.h          |   13 ++++++
 vl.c                     |    4 +-
 9 files changed, 236 insertions(+), 7 deletions(-)
 create mode 100755 replay/replay-char.c

diff --git a/include/sysemu/char.h b/include/sysemu/char.h
index 0bbd631..a7eb578 100644
--- a/include/sysemu/char.h
+++ b/include/sysemu/char.h
@@ -85,6 +85,7 @@ struct CharDriverState {
     int is_mux;
     guint fd_in_tag;
     QemuOpts *opts;
+    bool replay;
     QTAILQ_ENTRY(CharDriverState) next;
 };
 
@@ -126,6 +127,21 @@ CharDriverState *qemu_chr_new(const char *label, const char *filename,
                               void (*init)(struct CharDriverState *s));
 
 /**
+ * @qemu_chr_new_replay:
+ *
+ * Create a new character backend from a URI.
+ * All recieved data will be logged by replay module.
+ *
+ * @label the name of the backend
+ * @filename the URI
+ * @init not sure..
+ *
+ * Returns: a new character backend
+ */
+CharDriverState *qemu_chr_new_replay(const char *label, const char *filename,
+                                     void (*init)(struct CharDriverState *s));
+
+/**
  * @qemu_chr_delete:
  *
  * Destroy a character backend.
@@ -320,6 +336,15 @@ int qemu_chr_be_can_write(CharDriverState *s);
  */
 void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len);
 
+/**
+ * @qemu_chr_be_write_impl:
+ *
+ * Implementation of back end writing. Used by replay module.
+ *
+ * @buf a buffer to receive data from the front end
+ * @len the number of bytes to receive from the front end
+ */
+void qemu_chr_be_write_impl(CharDriverState *s, uint8_t *buf, int len);
 
 /**
  * @qemu_chr_be_event:
diff --git a/qemu-char.c b/qemu-char.c
index 55e372c..80d4809 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -28,6 +28,7 @@
 #include "sysemu/char.h"
 #include "hw/usb.h"
 #include "qmp-commands.h"
+#include "replay/replay.h"
 
 #include <unistd.h>
 #include <fcntl.h>
@@ -126,6 +127,9 @@ int qemu_chr_fe_write(CharDriverState *s, const uint8_t *buf, int len)
 
     qemu_mutex_lock(&s->chr_write_lock);
     ret = s->chr_write(s, buf, len);
+    if (s->replay) {
+        replay_data_int(&ret);
+    }
     qemu_mutex_unlock(&s->chr_write_lock);
     return ret;
 }
@@ -195,9 +199,18 @@ int qemu_chr_fe_read_all(CharDriverState *s, uint8_t *buf, int len)
 
 int qemu_chr_fe_ioctl(CharDriverState *s, int cmd, void *arg)
 {
-    if (!s->chr_ioctl)
-        return -ENOTSUP;
-    return s->chr_ioctl(s, cmd, arg);
+    int res;
+    if (!s->chr_ioctl) {
+        res = -ENOTSUP;
+    } else {
+        res = s->chr_ioctl(s, cmd, arg);
+        if (s->replay) {
+            fprintf(stderr, "Replay: ioctl is not supported for serial devices yet\n");
+            exit(1);
+        }
+    }
+
+    return res;
 }
 
 int qemu_chr_be_can_write(CharDriverState *s)
@@ -207,17 +220,34 @@ int qemu_chr_be_can_write(CharDriverState *s)
     return s->chr_can_read(s->handler_opaque);
 }
 
-void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+void qemu_chr_be_write_impl(CharDriverState *s, uint8_t *buf, int len)
 {
     if (s->chr_read) {
         s->chr_read(s->handler_opaque, buf, len);
     }
 }
 
+void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+{
+    if (s->replay) {
+        if (replay_mode == REPLAY_PLAY) {
+            fprintf(stderr, "Replay: calling qemu_chr_be_write in play mode\n");
+            exit(1);
+        }
+        replay_chr_be_write(s, buf, len);
+    } else {
+        qemu_chr_be_write_impl(s, buf, len);
+    }
+}
+
 int qemu_chr_fe_get_msgfd(CharDriverState *s)
 {
     int fd;
-    return (qemu_chr_fe_get_msgfds(s, &fd, 1) == 1) ? fd : -1;
+    int res = (qemu_chr_fe_get_msgfds(s, &fd, 1) == 1) ? fd : -1;
+    if (s->replay) {
+        replay_data_int(&res);
+    }
+    return res;
 }
 
 int qemu_chr_fe_get_msgfds(CharDriverState *s, int *fds, int len)
@@ -3609,6 +3639,21 @@ CharDriverState *qemu_chr_new(const char *label, const char *filename, void (*in
     return chr;
 }
 
+CharDriverState *qemu_chr_new_replay(const char *label, const char *filename,
+                                     void (*init)(struct CharDriverState *s))
+{   
+    if (replay_mode == REPLAY_PLAY && (strcmp(filename, "null")
+        && strcmp(filename, "vc:80Cx24C"))) {
+        fprintf(stderr, "Only \"-serial null\" can be used with replay\n");
+        exit(1);
+    }
+    CharDriverState *chr = qemu_chr_new(label, filename, init);
+    if (strcmp(filename, "vc:80Cx24C")) {
+        replay_register_char_driver(chr);
+    }
+    return chr;
+}
+
 void qemu_chr_fe_set_echo(struct CharDriverState *chr, bool echo)
 {
     if (chr->chr_set_echo) {
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 1d57e71..c54e550 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -5,3 +5,4 @@ obj-y += replay-time.o
 obj-y += replay-input.o
 obj-y += replay-net.o
 obj-y += replay-audio.o
+obj-y += replay-char.o
diff --git a/replay/replay-char.c b/replay/replay-char.c
new file mode 100755
index 0000000..ae4d9ed
--- /dev/null
+++ b/replay/replay-char.c
@@ -0,0 +1,99 @@
+/*
+ * replay-char.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "replay.h"
+#include "replay-internal.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/char.h"
+
+#define MAX_CHAR_DRIVERS MAX_SERIAL_PORTS
+/* Char drivers that generate qemu_chr_be_write events
+   that should be saved into the log. */
+static CharDriverState *char_drivers[MAX_CHAR_DRIVERS];
+
+/* Char event attributes. */
+typedef struct CharEvent {
+    int id;
+    uint8_t *buf;
+    size_t len;
+} CharEvent;
+
+static int find_char_driver(CharDriverState *chr)
+{
+    int i = 0;
+    while (i < MAX_CHAR_DRIVERS && char_drivers[i] != chr) {
+        ++i;
+    }
+
+    return i >= MAX_CHAR_DRIVERS ? -1 : i;
+}
+
+
+void replay_register_char_driver(CharDriverState *chr)
+{
+    chr->replay = true;
+    int i = find_char_driver(NULL);
+
+    if (i < 0) {
+        fprintf(stderr, "Replay: cannot register char driver\n");
+        exit(1);
+    } else {
+        char_drivers[i] = chr;
+    }
+}
+
+void replay_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+{
+    CharEvent *event = g_malloc0(sizeof(CharEvent));
+    
+    event->id = find_char_driver(s);
+    if (event->id < 0) {
+        fprintf(stderr, "Replay: cannot find char driver\n");
+        exit(1);
+    }
+    event->buf = g_malloc(len);
+    memcpy(event->buf, buf, len);
+    event->len = len;
+    
+    replay_add_event(REPLAY_ASYNC_EVENT_CHAR, event);
+}
+
+void replay_event_char_run(void *opaque)
+{
+    CharEvent *event = (CharEvent*)opaque;
+
+    qemu_chr_be_write_impl(char_drivers[event->id], event->buf, (int)event->len);
+    
+    g_free(event->buf);
+    g_free(event);
+}
+
+void replay_event_char_save(void *opaque)
+{
+    CharEvent *event = (CharEvent*)opaque;
+    
+    replay_put_byte(event->id);
+    replay_put_array(event->buf, event->len);
+}
+
+void *replay_event_char_read(void)
+{
+    CharEvent *event = g_malloc0(sizeof(CharEvent));
+    
+    event->id = replay_get_byte();
+    replay_get_array_alloc(&event->buf, &event->len);
+    
+    return event;
+}
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 077b012..6e0120e 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -56,6 +56,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_NETWORK:
         replay_net_send_packet(event->opaque);
         break;
+    case REPLAY_ASYNC_EVENT_CHAR:
+        replay_event_char_run(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -185,6 +188,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_NETWORK:
                 replay_net_save_packet(event->opaque);
                 break;
+            case REPLAY_ASYNC_EVENT_CHAR:
+                replay_event_char_save(event->opaque);
+                break;
             }
         }
 
@@ -255,6 +261,18 @@ void replay_read_events(int opt)
             replay_fetch_data_kind();
             /* continue with the next event */
             continue;
+        case REPLAY_ASYNC_EVENT_CHAR:
+            e.event_kind = read_event_kind;
+            e.opaque = replay_event_char_read();
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+
+            replay_run_event(&e);
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 0f001f4..dfcd5fd 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -38,6 +38,8 @@
 /* for async events */
 #define EVENT_ASYNC                 24
 #define EVENT_ASYNC_OPT             25
+/* for int data */
+#define EVENT_DATA_INT              26
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 /* for clock read/writes */
@@ -56,7 +58,8 @@
 #define REPLAY_ASYNC_EVENT_INPUT       2
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
 #define REPLAY_ASYNC_EVENT_NETWORK     4
-#define REPLAY_ASYNC_COUNT             5
+#define REPLAY_ASYNC_EVENT_CHAR        5
+#define REPLAY_ASYNC_COUNT             6
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -183,4 +186,13 @@ void replay_read_sound_out(void);
 void replay_read_sound_in(void);
 void replay_sound_flush_queue(void);
 
+/* Character devices */
+
+/*! Called to run char device event. */
+void replay_event_char_run(void *opaque);
+/*! Writes char event to the file. */
+void replay_event_char_save(void *opaque);
+/*! Reads char event from the file. */
+void *replay_event_char_read(void);
+
 #endif
diff --git a/replay/replay.c b/replay/replay.c
index 47a5809..8e15b04 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -595,3 +595,17 @@ void replay_finish(void)
     replay_net_free();
     replay_finish_events();
 }
+
+void replay_data_int(int *data)
+{
+    if (replay_file && replay_mode == REPLAY_PLAY) {
+        skip_async_events_until(EVENT_DATA_INT);
+        *data = replay_get_dword();
+        replay_check_error();
+        replay_has_unread_data = 0;
+    } else if (replay_file && replay_mode == REPLAY_SAVE) {
+        replay_save_instructions();
+        replay_put_event(EVENT_DATA_INT);
+        replay_put_dword(*data);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index 700b19a..533fa0a 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -23,6 +23,7 @@
 struct QemuOpts;
 struct InputEvent;
 struct NetClientState;
+struct CharDriverState;
 
 /* replay modes */
 #define REPLAY_NONE 0
@@ -149,4 +150,16 @@ void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz);
 void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz);
 #endif
 
+/* Character device */
+
+/*! Registers char driver to save it's events */
+void replay_register_char_driver(struct CharDriverState *chr);
+/*! Saves write to char device event to the log */
+void replay_chr_be_write(struct CharDriverState *s, uint8_t *buf, int len);
+
+/* Other data */
+
+/*! Writes or reads integer value to/from replay log. */
+void replay_data_int(int *data);
+
 #endif
diff --git a/vl.c b/vl.c
index f8c1e9e..9957bfe 100644
--- a/vl.c
+++ b/vl.c
@@ -2559,7 +2559,9 @@ static int serial_parse(const char *devname)
         exit(1);
     }
     snprintf(label, sizeof(label), "serial%d", index);
-    serial_hds[index] = qemu_chr_new(label, devname, NULL);
+    serial_hds[index] = (replay_mode == REPLAY_NONE ? qemu_chr_new
+                                                    : qemu_chr_new_replay)
+                        (label, devname, NULL);
     if (!serial_hds[index]) {
         fprintf(stderr, "qemu: could not connect serial device"
                 " to character backend '%s'\n", devname);

[Qemu-devel] [RFC PATCH v2 45/49] replay: USB passthrough

[Pavel Dovgalyuk]

It writes all external data, returned by libusb,
to the log. This data is read in replay mode instead of calling
libusb functions.
Command line option for connecting USB device to simulator should be
specified in both record and replay modes. In replay mode only log file
is read and USB device could be not connected to the host machine.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/usb/host-libusb.c     |  525 ++++++++++++++++++++++++++++++----------------
 include/hw/host-libusb.h |  105 +++++++++
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   49 ++++
 replay/replay-internal.h |   21 ++
 replay/replay-usb.c      |  188 ++++++++++++++++
 replay/replay.c          |   29 +++
 replay/replay.h          |   30 +++
 8 files changed, 765 insertions(+), 183 deletions(-)
 create mode 100755 include/hw/host-libusb.h
 create mode 100755 replay/replay-usb.c

diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index c189147..5283c64 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -40,98 +40,15 @@
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
 #include "trace.h"
+#include "qemu/log.h"
+#include "replay/replay.h"
 
 #include "hw/usb.h"
+#include "hw/host-libusb.h"
 
 /* ------------------------------------------------------------------------ */
 
-#define TYPE_USB_HOST_DEVICE "usb-host"
-#define USB_HOST_DEVICE(obj) \
-     OBJECT_CHECK(USBHostDevice, (obj), TYPE_USB_HOST_DEVICE)
-
-typedef struct USBHostDevice USBHostDevice;
-typedef struct USBHostRequest USBHostRequest;
-typedef struct USBHostIsoXfer USBHostIsoXfer;
-typedef struct USBHostIsoRing USBHostIsoRing;
-
-struct USBAutoFilter {
-    uint32_t bus_num;
-    uint32_t addr;
-    char     *port;
-    uint32_t vendor_id;
-    uint32_t product_id;
-};
-
-enum USBHostDeviceOptions {
-    USB_HOST_OPT_PIPELINE,
-};
-
-struct USBHostDevice {
-    USBDevice parent_obj;
-
-    /* properties */
-    struct USBAutoFilter             match;
-    int32_t                          bootindex;
-    uint32_t                         iso_urb_count;
-    uint32_t                         iso_urb_frames;
-    uint32_t                         options;
-    uint32_t                         loglevel;
-
-    /* state */
-    QTAILQ_ENTRY(USBHostDevice)      next;
-    int                              seen, errcount;
-    int                              bus_num;
-    int                              addr;
-    char                             port[16];
-
-    libusb_device                    *dev;
-    libusb_device_handle             *dh;
-    struct libusb_device_descriptor  ddesc;
-
-    struct {
-        bool                         detached;
-        bool                         claimed;
-    } ifs[USB_MAX_INTERFACES];
-
-    /* callbacks & friends */
-    QEMUBH                           *bh_nodev;
-    QEMUBH                           *bh_postld;
-    Notifier                         exit;
-
-    /* request queues */
-    QTAILQ_HEAD(, USBHostRequest)    requests;
-    QTAILQ_HEAD(, USBHostIsoRing)    isorings;
-};
-
-struct USBHostRequest {
-    USBHostDevice                    *host;
-    USBPacket                        *p;
-    bool                             in;
-    struct libusb_transfer           *xfer;
-    unsigned char                    *buffer;
-    unsigned char                    *cbuf;
-    unsigned int                     clen;
     bool                             usb3ep0quirk;
-    QTAILQ_ENTRY(USBHostRequest)     next;
-};
-
-struct USBHostIsoXfer {
-    USBHostIsoRing                   *ring;
-    struct libusb_transfer           *xfer;
-    bool                             copy_complete;
-    unsigned int                     packet;
-    QTAILQ_ENTRY(USBHostIsoXfer)     next;
-};
-
-struct USBHostIsoRing {
-    USBHostDevice                    *host;
-    USBEndpoint                      *ep;
-    QTAILQ_HEAD(, USBHostIsoXfer)    unused;
-    QTAILQ_HEAD(, USBHostIsoXfer)    inflight;
-    QTAILQ_HEAD(, USBHostIsoXfer)    copy;
-    QTAILQ_ENTRY(USBHostIsoRing)     next;
-};
-
 static QTAILQ_HEAD(, USBHostDevice) hostdevs =
     QTAILQ_HEAD_INITIALIZER(hostdevs);
 
@@ -143,6 +60,24 @@ static void usb_host_attach_kernel(USBHostDevice *s);
 
 /* ------------------------------------------------------------------------ */
 
+typedef struct USBHostTimer {
+    QEMUTimer *timer;
+} USBHostTimer;
+
+USBHostTimer usb_auto_timer;
+static int submitted_xfers = 0;
+
+static const VMStateDescription vmstate_usb_host_timer = {
+    .name = "usb-host-timer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_TIMER(timer, USBHostTimer),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 #define CONTROL_TIMEOUT  10000        /* 10 sec    */
 #define BULK_TIMEOUT         0        /* unlimited */
 #define INTR_TIMEOUT         0        /* unlimited */
@@ -214,6 +149,11 @@ static void usb_host_del_fd(int fd, void *user_data)
     qemu_set_fd_handler(fd, NULL, NULL, NULL);
 }
 
+bool usb_host_has_xfers(void)
+{
+    return submitted_xfers != 0;
+}
+
 static int usb_host_init(void)
 {
     const struct libusb_pollfd **poll;
@@ -226,18 +166,34 @@ static int usb_host_init(void)
     if (rc != 0) {
         return -1;
     }
-    libusb_set_debug(ctx, loglevel);
-
-    libusb_set_pollfd_notifiers(ctx, usb_host_add_fd,
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_set_debug(ctx, loglevel);
+        libusb_set_pollfd_notifiers(ctx, usb_host_add_fd,
                                 usb_host_del_fd,
                                 ctx);
-    poll = libusb_get_pollfds(ctx);
-    if (poll) {
-        for (i = 0; poll[i] != NULL; i++) {
-            usb_host_add_fd(poll[i]->fd, poll[i]->events, ctx);
+
+        poll = libusb_get_pollfds(ctx);
+        if (poll) {
+            for (i = 0; poll[i] != NULL; i++) {
+                usb_host_add_fd(poll[i]->fd, poll[i]->events, ctx);
+            }
+        }
+        free(poll);
+    }
+
+    /* replay: changed timer for working with VM clock instead of RT */
+    if (!usb_auto_timer.timer) {
+        usb_auto_timer.timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, usb_host_auto_check, NULL);
+        if (!usb_auto_timer.timer) {
+            return -1;
+        }
+        trace_usb_host_auto_scan_enabled();
+
+        if (replay_mode != REPLAY_NONE) {
+            vmstate_register(NULL, 0, &vmstate_usb_host_timer, &usb_auto_timer);
         }
     }
-    free(poll);
+
     return 0;
 }
 
@@ -252,6 +208,7 @@ static int usb_host_get_port(libusb_device *dev, char *port, size_t len)
 #else
     rc = libusb_get_port_path(ctx, dev, path, 7);
 #endif
+
     if (rc < 0) {
         return 0;
     }
@@ -307,6 +264,7 @@ static USBHostRequest *usb_host_req_alloc(USBHostDevice *s, USBPacket *p,
     r->host = s;
     r->p = p;
     r->in = in;
+    // allocate xfer's in REPLAY_PLAY too
     r->xfer = libusb_alloc_transfer(0);
     if (bufsize) {
         r->buffer = g_malloc(bufsize);
@@ -320,6 +278,7 @@ static void usb_host_req_free(USBHostRequest *r)
     if (r->host) {
         QTAILQ_REMOVE(&r->host->requests, r, next);
     }
+    // free xfer's in REPLAY_PLAY too
     libusb_free_transfer(r->xfer);
     g_free(r->buffer);
     g_free(r);
@@ -337,12 +296,13 @@ static USBHostRequest *usb_host_req_find(USBHostDevice *s, USBPacket *p)
     return NULL;
 }
 
-static void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
+void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
 {
     USBHostRequest *r = xfer->user_data;
     USBHostDevice  *s = r->host;
     bool disconnect = (xfer->status == LIBUSB_TRANSFER_NO_DEVICE);
 
+    --submitted_xfers;
     if (r->p == NULL) {
         goto out; /* request was canceled */
     }
@@ -354,7 +314,7 @@ static void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
 
         /* Fix up USB-3 ep0 maxpacket size to allow superspeed connected devices
          * to work redirected to a not superspeed capable hcd */
-        if (r->usb3ep0quirk && xfer->actual_length >= 18 &&
+        if (usb3ep0quirk && xfer->actual_length >= 18 &&
             r->cbuf[7] == 9) {
             r->cbuf[7] = 64;
         }
@@ -370,12 +330,13 @@ out:
     }
 }
 
-static void usb_host_req_complete_data(struct libusb_transfer *xfer)
+void usb_host_req_complete_data(struct libusb_transfer *xfer)
 {
     USBHostRequest *r = xfer->user_data;
     USBHostDevice  *s = r->host;
     bool disconnect = (xfer->status == LIBUSB_TRANSFER_NO_DEVICE);
 
+    --submitted_xfers;
     if (r->p == NULL) {
         goto out; /* request was canceled */
     }
@@ -419,17 +380,17 @@ static void usb_host_req_abort(USBHostRequest *r)
     QTAILQ_REMOVE(&r->host->requests, r, next);
     r->host = NULL;
 
-    if (inflight) {
+    if (inflight && replay_mode != REPLAY_PLAY) {
         libusb_cancel_transfer(r->xfer);
     }
 }
 
 /* ------------------------------------------------------------------------ */
 
-static void usb_host_req_complete_iso(struct libusb_transfer *transfer)
+void usb_host_req_complete_iso(struct libusb_transfer *transfer)
 {
     USBHostIsoXfer *xfer = transfer->user_data;
-
+    --submitted_xfers;
     if (!xfer) {
         /* USBHostIsoXfer released while inflight */
         g_free(transfer->buffer);
@@ -467,6 +428,7 @@ static USBHostIsoRing *usb_host_iso_alloc(USBHostDevice *s, USBEndpoint *ep)
     for (i = 0; i < s->iso_urb_count; i++) {
         xfer = g_new0(USBHostIsoXfer, 1);
         xfer->ring = ring;
+        // OK for replay too
         xfer->xfer = libusb_alloc_transfer(packets);
         xfer->xfer->dev_handle = s->dh;
         xfer->xfer->type = LIBUSB_TRANSFER_TYPE_ISOCHRONOUS;
@@ -475,7 +437,7 @@ static USBHostIsoRing *usb_host_iso_alloc(USBHostDevice *s, USBEndpoint *ep)
         if (ring->ep->pid == USB_TOKEN_IN) {
             xfer->xfer->endpoint |= USB_DIR_IN;
         }
-        xfer->xfer->callback = usb_host_req_complete_iso;
+        xfer->xfer->callback = replay_mode == REPLAY_NONE ? usb_host_req_complete_iso : replay_req_complete_iso;
         xfer->xfer->user_data = xfer;
 
         xfer->xfer->num_iso_packets = packets;
@@ -514,6 +476,7 @@ static void usb_host_iso_free_xfer(USBHostIsoXfer *xfer, bool inflight)
         xfer->xfer->user_data = NULL;
     } else {
         g_free(xfer->xfer->buffer);
+        // REPLAY_PLAY also allocates xfer
         libusb_free_transfer(xfer->xfer);
     }
     g_free(xfer);
@@ -549,12 +512,19 @@ static void usb_host_iso_free_all(USBHostDevice *s)
     }
 }
 
+
+unsigned char *usb_host_get_iso_packet_buffer(USBHostIsoXfer *xfer, int packet)
+{
+    // replay OK
+    return libusb_get_iso_packet_buffer_simple(xfer->xfer, packet);
+}
+
 static bool usb_host_iso_data_copy(USBHostIsoXfer *xfer, USBPacket *p)
 {
     unsigned int psize;
     unsigned char *buf;
 
-    buf = libusb_get_iso_packet_buffer_simple(xfer->xfer, xfer->packet);
+    buf = usb_host_get_iso_packet_buffer(xfer, xfer->packet);
     if (p->pid == USB_TOKEN_OUT) {
         psize = p->iov.size;
         if (psize > xfer->ring->ep->max_packet_size) {
@@ -600,7 +570,7 @@ static void usb_host_iso_data_in(USBHostDevice *s, USBPacket *p)
     while ((xfer = QTAILQ_FIRST(&ring->unused)) != NULL) {
         QTAILQ_REMOVE(&ring->unused, xfer, next);
         usb_host_iso_reset_xfer(xfer);
-        rc = libusb_submit_transfer(xfer->xfer);
+        REPLAY_DATA_INT(rc, libusb_submit_transfer(xfer->xfer));
         if (rc != 0) {
             usb_host_libusb_error("libusb_submit_transfer [iso]", rc);
             QTAILQ_INSERT_TAIL(&ring->unused, xfer, next);
@@ -608,6 +578,9 @@ static void usb_host_iso_data_in(USBHostDevice *s, USBPacket *p)
                 disconnect = true;
             }
             break;
+        } else {
+            replay_req_register_iso(xfer->xfer);
+            ++submitted_xfers;
         }
         if (QTAILQ_EMPTY(&ring->inflight)) {
             trace_usb_host_iso_start(s->bus_num, s->addr, p->ep->nr);
@@ -662,7 +635,7 @@ static void usb_host_iso_data_out(USBHostDevice *s, USBPacket *p)
     while ((xfer = QTAILQ_FIRST(&ring->copy)) != NULL &&
            xfer->copy_complete) {
         QTAILQ_REMOVE(&ring->copy, xfer, next);
-        rc = libusb_submit_transfer(xfer->xfer);
+        REPLAY_DATA_INT(rc, libusb_submit_transfer(xfer->xfer));
         if (rc != 0) {
             usb_host_libusb_error("libusb_submit_transfer [iso]", rc);
             QTAILQ_INSERT_TAIL(&ring->unused, xfer, next);
@@ -670,6 +643,9 @@ static void usb_host_iso_data_out(USBHostDevice *s, USBPacket *p)
                 disconnect = true;
             }
             break;
+        } else {
+            replay_req_register_iso(xfer->xfer);
+            ++submitted_xfers;
         }
         if (QTAILQ_EMPTY(&ring->inflight)) {
             trace_usb_host_iso_start(s->bus_num, s->addr, p->ep->nr);
@@ -697,6 +673,11 @@ static void usb_host_speed_compat(USBHostDevice *s)
     bool compat_full = true;
     uint8_t type;
     int rc, c, i, a, e;
+    bool ok = false;
+    if (replay_mode == REPLAY_PLAY) {
+        REPLAY_DATA_VAR(ok);
+        return;
+    }
 
     for (c = 0;; c++) {
         rc = libusb_get_config_descriptor(s->dev, c, &conf);
@@ -762,9 +743,12 @@ static void usb_host_ep_update(USBHostDevice *s)
         [USB_ENDPOINT_XFER_INT]     = "int",
     };
     USBDevice *udev = USB_DEVICE(s);
-    struct libusb_config_descriptor *conf;
-    const struct libusb_interface_descriptor *intf;
-    const struct libusb_endpoint_descriptor *endp;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_interface_descriptor dummy_intf;
+    struct libusb_endpoint_descriptor dummy_endp;
+    struct libusb_config_descriptor *conf = &dummy_conf;
+    struct libusb_interface_descriptor *intf;
+    struct libusb_endpoint_descriptor *endp;
 #ifdef HAVE_STREAMS
     struct libusb_ss_endpoint_companion_descriptor *endp_ss_comp;
 #endif
@@ -773,21 +757,36 @@ static void usb_host_ep_update(USBHostDevice *s)
     int rc, i, e;
 
     usb_ep_reset(udev);
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
     trace_usb_host_parse_config(s->bus_num, s->addr,
                                 conf->bConfigurationValue, true);
 
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
     for (i = 0; i < conf->bNumInterfaces; i++) {
-        assert(udev->altsetting[i] < conf->interface[i].num_altsetting);
-        intf = &conf->interface[i].altsetting[udev->altsetting[i]];
+        if (replay_mode != REPLAY_PLAY) {
+            assert(udev->altsetting[i] < conf->interface[i].num_altsetting);
+            intf = (struct libusb_interface_descriptor *)&conf->interface[i].altsetting[udev->altsetting[i]];
+        } else {
+            intf = &dummy_intf;
+        }
+        REPLAY_DATA_VAR(intf->bInterfaceNumber);
+        REPLAY_DATA_VAR(intf->bAlternateSetting);
+        REPLAY_DATA_VAR(intf->bNumEndpoints);
         trace_usb_host_parse_interface(s->bus_num, s->addr,
                                        intf->bInterfaceNumber,
                                        intf->bAlternateSetting, true);
         for (e = 0; e < intf->bNumEndpoints; e++) {
-            endp = &intf->endpoint[e];
+            if (replay_mode != REPLAY_PLAY) {
+                endp = (struct libusb_endpoint_descriptor *)&intf->endpoint[e];
+            } else {
+                endp = &dummy_endp;
+            }
+            REPLAY_DATA_VAR(endp->bEndpointAddress);
+            REPLAY_DATA_VAR(endp->bmAttributes);
+            REPLAY_DATA_VAR(endp->wMaxPacketSize);
 
             devep = endp->bEndpointAddress;
             pid = (devep & USB_DIR_IN) ? USB_TOKEN_IN : USB_TOKEN_OUT;
@@ -825,45 +824,76 @@ static void usb_host_ep_update(USBHostDevice *s)
         }
     }
 
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static int usb_host_open(USBHostDevice *s, libusb_device *dev)
 {
     USBDevice *udev = USB_DEVICE(s);
-    int bus_num = libusb_get_bus_number(dev);
-    int addr    = libusb_get_device_address(dev);
+    int bus_num;
+    REPLAY_DATA_INT(bus_num, libusb_get_bus_number(dev));
+    int addr;
+    REPLAY_DATA_INT(addr, libusb_get_device_address(dev));
     int rc;
+    int speed;
 
     trace_usb_host_open_started(bus_num, addr);
 
-    if (s->dh != NULL) {
+    if (s->is_open) {
         goto fail;
     }
-    rc = libusb_open(dev, &s->dh);
+    REPLAY_DATA_INT(rc, libusb_open(dev, &s->dh));
     if (rc != 0) {
         goto fail;
     }
+    if (replay_mode == REPLAY_PLAY) {
+        // invalid non-NULL pointer for checking conditions
+        s->dh = (void*)0xbad;
+    }
+    s->is_open = true;
 
+    usb_host_detach_kernel(s);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_get_device_descriptor(dev, &s->ddesc);
+    }
+    // TODO: all fields are read and written as int
+    REPLAY_DATA_VAR(s->ddesc.bLength);
+    REPLAY_DATA_VAR(s->ddesc.bDescriptorType);
+    REPLAY_DATA_VAR(s->ddesc.bcdUSB);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceClass);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceSubClass);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceProtocol);
+    REPLAY_DATA_VAR(s->ddesc.bMaxPacketSize0);
+    REPLAY_DATA_VAR(s->ddesc.idVendor);
+    REPLAY_DATA_VAR(s->ddesc.idProduct);
+    REPLAY_DATA_VAR(s->ddesc.bcdDevice);
+    REPLAY_DATA_VAR(s->ddesc.iManufacturer);
+    REPLAY_DATA_VAR(s->ddesc.iProduct);
+    REPLAY_DATA_VAR(s->ddesc.iSerialNumber);
+    REPLAY_DATA_VAR(s->ddesc.bNumConfigurations);
     s->dev     = dev;
     s->bus_num = bus_num;
     s->addr    = addr;
-
-    usb_host_detach_kernel(s);
-
-    libusb_get_device_descriptor(dev, &s->ddesc);
-    usb_host_get_port(s->dev, s->port, sizeof(s->port));
+    if (replay_mode != REPLAY_PLAY) {
+        usb_host_get_port(s->dev, s->port, sizeof(s->port));
+    }
+    replay_data_buffer((unsigned char *)s->port, sizeof(s->port));
 
     usb_ep_init(udev);
     usb_host_ep_update(s);
 
-    udev->speed     = speed_map[libusb_get_device_speed(dev)];
+    udev->speed     = speed_map[REPLAY_DATA_INT(speed, libusb_get_device_speed(dev))];
     usb_host_speed_compat(s);
 
     if (s->ddesc.iProduct) {
-        libusb_get_string_descriptor_ascii(s->dh, s->ddesc.iProduct,
+        if (replay_mode != REPLAY_PLAY) {
+            libusb_get_string_descriptor_ascii(s->dh, s->ddesc.iProduct,
                                            (unsigned char *)udev->product_desc,
                                            sizeof(udev->product_desc));
+        }
+        replay_data_buffer((unsigned char*)udev->product_desc, sizeof(udev->product_desc));
     } else {
         snprintf(udev->product_desc, sizeof(udev->product_desc),
                  "host:%d.%d", bus_num, addr);
@@ -879,10 +909,13 @@ static int usb_host_open(USBHostDevice *s, libusb_device *dev)
 
 fail:
     trace_usb_host_open_failure(bus_num, addr);
-    if (s->dh != NULL) {
-        libusb_close(s->dh);
+    if (s->is_open) {
+        if (replay_mode != REPLAY_PLAY) {
+            libusb_close(s->dh);
+        }
         s->dh = NULL;
         s->dev = NULL;
+        s->is_open = false;
     }
     return -1;
 }
@@ -896,11 +929,20 @@ static void usb_host_abort_xfers(USBHostDevice *s)
     }
 }
 
+static void usb_host_free_xfers(USBHostDevice *s)
+{
+    USBHostRequest *r;
+
+    while ((r = QTAILQ_FIRST(&s->requests)) != NULL) {
+        usb_host_req_free(r);
+    }
+}
+
 static int usb_host_close(USBHostDevice *s)
 {
     USBDevice *udev = USB_DEVICE(s);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         return -1;
     }
 
@@ -914,11 +956,20 @@ static int usb_host_close(USBHostDevice *s)
     }
 
     usb_host_release_interfaces(s);
-    libusb_reset_device(s->dh);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_reset_device(s->dh);
+    }
+
     usb_host_attach_kernel(s);
-    libusb_close(s->dh);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_close(s->dh);
+    }
+
+    //replay_unregister_usb_device(udev);
+
     s->dh = NULL;
     s->dev = NULL;
+    s->is_open = false;
 
     usb_host_auto_check(NULL);
     return 0;
@@ -933,7 +984,7 @@ static void usb_host_nodev_bh(void *opaque)
 static void usb_host_nodev(USBHostDevice *s)
 {
     if (!s->bh_nodev) {
-        s->bh_nodev = qemu_bh_new(usb_host_nodev_bh, s);
+        s->bh_nodev = qemu_bh_new_replay(usb_host_nodev_bh, s, s->bus_num);
     }
     qemu_bh_schedule(s->bh_nodev);
 }
@@ -942,7 +993,7 @@ static void usb_host_exit_notifier(struct Notifier *n, void *data)
 {
     USBHostDevice *s = container_of(n, USBHostDevice, exit);
 
-    if (s->dh) {
+    if (s->is_open) {
         usb_host_release_interfaces(s);
         usb_host_attach_kernel(s);
     }
@@ -1004,57 +1055,70 @@ static void usb_host_cancel_packet(USBDevice *udev, USBPacket *p)
     r = usb_host_req_find(s, p);
     if (r && r->p) {
         r->p = NULL; /* mark as dead */
-        libusb_cancel_transfer(r->xfer);
+        if (replay_mode != REPLAY_PLAY) {
+            libusb_cancel_transfer(r->xfer);
+        }
     }
 }
 
 static void usb_host_detach_kernel(USBHostDevice *s)
 {
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
     for (i = 0; i < conf->bNumInterfaces; i++) {
-        rc = libusb_kernel_driver_active(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_kernel_driver_active(s->dh, i));
         usb_host_libusb_error("libusb_kernel_driver_active", rc);
         if (rc != 1) {
             continue;
         }
         trace_usb_host_detach_kernel(s->bus_num, s->addr, i);
-        rc = libusb_detach_kernel_driver(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_detach_kernel_driver(s->dh, i));
         usb_host_libusb_error("libusb_detach_kernel_driver", rc);
         s->ifs[i].detached = true;
     }
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static void usb_host_attach_kernel(USBHostDevice *s)
 {
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
-    for (i = 0; i < conf->bNumInterfaces; i++) {
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
+    for (i = 0; i < conf->bNumInterfaces ; i++) {
         if (!s->ifs[i].detached) {
             continue;
         }
         trace_usb_host_attach_kernel(s->bus_num, s->addr, i);
-        libusb_attach_kernel_driver(s->dh, i);
+        if (replay_mode != REPLAY_PLAY) {
+            libusb_attach_kernel_driver(s->dh, i);
+        }
         s->ifs[i].detached = false;
     }
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
 {
     USBDevice *udev = USB_DEVICE(s);
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
     for (i = 0; i < USB_MAX_INTERFACES; i++) {
@@ -1065,7 +1129,7 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
 
     usb_host_detach_kernel(s);
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         if (rc == LIBUSB_ERROR_NOT_FOUND) {
             /* address state - ignore */
@@ -1073,10 +1137,11 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
         }
         return USB_RET_STALL;
     }
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
 
     for (i = 0; i < conf->bNumInterfaces; i++) {
         trace_usb_host_claim_interface(s->bus_num, s->addr, configuration, i);
-        rc = libusb_claim_interface(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_claim_interface(s->dh, i));
         usb_host_libusb_error("libusb_claim_interface", rc);
         if (rc != 0) {
             return USB_RET_STALL;
@@ -1087,7 +1152,9 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
     udev->ninterfaces   = conf->bNumInterfaces;
     udev->configuration = configuration;
 
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
     return USB_RET_SUCCESS;
 }
 
@@ -1101,7 +1168,7 @@ static void usb_host_release_interfaces(USBHostDevice *s)
             continue;
         }
         trace_usb_host_release_interface(s->bus_num, s->addr, i);
-        rc = libusb_release_interface(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_release_interface(s->dh, i));
         usb_host_libusb_error("libusb_release_interface", rc);
         s->ifs[i].claimed = false;
     }
@@ -1122,7 +1189,7 @@ static void usb_host_set_config(USBHostDevice *s, int config, USBPacket *p)
     trace_usb_host_set_config(s->bus_num, s->addr, config);
 
     usb_host_release_interfaces(s);
-    rc = libusb_set_configuration(s->dh, config);
+    REPLAY_DATA_INT(rc, libusb_set_configuration(s->dh, config));
     if (rc != 0) {
         usb_host_libusb_error("libusb_set_configuration", rc);
         p->status = USB_RET_STALL;
@@ -1153,7 +1220,7 @@ static void usb_host_set_interface(USBHostDevice *s, int iface, int alt,
         return;
     }
 
-    rc = libusb_set_interface_alt_setting(s->dh, iface, alt);
+    REPLAY_DATA_INT(rc, libusb_set_interface_alt_setting(s->dh, iface, alt));
     if (rc != 0) {
         usb_host_libusb_error("libusb_set_interface_alt_setting", rc);
         p->status = USB_RET_STALL;
@@ -1177,7 +1244,7 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
 
     trace_usb_host_req_control(s->bus_num, s->addr, p, request, value, index);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
         return;
@@ -1202,7 +1269,9 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
     case EndpointOutRequest | USB_REQ_CLEAR_FEATURE:
         if (value == 0) { /* clear halt */
             int pid = (index & USB_DIR_IN) ? USB_TOKEN_IN : USB_TOKEN_OUT;
-            libusb_clear_halt(s->dh, index);
+            if (replay_mode != REPLAY_PLAY) {
+                libusb_clear_halt(s->dh, index);
+            }
             usb_ep_set_halted(udev, pid, index & 0x0f, 0);
             trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
             return;
@@ -1222,13 +1291,15 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
     if (udev->speed == USB_SPEED_SUPER &&
         !((udev->port->speedmask & USB_SPEED_MASK_SUPER)) &&
         request == 0x8006 && value == 0x100 && index == 0) {
-        r->usb3ep0quirk = true;
+        usb3ep0quirk = true;
     }
 
+    // call in REPLAY_PLAY too - this function just copies data
     libusb_fill_control_transfer(r->xfer, s->dh, r->buffer,
-                                 usb_host_req_complete_ctrl, r,
-                                 CONTROL_TIMEOUT);
-    rc = libusb_submit_transfer(r->xfer);
+                                 replay_mode == REPLAY_NONE ? usb_host_req_complete_ctrl : replay_req_complete_ctrl,
+                                 r, CONTROL_TIMEOUT);
+
+    REPLAY_DATA_INT(rc, libusb_submit_transfer(r->xfer));
     if (rc != 0) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_complete(s->bus_num, s->addr, p,
@@ -1237,6 +1308,9 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
             usb_host_nodev(s);
         }
         return;
+    } else {
+        replay_req_register_ctrl(r->xfer);
+        ++submitted_xfers;
     }
 
     p->status = USB_RET_ASYNC;
@@ -1258,7 +1332,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
                             p->pid == USB_TOKEN_IN,
                             p->ep->nr, p->iov.size);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
         return;
@@ -1291,7 +1365,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         } else {
             libusb_fill_bulk_transfer(r->xfer, s->dh, ep,
                                       r->buffer, size,
-                                      usb_host_req_complete_data, r,
+                                      replay_mode == REPLAY_NONE ? usb_host_req_complete_data : replay_req_complete_data, r,
                                       BULK_TIMEOUT);
         }
         break;
@@ -1303,7 +1377,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         ep = p->ep->nr | (r->in ? USB_DIR_IN : 0);
         libusb_fill_interrupt_transfer(r->xfer, s->dh, ep,
                                        r->buffer, p->iov.size,
-                                       usb_host_req_complete_data, r,
+                                       replay_mode == REPLAY_NONE ? usb_host_req_complete_data : replay_req_complete_data, r,
                                        INTR_TIMEOUT);
         break;
     case USB_ENDPOINT_XFER_ISOC:
@@ -1322,7 +1396,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         return;
     }
 
-    rc = libusb_submit_transfer(r->xfer);
+    REPLAY_DATA_INT(rc, libusb_submit_transfer(r->xfer));
     if (rc != 0) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_complete(s->bus_num, s->addr, p,
@@ -1331,6 +1405,9 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
             usb_host_nodev(s);
         }
         return;
+    } else {
+        replay_req_register_data(r->xfer);
+        ++submitted_xfers;
     }
 
     p->status = USB_RET_ASYNC;
@@ -1350,7 +1427,7 @@ static void usb_host_handle_reset(USBDevice *udev)
 
     trace_usb_host_reset(s->bus_num, s->addr);
 
-    rc = libusb_reset_device(s->dh);
+    REPLAY_DATA_INT(rc, libusb_reset_device(s->dh));
     if (rc != 0) {
         usb_host_nodev(s);
     }
@@ -1425,7 +1502,7 @@ static void usb_host_post_load_bh(void *opaque)
     USBHostDevice *dev = opaque;
     USBDevice *udev = USB_DEVICE(dev);
 
-    if (dev->dh != NULL) {
+    if (dev->is_open) {
         usb_host_close(dev);
     }
     if (udev->attached) {
@@ -1445,6 +1522,37 @@ static int usb_host_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static int usb_host_post_load_replay(void *opaque, int version_id)
+{
+    USBHostDevice *dev = opaque;
+
+    usb_host_free_xfers(dev);
+    usb_host_iso_free_all(dev);
+    submitted_xfers = 0;
+    return 0;
+}
+
+static void usb_host_pre_save_replay(void *opaque)
+{
+    USBHostDevice *dev = opaque;
+
+    if (!QTAILQ_EMPTY(&dev->requests)) {
+        fprintf(stderr, "Replay: Saving VM state with non-empty USB requests queue\n");
+        exit(1);
+    }
+
+    if (!QTAILQ_EMPTY(&dev->isorings)) {
+        fprintf(stderr, "Replay: Saving VM state with non-empty USB iso rings queue\n");
+        exit(1);
+    }
+
+    // paranoidal checking
+    if (submitted_xfers != 0) {
+        fprintf(stderr, "Replay: Saving VM state with non-empty USB iso rings queue\n");
+        exit(1);
+    }
+}
+
 static const VMStateDescription vmstate_usb_host = {
     .name = "usb-host",
     .version_id = 1,
@@ -1456,6 +1564,51 @@ static const VMStateDescription vmstate_usb_host = {
     }
 };
 
+static const VMStateDescription vmstate_usb_host_interface = {
+    .name = "usb-host-interface",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_BOOL(detached, USBHostInterface),
+        VMSTATE_BOOL(claimed, USBHostInterface),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static const VMStateDescription vmstate_usb_host_replay = {
+    .name = "usb-host",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .post_load = usb_host_post_load_replay,
+    .pre_save = usb_host_pre_save_replay,
+    .fields = (VMStateField[]) {
+        VMSTATE_USB_DEVICE(parent_obj, USBHostDevice),
+        VMSTATE_INT32(seen, USBHostDevice),
+        VMSTATE_INT32(errcount, USBHostDevice),
+        VMSTATE_INT32(bus_num, USBHostDevice),
+        VMSTATE_INT32(addr, USBHostDevice),
+        VMSTATE_INT32(bus_num, USBHostDevice),
+        VMSTATE_BOOL(is_open, USBHostDevice),
+        VMSTATE_CHAR_ARRAY(port, USBHostDevice, 16),
+        VMSTATE_UINT8(ddesc.bLength, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDescriptorType, USBHostDevice),
+        VMSTATE_UINT16(ddesc.bcdUSB, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceClass, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceSubClass, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceProtocol, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bMaxPacketSize0, USBHostDevice),
+        VMSTATE_UINT16(ddesc.idVendor, USBHostDevice),
+        VMSTATE_UINT16(ddesc.idProduct, USBHostDevice),
+        VMSTATE_UINT16(ddesc.bcdDevice, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iManufacturer, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iProduct, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iSerialNumber, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bNumConfigurations, USBHostDevice),
+        VMSTATE_STRUCT_ARRAY(ifs, USBHostDevice, USB_MAX_INTERFACES, 1, vmstate_usb_host_interface, USBHostInterface),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static Property usb_host_dev_properties[] = {
     DEFINE_PROP_UINT32("hostbus",  USBHostDevice, match.bus_num,    0),
     DEFINE_PROP_UINT32("hostaddr", USBHostDevice, match.addr,       0),
@@ -1487,7 +1640,10 @@ static void usb_host_class_initfn(ObjectClass *klass, void *data)
     uc->flush_ep_queue = usb_host_flush_ep_queue;
     uc->alloc_streams  = usb_host_alloc_streams;
     uc->free_streams   = usb_host_free_streams;
-    dc->vmsd = &vmstate_usb_host;
+    if (replay_mode == REPLAY_NONE)
+        dc->vmsd = &vmstate_usb_host;
+    else
+        dc->vmsd = &vmstate_usb_host_replay;
     dc->props = usb_host_dev_properties;
     set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
 }
@@ -1508,7 +1664,6 @@ type_init(usb_host_register_types)
 
 /* ------------------------------------------------------------------------ */
 
-static QEMUTimer *usb_auto_timer;
 static VMChangeStateEntry *usb_vmstate;
 
 static void usb_host_vm_state(void *unused, int running, RunState state)
@@ -1525,70 +1680,74 @@ static void usb_host_auto_check(void *unused)
     libusb_device **devs = NULL;
     struct libusb_device_descriptor ddesc;
     int unconnected = 0;
-    int i, n;
+    int i, n, tmp;
 
     if (usb_host_init() != 0) {
         return;
     }
 
     if (runstate_is_running()) {
-        n = libusb_get_device_list(ctx, &devs);
+        REPLAY_DATA_INT(n, libusb_get_device_list(ctx, &devs));
         for (i = 0; i < n; i++) {
-            if (libusb_get_device_descriptor(devs[i], &ddesc) != 0) {
+            if (REPLAY_DATA_INT(tmp, libusb_get_device_descriptor(devs[i], &ddesc)) != 0) {
                 continue;
             }
-            if (ddesc.bDeviceClass == LIBUSB_CLASS_HUB) {
+            if (REPLAY_DATA_INT(tmp, ddesc.bDeviceClass) == LIBUSB_CLASS_HUB) {
                 continue;
             }
             QTAILQ_FOREACH(s, &hostdevs, next) {
                 f = &s->match;
                 if (f->bus_num > 0 &&
-                    f->bus_num != libusb_get_bus_number(devs[i])) {
+                    f->bus_num != REPLAY_DATA_INT(tmp, libusb_get_bus_number(devs[i]))) {
                     continue;
                 }
                 if (f->addr > 0 &&
-                    f->addr != libusb_get_device_address(devs[i])) {
+                    f->addr != REPLAY_DATA_INT(tmp, libusb_get_device_address(devs[i]))) {
                     continue;
                 }
                 if (f->port != NULL) {
                     char port[16] = "-";
-                    usb_host_get_port(devs[i], port, sizeof(port));
+                    if (replay_mode != REPLAY_PLAY) {
+                        usb_host_get_port(devs[i], port, sizeof(port));
+                    }
+                    replay_data_buffer((unsigned char *)port, sizeof(port));
                     if (strcmp(f->port, port) != 0) {
                         continue;
                     }
                 }
                 if (f->vendor_id > 0 &&
-                    f->vendor_id != ddesc.idVendor) {
+                    f->vendor_id != REPLAY_DATA_INT(tmp, ddesc.idVendor)) {
                     continue;
                 }
                 if (f->product_id > 0 &&
-                    f->product_id != ddesc.idProduct) {
+                    f->product_id != REPLAY_DATA_INT(tmp, ddesc.idProduct)) {
                     continue;
                 }
-
                 /* We got a match */
                 s->seen++;
                 if (s->errcount >= 3) {
                     continue;
                 }
-                if (s->dh != NULL) {
+                if (s->is_open) {
                     continue;
                 }
-                if (usb_host_open(s, devs[i]) < 0) {
+                if (usb_host_open(s, replay_mode == REPLAY_PLAY ? (libusb_device*)0xbad : devs[i]) < 0) {
                     s->errcount++;
                     continue;
                 }
                 break;
             }
         }
-        libusb_free_device_list(devs, 1);
+        if (replay_mode != REPLAY_PLAY) {
+            libusb_free_device_list(devs, 1);
+        }
 
         QTAILQ_FOREACH(s, &hostdevs, next) {
-            if (s->dh == NULL) {
+            if (!s->is_open) {
                 unconnected++;
             }
             if (s->seen == 0) {
-                if (s->dh) {
+                if (s->is_open) {
                     usb_host_close(s);
                 }
                 s->errcount = 0;
@@ -1608,17 +1767,19 @@ static void usb_host_auto_check(void *unused)
 #endif
     }
 
-    if (!usb_vmstate) {
+    if (!usb_vmstate && replay_mode == REPLAY_NONE) {
         usb_vmstate = qemu_add_vm_change_state_handler(usb_host_vm_state, NULL);
     }
-    if (!usb_auto_timer) {
-        usb_auto_timer = timer_new_ms(QEMU_CLOCK_REALTIME, usb_host_auto_check, NULL);
+
+
+    /*if (!usb_auto_timer) {
+        usb_auto_timer.timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, usb_host_auto_check, NULL);
         if (!usb_auto_timer) {
             return;
         }
         trace_usb_host_auto_scan_enabled();
-    }
-    timer_mod(usb_auto_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 2000);
+    }*/
+    timer_mod(usb_auto_timer.timer, qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 2000);
 }
 
 void usb_host_info(Monitor *mon, const QDict *qdict)
diff --git a/include/hw/host-libusb.h b/include/hw/host-libusb.h
new file mode 100755
index 0000000..2f5a9d7
--- /dev/null
+++ b/include/hw/host-libusb.h
@@ -0,0 +1,105 @@
+#ifndef HOST_LIBUSB_H
+#define HOST_LIBUSB_H
+
+#include <libusb.h>
+
+#include "qemu/notify.h"
+
+#define TYPE_USB_HOST_DEVICE "usb-host"
+#define USB_HOST_DEVICE(obj) \
+     OBJECT_CHECK(USBHostDevice, (obj), TYPE_USB_HOST_DEVICE)
+
+struct USBAutoFilter {
+    uint32_t bus_num;
+    uint32_t addr;
+    char     *port;
+    uint32_t vendor_id;
+    uint32_t product_id;
+};
+
+typedef struct USBHostDevice USBHostDevice;
+typedef struct USBHostRequest USBHostRequest;
+typedef struct USBHostIsoXfer USBHostIsoXfer;
+typedef struct USBHostIsoRing USBHostIsoRing;
+typedef struct USBHostInterface USBHostInterface;
+
+enum USBHostDeviceOptions {
+    USB_HOST_OPT_PIPELINE,
+};
+
+struct USBHostInterface {
+    bool                             detached;
+    bool                             claimed;
+};
+
+struct USBHostDevice {
+    USBDevice parent_obj;
+
+    /* properties */
+    struct USBAutoFilter             match;
+    int32_t                          bootindex;
+    uint32_t                         iso_urb_count;
+    uint32_t                         iso_urb_frames;
+    uint32_t                         options;
+    uint32_t                         loglevel;
+
+    /* state */
+    QTAILQ_ENTRY(USBHostDevice)      next;
+    int                              seen, errcount;
+    int                              bus_num;
+    int                              addr;
+    char                             port[16];
+    bool                             is_open;
+
+    libusb_device                    *dev;
+    libusb_device_handle             *dh;
+    struct libusb_device_descriptor  ddesc;
+
+    USBHostInterface ifs[USB_MAX_INTERFACES];
+
+    /* callbacks & friends */
+    QEMUBH                           *bh_nodev;
+    QEMUBH                           *bh_postld;
+    Notifier                         exit;
+
+    /* request queues */
+    QTAILQ_HEAD(, USBHostRequest)    requests;
+    QTAILQ_HEAD(, USBHostIsoRing)    isorings;
+};
+
+struct USBHostRequest {
+    USBHostDevice                    *host;
+    USBPacket                        *p;
+    bool                             in;
+    struct libusb_transfer           *xfer;
+    unsigned char                    *buffer;
+    unsigned char                    *cbuf;
+    unsigned int                     clen;
+    QTAILQ_ENTRY(USBHostRequest)     next;
+};
+
+struct USBHostIsoXfer {
+    USBHostIsoRing                   *ring;
+    struct libusb_transfer           *xfer;
+    bool                             copy_complete;
+    unsigned int                     packet;
+    QTAILQ_ENTRY(USBHostIsoXfer)     next;
+};
+
+struct USBHostIsoRing {
+    USBHostDevice                    *host;
+    USBEndpoint                      *ep;
+    QTAILQ_HEAD(, USBHostIsoXfer)    unused;
+    QTAILQ_HEAD(, USBHostIsoXfer)    inflight;
+    QTAILQ_HEAD(, USBHostIsoXfer)    copy;
+    QTAILQ_ENTRY(USBHostIsoRing)     next;
+};
+
+void usb_host_req_complete_ctrl(struct libusb_transfer *xfer);
+void usb_host_req_complete_data(struct libusb_transfer *xfer);
+void usb_host_req_complete_iso(struct libusb_transfer *xfer);
+unsigned char *usb_host_get_iso_packet_buffer(USBHostIsoXfer *xfer, int packet);
+
+bool usb_host_has_xfers(void);
+
+#endif
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index c54e550..f2e3bdc 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -6,3 +6,4 @@ obj-y += replay-input.o
 obj-y += replay-net.o
 obj-y += replay-audio.o
 obj-y += replay-char.o
+obj-y += replay-usb.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 6e0120e..c7b1215 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -59,6 +59,17 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_CHAR:
         replay_event_char_run(event->opaque);
         break;
+#ifdef CONFIG_USB_LIBUSB
+    case REPLAY_ASYNC_EVENT_USB_CTRL:
+        replay_event_usb_ctrl(event->opaque);
+        break;
+    case REPLAY_ASYNC_EVENT_USB_DATA:
+        replay_event_usb_data(event->opaque);
+        break;
+    case REPLAY_ASYNC_EVENT_USB_ISO:
+        replay_event_usb_iso(event->opaque);
+        break;
+#endif
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n", 
                 event->event_kind);
@@ -161,6 +172,11 @@ void replay_add_input_sync_event(void)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT_SYNC, NULL, NULL, 0);
 }
 
+void replay_add_usb_event(unsigned int event_kind, uint64_t id, void *opaque)
+{
+    replay_add_event_internal(event_kind, opaque, NULL, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -191,6 +207,17 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_CHAR:
                 replay_event_char_save(event->opaque);
                 break;
+#ifdef CONFIG_USB_LIBUSB
+            case REPLAY_ASYNC_EVENT_USB_CTRL:
+            case REPLAY_ASYNC_EVENT_USB_DATA:
+                replay_put_qword(event->id);
+                replay_event_save_usb_xfer(event->opaque);
+                break;
+            case REPLAY_ASYNC_EVENT_USB_ISO:
+                replay_put_qword(event->id);
+                replay_event_save_usb_iso_xfer(event->opaque);
+                break;
+#endif
             }
         }
 
@@ -273,6 +300,16 @@ void replay_read_events(int opt)
             replay_run_event(&e);
             /* continue with the next event */
             continue;
+#ifdef CONFIG_USB_LIBUSB
+        case REPLAY_ASYNC_EVENT_USB_CTRL:
+        case REPLAY_ASYNC_EVENT_USB_DATA:
+        case REPLAY_ASYNC_EVENT_USB_ISO:
+            if (read_id == -1) {
+                read_id = replay_get_qword();
+            }
+            /* read after finding event in the list */
+            break;
+#endif
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
@@ -293,6 +330,18 @@ void replay_read_events(int opt)
 
         if (event) {
             /* read event-specific reading data */
+#ifdef CONFIG_USB_LIBUSB
+            switch (read_event_kind)
+            {
+            case REPLAY_ASYNC_EVENT_USB_CTRL:
+            case REPLAY_ASYNC_EVENT_USB_DATA:
+                replay_event_read_usb_xfer(event->opaque);
+                break;
+            case REPLAY_ASYNC_EVENT_USB_ISO:
+                replay_event_read_usb_iso_xfer(event->opaque);
+                break;
+            }
+#endif
 
             QTAILQ_REMOVE(&events_list, event, events);
 
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index dfcd5fd..56b6ad9 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -40,6 +40,8 @@
 #define EVENT_ASYNC_OPT             25
 /* for int data */
 #define EVENT_DATA_INT              26
+/* for data buffer */
+#define EVENT_DATA_BUFFER           27
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 /* for clock read/writes */
@@ -59,7 +61,10 @@
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
 #define REPLAY_ASYNC_EVENT_NETWORK     4
 #define REPLAY_ASYNC_EVENT_CHAR        5
-#define REPLAY_ASYNC_COUNT             6
+#define REPLAY_ASYNC_EVENT_USB_CTRL    6
+#define REPLAY_ASYNC_EVENT_USB_DATA    7
+#define REPLAY_ASYNC_EVENT_USB_ISO     8
+#define REPLAY_ASYNC_COUNT             9
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -195,4 +200,18 @@ void replay_event_char_save(void *opaque);
 /*! Reads char event from the file. */
 void *replay_event_char_read(void);
 
+/* USB events */
+
+void replay_event_usb_ctrl(void *opaque);
+void replay_event_usb_data(void *opaque);
+void replay_event_usb_iso(void *opaque);
+void replay_event_save_usb_xfer(void *opaque);
+void replay_event_save_usb_iso_xfer(void *opaque);
+void replay_event_read_usb_xfer(void *opaque);
+void replay_event_read_usb_iso_xfer(void *opaque);
+void replay_event_skip_usb_xfer(void);
+void replay_event_skip_usb_iso_xfer(void);
+bool replay_usb_has_xfers(void);
+void replay_add_usb_event(unsigned int event_id, uint64_t id, void *opaque);
+
 #endif
diff --git a/replay/replay-usb.c b/replay/replay-usb.c
new file mode 100755
index 0000000..dae7f3d
--- /dev/null
+++ b/replay/replay-usb.c
@@ -0,0 +1,188 @@
+/*
+ * replay-usb.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+#include "hw/usb.h"
+
+#ifdef CONFIG_USB_LIBUSB
+#include "hw/host-libusb.h"
+
+static uint64_t replay_get_xfer_id(struct libusb_transfer *xfer)
+{
+    USBHostRequest *r = xfer->user_data;
+    USBHostDevice *host = r->host;
+
+    return ((uint64_t)host->match.vendor_id << 32)
+           | host->match.product_id;
+}
+
+static uint64_t replay_get_iso_xfer_id(struct libusb_transfer *xfer)
+{
+    USBHostIsoXfer *r = xfer->user_data;
+    USBHostDevice  *host = r->ring->host;
+
+    return ((uint64_t)host->match.vendor_id << 32)
+           | host->match.product_id;
+}
+
+void replay_req_complete_ctrl(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_CTRL,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_ctrl(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_CTRL,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_event_usb_ctrl(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_ctrl(xfer);
+}
+
+void replay_event_save_usb_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostRequest *r = xfer->user_data;
+    if (replay_mode == REPLAY_SAVE) {
+        replay_put_dword(xfer->status);
+        replay_put_dword(xfer->actual_length);
+        replay_put_array(xfer->buffer, r->in ? xfer->length : 0);
+    }
+}
+
+void replay_event_save_usb_iso_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostIsoXfer *iso = xfer->user_data;
+    int i;
+    if (replay_mode == REPLAY_SAVE) {
+        bool in = iso->ring->ep->pid == USB_TOKEN_IN;
+        replay_put_dword(xfer->status);
+        replay_put_dword(xfer->num_iso_packets);
+        for (i = 0 ; i < xfer->num_iso_packets ; ++i) {
+            /* all other fields of the packet are not used */
+            unsigned int len = xfer->iso_packet_desc[i].actual_length;
+            if (in) {
+                replay_put_array(usb_host_get_iso_packet_buffer(iso, i), len);
+            }
+        }
+    }
+}
+
+void replay_event_read_usb_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostRequest *r = xfer->user_data;
+
+    if (replay_mode == REPLAY_PLAY) {
+        xfer->status = replay_get_dword();
+        xfer->actual_length = replay_get_dword();
+        size_t sz;
+        replay_get_array(xfer->buffer, &sz);
+        if (r->in && xfer->length != (int)sz) {
+            fprintf(stderr, "Replay: trying to read USB control/data buffer with unexpected size\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_event_read_usb_iso_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostIsoXfer *iso = xfer->user_data;
+    int i;
+
+    if (replay_mode == REPLAY_PLAY) {
+        bool in = iso->ring->ep->pid == USB_TOKEN_IN;
+        xfer->status = replay_get_dword();
+        xfer->num_iso_packets = replay_get_dword();
+        for (i = 0 ; i < xfer->num_iso_packets ; ++i) {
+            /* all other fields of the packet are not used */
+            if (in) {
+                size_t sz;
+                replay_get_array(usb_host_get_iso_packet_buffer(iso, i), &sz);
+                xfer->iso_packet_desc[i].actual_length = (unsigned int)sz;
+            }
+        }
+    }
+}
+
+void replay_req_complete_data(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_DATA,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_data(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_DATA,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+
+void replay_event_usb_data(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_data(xfer);
+}
+
+void replay_req_complete_iso(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_SAVE) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_ISO,
+                             replay_get_iso_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_iso(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        USBHostIsoXfer *r = xfer->user_data;
+        USBHostDevice  *s = r->ring->host;
+
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_ISO,
+                             replay_get_iso_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_event_usb_iso(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_iso(xfer);
+}
+
+#endif
+
+bool replay_usb_has_xfers(void)
+{
+#ifdef CONFIG_USB_LIBUSB
+    return usb_host_has_xfers();
+#else
+    return false;
+#endif
+}
diff --git a/replay/replay.c b/replay/replay.c
index 8e15b04..a89ba91 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -101,6 +101,16 @@ static void replay_savevm(void *opaque)
     char name[128];
     uint64_t offset;
 
+#ifdef CONFIG_USB_LIBUSB
+    if (replay_usb_has_xfers()) {
+        /* Retry of saving VM state, when USB host controller is not ready.
+           We cannot save or interrupt non-finished transfers, so
+           just wait for finishing them later. */
+        timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 1);
+        return;
+    }
+#endif
+
     offset = ftello64(replay_file);
 
     replay_save_instructions();
@@ -609,3 +619,22 @@ void replay_data_int(int *data)
         replay_put_dword(*data);
     }
 }
+
+void replay_data_buffer(unsigned char *data, size_t size)
+{
+    if (replay_file && replay_mode == REPLAY_PLAY) {
+        size_t read_size = 0;
+        skip_async_events_until(EVENT_DATA_BUFFER);
+        replay_get_array(data, &read_size);
+        replay_check_error();
+        if (read_size != size) {
+            fprintf(stderr, "Replay: read non-matching size of the buffer\n");
+            exit(1);
+        }
+        replay_has_unread_data = 0;
+    } else if (replay_file && replay_mode == REPLAY_SAVE) {
+        replay_save_instructions();
+        replay_put_event(EVENT_DATA_BUFFER);
+        replay_put_array(data, size);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index 533fa0a..aa15803 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -24,6 +24,7 @@ struct QemuOpts;
 struct InputEvent;
 struct NetClientState;
 struct CharDriverState;
+struct libusb_transfer;
 
 /* replay modes */
 #define REPLAY_NONE 0
@@ -157,9 +158,38 @@ void replay_register_char_driver(struct CharDriverState *chr);
 /*! Saves write to char device event to the log */
 void replay_chr_be_write(struct CharDriverState *s, uint8_t *buf, int len);
 
+/* USB events */
+
+/*! Called in save mode when xfer for ctrl usb data is completed */
+void replay_req_complete_ctrl(struct libusb_transfer *xfer);
+/*! Called in play mode to register xfer 
+    to be completed by reading it from the log */
+void replay_req_register_ctrl(struct libusb_transfer *xfer);
+/*! Called in save mode when xfer for usb data request is completed */
+void replay_req_complete_data(struct libusb_transfer *xfer);
+/*! Called in play mode to register xfer 
+    to be completed by reading it from the log */
+void replay_req_register_data(struct libusb_transfer *xfer);
+/*! Called in save mode when iso xfer for usb data request is completed */
+void replay_req_complete_iso(struct libusb_transfer *xfer);
+/*! Called in play mode to register iso xfer 
+    to be completed by reading it from the log */
+void replay_req_register_iso(struct libusb_transfer *xfer);
+
 /* Other data */
 
 /*! Writes or reads integer value to/from replay log. */
 void replay_data_int(int *data);
+/*! Macro for using replay_data_int function */
+#define REPLAY_DATA_INT(var, value) \
+        ((replay_mode == REPLAY_NONE ? var = (value) \
+            : replay_mode == REPLAY_SAVE \
+                ? (var = (value), replay_data_int(&var), var) \
+                : /* PLAY */ (replay_data_int(&var), var)))
+/*! Writes or reads buffer to/from replay log. */
+void replay_data_buffer(unsigned char *data, size_t size);
+/*! Macro for using replay_data_buffer function */
+#define REPLAY_DATA_VAR(var) \
+                        replay_data_buffer((unsigned char*)&(var), sizeof(var))
 
 #endif

[Qemu-devel] [RFC PATCH v2 46/49] replay: replay_info command

[Pavel Dovgalyuk]

This patch adds support for replay_info monitor command. This command
returns the information about replay execution (replay mode and current step).

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx      |   13 +++++++++++++
 monitor.c            |   17 +++++++++++++++++
 qapi-schema.json     |   27 +++++++++++++++++++++++++++
 qmp-commands.hx      |   14 ++++++++++++++
 replay/Makefile.objs |    1 +
 replay/replay-qmp.c  |   29 +++++++++++++++++++++++++++++
 replay/replay.c      |   25 +++++++++++++++++++++++++
 replay/replay.h      |    4 ++++
 8 files changed, 130 insertions(+), 0 deletions(-)
 create mode 100755 replay/replay-qmp.c

diff --git a/hmp-commands.hx b/hmp-commands.hx
index d0943b1..19174f1 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1788,6 +1788,19 @@ STEXI
 show available trace events and their state
 ETEXI
 
+    {
+        .name       = "replay_info",
+        .args_type  = "",
+        .params     = "",
+        .help       = "show replay info",
+        .mhandler.cmd = do_replay_info,
+    },
+
+STEXI
+@item replay_info
+Shows information about replay process.
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index 5bc70a6..f336b91 100644
--- a/monitor.c
+++ b/monitor.c
@@ -73,6 +73,7 @@
 #include "block/qapi.h"
 #include "qapi/qmp-event.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 /* for pic/irq_info */
 #if defined(TARGET_SPARC)
@@ -1173,6 +1174,22 @@ static void do_watchdog_action(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_info(Monitor *mon, const QDict *qdict)
+{
+    switch (replay_mode) {
+    case REPLAY_NONE:
+        monitor_printf(mon, "Replay is not enabled\n");
+        break;
+    default:
+        monitor_printf(mon, "Replay mode: %s ", replay_get_mode_name());
+        if (replay_mode == REPLAY_PLAY) {
+            monitor_printf(mon, "(%s)", replay_get_play_submode_name());
+        }
+        monitor_printf(mon, "\n\tcurrent step: %" PRId64 "\n", replay_get_current_step());
+        break;
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index fee541a..c45f795 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3491,3 +3491,30 @@
 # Since: 2.1
 ##
 { 'command': 'rtc-reset-reinjection' }
+
+##
+# @ReplayInfo:
+#
+# Information about replay process
+#
+# @mode: replay mode (none, play, record)
+#
+# @submode: play submode
+#
+# @step: current step of record or play
+#
+# Since:  2.2
+##
+{ 'type': 'ReplayInfo',
+  'data': {'mode': 'str', 'submode': 'str', 'step': 'uint64'} }
+
+##
+# @replay_info
+#
+# Query the status of replay engine
+#
+# Returns: @ReplayInfo reflecting the status
+#
+# Since:  2.2
+##
+{ 'command': 'replay_info', 'returns': 'ReplayInfo' }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4be4765..d475633 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3755,3 +3755,17 @@ Example:
 <- { "return": {} }
 
 EQMP
+
+    {
+        .name       = "replay_info",
+        .args_type  = "",
+        .mhandler.cmd_new = qmp_marshal_input_replay_info,
+    },
+
+SQMP
+replay_info
+-----------
+
+Shows information about replay process.
+
+EQMP
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index f2e3bdc..1bc9621 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -7,3 +7,4 @@ obj-y += replay-net.o
 obj-y += replay-audio.o
 obj-y += replay-char.o
 obj-y += replay-usb.o
+obj-y += replay-qmp.o
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
new file mode 100755
index 0000000..966bd4d
--- /dev/null
+++ b/replay/replay-qmp.c
@@ -0,0 +1,29 @@
+/*
+ * replay-qmp.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "sysemu/sysemu.h"
+#include "qmp-commands.h"
+#include "qapi/qmp/qobject.h"
+#include "qapi/qmp-input-visitor.h"
+#include "replay/replay.h"
+#include "replay/replay-internal.h"
+
+ReplayInfo *qmp_replay_info(Error **errp)
+{
+    ReplayInfo *info = g_malloc0(sizeof(*info));
+
+    info->mode = g_strdup(replay_get_mode_name());
+    info->submode = g_strdup(replay_get_play_submode_name());
+    info->step = replay_get_current_step();
+
+    return info;
+}
diff --git a/replay/replay.c b/replay/replay.c
index a89ba91..f711c26 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -638,3 +638,28 @@ void replay_data_buffer(unsigned char *data, size_t size)
         replay_put_array(data, size);
     }
 }
+
+const char *replay_get_mode_name(void)
+{
+    switch (replay_mode) {
+    case REPLAY_NONE:
+        return "none";
+    case REPLAY_PLAY:
+        return "replay";
+    case REPLAY_SAVE:
+        return "record";
+    default:
+        return "unknown";
+    }
+}
+
+const char *replay_get_play_submode_name(void)
+{
+    switch (play_submode) {
+    case REPLAY_PLAY_NORMAL:
+        return "normal";
+    case REPLAY_PLAY_UNKNOWN:
+    default:
+        return "unknown";
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index aa15803..ee0460f 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -52,6 +52,10 @@ extern int replay_icount;
 
 /*! Returns replay play submode */
 int replay_get_play_submode(void);
+/*! Returns current mode name */
+const char *replay_get_mode_name(void);
+/*! Returns replay-play submode name */
+const char *replay_get_play_submode_name(void);
 
 /* Replay process control functions */
 

[Qemu-devel] [RFC PATCH v2 47/49] replay: replay_break command

[Pavel Dovgalyuk]

This patch adds support for replay_break monitor command. This command
sets the step (measured in executed instructions) where replay should be
stopped.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx     |   14 ++++++++++++++
 monitor.c           |   15 +++++++++++++++
 qapi-schema.json    |   11 +++++++++++
 qmp-commands.hx     |   23 +++++++++++++++++++++++
 replay/replay-qmp.c |   13 +++++++++++++
 replay/replay.c     |   24 +++++++++++++++++++++++-
 replay/replay.h     |    2 ++
 7 files changed, 101 insertions(+), 1 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 19174f1..3eaa80e 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1801,6 +1801,20 @@ STEXI
 Shows information about replay process.
 ETEXI
 
+    {
+        .name       = "replay_break",
+        .args_type  = "step:l",
+        .params     = "step",
+        .help       = "stop replaying at the specified replay step",
+        .mhandler.cmd = do_replay_break,
+    },
+
+STEXI
+@item replay_break @var{step}
+Stops replaying at the specified @var{step}.
+
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index f336b91..4710a59 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1190,6 +1190,21 @@ static void do_replay_info(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_break(Monitor *mon, const QDict *qdict)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        uint64_t step = qdict_get_int(qdict, "step");
+        if (step >= replay_get_current_step()) {
+            monitor_printf(mon, "Setting break at step: %" PRId64 "\n", step);
+            replay_set_break(step);
+        } else {
+            monitor_printf(mon, "Cannot stop on the preceding step.\n");
+        }
+    } else {
+        monitor_printf(mon, "You can stop at the specific step only in PLAY mode.\n");
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index c45f795..d0a4651 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3518,3 +3518,14 @@
 # Since:  2.2
 ##
 { 'command': 'replay_info', 'returns': 'ReplayInfo' }
+
+##
+# @replay_break
+#
+# Sets breakpoint at the specified step of replaying
+#
+# @step: step where breakpoint should be set
+#
+# Since:  2.2
+##
+{ 'command': 'replay_break', 'data': { 'step': 'uint64' } }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index d475633..575db76 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3769,3 +3769,26 @@ replay_info
 Shows information about replay process.
 
 EQMP
+
+    {
+        .name       = "replay_break",
+        .args_type  = "step:l",
+        .mhandler.cmd_new = qmp_marshal_input_replay_break,
+    },
+
+SQMP
+replay_break
+------------
+
+Sets breakpoint at the specified replay step.
+
+Arguments:
+
+- "step": step where breakpoint should be set
+
+Example:
+
+-> { "execute": "replay_break", "arguments": { "step": 1024 } }
+<- { "return": {} }
+
+EQMP
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
index 966bd4d..a9f30da 100755
--- a/replay/replay-qmp.c
+++ b/replay/replay-qmp.c
@@ -27,3 +27,16 @@ ReplayInfo *qmp_replay_info(Error **errp)
 
     return info;
 }
+
+void qmp_replay_break(uint64_t step, Error **errp)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        if (step >= replay_get_current_step()) {
+            replay_set_break(step);
+        } else {
+            error_setg(errp, "Cannot stop on the preceding step");
+        }
+    } else {
+        error_setg(errp, "replay_break can be used only in PLAY mode");
+    }
+}
diff --git a/replay/replay.c b/replay/replay.c
index f711c26..d6949d6 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -31,6 +31,8 @@ static char *replay_filename;
 char *replay_image_suffix;
 
 ReplayState replay_state;
+/*! Step for stopping execution at. */
+static uint64_t replay_break_step = -1;
 
 /* 
     Auto-saving for VM states data 
@@ -284,6 +286,19 @@ void replay_instruction(int process_events)
         } else if (replay_mode == REPLAY_PLAY) {
             skip_async_events_until(EVENT_INSTRUCTION);
             if (first_cpu->instructions_count >= 1) {
+                if (replay_get_current_step() == replay_break_step) {
+                    replay_break_step = -1;
+
+                    /* for stopping VM */
+                    if (play_submode == REPLAY_PLAY_NORMAL) {
+                        first_cpu->exception_index = EXCP_DEBUG;
+                        monitor_printf(default_mon, "Execution has stopped.\n");
+                        vm_stop(EXCP_DEBUG);
+                    }
+                    /* for breaking execution loop */
+                    cpu_exit(first_cpu);
+                    return;
+                }
                 ++replay_state.current_step;
                 --first_cpu->instructions_count;
                 if (first_cpu->instructions_count == 0) {
@@ -312,7 +327,8 @@ bool replay_has_async_request(void)
     }
 
     if (replay_mode == REPLAY_PLAY) {
-        if (skip_async_events(EVENT_ASYNC)) {
+        if (skip_async_events(EVENT_ASYNC)
+            || replay_get_current_step() == replay_break_step) {
             return true;
         }
 
@@ -470,6 +486,7 @@ static void replay_enable(const char *fname, int mode)
     replay_state.skipping_instruction = 0;
     replay_state.current_step = 0;
     current_saved_state = 0;
+    replay_break_step = -1;
 
     replay_net_init();
 
@@ -663,3 +680,8 @@ const char *replay_get_play_submode_name(void)
         return "unknown";
     }
 }
+
+void replay_set_break(uint64_t step)
+{
+    replay_break_step = step;
+}
diff --git a/replay/replay.h b/replay/replay.h
index ee0460f..9a20a3b 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -65,6 +65,8 @@ void replay_configure(struct QemuOpts *opts, int mode);
 void replay_init_timer(void);
 /*! Closes replay log file and frees other resources. */
 void replay_finish(void);
+/*! Sets step where execution should be stopped. */
+void replay_set_break(uint64_t step);
 
 /* Processing the instructions */
 

[Qemu-devel] [RFC PATCH v2 48/49] replay: replay_seek_step command

[Pavel Dovgalyuk]

This patch adds support for replay_seek_step monitor command. This command
loads one of the snapshots and replays the execution until the specified
step is met.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx          |   14 ++++++++++++++
 monitor.c                |   17 +++++++++++++++++
 qapi-schema.json         |   11 +++++++++++
 qmp-commands.hx          |   23 +++++++++++++++++++++++
 replay/replay-internal.h |    3 +++
 replay/replay-qmp.c      |   15 +++++++++++++++
 replay/replay.c          |   43 +++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |    4 ++++
 8 files changed, 130 insertions(+), 0 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 3eaa80e..57ce5da 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1815,6 +1815,20 @@ Stops replaying at the specified @var{step}.
 
 ETEXI
 
+    {
+        .name       = "replay_seek",
+        .args_type  = "step:l",
+        .params     = "step",
+        .help       = "seeks the replay log to the specified step",
+        .mhandler.cmd = do_replay_seek,
+    },
+
+STEXI
+@item replay_seek @var{step}
+Seeks the replay log to the specified @var{step}.
+
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index 4710a59..031decf 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1205,6 +1205,23 @@ static void do_replay_break(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_seek(Monitor *mon, const QDict *qdict)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        uint64_t step = qdict_get_int(qdict, "step");
+        if (replay_seek_step(step)) {
+            monitor_printf(mon, "Seeking for step: %" PRId64 "\n", step);
+            if (!runstate_is_running()) {
+                vm_start();
+            }
+        } else {
+            monitor_printf(mon, "Cannot seek to the specified step.\n");
+        }
+    } else {
+        monitor_printf(mon, "You can go to the specific step only in PLAY mode.\n");
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index d0a4651..9d3fda3 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3529,3 +3529,14 @@
 # Since:  2.2
 ##
 { 'command': 'replay_break', 'data': { 'step': 'uint64' } }
+
+##
+# @replay_seek
+#
+# Seeks the replay log to the specified step
+#
+# @step: destination replay step 
+#
+# Since:  2.2
+##
+{ 'command': 'replay_seek', 'data': { 'step': 'uint64' } }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 575db76..a144327 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3792,3 +3792,26 @@ Example:
 <- { "return": {} }
 
 EQMP
+
+    {
+        .name       = "replay_seek",
+        .args_type  = "step:l",
+        .mhandler.cmd_new = qmp_marshal_input_replay_seek,
+    },
+
+SQMP
+replay_seek
+-----------
+
+Seeks the replay log to the specified step.
+
+Arguments:
+
+- "step": destination replay step
+
+Example:
+
+-> { "execute": "replay_seek", "arguments": { "step": 1024 } }
+<- { "return": {} }
+
+EQMP
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 56b6ad9..c7655e7 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -135,6 +135,9 @@ void skip_async_events_until(unsigned int kind);
     If the parameter is -1, the clock value is read to the cache anyway. */
 void replay_read_next_clock(unsigned int kind);
 
+/*! Finds saved state info which is nearest before the specified step. */
+SavedStateInfo *find_nearest_state(uint64_t step);
+
 /* Asynchronous events queue */
 
 /*! Initializes events' processing internals */
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
index a9f30da..5cb04fb 100755
--- a/replay/replay-qmp.c
+++ b/replay/replay-qmp.c
@@ -40,3 +40,18 @@ void qmp_replay_break(uint64_t step, Error **errp)
         error_setg(errp, "replay_break can be used only in PLAY mode");
     }
 }
+
+void qmp_replay_seek(uint64_t step, Error **errp)
+{
+    if (replay_mode == REPLAY_PLAY) {
+        if (replay_seek_step(step)) {
+            if (!runstate_is_running()) {
+                vm_start();
+            }
+        } else {
+            error_setg(errp, "Cannot seek to the specified step");
+        }
+    } else {
+        error_setg(errp, "replay_seek can be used only in PLAY mode");
+    }
+}
diff --git a/replay/replay.c b/replay/replay.c
index d6949d6..e6b3e39 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -685,3 +685,46 @@ void replay_set_break(uint64_t step)
 {
     replay_break_step = step;
 }
+
+SavedStateInfo *find_nearest_state(uint64_t step)
+{
+    SavedStateInfo *first = saved_states;
+    SavedStateInfo *last = saved_states + saved_states_count;
+    while (first < last - 1) {
+        SavedStateInfo *next = first + (last - first) / 2;
+        if (next->step > step) {
+            last = next;
+        } else {
+            first = next;
+        }
+    }
+
+    return first;
+}
+
+int replay_seek_step(uint64_t step)
+{
+    if (replay_mode != REPLAY_PLAY) {
+        return 0;
+    }
+
+    /* load VM state, if possible */
+    if (saved_states_count > 0) {
+        /* find VM state to load */
+        SavedStateInfo *first = find_nearest_state(step);
+
+        if (first->step <= step 
+            && (replay_get_current_step() > step 
+                || replay_get_current_step() < first->step)) {
+            replay_loadvm(first - saved_states);
+        }
+    }
+
+    /* setup the breakpoint */
+    if (step >= replay_get_current_step()) {
+        replay_set_break(step);
+        return 1;
+    }
+
+    return 0;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 9a20a3b..356e99b 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -67,6 +67,10 @@ void replay_init_timer(void);
 void replay_finish(void);
 /*! Sets step where execution should be stopped. */
 void replay_set_break(uint64_t step);
+/*! Seeks to the specified step. 
+    Loads VM state, if possible, and sets break to specified step.
+    Returns nonzero, when seeking was successful. */
+int replay_seek_step(uint64_t step);
 
 /* Processing the instructions */
 

[Qemu-devel] [RFC PATCH v2 49/49] gdbstub: reverse debugging

[Pavel Dovgalyuk]

This patch introduces support of reverse debugging through the gdb remote
protocol. Patch adds reverse-stepi and reverse-continue commands support
to qemu. Other reverse commands should also work, because they reuse these ones.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 exec.c                      |    7 ++
 gdbstub.c                   |   79 ++++++++++++++++++-----
 replay/Makefile.objs        |    1 
 replay/replay-debug.c       |  148 +++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h    |    4 +
 replay/replay.c             |    5 +
 replay/replay.h             |   13 ++++
 target-arm/helper.h         |    1 
 target-arm/replay_helper.c  |    5 +
 target-arm/translate.c      |   14 +++-
 target-i386/helper.h        |    1 
 target-i386/replay_helper.c |    5 +
 target-i386/translate.c     |   12 ++-
 13 files changed, 269 insertions(+), 26 deletions(-)
 create mode 100755 replay/replay-debug.c

diff --git a/exec.c b/exec.c
index 050e0a8..8255d96 100644
--- a/exec.c
+++ b/exec.c
@@ -1613,6 +1613,13 @@ static void check_watchpoint(int offset, int len_mask, int flags)
     QTAILQ_FOREACH(wp, &cpu->watchpoints, entry) {
         if ((vaddr == (wp->vaddr & len_mask) ||
              (vaddr & wp->len_mask) == wp->vaddr) && (wp->flags & flags)) {
+            /* Don't actually process a watchpoint, it will be processed,
+               when reverse execution stops. */
+            if (replay_get_play_submode() == REPLAY_PLAY_REVERSE) {
+                wp->flags &= ~BP_WATCHPOINT_HIT;
+                replay_reverse_breakpoint();
+                continue;
+            }
             wp->flags |= BP_WATCHPOINT_HIT;
             if (!cpu->watchpoint_hit) {
                 cpu->watchpoint_hit = wp;
diff --git a/gdbstub.c b/gdbstub.c
index 8afe0b7..756b846 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -40,6 +40,7 @@
 #include "cpu.h"
 #include "qemu/sockets.h"
 #include "sysemu/kvm.h"
+#include "replay/replay.h"
 
 static inline int target_memory_rw_debug(CPUState *cpu, target_ulong addr,
                                          uint8_t *buf, int len, bool is_write)
@@ -313,6 +314,19 @@ typedef struct GDBState {
  */
 static int sstep_flags = SSTEP_ENABLE|SSTEP_NOIRQ|SSTEP_NOTIMER;
 
+/*! Retrieves flags for single step mode. */
+static int get_sstep_flags(void)
+{
+    /* In replay mode all events written into the log should be replayed.
+     * That is why NOIRQ flag is removed in this mode.
+     */
+    if (replay_mode != REPLAY_NONE) {
+        return SSTEP_ENABLE;
+    } else {
+        return sstep_flags;
+    }
+}
+
 static GDBState *gdbserver_state;
 
 bool gdb_has_xml;
@@ -835,7 +849,7 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
                     s->c_cpu = cpu;
                 }
                 if (res == 's') {
-                    cpu_single_step(s->c_cpu, sstep_flags);
+                    cpu_single_step(s->c_cpu, get_sstep_flags());
                 }
                 s->signal = res_signal;
                 gdb_continue(s);
@@ -863,9 +877,29 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             addr = strtoull(p, (char **)&p, 16);
             gdb_set_cpu_pc(s, addr);
         }
-        cpu_single_step(s->c_cpu, sstep_flags);
+        cpu_single_step(s->c_cpu, get_sstep_flags());
         gdb_continue(s);
 	return RS_IDLE;
+    case 'b':
+        /* backward debugging commands */
+        if (replay_mode == REPLAY_PLAY
+            && replay_get_play_submode() == REPLAY_PLAY_NORMAL) {
+            switch (*p) {
+            case 's':
+                replay_reverse_step();
+                gdb_continue(s);
+	            return RS_IDLE;
+            case 'c':
+                replay_reverse_continue();
+                gdb_continue(s);
+	            return RS_IDLE;
+            default:
+                goto unknown_command;
+            }
+        } else {
+            put_packet (s, "E22");
+        }
+        goto unknown_command;
     case 'F':
         {
             target_ulong ret;
@@ -937,8 +971,6 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
         if (target_memory_rw_debug(s->g_cpu, addr, mem_buf, len,
                                    true) != 0) {
             put_packet(s, "E14");
-        } else {
-            put_packet(s, "OK");
         }
         break;
     case 'p':
@@ -1035,18 +1067,23 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             put_packet(s, buf);
             break;
         } else if (strncmp(p,"qemu.sstep",10) == 0) {
-            /* Display or change the sstep_flags */
-            p += 10;
-            if (*p != '=') {
-                /* Display current setting */
-                snprintf(buf, sizeof(buf), "0x%x", sstep_flags);
-                put_packet(s, buf);
-                break;
+            if (replay_mode == REPLAY_NONE) {
+                /* Display or change the sstep_flags */
+                p += 10;
+                if (*p != '=') {
+                    /* Display current setting */
+                    snprintf(buf, sizeof(buf), "0x%x", sstep_flags);
+                    put_packet(s, buf);
+                    break;
+                }
+                p++;
+                type = strtoul(p, (char **)&p, 16);
+                sstep_flags = type;
+                put_packet(s, "OK");
+            } else {
+                /* Cannot change sstep flags in replay mode */
+                put_packet(s, "E22");
             }
-            p++;
-            type = strtoul(p, (char **)&p, 16);
-            sstep_flags = type;
-            put_packet(s, "OK");
             break;
         } else if (strcmp(p,"C") == 0) {
             /* "Current thread" remains vague in the spec, so always return
@@ -1113,6 +1150,9 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             if (cc->gdb_core_xml_file != NULL) {
                 pstrcat(buf, sizeof(buf), ";qXfer:features:read+");
             }
+            if (replay_mode == REPLAY_PLAY) {
+                pstrcat(buf, sizeof(buf), ";ReverseStep+;ReverseContinue+");
+            }
             put_packet(s, buf);
             break;
         }
@@ -1174,8 +1214,13 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
 
 void gdb_set_stop_cpu(CPUState *cpu)
 {
-    gdbserver_state->c_cpu = cpu;
-    gdbserver_state->g_cpu = cpu;
+    /* DEBUG interrupts are also used by replay module.
+       In some cases gdb is not connected, when replay is used.
+       This check is added to prevent faults in such cases. */
+    if (gdbserver_state) {
+        gdbserver_state->c_cpu = cpu;
+        gdbserver_state->g_cpu = cpu;
+    }
 }
 
 #ifndef CONFIG_USER_ONLY
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 1bc9621..af3baa4 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -8,3 +8,4 @@ obj-y += replay-audio.o
 obj-y += replay-char.o
 obj-y += replay-usb.o
 obj-y += replay-qmp.o
+obj-y += replay-debug.o
diff --git a/replay/replay-debug.c b/replay/replay-debug.c
new file mode 100755
index 0000000..c54a0d3
--- /dev/null
+++ b/replay/replay-debug.c
@@ -0,0 +1,148 @@
+/*
+ * replay-debug.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming 
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "exec/cpu-defs.h"
+
+#include "replay.h"
+#include "replay-internal.h"
+
+/* Reverse debugging data */
+
+/* Saved handler of the debug exception */
+static CPUDebugExcpHandler *prev_debug_excp_handler;
+/* Step of the last breakpoint hit. Used for seeking in reverse continue mode. */
+static uint64_t last_breakpoint_step;
+/* Start step, where reverse continue begins,
+   or target step for reverse stepping.*/
+static uint64_t last_reverse_step;
+/* Start step, where reverse continue begins.*/
+static uint64_t start_reverse_step;
+/* Previously loaded step for reverse continue */
+static SavedStateInfo *reverse_state;
+
+/*! Breakpoint handler for pass2 of reverse continue.
+    Stops the execution at previously saved breakpoint step. */
+static void reverse_continue_pass2_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_breakpoint_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        CPUDebugExcpHandler *handler = prev_debug_excp_handler;
+        prev_debug_excp_handler = NULL;
+
+        play_submode = REPLAY_PLAY_NORMAL;
+
+        cpu->exception_index = EXCP_DEBUG;
+        /* invoke the breakpoint */
+        cpu_set_debug_excp_handler(handler);
+        handler(env);
+        cpu_exit(cpu);
+    }
+}
+
+/*! Breakpoint handler for pass1 of reverse continue.
+    Saves last breakpoint hit and switches to pass2 when starting point is reached. */
+static void reverse_continue_pass1_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_reverse_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        /* repeat first pass if breakpoint was not found
+           on current iteration */
+        if (last_breakpoint_step == reverse_state->step - 1
+            && reverse_state != saved_states) {
+            last_reverse_step = reverse_state->step;
+            /* load previous state */
+            --reverse_state;
+            last_breakpoint_step = reverse_state->step - 1;
+            replay_seek_step(reverse_state->step);
+            /* set break should be after seek, because seek resets break */
+            replay_set_break(last_reverse_step);
+			cpu_loop_exit(cpu);
+        } else {
+            /* this condition is needed, when no breakpoints were found */
+            if (last_breakpoint_step == reverse_state->step - 1) {
+                ++last_breakpoint_step;
+            }
+            cpu_set_debug_excp_handler(reverse_continue_pass2_breakpoint_handler);
+			
+            reverse_continue_pass2_breakpoint_handler(env);
+            replay_seek_step(last_breakpoint_step);
+            cpu_loop_exit(cpu);
+        }
+    } else {
+        /* skip watchpoint/breakpoint at the current step
+           to allow reverse continue */
+        last_breakpoint_step = replay_get_current_step();
+    }
+}
+
+void replay_reverse_breakpoint(void)
+{
+    /* we started reverse execution from a breakpoint */
+    if (replay_get_current_step() != start_reverse_step) {
+        last_breakpoint_step = replay_get_current_step();
+    }
+}
+
+void replay_reverse_continue(void)
+{
+    if (replay_mode == REPLAY_PLAY && play_submode == REPLAY_PLAY_NORMAL) {
+        tb_flush_all();
+        play_submode = REPLAY_PLAY_REVERSE;
+
+        last_reverse_step = replay_get_current_step();
+        start_reverse_step = replay_get_current_step();
+        /* load initial state */
+        reverse_state = find_nearest_state(replay_get_current_step());
+        replay_seek_step(reverse_state->step);
+        /* run to current step */
+        replay_set_break(last_reverse_step);
+        /* decrement to allow breaking at the first step */
+        last_breakpoint_step = reverse_state->step - 1;
+        prev_debug_excp_handler =
+            cpu_set_debug_excp_handler(reverse_continue_pass1_breakpoint_handler);
+    }
+}
+
+/*! Breakpoint handler for reverse stepping.
+    Stops at the desired step and skips other breakpoints. */
+static void reverse_step_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_reverse_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        CPUDebugExcpHandler *handler = prev_debug_excp_handler;
+        prev_debug_excp_handler = NULL;
+
+        play_submode = REPLAY_PLAY_NORMAL;
+
+        cpu->exception_index = EXCP_DEBUG;
+        /* invoke the breakpoint */
+        cpu_set_debug_excp_handler(handler);
+        handler(env);
+        cpu_exit(cpu);
+    }
+}
+
+void replay_reverse_step(void)
+{
+    if (replay_mode == REPLAY_PLAY && play_submode == REPLAY_PLAY_NORMAL
+        && replay_get_current_step() > 0) {
+        tb_flush_all();
+        play_submode = REPLAY_PLAY_REVERSE;
+
+        last_reverse_step = replay_get_current_step() - 1;
+        replay_seek_step(last_reverse_step);
+
+        prev_debug_excp_handler =
+            cpu_set_debug_excp_handler(reverse_step_breakpoint_handler);
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index c7655e7..e53913b 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -88,7 +88,11 @@ struct SavedStateInfo {
 };
 /*! Reference to the saved state */
 typedef struct SavedStateInfo SavedStateInfo;
+/* List of the saved states information */
+extern SavedStateInfo *saved_states;
 
+/*! Stores current submode for PLAY mode */
+extern int play_submode;
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
 
diff --git a/replay/replay.c b/replay/replay.c
index e6b3e39..b4162f3 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -294,6 +294,9 @@ void replay_instruction(int process_events)
                         first_cpu->exception_index = EXCP_DEBUG;
                         monitor_printf(default_mon, "Execution has stopped.\n");
                         vm_stop(EXCP_DEBUG);
+                    } else if (play_submode == REPLAY_PLAY_REVERSE) {
+                        cpu_handle_debug_exception(first_cpu->env_ptr);
+                        return;
                     }
                     /* for breaking execution loop */
                     cpu_exit(first_cpu);
@@ -675,6 +678,8 @@ const char *replay_get_play_submode_name(void)
     switch (play_submode) {
     case REPLAY_PLAY_NORMAL:
         return "normal";
+    case REPLAY_PLAY_REVERSE:
+        return "reverse";
     case REPLAY_PLAY_UNKNOWN:
     default:
         return "unknown";
diff --git a/replay/replay.h b/replay/replay.h
index 356e99b..565af2f 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -34,6 +34,8 @@ struct libusb_transfer;
 #define REPLAY_PLAY_UNKNOWN 0
 /* Normal log replaying */
 #define REPLAY_PLAY_NORMAL  1
+/* Replaying for reverse execution */
+#define REPLAY_PLAY_REVERSE 2
 
 /* replay clock kinds */
 /* rdtsc */
@@ -186,6 +188,17 @@ void replay_req_complete_iso(struct libusb_transfer *xfer);
     to be completed by reading it from the log */
 void replay_req_register_iso(struct libusb_transfer *xfer);
 
+/* Reverse debugging */
+
+/*! Initializes reverse continue execution.
+    Called by debugger module. */
+void replay_reverse_continue(void);
+/*! Initializes reverse step execution.
+    Called by debugger module. */
+void replay_reverse_step(void);
+/* Called instead of invoking breakpoint in reverse continue mode. */
+void replay_reverse_breakpoint(void);
+
 /* Other data */
 
 /*! Writes or reads integer value to/from replay log. */
diff --git a/target-arm/helper.h b/target-arm/helper.h
index 0233b64..802ff9d 100644
--- a/target-arm/helper.h
+++ b/target-arm/helper.h
@@ -531,3 +531,4 @@ DEF_HELPER_FLAGS_2(neon_pmull_64_hi, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 #endif
 
 DEF_HELPER_1(replay_instruction, i32, env)
+DEF_HELPER_0(reverse_breakpoint, void)
diff --git a/target-arm/replay_helper.c b/target-arm/replay_helper.c
index 736cd51..d342326 100755
--- a/target-arm/replay_helper.c
+++ b/target-arm/replay_helper.c
@@ -31,3 +31,8 @@ uint32_t helper_replay_instruction(CPUARMState *env)
     replay_instruction(timer);
     return timer;
 }
+
+void helper_reverse_breakpoint(void)
+{
+    replay_reverse_breakpoint();
+}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index dd40998..bd04cd6 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -11021,11 +11021,15 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
         if (unlikely(!QTAILQ_EMPTY(&cs->breakpoints))) {
             QTAILQ_FOREACH(bp, &cs->breakpoints, entry) {
                 if (bp->pc == dc->pc) {
-                    gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
-                    /* Advance PC so that clearing the breakpoint will
-                       invalidate this TB.  */
-                    dc->pc += 2;
-                    goto done_generating;
+                    if (replay_get_play_submode() == REPLAY_PLAY_REVERSE) {
+                        gen_helper_reverse_breakpoint();
+                    } else {
+                        gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
+                        /* Advance PC so that clearing the breakpoint will
+                           invalidate this TB.  */
+                        dc->pc += 2;
+                        goto done_generating;
+                    }
                 }
             }
         }
diff --git a/target-i386/helper.h b/target-i386/helper.h
index 058302b..9aacef1 100644
--- a/target-i386/helper.h
+++ b/target-i386/helper.h
@@ -219,3 +219,4 @@ DEF_HELPER_3(rcrq, tl, env, tl, tl)
 #endif
 
 DEF_HELPER_1(replay_instruction, i32, env)
+DEF_HELPER_0(reverse_breakpoint, void)
diff --git a/target-i386/replay_helper.c b/target-i386/replay_helper.c
index 8b82eee..675bc17 100755
--- a/target-i386/replay_helper.c
+++ b/target-i386/replay_helper.c
@@ -31,3 +31,8 @@ uint32_t helper_replay_instruction(CPUX86State *env)
     replay_instruction(timer);
     return timer;
 }
+
+void helper_reverse_breakpoint(void)
+{
+    replay_reverse_breakpoint();
+}
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 6ae3fc3..104a569 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -2534,10 +2534,14 @@ static void gen_interrupt(DisasContext *s, int intno,
 
 static void gen_debug(DisasContext *s, target_ulong cur_eip)
 {
-    gen_update_cc_op(s);
-    gen_jmp_im(cur_eip);
-    gen_helper_debug(cpu_env);
-    s->is_jmp = DISAS_TB_JUMP;
+    if (replay_get_play_submode() == REPLAY_PLAY_REVERSE) {
+        gen_helper_reverse_breakpoint();
+    } else {
+        gen_update_cc_op(s);
+        gen_jmp_im(cur_eip);
+        gen_helper_debug(cpu_env);
+        s->is_jmp = DISAS_TB_JUMP;
+    }
 }
 
 /* generate a generic end of block. Trace exception is also generated

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Frederic Konrad]

On 17/07/2014 13:01, Pavel Dovgalyuk wrote:
> This set of patches is related to the reverse execution and deterministic
> replay of qemu execution  Our implementation of deterministic replay can
> be used for deterministic and reverse debugging of guest code through gdb
> remote interface.
>
> Execution recording writes non-deterministic events log, which can be later
> used for replaying the execution anywhere and for unlimited number of times.
> It also supports checkpointing for faster rewinding during reverse debugging.
> Execution replaying reads the log and replays all non-deterministic events
> including external input, hardware clocks, and interrupts.
>
> Reverse execution has the following features:
>   * Deterministically replays whole system execution and all contents of the memory,
>     state of the hadrware devices, clocks, and screen of the VM.
>   * Writes execution log into the file for latter replaying for multiple times
>     on different machines.
>   * Supports i386, x86_64, and ARM hardware platforms.
>   * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
>     audio devices, serial interfaces, and physical USB devices connected to the emulator.
>   * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
>   * Supports auto-checkpointing for convenient reverse debugging.
>
> Usage of the record/replay:
>   * First, record the execution, by adding '-record fname=replay.bin' to the
>     command line.
>   * Then you can replay it for the multiple times by using another command
>     line option: '-replay fname=replay.bin'
>   * Virtual machine should have at least one virtual disk, which is used to
>     store checkpoints. If you want to enable automatic checkpointing, simply
>     add ',period=XX' to record options, where XX is the checkpointing period
>     in seconds.
>   * Using of the network adapters in record/replay mode is possible with
>     the following command-line options:
>     - '-net user' (or another host adapter) in record mode
>     - '-net replay' in replay mode. Every host network adapter should be
>       replaced by 'replay' when replaying the execution.
>   * Reverse debugging can be used through gdb remote interface.
>     reverse-stepi and reverse-continue commands are supported. Other reverse
>     commands should also work, because they reuse these ones.
>   * Monitor is extended by the following commands:
>     - replay_info - prints information about replay mode and current step
>       (number of instructions executed)
>     - replay_break - sets "breakpoint" at the specified instructions count.
>     - replay_seek - rewinds (using the checkpoints, if possible) to the
>       specified step of replay log.
>
> Paper with short description of deterministic replay implementation:
> http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
>
> Modifications of qemu include:
>   * adding missed fields of the virtual devices' states to the vmstate
>     structures to allow deterministic saving and restoring the VM state
>   * adding virtual clock-based timers to vmstate structures, because virtual
>     clock is the part of the virtual machine state
>   * modification of block layer to support automatic creation of the overlay
>     files to store the changes and snapshots while recording
>   * disabling of system reset while loading VM state to avoid generating of
>     interrupts by reset handlers
>   * adding warpers for clock and time functions to save their return
>     values in the log
>   * saving different asynchronous events (e.g. system shutdown) into the log
>   * synchronization of the bottom halves execution
>   * synchronization of the threads from thread pool
>   * recording/replaying user input (mouse and keyboard), input from virtual
>     serial ports, incoming network packets, input from connected USB devices
>   * adding HMP/QMP commands to monitor for controlling replay execution
>
> v2 changes:
>   * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
>   * Added QMP versions of replay commands (as suggested by Eric Blake)
>   * Removed some optional features of replay to make patches cleaner
>   * Minor changes and code cleanup were made

Thanks for reshaping the patches.
We are a little busy just now, but I will review that as soon as possible.

Thanks,
Fred

>
> ---
>
> Pavel Dovgalyuk (49):
>        acpi: accurate overflow check
>        integratorcp: adding vmstate for save/restore
>        pcspk: adding vmstate for save/restore
>        fdc: adding vmstate for save/restore
>        parallel: adding vmstate for save/restore
>        serial: fixing vmstate for save/restore
>        kvmapic: fixing loading vmstate
>        hpet: fixing saving and loading process
>        pckbd: adding new fields to vmstate
>        rtl8139: adding new fields to vmstate
>        piix: do not raise irq while loading vmstate
>        mc146818rtc: add missed field to vmstate
>        pl031: add missed field to vmstate
>        ide pci: reset status field before loading the vmstate
>        softmmu: fixing usage of cpu_st/ld* from helpers
>        target: save cpu state fields
>        target-i386: update fp status fix
>        migration: add vmstate for int8 and char arrays
>        replay: global variables and function stubs
>        block: add suffix parameter to bdrv_open functions
>        sysemu: system functions for replay
>        replay: internal functions for replay log
>        cpu: invent instruction count for accurate replay
>        target-arm: instructions counting code for replay
>        target-i386: instructions counting code for replay
>        replay: interrupts and exceptions
>        vga: do not use virtual clock for blinking cursor
>        replay: asynchronous events infrastructure
>        replay: recording and replaying clock ticks
>        replay: recording and replaying different timers
>        replay: shutdown event
>        replay: checkpoints
>        replay: bottom halves
>        replay: replay aio requests
>        replay: thread pool
>        pl031: vmstate in replay mode
>        replay: initialization and deinitialization
>        replay: command line options
>        replay: snapshotting the virtual machine
>        replay: recording of the user input
>        tap-win32: destroy the thread at exit
>        replay: network packets record/replay
>        replay: audio data record/replay
>        replay: serial port
>        replay: USB passthrough
>        replay: replay_info command
>        replay: replay_break command
>        replay: replay_seek_step command
>        gdbstub: reverse debugging
>
>
>   Makefile.target                  |    1
>   arch_init.c                      |    8
>   async.c                          |   45 ++
>   audio/audio.c                    |   14 +
>   audio/audio_win_int.h            |    3
>   audio/winwaveaudio.c             |  167 +++++++--
>   block.c                          |  134 ++++++-
>   block/blkdebug.c                 |    2
>   block/blkverify.c                |    4
>   block/cow.c                      |    2
>   block/qcow.c                     |    2
>   block/qcow2.c                    |   10 -
>   block/qed.c                      |    2
>   block/raw-posix.c                |    6
>   block/raw-win32.c                |    4
>   block/sheepdog.c                 |    4
>   block/vmdk.c                     |    8
>   block/vvfat.c                    |    2
>   blockdev.c                       |   11 -
>   cpu-exec.c                       |   34 +-
>   cpus.c                           |   81 ++++
>   dma-helpers.c                    |   10 -
>   exec.c                           |   12 +
>   gdbstub.c                        |   79 +++-
>   hmp-commands.hx                  |   41 ++
>   hw/acpi/core.c                   |    7
>   hw/arm/integratorcp.c            |   38 ++
>   hw/audio/pcspk.c                 |   19 +
>   hw/block/fdc.c                   |   11 -
>   hw/block/virtio-blk.c            |   10 -
>   hw/char/parallel.c               |   22 +
>   hw/char/serial.c                 |  115 ++++--
>   hw/display/vga.c                 |    4
>   hw/i386/kvmvapic.c               |   22 +
>   hw/ide/ahci.c                    |    4
>   hw/ide/atapi.c                   |    9
>   hw/ide/core.c                    |   18 +
>   hw/ide/pci.c                     |   10 +
>   hw/input/pckbd.c                 |    3
>   hw/intc/apic_common.c            |    5
>   hw/net/rtl8139.c                 |    5
>   hw/pci-host/piix.c               |   22 +
>   hw/timer/arm_timer.c             |    2
>   hw/timer/hpet.c                  |   13 -
>   hw/timer/mc146818rtc.c           |    5
>   hw/timer/pl031.c                 |   35 +-
>   hw/usb/hcd-uhci.c                |    2
>   hw/usb/host-libusb.c             |  525 ++++++++++++++++++---------
>   include/block/aio.h              |   17 +
>   include/block/block.h            |   21 +
>   include/block/thread-pool.h      |    4
>   include/exec/cpu-defs.h          |    1
>   include/exec/cpu_ldst_template.h |   28 +
>   include/exec/exec-all.h          |   31 ++
>   include/hw/host-libusb.h         |  105 +++++
>   include/migration/vmstate.h      |   13 +
>   include/qemu-common.h            |    3
>   include/qemu/main-loop.h         |    1
>   include/qemu/timer.h             |   48 ++
>   include/qom/cpu.h                |    4
>   include/sysemu/char.h            |   25 +
>   include/sysemu/cpus.h            |    1
>   include/sysemu/sysemu.h          |    2
>   include/ui/input.h               |    2
>   main-loop.c                      |    5
>   monitor.c                        |   49 +++
>   net/Makefile.objs                |    1
>   net/clients.h                    |    3
>   net/dump.c                       |    6
>   net/hub.c                        |    1
>   net/net-replay.c                 |   66 +++
>   net/net.c                        |    7
>   net/slirp.c                      |   14 +
>   net/socket.c                     |   35 ++
>   net/tap-win32.c                  |   25 +
>   net/tap.c                        |   23 +
>   net/vde.c                        |   14 +
>   qapi-schema.json                 |   62 +++
>   qemu-char.c                      |   55 +++
>   qemu-img.c                       |    6
>   qemu-io-cmds.c                   |    2
>   qemu-io.c                        |    4
>   qemu-nbd.c                       |    2
>   qemu-options.hx                  |   28 +
>   qemu-timer.c                     |   45 ++
>   qmp-commands.hx                  |   60 +++
>   replay/Makefile.objs             |   11 +
>   replay/replay-audio.c            |  228 ++++++++++++
>   replay/replay-char.c             |   99 +++++
>   replay/replay-debug.c            |  148 ++++++++
>   replay/replay-events.c           |  381 ++++++++++++++++++++
>   replay/replay-input.c            |  107 ++++++
>   replay/replay-internal.c         |  159 ++++++++
>   replay/replay-internal.h         |  224 ++++++++++++
>   replay/replay-net.c              |  190 ++++++++++
>   replay/replay-qmp.c              |   57 +++
>   replay/replay-time.c             |  181 +++++++++
>   replay/replay-usb.c              |  188 ++++++++++
>   replay/replay.c                  |  735 ++++++++++++++++++++++++++++++++++++++
>   replay/replay.h                  |  218 +++++++++++
>   savevm.c                         |   32 +-
>   slirp/slirp.c                    |    9
>   softmmu_template.h               |   18 +
>   stubs/Makefile.objs              |    1
>   stubs/replay.c                   |   42 ++
>   target-arm/Makefile.objs         |    1
>   target-arm/helper.h              |    3
>   target-arm/machine.c             |    5
>   target-arm/replay_helper.c       |   38 ++
>   target-arm/translate.c           |   62 +++
>   target-i386/Makefile.objs        |    1
>   target-i386/cpu.c                |    1
>   target-i386/cpu.h                |    8
>   target-i386/fpu_helper.c         |    5
>   target-i386/helper.h             |    3
>   target-i386/machine.c            |    7
>   target-i386/replay_helper.c      |   38 ++
>   target-i386/translate.c          |   91 ++++-
>   tests/test-thread-pool.c         |    7
>   thread-pool.c                    |   53 ++-
>   trace-events                     |    2
>   translate-all.c                  |   15 +
>   ui/input.c                       |   79 +++-
>   util/iov.c                       |    4
>   vl.c                             |  108 +++++-
>   vmstate.c                        |    6
>   126 files changed, 5433 insertions(+), 543 deletions(-)
>   create mode 100755 include/hw/host-libusb.h
>   create mode 100755 net/net-replay.c
>   create mode 100755 replay/Makefile.objs
>   create mode 100755 replay/replay-audio.c
>   create mode 100755 replay/replay-char.c
>   create mode 100755 replay/replay-debug.c
>   create mode 100755 replay/replay-events.c
>   create mode 100755 replay/replay-input.c
>   create mode 100755 replay/replay-internal.c
>   create mode 100755 replay/replay-internal.h
>   create mode 100755 replay/replay-net.c
>   create mode 100755 replay/replay-qmp.c
>   create mode 100755 replay/replay-time.c
>   create mode 100755 replay/replay-usb.c
>   create mode 100755 replay/replay.c
>   create mode 100755 replay/replay.h
>   create mode 100755 stubs/replay.c
>   create mode 100755 target-arm/replay_helper.c
>   create mode 100755 target-i386/replay_helper.c
>

Re: [Qemu-devel] [RFC PATCH v2 46/49] replay: replay_info command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2148 bytes --]

On 07/17/2014 05:06 AM, Pavel Dovgalyuk wrote:
> This patch adds support for replay_info monitor command. This command
> returns the information about replay execution (replay mode and current step).
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---

> +++ b/qapi-schema.json
> @@ -3491,3 +3491,30 @@
>  # Since: 2.1
>  ##
>  { 'command': 'rtc-reset-reinjection' }
> +
> +##
> +# @ReplayInfo:
> +#
> +# Information about replay process
> +#
> +# @mode: replay mode (none, play, record)
> +#
> +# @submode: play submode

What are the valid submodes?  Does the set of submodes differ according
to the main mode?  If so, is it better to use a flat union, where the
subtype is strongly typed to the correct limited possibilities according
to the mode?

> +#
> +# @step: current step of record or play
> +#
> +# Since:  2.2
> +##
> +{ 'type': 'ReplayInfo',
> +  'data': {'mode': 'str', 'submode': 'str', 'step': 'uint64'} }

Using 'str' to open-code a finite set of strings is not nice.  Please add:
 { 'enum': 'ReplayMode', 'data': [ 'none', 'play', 'record' ] }
then use 'mode': 'ReplayMode'.


> +++ b/qmp-commands.hx
> @@ -3755,3 +3755,17 @@ Example:
>  <- { "return": {} }
>  
>  EQMP
> +
> +    {
> +        .name       = "replay_info",
> +        .args_type  = "",
> +        .mhandler.cmd_new = qmp_marshal_input_replay_info,
> +    },
> +
> +SQMP
> +replay_info
> +-----------
> +
> +Shows information about replay process.
> +

No example?


> +
> +const char *replay_get_mode_name(void)
> +{
> +    switch (replay_mode) {
> +    case REPLAY_NONE:
> +        return "none";
> +    case REPLAY_PLAY:
> +        return "replay";
> +    case REPLAY_SAVE:
> +        return "record";
> +    default:
> +        return "unknown";
> +    }
> +}
> +

No need to open-code this.  If you properly set up an enum type in the
.json file, then you get the conversions between enum values and strings
for free from the generated QAPI framework.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [RFC PATCH v2 46/49] replay: replay_info command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

On 07/17/2014 05:06 AM, Pavel Dovgalyuk wrote:
> This patch adds support for replay_info monitor command. This command
> returns the information about replay execution (replay mode and current step).
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---

Oh, missed one.


> +++ b/qapi-schema.json

> +##
> +# @replay_info

Please name this replay-info. HMP still uses '_', but QMP should prefer '-'.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [RFC PATCH v2 47/49] replay: replay_break command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

On 07/17/2014 05:06 AM, Pavel Dovgalyuk wrote:
> This patch adds support for replay_break monitor command. This command
> sets the step (measured in executed instructions) where replay should be
> stopped.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hmp-commands.hx     |   14 ++++++++++++++
>  monitor.c           |   15 +++++++++++++++
>  qapi-schema.json    |   11 +++++++++++
>  qmp-commands.hx     |   23 +++++++++++++++++++++++
>  replay/replay-qmp.c |   13 +++++++++++++
>  replay/replay.c     |   24 +++++++++++++++++++++++-
>  replay/replay.h     |    2 ++
>  7 files changed, 101 insertions(+), 1 deletions(-)
> 

> +++ b/qapi-schema.json
> @@ -3518,3 +3518,14 @@
>  # Since:  2.2
>  ##
>  { 'command': 'replay_info', 'returns': 'ReplayInfo' }
> +
> +##
> +# @replay_break

replay-break in QMP

> +#
> +# Sets breakpoint at the specified step of replaying
> +#
> +# @step: step where breakpoint should be set
> +#
> +# Since:  2.2

Is the double space intentional?

This is a very high-level description - reading just this definition, I
don't know what typical settings are, and whether changing the number to
be larger or smaller makes a difference.  Might be worth a bit more docs.

I hate write-only interfaces - how do I query what the current
breakpoint step is?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [RFC PATCH v2 48/49] replay: replay_seek_step command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]

On 07/17/2014 05:06 AM, Pavel Dovgalyuk wrote:
> This patch adds support for replay_seek_step monitor command. This command
> loads one of the snapshots and replays the execution until the specified
> step is met.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hmp-commands.hx          |   14 ++++++++++++++
>  monitor.c                |   17 +++++++++++++++++
>  qapi-schema.json         |   11 +++++++++++
>  qmp-commands.hx          |   23 +++++++++++++++++++++++
>  replay/replay-internal.h |    3 +++
>  replay/replay-qmp.c      |   15 +++++++++++++++
>  replay/replay.c          |   43 +++++++++++++++++++++++++++++++++++++++++++
>  replay/replay.h          |    4 ++++
>  8 files changed, 130 insertions(+), 0 deletions(-)
> 

> +++ b/qapi-schema.json
> @@ -3529,3 +3529,14 @@
>  # Since:  2.2
>  ##
>  { 'command': 'replay_break', 'data': { 'step': 'uint64' } }
> +
> +##
> +# @replay_seek

replay-seek

> +#
> +# Seeks the replay log to the specified step
> +#
> +# @step: destination replay step 
> +#
> +# Since:  2.2
> +##

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Paolo Bonzini]

Il 17/07/2014 13:01, Pavel Dovgalyuk ha scritto:
> This set of patches is related to the reverse execution and deterministic 
> replay of qemu execution  Our implementation of deterministic replay can 
> be used for deterministic and reverse debugging of guest code through gdb 
> remote interface.
> 
> Execution recording writes non-deterministic events log, which can be later 
> used for replaying the execution anywhere and for unlimited number of times. 
> It also supports checkpointing for faster rewinding during reverse debugging. 
> Execution replaying reads the log and replays all non-deterministic events 
> including external input, hardware clocks, and interrupts.

>From a first look:

- patches 1-14-15-17-41 are okay :)

- patches 2 to 13 probably should try to use subsections, so that VMs
that do not use the devices try not to save the extra data and keep
backwards migration compatibility (at least try to)

- patch 16 should also use subsections, and perhaps apply to all other
CPUs too?

- patches 23-24-25 perhaps could try using icount, like Konrad's patch do?

- patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
is stopped

- the others I haven't yet looked at, but they look like something that
would bitrot really, really fast.  For patch 33, I think changing
INSERT_HEAD to INSERT_TAIL would be just fine, and I wonder if it's the
same for other patches here.  How do you plan on testing them and
keeping them up to date?

Paolo

> Reverse execution has the following features:
>  * Deterministically replays whole system execution and all contents of the memory,
>    state of the hadrware devices, clocks, and screen of the VM.
>  * Writes execution log into the file for latter replaying for multiple times 
>    on different machines.
>  * Supports i386, x86_64, and ARM hardware platforms.
>  * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
>    audio devices, serial interfaces, and physical USB devices connected to the emulator.
>  * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
>  * Supports auto-checkpointing for convenient reverse debugging.
> 
> Usage of the record/replay:
>  * First, record the execution, by adding '-record fname=replay.bin' to the
>    command line.
>  * Then you can replay it for the multiple times by using another command
>    line option: '-replay fname=replay.bin'
>  * Virtual machine should have at least one virtual disk, which is used to
>    store checkpoints. If you want to enable automatic checkpointing, simply
>    add ',period=XX' to record options, where XX is the checkpointing period
>    in seconds.
>  * Using of the network adapters in record/replay mode is possible with 
>    the following command-line options:
>    - '-net user' (or another host adapter) in record mode
>    - '-net replay' in replay mode. Every host network adapter should be
>      replaced by 'replay' when replaying the execution.
>  * Reverse debugging can be used through gdb remote interface.
>    reverse-stepi and reverse-continue commands are supported. Other reverse
>    commands should also work, because they reuse these ones.
>  * Monitor is extended by the following commands:
>    - replay_info - prints information about replay mode and current step
>      (number of instructions executed)
>    - replay_break - sets "breakpoint" at the specified instructions count.
>    - replay_seek - rewinds (using the checkpoints, if possible) to the
>      specified step of replay log.
> 
> Paper with short description of deterministic replay implementation:
> http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
> 
> Modifications of qemu include:
>  * adding missed fields of the virtual devices' states to the vmstate 
>    structures to allow deterministic saving and restoring the VM state
>  * adding virtual clock-based timers to vmstate structures, because virtual 
>    clock is the part of the virtual machine state
>  * modification of block layer to support automatic creation of the overlay
>    files to store the changes and snapshots while recording
>  * disabling of system reset while loading VM state to avoid generating of
>    interrupts by reset handlers
>  * adding warpers for clock and time functions to save their return
>    values in the log
>  * saving different asynchronous events (e.g. system shutdown) into the log
>  * synchronization of the bottom halves execution
>  * synchronization of the threads from thread pool
>  * recording/replaying user input (mouse and keyboard), input from virtual
>    serial ports, incoming network packets, input from connected USB devices
>  * adding HMP/QMP commands to monitor for controlling replay execution
> 
> v2 changes:
>  * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
>  * Added QMP versions of replay commands (as suggested by Eric Blake)
>  * Removed some optional features of replay to make patches cleaner
>  * Minor changes and code cleanup were made
> 
> ---
> 
> Pavel Dovgalyuk (49):
>       acpi: accurate overflow check
>       integratorcp: adding vmstate for save/restore
>       pcspk: adding vmstate for save/restore
>       fdc: adding vmstate for save/restore
>       parallel: adding vmstate for save/restore
>       serial: fixing vmstate for save/restore
>       kvmapic: fixing loading vmstate
>       hpet: fixing saving and loading process
>       pckbd: adding new fields to vmstate
>       rtl8139: adding new fields to vmstate
>       piix: do not raise irq while loading vmstate
>       mc146818rtc: add missed field to vmstate
>       pl031: add missed field to vmstate
>       ide pci: reset status field before loading the vmstate
>       softmmu: fixing usage of cpu_st/ld* from helpers
>       target: save cpu state fields
>       target-i386: update fp status fix
>       migration: add vmstate for int8 and char arrays
>       replay: global variables and function stubs
>       block: add suffix parameter to bdrv_open functions
>       sysemu: system functions for replay
>       replay: internal functions for replay log
>       cpu: invent instruction count for accurate replay
>       target-arm: instructions counting code for replay
>       target-i386: instructions counting code for replay
>       replay: interrupts and exceptions
>       vga: do not use virtual clock for blinking cursor
>       replay: asynchronous events infrastructure
>       replay: recording and replaying clock ticks
>       replay: recording and replaying different timers
>       replay: shutdown event
>       replay: checkpoints
>       replay: bottom halves
>       replay: replay aio requests
>       replay: thread pool
>       pl031: vmstate in replay mode
>       replay: initialization and deinitialization
>       replay: command line options
>       replay: snapshotting the virtual machine
>       replay: recording of the user input
>       tap-win32: destroy the thread at exit
>       replay: network packets record/replay
>       replay: audio data record/replay
>       replay: serial port
>       replay: USB passthrough
>       replay: replay_info command
>       replay: replay_break command
>       replay: replay_seek_step command
>       gdbstub: reverse debugging
> 
> 
>  Makefile.target                  |    1 
>  arch_init.c                      |    8 
>  async.c                          |   45 ++
>  audio/audio.c                    |   14 +
>  audio/audio_win_int.h            |    3 
>  audio/winwaveaudio.c             |  167 +++++++--
>  block.c                          |  134 ++++++-
>  block/blkdebug.c                 |    2 
>  block/blkverify.c                |    4 
>  block/cow.c                      |    2 
>  block/qcow.c                     |    2 
>  block/qcow2.c                    |   10 -
>  block/qed.c                      |    2 
>  block/raw-posix.c                |    6 
>  block/raw-win32.c                |    4 
>  block/sheepdog.c                 |    4 
>  block/vmdk.c                     |    8 
>  block/vvfat.c                    |    2 
>  blockdev.c                       |   11 -
>  cpu-exec.c                       |   34 +-
>  cpus.c                           |   81 ++++
>  dma-helpers.c                    |   10 -
>  exec.c                           |   12 +
>  gdbstub.c                        |   79 +++-
>  hmp-commands.hx                  |   41 ++
>  hw/acpi/core.c                   |    7 
>  hw/arm/integratorcp.c            |   38 ++
>  hw/audio/pcspk.c                 |   19 +
>  hw/block/fdc.c                   |   11 -
>  hw/block/virtio-blk.c            |   10 -
>  hw/char/parallel.c               |   22 +
>  hw/char/serial.c                 |  115 ++++--
>  hw/display/vga.c                 |    4 
>  hw/i386/kvmvapic.c               |   22 +
>  hw/ide/ahci.c                    |    4 
>  hw/ide/atapi.c                   |    9 
>  hw/ide/core.c                    |   18 +
>  hw/ide/pci.c                     |   10 +
>  hw/input/pckbd.c                 |    3 
>  hw/intc/apic_common.c            |    5 
>  hw/net/rtl8139.c                 |    5 
>  hw/pci-host/piix.c               |   22 +
>  hw/timer/arm_timer.c             |    2 
>  hw/timer/hpet.c                  |   13 -
>  hw/timer/mc146818rtc.c           |    5 
>  hw/timer/pl031.c                 |   35 +-
>  hw/usb/hcd-uhci.c                |    2 
>  hw/usb/host-libusb.c             |  525 ++++++++++++++++++---------
>  include/block/aio.h              |   17 +
>  include/block/block.h            |   21 +
>  include/block/thread-pool.h      |    4 
>  include/exec/cpu-defs.h          |    1 
>  include/exec/cpu_ldst_template.h |   28 +
>  include/exec/exec-all.h          |   31 ++
>  include/hw/host-libusb.h         |  105 +++++
>  include/migration/vmstate.h      |   13 +
>  include/qemu-common.h            |    3 
>  include/qemu/main-loop.h         |    1 
>  include/qemu/timer.h             |   48 ++
>  include/qom/cpu.h                |    4 
>  include/sysemu/char.h            |   25 +
>  include/sysemu/cpus.h            |    1 
>  include/sysemu/sysemu.h          |    2 
>  include/ui/input.h               |    2 
>  main-loop.c                      |    5 
>  monitor.c                        |   49 +++
>  net/Makefile.objs                |    1 
>  net/clients.h                    |    3 
>  net/dump.c                       |    6 
>  net/hub.c                        |    1 
>  net/net-replay.c                 |   66 +++
>  net/net.c                        |    7 
>  net/slirp.c                      |   14 +
>  net/socket.c                     |   35 ++
>  net/tap-win32.c                  |   25 +
>  net/tap.c                        |   23 +
>  net/vde.c                        |   14 +
>  qapi-schema.json                 |   62 +++
>  qemu-char.c                      |   55 +++
>  qemu-img.c                       |    6 
>  qemu-io-cmds.c                   |    2 
>  qemu-io.c                        |    4 
>  qemu-nbd.c                       |    2 
>  qemu-options.hx                  |   28 +
>  qemu-timer.c                     |   45 ++
>  qmp-commands.hx                  |   60 +++
>  replay/Makefile.objs             |   11 +
>  replay/replay-audio.c            |  228 ++++++++++++
>  replay/replay-char.c             |   99 +++++
>  replay/replay-debug.c            |  148 ++++++++
>  replay/replay-events.c           |  381 ++++++++++++++++++++
>  replay/replay-input.c            |  107 ++++++
>  replay/replay-internal.c         |  159 ++++++++
>  replay/replay-internal.h         |  224 ++++++++++++
>  replay/replay-net.c              |  190 ++++++++++
>  replay/replay-qmp.c              |   57 +++
>  replay/replay-time.c             |  181 +++++++++
>  replay/replay-usb.c              |  188 ++++++++++
>  replay/replay.c                  |  735 ++++++++++++++++++++++++++++++++++++++
>  replay/replay.h                  |  218 +++++++++++
>  savevm.c                         |   32 +-
>  slirp/slirp.c                    |    9 
>  softmmu_template.h               |   18 +
>  stubs/Makefile.objs              |    1 
>  stubs/replay.c                   |   42 ++
>  target-arm/Makefile.objs         |    1 
>  target-arm/helper.h              |    3 
>  target-arm/machine.c             |    5 
>  target-arm/replay_helper.c       |   38 ++
>  target-arm/translate.c           |   62 +++
>  target-i386/Makefile.objs        |    1 
>  target-i386/cpu.c                |    1 
>  target-i386/cpu.h                |    8 
>  target-i386/fpu_helper.c         |    5 
>  target-i386/helper.h             |    3 
>  target-i386/machine.c            |    7 
>  target-i386/replay_helper.c      |   38 ++
>  target-i386/translate.c          |   91 ++++-
>  tests/test-thread-pool.c         |    7 
>  thread-pool.c                    |   53 ++-
>  trace-events                     |    2 
>  translate-all.c                  |   15 +
>  ui/input.c                       |   79 +++-
>  util/iov.c                       |    4 
>  vl.c                             |  108 +++++-
>  vmstate.c                        |    6 
>  126 files changed, 5433 insertions(+), 543 deletions(-)
>  create mode 100755 include/hw/host-libusb.h
>  create mode 100755 net/net-replay.c
>  create mode 100755 replay/Makefile.objs
>  create mode 100755 replay/replay-audio.c
>  create mode 100755 replay/replay-char.c
>  create mode 100755 replay/replay-debug.c
>  create mode 100755 replay/replay-events.c
>  create mode 100755 replay/replay-input.c
>  create mode 100755 replay/replay-internal.c
>  create mode 100755 replay/replay-internal.h
>  create mode 100755 replay/replay-net.c
>  create mode 100755 replay/replay-qmp.c
>  create mode 100755 replay/replay-time.c
>  create mode 100755 replay/replay-usb.c
>  create mode 100755 replay/replay.c
>  create mode 100755 replay/replay.h
>  create mode 100755 stubs/replay.c
>  create mode 100755 target-arm/replay_helper.c
>  create mode 100755 target-i386/replay_helper.c
> 

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Il 17/07/2014 13:01, Pavel Dovgalyuk ha scritto:
> > This set of patches is related to the reverse execution and deterministic
> > replay of qemu execution  Our implementation of deterministic replay can
> > be used for deterministic and reverse debugging of guest code through gdb
> > remote interface.
> >
> > Execution recording writes non-deterministic events log, which can be later
> > used for replaying the execution anywhere and for unlimited number of times.
> > It also supports checkpointing for faster rewinding during reverse debugging.
> > Execution replaying reads the log and replays all non-deterministic events
> > including external input, hardware clocks, and interrupts.
> 
> From a first look:
> 
> - patches 2 to 13 probably should try to use subsections, so that VMs
> that do not use the devices try not to save the extra data and keep
> backwards migration compatibility (at least try to)

 Could you give me and example?
 As I know, subsection is loaded when some predicate function returns true. How can I 
construct such a function for integratorcp module? What kind of condition should it check?
 In this module I just added missed vmstates (it does not saved/restored at all
by the master version).

> - patch 16 should also use subsections, and perhaps apply to all other
> CPUs too?

 We implemented replay only for i386 and ARM. If we'll change other targets, it will not
add record/replay capabilities to them, but can confuse the reviewers.

> - patches 23-24-25 perhaps could try using icount, like Konrad's patch do?

 Using faster icount (like in Konrad's patches) is the our next aim. It obviously will 
increase the speed of recording process. But now I submitted slower, but more conservative
version of icount which we had already tested.

> - patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
> is stopped

 Right, this is kind of hack. I haven't found better solution yet.

> - the others I haven't yet looked at, but they look like something that
> would bitrot really, really fast.  For patch 33, I think changing
> INSERT_HEAD to INSERT_TAIL would be just fine, and I wonder if it's the
> same for other patches here.  How do you plan on testing them and
> keeping them up to date?

 We're constantly keeping these patches up to date, because we are using deterministic replay
and reverse debugging for solving our tasks. We re-test all the features when pulling new
patches from the master branch.

Pavel Dovgalyuk

> 
> > Reverse execution has the following features:
> >  * Deterministically replays whole system execution and all contents of the memory,
> >    state of the hadrware devices, clocks, and screen of the VM.
> >  * Writes execution log into the file for latter replaying for multiple times
> >    on different machines.
> >  * Supports i386, x86_64, and ARM hardware platforms.
> >  * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
> >    audio devices, serial interfaces, and physical USB devices connected to the emulator.
> >  * Provides support for gdb reverse debugging commands like reverse-step and reverse-
> continue.
> >  * Supports auto-checkpointing for convenient reverse debugging.
> >
> > Usage of the record/replay:
> >  * First, record the execution, by adding '-record fname=replay.bin' to the
> >    command line.
> >  * Then you can replay it for the multiple times by using another command
> >    line option: '-replay fname=replay.bin'
> >  * Virtual machine should have at least one virtual disk, which is used to
> >    store checkpoints. If you want to enable automatic checkpointing, simply
> >    add ',period=XX' to record options, where XX is the checkpointing period
> >    in seconds.
> >  * Using of the network adapters in record/replay mode is possible with
> >    the following command-line options:
> >    - '-net user' (or another host adapter) in record mode
> >    - '-net replay' in replay mode. Every host network adapter should be
> >      replaced by 'replay' when replaying the execution.
> >  * Reverse debugging can be used through gdb remote interface.
> >    reverse-stepi and reverse-continue commands are supported. Other reverse
> >    commands should also work, because they reuse these ones.
> >  * Monitor is extended by the following commands:
> >    - replay_info - prints information about replay mode and current step
> >      (number of instructions executed)
> >    - replay_break - sets "breakpoint" at the specified instructions count.
> >    - replay_seek - rewinds (using the checkpoints, if possible) to the
> >      specified step of replay log.
> >
> > Paper with short description of deterministic replay implementation:
> > http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
> >
> > Modifications of qemu include:
> >  * adding missed fields of the virtual devices' states to the vmstate
> >    structures to allow deterministic saving and restoring the VM state
> >  * adding virtual clock-based timers to vmstate structures, because virtual
> >    clock is the part of the virtual machine state
> >  * modification of block layer to support automatic creation of the overlay
> >    files to store the changes and snapshots while recording
> >  * disabling of system reset while loading VM state to avoid generating of
> >    interrupts by reset handlers
> >  * adding warpers for clock and time functions to save their return
> >    values in the log
> >  * saving different asynchronous events (e.g. system shutdown) into the log
> >  * synchronization of the bottom halves execution
> >  * synchronization of the threads from thread pool
> >  * recording/replaying user input (mouse and keyboard), input from virtual
> >    serial ports, incoming network packets, input from connected USB devices
> >  * adding HMP/QMP commands to monitor for controlling replay execution
> >
> > v2 changes:
> >  * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
> >  * Added QMP versions of replay commands (as suggested by Eric Blake)
> >  * Removed some optional features of replay to make patches cleaner
> >  * Minor changes and code cleanup were made
> >
> > ---
> >
> > Pavel Dovgalyuk (49):
> >       acpi: accurate overflow check
> >       integratorcp: adding vmstate for save/restore
> >       pcspk: adding vmstate for save/restore
> >       fdc: adding vmstate for save/restore
> >       parallel: adding vmstate for save/restore
> >       serial: fixing vmstate for save/restore
> >       kvmapic: fixing loading vmstate
> >       hpet: fixing saving and loading process
> >       pckbd: adding new fields to vmstate
> >       rtl8139: adding new fields to vmstate
> >       piix: do not raise irq while loading vmstate
> >       mc146818rtc: add missed field to vmstate
> >       pl031: add missed field to vmstate
> >       ide pci: reset status field before loading the vmstate
> >       softmmu: fixing usage of cpu_st/ld* from helpers
> >       target: save cpu state fields
> >       target-i386: update fp status fix
> >       migration: add vmstate for int8 and char arrays
> >       replay: global variables and function stubs
> >       block: add suffix parameter to bdrv_open functions
> >       sysemu: system functions for replay
> >       replay: internal functions for replay log
> >       cpu: invent instruction count for accurate replay
> >       target-arm: instructions counting code for replay
> >       target-i386: instructions counting code for replay
> >       replay: interrupts and exceptions
> >       vga: do not use virtual clock for blinking cursor
> >       replay: asynchronous events infrastructure
> >       replay: recording and replaying clock ticks
> >       replay: recording and replaying different timers
> >       replay: shutdown event
> >       replay: checkpoints
> >       replay: bottom halves
> >       replay: replay aio requests
> >       replay: thread pool
> >       pl031: vmstate in replay mode
> >       replay: initialization and deinitialization
> >       replay: command line options
> >       replay: snapshotting the virtual machine
> >       replay: recording of the user input
> >       tap-win32: destroy the thread at exit
> >       replay: network packets record/replay
> >       replay: audio data record/replay
> >       replay: serial port
> >       replay: USB passthrough
> >       replay: replay_info command
> >       replay: replay_break command
> >       replay: replay_seek_step command
> >       gdbstub: reverse debugging
> >
> >
> >  Makefile.target                  |    1
> >  arch_init.c                      |    8
> >  async.c                          |   45 ++
> >  audio/audio.c                    |   14 +
> >  audio/audio_win_int.h            |    3
> >  audio/winwaveaudio.c             |  167 +++++++--
> >  block.c                          |  134 ++++++-
> >  block/blkdebug.c                 |    2
> >  block/blkverify.c                |    4
> >  block/cow.c                      |    2
> >  block/qcow.c                     |    2
> >  block/qcow2.c                    |   10 -
> >  block/qed.c                      |    2
> >  block/raw-posix.c                |    6
> >  block/raw-win32.c                |    4
> >  block/sheepdog.c                 |    4
> >  block/vmdk.c                     |    8
> >  block/vvfat.c                    |    2
> >  blockdev.c                       |   11 -
> >  cpu-exec.c                       |   34 +-
> >  cpus.c                           |   81 ++++
> >  dma-helpers.c                    |   10 -
> >  exec.c                           |   12 +
> >  gdbstub.c                        |   79 +++-
> >  hmp-commands.hx                  |   41 ++
> >  hw/acpi/core.c                   |    7
> >  hw/arm/integratorcp.c            |   38 ++
> >  hw/audio/pcspk.c                 |   19 +
> >  hw/block/fdc.c                   |   11 -
> >  hw/block/virtio-blk.c            |   10 -
> >  hw/char/parallel.c               |   22 +
> >  hw/char/serial.c                 |  115 ++++--
> >  hw/display/vga.c                 |    4
> >  hw/i386/kvmvapic.c               |   22 +
> >  hw/ide/ahci.c                    |    4
> >  hw/ide/atapi.c                   |    9
> >  hw/ide/core.c                    |   18 +
> >  hw/ide/pci.c                     |   10 +
> >  hw/input/pckbd.c                 |    3
> >  hw/intc/apic_common.c            |    5
> >  hw/net/rtl8139.c                 |    5
> >  hw/pci-host/piix.c               |   22 +
> >  hw/timer/arm_timer.c             |    2
> >  hw/timer/hpet.c                  |   13 -
> >  hw/timer/mc146818rtc.c           |    5
> >  hw/timer/pl031.c                 |   35 +-
> >  hw/usb/hcd-uhci.c                |    2
> >  hw/usb/host-libusb.c             |  525 ++++++++++++++++++---------
> >  include/block/aio.h              |   17 +
> >  include/block/block.h            |   21 +
> >  include/block/thread-pool.h      |    4
> >  include/exec/cpu-defs.h          |    1
> >  include/exec/cpu_ldst_template.h |   28 +
> >  include/exec/exec-all.h          |   31 ++
> >  include/hw/host-libusb.h         |  105 +++++
> >  include/migration/vmstate.h      |   13 +
> >  include/qemu-common.h            |    3
> >  include/qemu/main-loop.h         |    1
> >  include/qemu/timer.h             |   48 ++
> >  include/qom/cpu.h                |    4
> >  include/sysemu/char.h            |   25 +
> >  include/sysemu/cpus.h            |    1
> >  include/sysemu/sysemu.h          |    2
> >  include/ui/input.h               |    2
> >  main-loop.c                      |    5
> >  monitor.c                        |   49 +++
> >  net/Makefile.objs                |    1
> >  net/clients.h                    |    3
> >  net/dump.c                       |    6
> >  net/hub.c                        |    1
> >  net/net-replay.c                 |   66 +++
> >  net/net.c                        |    7
> >  net/slirp.c                      |   14 +
> >  net/socket.c                     |   35 ++
> >  net/tap-win32.c                  |   25 +
> >  net/tap.c                        |   23 +
> >  net/vde.c                        |   14 +
> >  qapi-schema.json                 |   62 +++
> >  qemu-char.c                      |   55 +++
> >  qemu-img.c                       |    6
> >  qemu-io-cmds.c                   |    2
> >  qemu-io.c                        |    4
> >  qemu-nbd.c                       |    2
> >  qemu-options.hx                  |   28 +
> >  qemu-timer.c                     |   45 ++
> >  qmp-commands.hx                  |   60 +++
> >  replay/Makefile.objs             |   11 +
> >  replay/replay-audio.c            |  228 ++++++++++++
> >  replay/replay-char.c             |   99 +++++
> >  replay/replay-debug.c            |  148 ++++++++
> >  replay/replay-events.c           |  381 ++++++++++++++++++++
> >  replay/replay-input.c            |  107 ++++++
> >  replay/replay-internal.c         |  159 ++++++++
> >  replay/replay-internal.h         |  224 ++++++++++++
> >  replay/replay-net.c              |  190 ++++++++++
> >  replay/replay-qmp.c              |   57 +++
> >  replay/replay-time.c             |  181 +++++++++
> >  replay/replay-usb.c              |  188 ++++++++++
> >  replay/replay.c                  |  735 ++++++++++++++++++++++++++++++++++++++
> >  replay/replay.h                  |  218 +++++++++++
> >  savevm.c                         |   32 +-
> >  slirp/slirp.c                    |    9
> >  softmmu_template.h               |   18 +
> >  stubs/Makefile.objs              |    1
> >  stubs/replay.c                   |   42 ++
> >  target-arm/Makefile.objs         |    1
> >  target-arm/helper.h              |    3
> >  target-arm/machine.c             |    5
> >  target-arm/replay_helper.c       |   38 ++
> >  target-arm/translate.c           |   62 +++
> >  target-i386/Makefile.objs        |    1
> >  target-i386/cpu.c                |    1
> >  target-i386/cpu.h                |    8
> >  target-i386/fpu_helper.c         |    5
> >  target-i386/helper.h             |    3
> >  target-i386/machine.c            |    7
> >  target-i386/replay_helper.c      |   38 ++
> >  target-i386/translate.c          |   91 ++++-
> >  tests/test-thread-pool.c         |    7
> >  thread-pool.c                    |   53 ++-
> >  trace-events                     |    2
> >  translate-all.c                  |   15 +
> >  ui/input.c                       |   79 +++-
> >  util/iov.c                       |    4
> >  vl.c                             |  108 +++++-
> >  vmstate.c                        |    6
> >  126 files changed, 5433 insertions(+), 543 deletions(-)
> >  create mode 100755 include/hw/host-libusb.h
> >  create mode 100755 net/net-replay.c
> >  create mode 100755 replay/Makefile.objs
> >  create mode 100755 replay/replay-audio.c
> >  create mode 100755 replay/replay-char.c
> >  create mode 100755 replay/replay-debug.c
> >  create mode 100755 replay/replay-events.c
> >  create mode 100755 replay/replay-input.c
> >  create mode 100755 replay/replay-internal.c
> >  create mode 100755 replay/replay-internal.h
> >  create mode 100755 replay/replay-net.c
> >  create mode 100755 replay/replay-qmp.c
> >  create mode 100755 replay/replay-time.c
> >  create mode 100755 replay/replay-usb.c
> >  create mode 100755 replay/replay.c
> >  create mode 100755 replay/replay.h
> >  create mode 100755 stubs/replay.c
> >  create mode 100755 target-arm/replay_helper.c
> >  create mode 100755 target-i386/replay_helper.c
> >

Re: [Qemu-devel] [RFC PATCH v2 08/49] hpet: fixing saving and loading process

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> VM clock does not run while saving, so there is no need for saving the ticks
> in HPET. Also added saving of hpet_offset field.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/timer/hpet.c |   13 +------------
>  1 files changed, 1 insertions(+), 12 deletions(-)
> 
> diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c
> index e160e8f..73401b9 100644
> --- a/hw/timer/hpet.c
> +++ b/hw/timer/hpet.c
> @@ -222,14 +222,6 @@ static void update_irq(struct HPETTimer *timer, int set)
>      }
>  }
>  
> -static void hpet_pre_save(void *opaque)
> -{
> -    HPETState *s = opaque;
> -
> -    /* save current counter value */
> -    s->hpet_counter = hpet_get_ticks(s);
> -}
> -
>  static int hpet_pre_load(void *opaque)
>  {
>      HPETState *s = opaque;
> @@ -255,9 +247,6 @@ static int hpet_post_load(void *opaque, int version_id)
>  {
>      HPETState *s = opaque;
>  
> -    /* Recalculate the offset between the main counter and guest time */
> -    s->hpet_offset = ticks_to_ns(s->hpet_counter) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
> -
>      /* Push number of timers into capability returned via HPET_ID */
>      s->capability &= ~HPET_ID_NUM_TIM_MASK;
>      s->capability |= (s->num_timers - 1) << HPET_ID_NUM_TIM_SHIFT;
> @@ -308,13 +297,13 @@ static const VMStateDescription vmstate_hpet = {
>      .name = "hpet",
>      .version_id = 2,
>      .minimum_version_id = 1,
> -    .pre_save = hpet_pre_save,
>      .pre_load = hpet_pre_load,
>      .post_load = hpet_post_load,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT64(config, HPETState),
>          VMSTATE_UINT64(isr, HPETState),
>          VMSTATE_UINT64(hpet_counter, HPETState),
> +        VMSTATE_UINT64(hpet_offset, HPETState),

This needs a version bump.

Paolo

>          VMSTATE_UINT8_V(num_timers, HPETState, 2),
>          VMSTATE_VALIDATE("num_timers in range", hpet_validate_num_timers),
>          VMSTATE_STRUCT_VARRAY_UINT8(timer, HPETState, num_timers, 0,
> 
> 
> 

Re: [Qemu-devel] [RFC PATCH v2 07/49] kvmapic: fixing loading vmstate

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> vapic state should not be synchronized with APIC while loading,
> because APIC state could be not loaded yet at that moment.
> We just save vapic_paddr in APIC VMState instead of synchronization.

This comment is now obsolete:

include/hw/i386/apic_internal.h:    hwaddr vapic_paddr; /* note: persistence via kvmvapic */

> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/i386/kvmvapic.c    |   22 +++++++++++++++++++++-
>  hw/intc/apic_common.c |    5 ++++-
>  2 files changed, 25 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
> index cb855c7..417ab6a 100644
> --- a/hw/i386/kvmvapic.c
> +++ b/hw/i386/kvmvapic.c
> @@ -351,6 +351,24 @@ static int get_kpcr_number(X86CPU *cpu)
>      return kpcr.number;
>  }
>  
> +static int vapic_enable_post_load(VAPICROMState *s, X86CPU *cpu)
> +{
> +    int cpu_number = get_kpcr_number(cpu);
> +    hwaddr vapic_paddr;
> +    static const uint8_t enabled = 1;
> +
> +    if (cpu_number < 0) {
> +        return -1;
> +    }
> +    vapic_paddr = s->vapic_paddr +
> +        (((hwaddr)cpu_number) << VAPIC_CPU_SHIFT);
> +    cpu_physical_memory_rw(vapic_paddr + offsetof(VAPICState, enabled),
> +                           (void *)&enabled, sizeof(enabled), 1);
> +    s->state = VAPIC_ACTIVE;
> +
> +    return 0;
> +}
> +
>  static int vapic_enable(VAPICROMState *s, X86CPU *cpu)
>  {
>      int cpu_number = get_kpcr_number(cpu);
> @@ -731,7 +749,9 @@ static void do_vapic_enable(void *data)
>      VAPICROMState *s = data;
>      X86CPU *cpu = X86_CPU(first_cpu);
>  
> -    vapic_enable(s, cpu);
> +    /* Do not synchronize with APIC, because it was not loaded yet.
> +       Just call the enable function which does not have synchronization. */
> +    vapic_enable_post_load(s, cpu);
>  }
>  
>  static int vapic_post_load(void *opaque, int version_id)
> diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> index ce3d903..9d75ee0 100644
> --- a/hw/intc/apic_common.c
> +++ b/hw/intc/apic_common.c
> @@ -347,7 +347,7 @@ static int apic_dispatch_post_load(void *opaque, int version_id)
>  
>  static const VMStateDescription vmstate_apic_common = {
>      .name = "apic",
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 3,
>      .minimum_version_id_old = 1,
>      .load_state_old = apic_load_old,
> @@ -374,6 +374,9 @@ static const VMStateDescription vmstate_apic_common = {
>          VMSTATE_INT64(next_time, APICCommonState),
>          VMSTATE_INT64(timer_expiry,
>                        APICCommonState), /* open-coded timer state */
> +        VMSTATE_INT32_V(sipi_vector, APICCommonState, 4),
> +        VMSTATE_INT32_V(wait_for_sipi, APICCommonState, 4),

This could be a subsection.  sipi_vector is only used (needed) if wait_for_sipi != 0.

> +        VMSTATE_UINT64_V(vapic_paddr, APICCommonState, 4),

Here you could also use a subsection, where the "needed" function returns false if vapic_paddr == 0.

Paolo


>          VMSTATE_END_OF_LIST()
>      }
>  };
> 
> 
> 

Re: [Qemu-devel] [RFC PATCH v2 09/49] pckbd: adding new fields to vmstate

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> This patch adds outport to VMState to allow correct saving and restoring
> the state of PC keyboard controller.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/input/pckbd.c |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/hw/input/pckbd.c b/hw/input/pckbd.c
> index ca1cffc..19f6658 100644
> --- a/hw/input/pckbd.c
> +++ b/hw/input/pckbd.c
> @@ -371,13 +371,14 @@ static void kbd_reset(void *opaque)
>  
>  static const VMStateDescription vmstate_kbd = {
>      .name = "pckbd",
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 3,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT8(write_cmd, KBDState),
>          VMSTATE_UINT8(status, KBDState),
>          VMSTATE_UINT8(mode, KBDState),
>          VMSTATE_UINT8(pending, KBDState),
> +        VMSTATE_UINT8_V(outport, KBDState, 4),
>          VMSTATE_END_OF_LIST()
>      }
>  };
> 
> 
> 

Again it would be nice to use a subsection.  You can use as the
"default" value

    KBD_OUT_RESET | KBD_OUT_A20
     | (kbd->status & KBD_STAT_OBF ? KBD_OUT_OBF : 0)
     | (kbd->status & KBD_STAT_MOUSE_OBF ? KBD_OUT_MOUSE_OBF : 0)

If the value of outport matches this, you need not write it.  It's not
trivial, but you could do it like this:

- needed: return false if outport doesn't have the value above

- subsection post_load: set kbd->outport_present = 1

- device post_load: reconstruct outport if kbd->outport_present == 0

Paolo

Re: [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> This patch adds virtual clock-dependent timers to VMState to allow correct
> saving and restoring the state of RTL8139 network controller.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/net/rtl8139.c |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c
> index 90bc5ec..992caf0 100644
> --- a/hw/net/rtl8139.c
> +++ b/hw/net/rtl8139.c
> @@ -3289,7 +3289,7 @@ static void rtl8139_pre_save(void *opaque)
>  
>  static const VMStateDescription vmstate_rtl8139 = {
>      .name = "rtl8139",
> -    .version_id = 4,
> +    .version_id = 5,
>      .minimum_version_id = 3,
>      .post_load = rtl8139_post_load,
>      .pre_save  = rtl8139_pre_save,
> @@ -3363,6 +3363,9 @@ static const VMStateDescription vmstate_rtl8139 = {
>          VMSTATE_STRUCT(tally_counters, RTL8139State, 0,
>                         vmstate_tally_counters, RTL8139TallyCounters),
>  
> +        VMSTATE_TIMER_V(timer, RTL8139State, 5),

timer need not be migrated, because it is reinstated by rtl8139_post_load.

> +        VMSTATE_INT64_V(TimerExpire, RTL8139State, 5),

This can be in a subsection, migrated only if non-zero.

Paolo

>          VMSTATE_UINT32_V(cplus_enabled, RTL8139State, 4),
>          VMSTATE_END_OF_LIST()
>      },
> 
> 
> 

Re: [Qemu-devel] [RFC PATCH v2 12/49] mc146818rtc: add missed field to vmstate

[Paolo Bonzini]

Il 17/07/2014 13:03, Pavel Dovgalyuk ha scritto:
> This patch adds irq_reinject_on_ack_count field to VMState to allow correct
> saving/loading the state of MC146818 RTC.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/timer/mc146818rtc.c |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> index 9d817ca..c204abb 100644
> --- a/hw/timer/mc146818rtc.c
> +++ b/hw/timer/mc146818rtc.c
> @@ -735,7 +735,7 @@ static int rtc_post_load(void *opaque, int version_id)
>  
>  static const VMStateDescription vmstate_rtc = {
>      .name = "mc146818rtc",
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 1,
>      .post_load = rtc_post_load,
>      .fields = (VMStateField[]) {
> @@ -752,6 +752,7 @@ static const VMStateDescription vmstate_rtc = {
>          VMSTATE_INT64_V(offset, RTCState, 3),
>          VMSTATE_TIMER_V(update_timer, RTCState, 3),
>          VMSTATE_UINT64_V(next_alarm_time, RTCState, 3),
> +        VMSTATE_UINT16_V(irq_reinject_on_ack_count, RTCState, 4),

Also can be a subsection, migrated only if nonzero.

Paolo

>          VMSTATE_END_OF_LIST()
>      }
>  };
> 
> 
> 

Re: [Qemu-devel] [RFC PATCH v2 04/49] fdc: adding vmstate for save/restore

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> VMState added by this patch preserves correct
> loading of the FDC device state.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/block/fdc.c |   11 +++++++++--
>  1 files changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/block/fdc.c b/hw/block/fdc.c
> index 490d127..132310a 100644
> --- a/hw/block/fdc.c
> +++ b/hw/block/fdc.c
> @@ -697,12 +697,17 @@ static const VMStateDescription vmstate_fdrive_media_rate = {
>  
>  static const VMStateDescription vmstate_fdrive = {
>      .name = "fdrive",
> -    .version_id = 1,
> +    .version_id = 2,
>      .minimum_version_id = 1,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT8(head, FDrive),
>          VMSTATE_UINT8(track, FDrive),
>          VMSTATE_UINT8(sect, FDrive),
> +        VMSTATE_UINT8_V(last_sect, FDrive, 2),
> +        VMSTATE_UINT8_V(max_track, FDrive, 2),
> +        VMSTATE_UINT16_V(bps, FDrive, 2),
> +        VMSTATE_UINT8_V(ro, FDrive, 2),
> +        VMSTATE_UINT8_V(perpendicular, FDrive, 2),

Perpendicular can be added to a subsection, migrated only if nonzero.
The others can be reconstructed by calling fd_revalidate in
vmstate_fdrive's post_load callback.


>          VMSTATE_END_OF_LIST()
>      },
>      .subsections = (VMStateSubsection[]) {
> @@ -736,7 +741,7 @@ static int fdc_post_load(void *opaque, int version_id)
>  
>  static const VMStateDescription vmstate_fdc = {
>      .name = "fdc",
> -    .version_id = 2,
> +    .version_id = 3,
>      .minimum_version_id = 2,
>      .pre_save = fdc_pre_save,
>      .post_load = fdc_post_load,
> @@ -769,6 +774,8 @@ static const VMStateDescription vmstate_fdc = {
>          VMSTATE_UINT8_EQUAL(num_floppies, FDCtrl),
>          VMSTATE_STRUCT_ARRAY(drives, FDCtrl, MAX_FD, 1,
>                               vmstate_fdrive, FDrive),
> +        VMSTATE_INT32_V(reset_sensei, FDCtrl, 3),

Subsection, only migrated if nonzero.

> +        VMSTATE_TIMER_V(result_timer, FDCtrl, 3),

Subsection, only migrated if pending.

Paolo

>          VMSTATE_END_OF_LIST()
>      }
>  };
> 
> 
> 

Re: [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> > This patch adds virtual clock-dependent timers to VMState to allow correct
> > saving and restoring the state of RTL8139 network controller.
> >
> > Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> > ---
> >  hw/net/rtl8139.c |    5 ++++-
> >  1 files changed, 4 insertions(+), 1 deletions(-)
> >
> > diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c
> > index 90bc5ec..992caf0 100644
> > --- a/hw/net/rtl8139.c
> > +++ b/hw/net/rtl8139.c
> > @@ -3289,7 +3289,7 @@ static void rtl8139_pre_save(void *opaque)
> >
> >  static const VMStateDescription vmstate_rtl8139 = {
> >      .name = "rtl8139",
> > -    .version_id = 4,
> > +    .version_id = 5,
> >      .minimum_version_id = 3,
> >      .post_load = rtl8139_post_load,
> >      .pre_save  = rtl8139_pre_save,
> > @@ -3363,6 +3363,9 @@ static const VMStateDescription vmstate_rtl8139 = {
> >          VMSTATE_STRUCT(tally_counters, RTL8139State, 0,
> >                         vmstate_tally_counters, RTL8139TallyCounters),
> >
> > +        VMSTATE_TIMER_V(timer, RTL8139State, 5),
> 
> timer need not be migrated, because it is reinstated by rtl8139_post_load.
> 

  That's true for normal execution.
  In replay execution mode post_load can be called before cached virtual clock
values are loaded. This may cause invalid setting of the timer and raising
an IRQ which didn't happen in record mode.
  I will update this patch to fix post_load function and avoid this 
non-deterministic behavior.

Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 2,
>      .pre_save = serial_pre_save,
>      .post_load = serial_post_load,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT16_V(divider, SerialState, 2),
>          VMSTATE_UINT8(rbr, SerialState),
> +        VMSTATE_UINT8_V(thr, SerialState, 4),
> +        VMSTATE_UINT8_V(tsr, SerialState, 4),
>          VMSTATE_UINT8(ier, SerialState),
>          VMSTATE_UINT8(iir, SerialState),
>          VMSTATE_UINT8(lcr, SerialState),
> @@ -613,6 +627,15 @@ const VMStateDescription vmstate_serial = {
>          VMSTATE_UINT8(msr, SerialState),
>          VMSTATE_UINT8(scr, SerialState),
>          VMSTATE_UINT8_V(fcr_vmstate, SerialState, 3),
> +        VMSTATE_INT32_V(thr_ipending, SerialState, 4),

Subsection, only migrated if it doesn't match "(s->iir & UART_IIR_ID) ==
UART_IIR_THRI".

> +        VMSTATE_INT32_V(last_break_enable, SerialState, 4),

Can be reconstructed in the post_load callback from s->lcr.

> +        VMSTATE_INT32_V(tsr_retry, SerialState, 4),

Subsection, only migrated if nonzero.  thr/tsr can be in this subsection
as well.

> +        VMSTATE_STRUCT(recv_fifo, SerialState, 4, vmstate_fifo8, Fifo8),
> +        VMSTATE_STRUCT(xmit_fifo, SerialState, 4, vmstate_fifo8, Fifo8),

Two subsections, only transmitted if nonempty.

> +        VMSTATE_TIMER_V(fifo_timeout_timer, SerialState, 4),

Subsection, only transmitted if pending.

> +        VMSTATE_INT32_V(timeout_ipending, SerialState, 4),

Subsection, transmitted only if nonzero.

> +        VMSTATE_INT32_V(poll_msl, SerialState, 4),
> +        VMSTATE_TIMER_V(modem_status_poll, SerialState, 4),

Both in a subsection, only migrated if poll_msl is not -1.

Paolo

>          VMSTATE_END_OF_LIST()

Re: [Qemu-devel] [RFC PATCH v2 05/49] parallel: adding vmstate for save/restore

[Paolo Bonzini]

Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> +    .fields      = (VMStateField []) {
> +        VMSTATE_UINT8(state.dataw, ISAParallelState),
> +        VMSTATE_UINT8(state.datar, ISAParallelState),
> +        VMSTATE_UINT8(state.status, ISAParallelState),
> +        VMSTATE_UINT8(state.control, ISAParallelState),
> +        VMSTATE_INT32(state.irq_pending, ISAParallelState),
> +        VMSTATE_INT32(state.hw_driver, ISAParallelState),

Static, doesn't need migration.

> +        VMSTATE_INT32(state.epp_timeout, ISAParallelState),
> +        VMSTATE_INT32(state.it_shift, ISAParallelState),

Static, doesn't need migration.

> +        VMSTATE_END_OF_LIST()

Re: [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Paolo Bonzini]

Il 28/07/2014 11:54, Pavel Dovgaluk ha scritto:
>>> > > +        VMSTATE_TIMER_V(timer, RTL8139State, 5),
>> > 
>> > timer need not be migrated, because it is reinstated by rtl8139_post_load.
>> > 
>   That's true for normal execution.
>   In replay execution mode post_load can be called before cached virtual clock
> values are loaded. This may cause invalid setting of the timer and raising
> an IRQ which didn't happen in record mode.
>   I will update this patch to fix post_load function and avoid this 
> non-deterministic behavior.

This is what worries me of this series.  These invariants are not
documented anywhere, and people will break them unless you add
assertions that also hold in normal mode.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Paolo Bonzini]

Il 28/07/2014 09:50, Pavel Dovgaluk ha scritto:
>> - patches 2 to 13 probably should try to use subsections, so that VMs
>> that do not use the devices try not to save the extra data and keep
>> backwards migration compatibility (at least try to)
> 
>  Could you give me and example?
>  As I know, subsection is loaded when some predicate function returns true. How can I 
> construct such a function for integratorcp module? What kind of condition should it check?
>  In this module I just added missed vmstates (it does not saved/restored at all
> by the master version).

I tried to review the x86 ones.  These are the most interesting ones,
because it's the only board where we support cross-version migration.

Parallel is (quite obviously) unsalvageable, and I think HPET is too.

>> - patch 16 should also use subsections, and perhaps apply to all other
>> CPUs too?
> 
>  We implemented replay only for i386 and ARM. If we'll change other targets, it will not
> add record/replay capabilities to them, but can confuse the reviewers.

Why are these fields only required by record/replay?

>> - patches 23-24-25 perhaps could try using icount, like Konrad's patch do?
> 
>  Using faster icount (like in Konrad's patches) is the our next aim. It obviously will 
> increase the speed of recording process. But now I submitted slower, but more conservative
> version of icount which we had already tested.

I see.

>> - patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
>> is stopped
> 
>  Right, this is kind of hack. I haven't found better solution yet.

I think it's okay to use REALTIME, just add runstate_is_running() to the
"if (now >= s->cursor_blink_time)".

That said, "every read to virtual clock is written to the
replay log" worries me a bit from the point of view of thread-safety.
Do you need to log all reads because you don't use icount?  Reads can
only happen at given points if you use icount, and you could simply log
the icount value at each synchronization point.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 07/49] kvmapic: fixing loading vmstate

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> > diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> > index ce3d903..9d75ee0 100644
> > --- a/hw/intc/apic_common.c
> > +++ b/hw/intc/apic_common.c
> > @@ -347,7 +347,7 @@ static int apic_dispatch_post_load(void *opaque, int version_id)
> >
> >  static const VMStateDescription vmstate_apic_common = {
> >      .name = "apic",
> > -    .version_id = 3,
> > +    .version_id = 4,
> >      .minimum_version_id = 3,
> >      .minimum_version_id_old = 1,
> >      .load_state_old = apic_load_old,
> > @@ -374,6 +374,9 @@ static const VMStateDescription vmstate_apic_common = {
> >          VMSTATE_INT64(next_time, APICCommonState),
> >          VMSTATE_INT64(timer_expiry,
> >                        APICCommonState), /* open-coded timer state */
> > +        VMSTATE_INT32_V(sipi_vector, APICCommonState, 4),
> > +        VMSTATE_INT32_V(wait_for_sipi, APICCommonState, 4),
> 
> This could be a subsection.  sipi_vector is only used (needed) if wait_for_sipi != 0.

  Right, sipi_vector is used when wait_for_sipi != 0. But we can set sipi_vector to non-zero,
save the snapshot, and then set wait_for_sipi. If that snapshot will be loaded, sipi_vector become incorrect.
Isn't this scenario possible?

Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 07/49] kvmapic: fixing loading vmstate

[Paolo Bonzini]

Il 29/07/2014 14:03, Pavel Dovgaluk ha scritto:
>> > 
>> > This could be a subsection.  sipi_vector is only used (needed) if wait_for_sipi != 0.
>   Right, sipi_vector is used when wait_for_sipi != 0. But we can set sipi_vector to non-zero,
> save the snapshot, and then set wait_for_sipi. If that snapshot will be loaded, sipi_vector become incorrect.
> Isn't this scenario possible?

sipi_vector will not be used until CPU_INTERRUPT_SIPI is set, and then
sipi_vector will have been overwritten with a new value.  The
architecture guarantees that.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore

[Pavel Dovgaluk]

> -----Original Message-----
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Sent: Monday, July 28, 2014 1:59 PM
> To: Pavel Dovgalyuk; qemu-devel@nongnu.org
> Cc: peter.maydell@linaro.org; peter.crosthwaite@xilinx.com; mark.burton@greensocs.com;
> real@ispras.ru; batuzovk@ispras.ru; fred.konrad@greensocs.com
> Subject: Re: [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore
> 
> Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
> > -    .version_id = 3,
> > +    .version_id = 4,
> >      .minimum_version_id = 2,
> >      .pre_save = serial_pre_save,
> >      .post_load = serial_post_load,
> >      .fields = (VMStateField[]) {
> >          VMSTATE_UINT16_V(divider, SerialState, 2),
> >          VMSTATE_UINT8(rbr, SerialState),
> > +        VMSTATE_UINT8_V(thr, SerialState, 4),
> > +        VMSTATE_UINT8_V(tsr, SerialState, 4),
> >          VMSTATE_UINT8(ier, SerialState),
> >          VMSTATE_UINT8(iir, SerialState),
> >          VMSTATE_UINT8(lcr, SerialState),
> > @@ -613,6 +627,15 @@ const VMStateDescription vmstate_serial = {
> >          VMSTATE_UINT8(msr, SerialState),
> >          VMSTATE_UINT8(scr, SerialState),
> >          VMSTATE_UINT8_V(fcr_vmstate, SerialState, 3),
> > +        VMSTATE_INT32_V(thr_ipending, SerialState, 4),
> 
> Subsection, only migrated if it doesn't match "(s->iir & UART_IIR_ID) ==
> UART_IIR_THRI".

 thr_ipending is set here:
    if (s->lsr & UART_LSR_THRE) {
        s->lsr |= UART_LSR_TEMT;
        s->thr_ipending = 1;
        serial_update_irq(s);
    }

 serial_update_irq has several if-else branches. One of them sets tmp_iir = UART_IIR_THRI, as you said.
Couldn't be possible, that this branch will not be reached, because one of the previous ones will be executed?

Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> >> - patch 16 should also use subsections, and perhaps apply to all other
> >> CPUs too?
> >
> >  We implemented replay only for i386 and ARM. If we'll change other targets, it will not
> > add record/replay capabilities to them, but can confuse the reviewers.
> 
> Why are these fields only required by record/replay?

No, these fields can cause non-determinism for normal execution mode too.
But these changes are not related to reverse execution. I can make patches for them
and submit them separately, if needed.

> >> - patches 23-24-25 perhaps could try using icount, like Konrad's patch do?
> >
> >  Using faster icount (like in Konrad's patches) is the our next aim. It obviously will
> > increase the speed of recording process. But now I submitted slower, but more conservative
> > version of icount which we had already tested.
> 
> I see.
> 
> >> - patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
> >> is stopped
> >
> >  Right, this is kind of hack. I haven't found better solution yet.
> 
> I think it's okay to use REALTIME, just add runstate_is_running() to the
> "if (now >= s->cursor_blink_time)".
> 
> That said, "every read to virtual clock is written to the
> replay log" worries me a bit from the point of view of thread-safety.
> Do you need to log all reads because you don't use icount?  Reads can
> only happen at given points if you use icount, and you could simply log
> the icount value at each synchronization point.

We made two implementations of virtual clock. One uses host realtime clock
and thus should be written into the log. Another one uses icount and calculated
in one of the replay modules (similar to regular icount implementation).
This patch series contains only first implementation, because it looks more convenient:
user does not have to specify icount shift value, when starting the recording or replaying.

Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> Il 28/07/2014 11:54, Pavel Dovgaluk ha scritto:
> >>> > > +        VMSTATE_TIMER_V(timer, RTL8139State, 5),
> >> >
> >> > timer need not be migrated, because it is reinstated by rtl8139_post_load.
> >> >
> >   That's true for normal execution.
> >   In replay execution mode post_load can be called before cached virtual clock
> > values are loaded. This may cause invalid setting of the timer and raising
> > an IRQ which didn't happen in record mode.
> >   I will update this patch to fix post_load function and avoid this
> > non-deterministic behavior.
> 
> This is what worries me of this series.  These invariants are not
> documented anywhere, and people will break them unless you add
> assertions that also hold in normal mode.

Assertions is a good idea, we added such warning message to qemu_get_timedate function
to be sure, that it is used correctly with replay.

Another thing, that could help for making snapshots - find a way to load replay structures
before all other ones. Are there any priorities in migration states list?
Priorities could also solve some other issues, because sometimes post_load function
of one device uses other devices' functions. And the second ones could be not loaded yet.

Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore

[Paolo Bonzini]

Il 30/07/2014 09:01, Pavel Dovgaluk ha scritto:
> 
>> -----Original Message-----
>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
>> Sent: Monday, July 28, 2014 1:59 PM
>> To: Pavel Dovgalyuk; qemu-devel@nongnu.org
>> Cc: peter.maydell@linaro.org; peter.crosthwaite@xilinx.com; mark.burton@greensocs.com;
>> real@ispras.ru; batuzovk@ispras.ru; fred.konrad@greensocs.com
>> Subject: Re: [RFC PATCH v2 06/49] serial: fixing vmstate for save/restore
>>
>> Il 17/07/2014 13:02, Pavel Dovgalyuk ha scritto:
>>> -    .version_id = 3,
>>> +    .version_id = 4,
>>>      .minimum_version_id = 2,
>>>      .pre_save = serial_pre_save,
>>>      .post_load = serial_post_load,
>>>      .fields = (VMStateField[]) {
>>>          VMSTATE_UINT16_V(divider, SerialState, 2),
>>>          VMSTATE_UINT8(rbr, SerialState),
>>> +        VMSTATE_UINT8_V(thr, SerialState, 4),
>>> +        VMSTATE_UINT8_V(tsr, SerialState, 4),
>>>          VMSTATE_UINT8(ier, SerialState),
>>>          VMSTATE_UINT8(iir, SerialState),
>>>          VMSTATE_UINT8(lcr, SerialState),
>>> @@ -613,6 +627,15 @@ const VMStateDescription vmstate_serial = {
>>>          VMSTATE_UINT8(msr, SerialState),
>>>          VMSTATE_UINT8(scr, SerialState),
>>>          VMSTATE_UINT8_V(fcr_vmstate, SerialState, 3),
>>> +        VMSTATE_INT32_V(thr_ipending, SerialState, 4),
>>
>> Subsection, only migrated if it doesn't match "(s->iir & UART_IIR_ID) ==
>> UART_IIR_THRI".
> 
>  thr_ipending is set here:
>     if (s->lsr & UART_LSR_THRE) {
>         s->lsr |= UART_LSR_TEMT;
>         s->thr_ipending = 1;
>         serial_update_irq(s);
>     }
> 
>  serial_update_irq has several if-else branches. One of them sets tmp_iir = UART_IIR_THRI, as you said.
> Couldn't be possible, that this branch will not be reached, because one of the previous ones will be executed?

Yes, what I wrote is equivalent to the following:

int serial_pre_load(void *opaque)
{
    SerialState *s = opaque;
    s->thr_ipending = -1;
    return 0;
}

int serial_post_load(void *opaque)
{
    SerialState *s = opaque;
    ...
    if (s->thr_ipending == -1) {
        s->thr_ipending = ((s->iir & UART_IIR_ID) == UART_IIR_THRI);
    }
    ...
}

bool thr_ipending_needed(void *opaque)
{
    SerialState *s = opaque;
    bool expected_value = ((s->iir & UART_IIR_ID) == UART_IIR_THRI);
    return s->thr_pending != expected_value;
}

The case you mention is exactly the one that will cause thr_ipending to
be migrated.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Paolo Bonzini]

Il 30/07/2014 09:44, Pavel Dovgaluk ha scritto:
>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
>>>> - patch 16 should also use subsections, and perhaps apply to all other
>>>> CPUs too?
>>>
>>>  We implemented replay only for i386 and ARM. If we'll change other targets, it will not
>>> add record/replay capabilities to them, but can confuse the reviewers.
>>
>> Why are these fields only required by record/replay?
> 
> No, these fields can cause non-determinism for normal execution mode too.
> But these changes are not related to reverse execution. I can make patches for them
> and submit them separately, if needed.

Yes, thanks.  If it causes nondeterminism for normal execution mode too,
it's better to apply them to all targets.  If possible use a subsection,
though.

>>>> - patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
>>>> is stopped
>>>
>>>  Right, this is kind of hack. I haven't found better solution yet.
>>
>> I think it's okay to use REALTIME, just add runstate_is_running() to the
>> "if (now >= s->cursor_blink_time)".
>>
>> That said, "every read to virtual clock is written to the
>> replay log" worries me a bit from the point of view of thread-safety.
>> Do you need to log all reads because you don't use icount?  Reads can
>> only happen at given points if you use icount, and you could simply log
>> the icount value at each synchronization point.
> 
> We made two implementations of virtual clock. One uses host realtime clock
> and thus should be written into the log. Another one uses icount and calculated
> in one of the replay modules (similar to regular icount implementation).
> This patch series contains only first implementation, because it looks more convenient:
> user does not have to specify icount shift value, when starting the recording or replaying.

I think from the upstreaming point of view it's better to stick with
what makes the code simpler.  Start by submitting only the icount-based
implementation, the other can come later.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 10/49] rtl8139: adding new fields to vmstate

[Paolo Bonzini]

Il 30/07/2014 10:24, Pavel Dovgaluk ha scritto:
> Assertions is a good idea, we added such warning message to qemu_get_timedate function
> to be sure, that it is used correctly with replay.
> 
> Another thing, that could help for making snapshots - find a way to load replay structures
> before all other ones. Are there any priorities in migration states list?

No, it is just about vmstate registration order.  In general bus parents
come before bus children for obvious reasons.

Paolo

> Priorities could also solve some other issues, because sometimes post_load function
> of one device uses other devices' functions. And the second ones could be not loaded yet.

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Frederic Konrad]

On 30/07/2014 11:25, Paolo Bonzini wrote:
> Il 30/07/2014 09:44, Pavel Dovgaluk ha scritto:
>>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
>>>>> - patch 16 should also use subsections, and perhaps apply to all other
>>>>> CPUs too?
>>>>   We implemented replay only for i386 and ARM. If we'll change other targets, it will not
>>>> add record/replay capabilities to them, but can confuse the reviewers.
>>> Why are these fields only required by record/replay?
>> No, these fields can cause non-determinism for normal execution mode too.
>> But these changes are not related to reverse execution. I can make patches for them
>> and submit them separately, if needed.
> Yes, thanks.  If it causes nondeterminism for normal execution mode too,
> it's better to apply them to all targets.  If possible use a subsection,
> though.
>
>>>>> - patch 27 makes sense but VIRTUAL is used to skip blinking when the VM
>>>>> is stopped
>>>>   Right, this is kind of hack. I haven't found better solution yet.
>>> I think it's okay to use REALTIME, just add runstate_is_running() to the
>>> "if (now >= s->cursor_blink_time)".
>>>
>>> That said, "every read to virtual clock is written to the
>>> replay log" worries me a bit from the point of view of thread-safety.
>>> Do you need to log all reads because you don't use icount?  Reads can
>>> only happen at given points if you use icount, and you could simply log
>>> the icount value at each synchronization point.
>> We made two implementations of virtual clock. One uses host realtime clock
>> and thus should be written into the log. Another one uses icount and calculated
>> in one of the replay modules (similar to regular icount implementation).
>> This patch series contains only first implementation, because it looks more convenient:
>> user does not have to specify icount shift value, when starting the recording or replaying.
> I think from the upstreaming point of view it's better to stick with
> what makes the code simpler.  Start by submitting only the icount-based
> implementation, the other can come later.
>
> Paolo

Hi all,

I think that's actually our implementation cover no?
(http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00677.html)

Fred

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Paolo Bonzini]

Il 30/07/2014 15:19, Frederic Konrad ha scritto:
>> Start by submitting only the icount-based
>> implementation, the other can come later.
>>
>> Paolo
> 
> Hi all,
> 
> I think that's actually our implementation cover no?
> (http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00677.html)

Yes, Pavel's series covers the missing bits of yours AFAIU.

Paolo

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Frederic Konrad]

On 30/07/2014 15:35, Paolo Bonzini wrote:
> Il 30/07/2014 15:19, Frederic Konrad ha scritto:
>>> Start by submitting only the icount-based
>>> implementation, the other can come later.
>>>
>>> Paolo
>> Hi all,
>>
>> I think that's actually our implementation cover no?
>> (http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00677.html)
> Yes, Pavel's series covers the missing bits of yours AFAIU.
>
> Paolo

I think that our first patch set part (icount fixes and icount based 
clock) is fully reviewed and can be upstreamed,
but since the reverse execution mechanism itself is very similar between 
our series and Pavels, while Pavel’s
set goes further in terms of addressing IO replay,  I suggest Pavel uses 
our icount counter with his mechanism.

In addition maybe this series can be split eg:
the migration related series/the mechanism itself so it will be easier 
for me to review your mechanism?

Thanks,
Fred

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Pavel Dovgaluk]

> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> Il 30/07/2014 09:44, Pavel Dovgaluk ha scritto:
> >> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> >>>> - patch 16 should also use subsections, and perhaps apply to all other
> >>>> CPUs too?
> >>>
> >>>  We implemented replay only for i386 and ARM. If we'll change other targets, it will not
> >>> add record/replay capabilities to them, but can confuse the reviewers.
> >>
> >> Why are these fields only required by record/replay?
> >
> > No, these fields can cause non-determinism for normal execution mode too.
> > But these changes are not related to reverse execution. I can make patches for them
> > and submit them separately, if needed.
> 
> Yes, thanks.  If it causes nondeterminism for normal execution mode too,
> it's better to apply them to all targets.  If possible use a subsection,
> though.

I've sent this change as a separate patch.
Instead of changing every virtual machine's vmstate, I added exception_index to the common
exec.c module.
 
Pavel Dovgalyuk

Re: [Qemu-devel] [RFC PATCH v2 16/49] target: save cpu state fields

[Andreas Färber]

Am 17.07.2014 13:03, schrieb Pavel Dovgalyuk:
> This patch adds interrupt fields to VMState for correct saving the CPU state.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  target-arm/machine.c  |    5 ++++-
>  target-i386/machine.c |    5 ++++-
>  2 files changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/target-arm/machine.c b/target-arm/machine.c
> index 3bcc7cc..29bfc10 100644
> --- a/target-arm/machine.c
> +++ b/target-arm/machine.c
> @@ -218,7 +218,7 @@ static int cpu_post_load(void *opaque, int version_id)
>  
>  const VMStateDescription vmstate_arm_cpu = {
>      .name = "cpu",
> -    .version_id = 20,
> +    .version_id = 21,
>      .minimum_version_id = 20,
>      .pre_save = cpu_pre_save,
>      .post_load = cpu_post_load,
> @@ -259,6 +259,9 @@ const VMStateDescription vmstate_arm_cpu = {
>          VMSTATE_UINT64(env.exception.vaddress, ARMCPU),
>          VMSTATE_TIMER(gt_timer[GTIMER_PHYS], ARMCPU),
>          VMSTATE_TIMER(gt_timer[GTIMER_VIRT], ARMCPU),
> +        /* Fields required by replay */
> +        VMSTATE_UINT32_V(parent_obj.interrupt_request, ARMCPU, 21),
> +        VMSTATE_INT32_V(parent_obj.exception_index, ARMCPU, 21),
>          VMSTATE_END_OF_LIST()
>      },
>      .subsections = (VMStateSubsection[]) {
> diff --git a/target-i386/machine.c b/target-i386/machine.c
> index 16d2f6a..9dfac33 100644
> --- a/target-i386/machine.c
> +++ b/target-i386/machine.c
> @@ -605,7 +605,7 @@ static const VMStateDescription vmstate_msr_hyperv_time = {
>  
>  VMStateDescription vmstate_x86_cpu = {
>      .name = "cpu",
> -    .version_id = 12,
> +    .version_id = 13,
>      .minimum_version_id = 3,
>      .pre_save = cpu_pre_save,
>      .post_load = cpu_post_load,
> @@ -702,6 +702,9 @@ VMStateDescription vmstate_x86_cpu = {
>          VMSTATE_UINT64_V(env.xcr0, X86CPU, 12),
>          VMSTATE_UINT64_V(env.xstate_bv, X86CPU, 12),
>          VMSTATE_YMMH_REGS_VARS(env.ymmh_regs, X86CPU, CPU_NB_REGS, 12),
> +        /* Fields required by replay */
> +        VMSTATE_UINT32_V(parent_obj.interrupt_request, X86CPU, 13),
> +        VMSTATE_INT32_V(parent_obj.exception_index, X86CPU, 13),
>          VMSTATE_END_OF_LIST()
>          /* The above list is not sorted /wrt version numbers, watch out! */
>      },

Instead of messing with parent_obj here and above, can you please add
common CPUState stuff to vmstate_cpu_common in exec.c?

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Frederic Konrad]

On 30/07/2014 16:51, Frederic Konrad wrote:
> On 30/07/2014 15:35, Paolo Bonzini wrote:
>> Il 30/07/2014 15:19, Frederic Konrad ha scritto:
>>>> Start by submitting only the icount-based
>>>> implementation, the other can come later.
>>>>
>>>> Paolo
>>> Hi all,
>>>
>>> I think that's actually our implementation cover no?
>>> (http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00677.html)
>> Yes, Pavel's series covers the missing bits of yours AFAIU.
>>
>> Paolo
>
> I think that our first patch set part (icount fixes and icount based 
> clock) is fully reviewed and can be upstreamed,
> but since the reverse execution mechanism itself is very similar 
> between our series and Pavels, while Pavel’s
> set goes further in terms of addressing IO replay,  I suggest Pavel 
> uses our icount counter with his mechanism.
>
> In addition maybe this series can be split eg:
> the migration related series/the mechanism itself so it will be easier 
> for me to review your mechanism?
>
> Thanks,
> Fred
>
Is that making sense?

Fred

Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description

[Paolo Bonzini]

Il 31/07/2014 15:05, Frederic Konrad ha scritto:
>>>
>>
>> I think that our first patch set part (icount fixes and icount based
>> clock) is fully reviewed and can be upstreamed,
>> but since the reverse execution mechanism itself is very similar
>> between our series and Pavels, while Pavel’s
>> set goes further in terms of addressing IO replay,  I suggest Pavel
>> uses our icount counter with his mechanism.
>>
>> In addition maybe this series can be split eg:
>> the migration related series/the mechanism itself so it will be easier
>> for me to review your mechanism?
>
> Is that making sense?

Yes, definitely.  If you rebase reverse execution on top of branch
uq/master of git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git I can
include it in the same pull request as icount alignment (which is now
ready for inclusion in 2.2).

Paolo