[Qemu-devel] [RFC V7 00/10] Reverse execution.

[Qemu-devel] [RFC V7 00/10] Reverse execution.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

Hi everybody,

This is the seventh version of this RFC (see the changes below).

The first patches:
  migration: make qemu_savevm_state public.
  icount: introduce icount timer.
  icount: check for icount clock deadline when cpu loop exits.
  icount: make icount extra computed on icount clock as well.

are various preparation patches for reverse execution it creates the icount
clock.

The last patches:
  trace-events: add reverse-execution events.
  introduce reverse execution mechanism.
  gdbstub: allow reverse execution in gdb stub.
  cpu-exec: trigger a debug request when rexec stops.
  rexec: synchronize icount on the next event.
  rexec: allow to enable reverse execution.

are reverse execution introduction.

They can be clone at: git://git.greensocs.com/qemu_cexe.git:rexec_v7

This implementation of reverse execution works with instruction counting:

A new clock is implemented which is icount clock. It grows each time an
instruction is executed and is totally independant of host clock.

Snapshots are taken regularly (based on icount clock) with help of migration
code and written on the disk.

When user wants to use reverse-stepi:
 * Last snapshot is reloaded.
 * A stop callback is created to be triggered at the previous instruction.

This stop callback generates a debug exception so QEMU stops in debug mode.

Command line:
 * rexec suboption is added to icount to enable reverse execution, it needs
   icount=N and doesn't support auto mode.

About non determinism in QEMU:
 * This implementation doesn't take IO in account so any IO will cause non
   determinism and break reverse execution.

 * The icount warp mechanism have been disabled when reverse execution is
   enabled so the time grow differently inside the VM.

Testing:
 * It has been tested on ARM without any IO such as network or asynchronous file
   access to keep the deterministic behaviour of icount.

Known issues:
 * On ARM stepi seems to do some additional steps which are added to icount
   counter so reverse-stepi just after stepi is broken.

 * The IO replay explained above.

Changes:
 v6 -> v7:
  * Rebase.
 v5 -> v6:
  * Boolize some variable (As suggested by Li Guang).
  * s/snap_timer/snapshot_timer/g (As suggested by Li Guang).
  * Choice a better name for the snapshot file. (As suggested by Li Guang).
  * Boolize stepping_back, etc.. (As suggested by Li Guang).
  * Cherry-pick icount: Add QemuOpts for icount. (As suggested by Paolo).
  * Making reverse execution option using QEMUOpts (As sugested by Paolo).
  * s/cexe/rexec/g (As nobody like cexe :)).
 
 v4 -> v5:
  * Fix a coding style issue (As suggested by Juan).

 v3 -> v4:
  * Fix icount_state_needed (As suggested by Amit).
  * Rebase.

 v2 -> v3:
  * Use trace instead of debug printfs (As suggested by Lluis).

 v1 -> v2:
  * Use subsection for icount migration (As suggested by Paolo).

  * Use with_bias parameters to get_icount instead of get_icount_wo_bias
    function (As suggested by Paolo).


KONRAD Frederic (10):
  migration: make qemu_savevm_state public.
  icount: introduce icount timer.
  icount: check for icount clock deadline when cpu loop exits.
  icount: make icount extra computed on icount clock as well.
  trace-events: add reverse-execution events.
  introduce reverse execution mechanism.
  gdbstub: allow reverse execution in gdb stub.
  cpu-exec: trigger a debug request when rexec stops.
  rexec: synchronize icount on the next event.
  rexec: allow to enable reverse execution.

 Makefile.target                |   1 +
 cpu-exec.c                     |  13 ++
 cpus.c                         |  76 ++++++++--
 gdbstub.c                      |  31 ++++-
 include/qemu/timer.h           |  17 ++-
 include/reverse-execution.h    |  43 ++++++
 include/sysemu/sysemu.h        |   1 +
 main-loop.c                    |  10 ++
 qemu-options.hx                |   5 +-
 qemu-timer.c                   |   8 +-
 reverse-execution.c            | 308 +++++++++++++++++++++++++++++++++++++++++
 savevm.c                       |   2 +-
 stubs/Makefile.objs            |   1 +
 stubs/cpu-get-icount.c         |  10 +-
 stubs/reverse-execution-stub.c |  32 +++++
 trace-events                   |   6 +
 vl.c                           |  16 ++-
 17 files changed, 561 insertions(+), 19 deletions(-)
 create mode 100644 include/reverse-execution.h
 create mode 100644 reverse-execution.c
 create mode 100644 stubs/reverse-execution-stub.c

-- 
1.9.0

[Qemu-devel] [RFC V7 01/10] migration: make qemu_savevm_state public.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This makes qemu_savevm_state public for reverse-execution.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
---
 include/sysemu/sysemu.h | 1 +
 savevm.c                | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index d8539fd..bc52525 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -81,6 +81,7 @@ void do_info_snapshots(Monitor *mon, const QDict *qdict);
 
 void qemu_announce_self(void);
 
+int qemu_savevm_state(QEMUFile *f);
 bool qemu_savevm_state_blocked(Error **errp);
 void qemu_savevm_state_begin(QEMUFile *f,
                              const MigrationParams *params);
diff --git a/savevm.c b/savevm.c
index e19ae0a..0286f9b 100644
--- a/savevm.c
+++ b/savevm.c
@@ -779,7 +779,7 @@ void qemu_savevm_state_cancel(void)
     }
 }
 
-static int qemu_savevm_state(QEMUFile *f)
+int qemu_savevm_state(QEMUFile *f)
 {
     int ret;
     MigrationParams params = {
-- 
1.9.0

[Qemu-devel] [RFC V7 02/10] icount: introduce icount timer.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This introduces a new timer based only on instruction counter and without any
compensation.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 cpus.c                 | 21 +++++++++++++--------
 include/qemu/timer.h   |  9 ++++++++-
 qemu-timer.c           |  8 +++++++-
 stubs/cpu-get-icount.c |  2 +-
 4 files changed, 29 insertions(+), 11 deletions(-)

diff --git a/cpus.c b/cpus.c
index 0f7d0ea..0ae6798 100644
--- a/cpus.c
+++ b/cpus.c
@@ -137,7 +137,7 @@ typedef struct TimersState {
 static TimersState timers_state;
 
 /* Return the virtual CPU time, based on the instruction counter.  */
-static int64_t cpu_get_icount_locked(void)
+static int64_t cpu_get_icount_locked(int with_bias)
 {
     int64_t icount;
     CPUState *cpu = current_cpu;
@@ -149,17 +149,22 @@ static int64_t cpu_get_icount_locked(void)
         }
         icount -= (cpu->icount_decr.u16.low + cpu->icount_extra);
     }
-    return timers_state.qemu_icount_bias + cpu_icount_to_ns(icount);
+
+    if (with_bias) {
+        return timers_state.qemu_icount_bias + cpu_icount_to_ns(icount);
+    } else {
+        return cpu_icount_to_ns(icount);
+    }
 }
 
-int64_t cpu_get_icount(void)
+int64_t cpu_get_icount(int with_bias)
 {
     int64_t icount;
     unsigned start;
 
     do {
         start = seqlock_read_begin(&timers_state.vm_clock_seqlock);
-        icount = cpu_get_icount_locked();
+        icount = cpu_get_icount_locked(with_bias);
     } while (seqlock_read_retry(&timers_state.vm_clock_seqlock, start));
 
     return icount;
@@ -177,7 +182,7 @@ int64_t cpu_get_ticks(void)
     int64_t ticks;
 
     if (use_icount) {
-        return cpu_get_icount();
+        return cpu_get_icount(true);
     }
 
     ticks = timers_state.cpu_ticks_offset;
@@ -292,7 +297,7 @@ static void icount_adjust(void)
 
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     cur_time = cpu_get_clock_locked();
-    cur_icount = cpu_get_icount_locked();
+    cur_icount = cpu_get_icount_locked(true);
 
     delta = cur_icount - cur_time;
     /* FIXME: This is a very crude algorithm, somewhat prone to oscillation.  */
@@ -355,7 +360,7 @@ static void icount_warp_rt(void *opaque)
              * far ahead of real time.
              */
             int64_t cur_time = cpu_get_clock_locked();
-            int64_t cur_icount = cpu_get_icount_locked();
+            int64_t cur_icount = cpu_get_icount_locked(true);
             int64_t delta = cur_time - cur_icount;
             warp_delta = MIN(warp_delta, delta);
         }
@@ -1555,7 +1560,7 @@ void dump_drift_info(FILE *f, fprintf_function cpu_fprintf)
     }
 
     cpu_fprintf(f, "Host - Guest clock  %"PRIi64" ms\n",
-                (cpu_get_clock() - cpu_get_icount())/SCALE_MS);
+                (cpu_get_clock() - cpu_get_icount(true))/SCALE_MS);
     if (icount_align_option) {
         cpu_fprintf(f, "Max guest delay     %"PRIi64" ms\n", -max_delay/SCALE_MS);
         cpu_fprintf(f, "Max guest advance   %"PRIi64" ms\n", max_advance/SCALE_MS);
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 5f5210d..16bafde 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -36,12 +36,19 @@
  * is suspended, and it will reflect system time changes the host may
  * undergo (e.g. due to NTP). The host clock has the same precision as
  * the virtual clock.
+ *
+ * @QEMU_CLOCK_ICOUNT: icount clock
+ *
+ * The icount clock is based on instruction counter without any compensation for
+ * speed. It will run only when instruction are executed and make only sense in
+ * icount mode.
  */
 
 typedef enum {
     QEMU_CLOCK_REALTIME = 0,
     QEMU_CLOCK_VIRTUAL = 1,
     QEMU_CLOCK_HOST = 2,
+    QEMU_CLOCK_ICOUNT = 3,
     QEMU_CLOCK_MAX
 } QEMUClockType;
 
@@ -743,7 +750,7 @@ static inline int64_t get_clock(void)
 #endif
 
 /* icount */
-int64_t cpu_get_icount(void);
+int64_t cpu_get_icount(int with_bias);
 int64_t cpu_get_clock(void);
 int64_t cpu_get_clock_offset(void);
 int64_t cpu_icount_to_ns(int64_t icount);
diff --git a/qemu-timer.c b/qemu-timer.c
index 00a5d35..3be80b1 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -554,7 +554,7 @@ int64_t qemu_clock_get_ns(QEMUClockType type)
     default:
     case QEMU_CLOCK_VIRTUAL:
         if (use_icount) {
-            return cpu_get_icount();
+            return cpu_get_icount(true);
         } else {
             return cpu_get_clock();
         }
@@ -566,6 +566,12 @@ int64_t qemu_clock_get_ns(QEMUClockType type)
             notifier_list_notify(&clock->reset_notifiers, &now);
         }
         return now;
+    case QEMU_CLOCK_ICOUNT:
+        if (use_icount) {
+            return cpu_get_icount(false);
+        } else {
+            return -1;
+        }
     }
 }
 
diff --git a/stubs/cpu-get-icount.c b/stubs/cpu-get-icount.c
index d685859..1968de7 100644
--- a/stubs/cpu-get-icount.c
+++ b/stubs/cpu-get-icount.c
@@ -3,7 +3,7 @@
 
 int use_icount;
 
-int64_t cpu_get_icount(void)
+int64_t cpu_get_icount(int with_bias)
 {
     abort();
 }
-- 
1.9.0

[Qemu-devel] [RFC V7 03/10] icount: check for icount clock deadline when cpu loop exits.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

Notify events on icount clock when CPU loop exits.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cpus.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/cpus.c b/cpus.c
index 0ae6798..2e9e2ea 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1030,6 +1030,11 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
             if (deadline == 0) {
                 qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
             }
+
+            deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_ICOUNT);
+            if (deadline == 0) {
+                qemu_clock_notify(QEMU_CLOCK_ICOUNT);
+            }
         }
         qemu_tcg_wait_io_event();
     }
-- 
1.9.0

[Qemu-devel] [RFC V7 04/10] icount: make icount extra computed on icount clock as well.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This takes icount clock in account for icount extra computation so icount
clock's timers will be triggered at the exact time.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cpus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/cpus.c b/cpus.c
index 2e9e2ea..834d8b9 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1325,6 +1325,7 @@ static int tcg_cpu_exec(CPUArchState *env)
     if (use_icount) {
         int64_t count;
         int64_t deadline;
+        int64_t icount_deadline;
         int decr;
         timers_state.qemu_icount -= (cpu->icount_decr.u16.low
                                     + cpu->icount_extra);
@@ -1341,6 +1342,15 @@ static int tcg_cpu_exec(CPUArchState *env)
             deadline = INT32_MAX;
         }
 
+        /*
+         * Take icount clock deadline in account too, and keep the nearest
+         * deadline.
+         */
+        icount_deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_ICOUNT);
+        if ((icount_deadline >= 0) && (icount_deadline < deadline)) {
+            deadline = icount_deadline;
+        }
+
         count = qemu_icount_round(deadline);
         timers_state.qemu_icount += count;
         decr = (count > 0xffff) ? 0xffff : count;
-- 
1.9.0

[Qemu-devel] [RFC V7 05/10] trace-events: add reverse-execution events.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This adds some trace-events for reverse execution.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 trace-events | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/trace-events b/trace-events
index fb58963..573dea9 100644
--- a/trace-events
+++ b/trace-events
@@ -1318,3 +1318,9 @@ mhp_pc_dimm_assigned_address(uint64_t addr) "0x%"PRIx64
 # target-s390x/kvm.c
 kvm_enable_cmma(int rc) "CMMA: enabling with result code %d"
 kvm_clear_cmma(int rc) "CMMA: clearing with result code %d"
+
+# reverse-execution.c
+snap_callback(uint64_t time, const char *filename) "snapshot @%li -> %s"
+stop_callback(uint64_t time) "stopping @%li"
+rexec_stop_reverse_continue(void) "stop reverse continue."
+rexec_step_backward(uint64_t time, uint64_t stop) "stepping backward @%li stop @%li"
-- 
1.9.0

[Qemu-devel] [RFC V7 06/10] introduce reverse execution mechanism.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This introduces the basic reverse-execution mechanism.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 Makefile.target             |   1 +
 cpus.c                      |   6 +
 include/reverse-execution.h |  41 ++++++
 reverse-execution.c         | 308 ++++++++++++++++++++++++++++++++++++++++++++
 vl.c                        |   7 +-
 5 files changed, 362 insertions(+), 1 deletion(-)
 create mode 100644 include/reverse-execution.h
 create mode 100644 reverse-execution.c

diff --git a/Makefile.target b/Makefile.target
index 1e8d7ab..8155d0c 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -127,6 +127,7 @@ endif #CONFIG_BSD_USER
 # System emulator target
 ifdef CONFIG_SOFTMMU
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o numa.o
+obj-y += reverse-execution.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
diff --git a/cpus.c b/cpus.c
index 834d8b9..39f4e98 100644
--- a/cpus.c
+++ b/cpus.c
@@ -64,6 +64,8 @@
 
 #endif /* CONFIG_LINUX */
 
+#include "reverse-execution.h"
+
 static CPUState *next_cpu;
 int64_t max_delay;
 int64_t max_advance;
@@ -625,7 +627,11 @@ static bool cpu_can_run(CPUState *cpu)
 
 static void cpu_handle_guest_debug(CPUState *cpu)
 {
+    if (rexec_is_continuing_backward()) {
+        rexec_step_done();
+    }
     gdb_set_stop_cpu(cpu);
+    rexec_stop_stepping_back_mode();
     qemu_system_debug_request();
     cpu->stopped = true;
 }
diff --git a/include/reverse-execution.h b/include/reverse-execution.h
new file mode 100644
index 0000000..9951549
--- /dev/null
+++ b/include/reverse-execution.h
@@ -0,0 +1,41 @@
+/*
+ *  reverse execution.
+ *
+ *  Copyright (C) 2014 : GreenSocs Ltd
+ *      http://www.greensocs.com/ , email: info@greensocs.com
+ *
+ *  Developed by :
+ *  Frederic Konrad   <fred.konrad@greensocs.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef REVERSE_EXECUTION
+#define REVERSE_EXECUTION
+
+void rexec_setup(void);
+void rexec_step_backward(CPUState *cpu, uint64_t steps);
+void rexec_stop_stepping_back_mode(void);
+void rexec_continue_backward(CPUState *cpu);
+int rexec_is_continuing_backward(void);
+void rexec_next_reverse_continue_step(void);
+void rexec_stop_reverse_continue(void);
+void rexec_step_done(void);
+bool rexec_is_step_done(void);
+bool rexec_is_enabled(void);
+void rexec_cleanup(void);
+bool rexec_dbg_requested(void);
+
+#endif /* REVERSE_EXECUTION */
diff --git a/reverse-execution.c b/reverse-execution.c
new file mode 100644
index 0000000..4c97f39
--- /dev/null
+++ b/reverse-execution.c
@@ -0,0 +1,308 @@
+/*
+ *  reverse execution.
+ *
+ *  Copyright (C) 2014 : GreenSocs Ltd
+ *      http://www.greensocs.com/ , email: info@greensocs.com
+ *
+ *  Developed by :
+ *  Frederic Konrad   <fred.konrad@greensocs.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/timer.h"
+#include "sysemu/sysemu.h"
+#include "migration/qemu-file.h"
+
+#include "reverse-execution.h"
+
+#include "trace.h"  /* needed for trace event prototype */
+
+#define SNAPSHOT_FILENAME "snapshot-step"
+
+typedef struct snapshot_entry {
+    uint32_t id;
+    int64_t time;
+    QLIST_ENTRY(snapshot_entry) next;
+} snapshot_entry;
+
+static QLIST_HEAD(, snapshot_entry) snapshot = QLIST_HEAD_INITIALIZER(snapshot);
+
+QEMUTimer *snapshot_timer;
+QEMUTimer *stop_timer;
+
+struct rexec_state {
+    bool stepping_back;
+    bool continue_backward_mode;
+    bool singlestep_was_enabled;
+    bool step_done;
+    bool stop_requested;
+};
+
+static bool rexec_enabled;
+struct rexec_state rexec_state;
+
+static snapshot_entry *new_snapshot(void)
+{
+    snapshot_entry *snap = NULL;
+    snap = g_malloc(sizeof(snapshot_entry));
+    assert(snap);
+
+    if (QLIST_FIRST(&snapshot) != NULL) {
+        snap->id = QLIST_FIRST(&snapshot)->id + 1;
+    } else {
+        snap->id = 0;
+    }
+
+    QLIST_INSERT_HEAD(&snapshot, snap, next);
+    return snap;
+}
+
+/*
+ * Timer callback called when a snapshot must be done.
+ */
+static void snap_callback(void *opaque)
+{
+    QEMUFile *file = NULL;
+    int saved_vm_running;
+    snapshot_entry *snap = NULL;
+    CPUArchState *cpu = NULL;
+    char filename[20];
+
+    cpu = qemu_get_cpu(0)->env_ptr;
+    assert(cpu != NULL);
+
+    if (!rexec_state.stepping_back) {
+        snap = new_snapshot();
+
+        saved_vm_running = runstate_is_running();
+        vm_stop(RUN_STATE_SAVE_VM);
+        snap->time = qemu_clock_get_ns(QEMU_CLOCK_ICOUNT);
+        sprintf(filename, "%s%04u", SNAPSHOT_FILENAME, snap->id);
+
+        trace_snap_callback(snap->time, filename);
+
+        file = qemu_fopen(filename, "wb");
+        qemu_savevm_state(file);
+        qemu_fclose(file);
+
+        if (saved_vm_running) {
+            vm_start();
+        }
+        timer_mod_ns(snapshot_timer, snap->time + 100000000);
+    }
+}
+
+/*
+ * Timer callback called when the VM have to stop.
+ */
+static void stop_callback(void *opaque)
+{
+    trace_stop_callback(qemu_clock_get_ns(QEMU_CLOCK_ICOUNT));
+    rexec_state.stop_requested = true;
+}
+
+void rexec_setup(void)
+{
+    snapshot_timer = timer_new_ns(QEMU_CLOCK_ICOUNT, snap_callback, NULL);
+    stop_timer = timer_new_ns(QEMU_CLOCK_ICOUNT, stop_callback, NULL);
+
+    timer_mod_ns(snapshot_timer, qemu_clock_get_ns(QEMU_CLOCK_ICOUNT));
+    rexec_enabled = true;
+    rexec_state.stepping_back = false;
+    rexec_state.continue_backward_mode = false;
+    rexec_state.stop_requested = false;
+}
+
+void rexec_stop_stepping_back_mode(void)
+{
+    if (rexec_state.stepping_back) {
+        singlestep = rexec_state.singlestep_was_enabled;
+        rexec_state.stepping_back = false;
+    }
+    rexec_state.stop_requested = false;
+}
+
+static void rexec_start_stepping_back_mode(CPUState *cpu)
+{
+    assert(!rexec_state.stepping_back);
+    /*
+     * Flushing tb.
+     * FIXME: might not be necessary with counter.
+     */
+    tb_flush(cpu->env_ptr);
+
+    /*
+     * Single step to the right PC.
+     */
+    rexec_state.singlestep_was_enabled = singlestep;
+    singlestep = 1;
+
+    rexec_state.stepping_back = true;
+}
+
+/**
+ * \func rexec_step_backward
+ * \param cpu GDBStub's cpu.
+ * \param steps Number of steps to step back.
+ * \brief Steps backward: "reverse-step" in GDB.
+ *
+ */
+void rexec_step_backward(CPUState *cpu, uint64_t steps)
+{
+    QEMUFile *file = NULL;
+    char filename[256];
+    snapshot_entry *snap = QLIST_FIRST(&snapshot);
+
+    int64_t stop_time = qemu_clock_get_ns(QEMU_CLOCK_ICOUNT)
+                      - cpu_icount_to_ns(steps);
+
+    /*
+     * FIXME: Remove the file?
+     */
+    while ((stop_time > 0) && ((snap = QLIST_FIRST(&snapshot)) != NULL)
+                           && (snap->time >= stop_time)) {
+        /*
+         * Remove the snapshot from the list and mod the snapshot timer to its
+         * time. This will cause the snapshot to be taken at the same value in
+         * case of a forward execution.
+         */
+        QLIST_REMOVE(snap, next);
+        timer_mod_ns(snapshot_timer, snap->time);
+        g_free(snap);
+    }
+
+    if ((stop_time <= 0) || (snap == NULL)) {
+        /*
+         * This happens when an instruction behind the first snapshot is asked.
+         * Just trigger a debug event so it won't move.
+         */
+        rexec_state.stop_requested = true;
+        vm_start();
+        return;
+    }
+
+    sprintf(filename, "%s%04u", SNAPSHOT_FILENAME, snap->id);
+
+    /*
+     * Load the previous state.
+     */
+    vm_stop(RUN_STATE_RESTORE_VM);
+
+    trace_rexec_step_backward(qemu_clock_get_ns(QEMU_CLOCK_ICOUNT), stop_time);
+
+    file = qemu_fopen(filename, "rb");
+    qemu_loadvm_state(file);
+    qemu_fclose(file);
+
+    /*
+     * Mod the timer so it will stop at the exact instruction.
+     */
+    timer_mod_ns(stop_timer, stop_time);
+
+    rexec_start_stepping_back_mode(cpu);
+    /*
+     * Restart the vm.
+     */
+    vm_start();
+}
+
+/**
+ * \func rexec_continue_backward
+ * \brief Continue execution backward.
+ * \param cpu GDB's stub cpu.
+ *
+ */
+void rexec_continue_backward(CPUState *cpu)
+{
+    rexec_state.continue_backward_mode = true;
+    rexec_state.step_done = false;
+    rexec_step_backward(cpu, 1);
+}
+
+/**
+ * \func rexec_is_continuing_backward
+ * \brief Check if we are continuing backward.
+ * \return Return true if we are continuing backward.
+ *
+ */
+int rexec_is_continuing_backward(void)
+{
+    return rexec_state.continue_backward_mode;
+}
+
+void rexec_next_reverse_continue_step(void)
+{
+    CPUState *cpu = qemu_get_cpu(0);
+
+    assert(cpu != NULL);
+    rexec_state.step_done = false;
+
+    /*
+     * FIXME:
+     *         - Stop at breakpoint in reverse order.
+     *         - The reverse execution speed is not constant as the snapshot
+     *           replay is not constant.
+     */
+    rexec_step_backward(cpu, 10000000);
+}
+
+void rexec_stop_reverse_continue(void)
+{
+    if (rexec_state.continue_backward_mode) {
+        trace_rexec_stop_reverse_continue();
+        rexec_state.continue_backward_mode = false;
+        rexec_state.step_done = false;
+        rexec_stop_stepping_back_mode();
+    }
+}
+
+void rexec_step_done(void)
+{
+    rexec_state.step_done = true;
+}
+
+bool rexec_is_step_done(void)
+{
+    return rexec_state.step_done;
+}
+
+bool rexec_is_enabled(void)
+{
+    return rexec_enabled;
+}
+
+void rexec_cleanup(void)
+{
+    snapshot_entry *snap = QLIST_FIRST(&snapshot);
+
+    /*
+     * FIXME: Remove the file?
+     */
+    while ((snap = QLIST_FIRST(&snapshot)) != NULL) {
+        /*
+         * Remove the snapshot from the list and mod the snapshot timer to its
+         * time. This will cause the snapshot to be taken at the same value in
+         * case of a forward execution.
+         */
+        QLIST_REMOVE(snap, next);
+        g_free(snap);
+    }
+}
+
+bool rexec_dbg_requested(void)
+{
+    return rexec_state.stop_requested;
+}
diff --git a/vl.c b/vl.c
index 9c9acf5..f69d1f1 100644
--- a/vl.c
+++ b/vl.c
@@ -119,6 +119,8 @@ int main(int argc, char **argv)
 #include "qom/object_interfaces.h"
 #include "qapi-event.h"
 
+#include "reverse-execution.h"
+
 #define DEFAULT_RAM_SIZE 128
 
 #define MAX_VIRTIO_CONSOLES 1
@@ -1975,7 +1977,7 @@ void qemu_system_debug_request(void)
 static bool main_loop_should_exit(void)
 {
     RunState r;
-    if (qemu_debug_requested()) {
+    if (qemu_debug_requested() && !rexec_is_continuing_backward()) {
         vm_stop(RUN_STATE_DEBUG);
     }
     if (qemu_suspend_requested()) {
@@ -2025,6 +2027,9 @@ static void main_loop(void)
     int64_t ti;
 #endif
     do {
+        if (rexec_is_continuing_backward() && rexec_is_step_done()) {
+            rexec_next_reverse_continue_step();
+        }
         nonblocking = !kvm_enabled() && !xen_enabled() && last_io > 0;
 #ifdef CONFIG_PROFILER
         ti = profile_getclock();
-- 
1.9.0

[Qemu-devel] [RFC V7 07/10] gdbstub: allow reverse execution in gdb stub.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This allows gdb to reverse step QEMU: reverse-stepi and reverse-cont commands
are allowed.

When step_backward is called, QEMU restores a snapshot before the actual
instruction and stops (with a debug exit) when the previous instruction is
reached.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 gdbstub.c | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/gdbstub.c b/gdbstub.c
index 8afe0b7..4564988 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -33,6 +33,7 @@
 #include "sysemu/char.h"
 #include "sysemu/sysemu.h"
 #include "exec/gdbstub.h"
+#include "reverse-execution.h"
 #endif
 
 #define MAX_PACKET_LENGTH 4096
@@ -1113,6 +1114,17 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             if (cc->gdb_core_xml_file != NULL) {
                 pstrcat(buf, sizeof(buf), ";qXfer:features:read+");
             }
+
+            #ifndef CONFIG_USER_ONLY
+            /*
+             * When reverse execution is enabled those additional features must
+             * be set so GDB allows reverse-stepi and reverse-continue command.
+             */
+            if (rexec_is_enabled()) {
+                pstrcat(buf, sizeof(buf), ";ReverseStep+;ReverseContinue+");
+            }
+            #endif /* !CONFIG_USER_ONLY */
+
             put_packet(s, buf);
             break;
         }
@@ -1161,7 +1173,23 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
         }
         /* Unrecognised 'q' command.  */
         goto unknown_command;
-
+    #ifndef CONFIG_USER_ONLY
+    case 'b':
+        /* Reverse execution. */
+        switch (*p) {
+        case 's':
+            rexec_step_backward(s->c_cpu, 1);
+            break;
+        case 'c':
+            rexec_continue_backward(s->c_cpu);
+            break;
+        default:
+            buf[0] = '\0';
+            put_packet(s, buf);
+            break;
+        }
+        break;
+    #endif /* !CONFIG_USER_ONLY */
     default:
     unknown_command:
         /* put empty packet */
@@ -1221,6 +1249,7 @@ static void gdb_vm_state_change(void *opaque, int running, RunState state)
         ret = GDB_SIGNAL_TRAP;
         break;
     case RUN_STATE_PAUSED:
+        rexec_stop_reverse_continue();
         ret = GDB_SIGNAL_INT;
         break;
     case RUN_STATE_SHUTDOWN:
-- 
1.9.0

[Qemu-devel] [RFC V7 08/10] cpu-exec: trigger a debug request when rexec stops.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This allows QEMU to trigger a debug exception when rexec_dbg_requested
is set.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 cpu-exec.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/cpu-exec.c b/cpu-exec.c
index bd93165..be2070d 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -24,6 +24,7 @@
 #include "qemu/atomic.h"
 #include "sysemu/qtest.h"
 #include "qemu/timer.h"
+#include "reverse-execution.h"
 
 /* -icount align implementation. */
 
@@ -333,6 +334,18 @@ int cpu_exec(CPUArchState *env)
     /* This must be volatile so it is not trashed by longjmp() */
     volatile bool have_tb_lock = false;
 
+    #ifndef CONFIG_USER_ONLY
+    if (rexec_is_enabled() && rexec_dbg_requested()) {
+        /*
+         * Reverse execution need to stop right now.
+         * So just generate a EXCP_DEBUG.
+         */
+        cpu->exception_index = EXCP_DEBUG;
+        cpu_handle_debug_exception(env);
+        return EXCP_DEBUG;
+    }
+    #endif /* !CONFIG_USER_ONLY */
+
     if (cpu->halted) {
         if (!cpu_has_work(cpu)) {
             return EXCP_HALTED;
-- 
1.9.0

[Qemu-devel] [RFC V7 09/10] rexec: synchronize icount on the next event.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

We don't want to warp on host clock as it is not deterministic for replay.
So this patch warp icount on the next QEMU_VIRTUAL_CLOCK event if reverse
execution is enabled.

The normal behaviour is kept when reverse execution is disabled.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 cpus.c                         | 19 +++++++++++++++++--
 include/qemu/timer.h           |  8 ++++++++
 include/reverse-execution.h    |  2 ++
 main-loop.c                    | 10 ++++++++++
 stubs/Makefile.objs            |  1 +
 stubs/cpu-get-icount.c         |  8 ++++++++
 stubs/reverse-execution-stub.c | 32 ++++++++++++++++++++++++++++++++
 7 files changed, 78 insertions(+), 2 deletions(-)
 create mode 100644 stubs/reverse-execution-stub.c

diff --git a/cpus.c b/cpus.c
index 39f4e98..742fc84 100644
--- a/cpus.c
+++ b/cpus.c
@@ -341,8 +341,10 @@ static int64_t qemu_icount_round(int64_t count)
     return (count + (1 << icount_time_shift) - 1) >> icount_time_shift;
 }
 
-static void icount_warp_rt(void *opaque)
+void icount_warp_rt(void *opaque)
 {
+    int64_t next_vm_deadline = -1;
+
     /* The icount_warp_timer is rescheduled soon after vm_clock_warp_start
      * changes from -1 to another value, so the race here is okay.
      */
@@ -350,6 +352,13 @@ static void icount_warp_rt(void *opaque)
         return;
     }
 
+    if (rexec_is_enabled()) {
+        /*
+         * We need this because the standard warp_delta is not deterministic.
+         */
+        next_vm_deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL);
+    }
+
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     if (runstate_is_running()) {
         int64_t clock = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -366,7 +375,13 @@ static void icount_warp_rt(void *opaque)
             int64_t delta = cur_time - cur_icount;
             warp_delta = MIN(warp_delta, delta);
         }
-        timers_state.qemu_icount_bias += warp_delta;
+        if (rexec_is_enabled()) {
+            if (next_vm_deadline > 0) {
+                timers_state.qemu_icount_bias += next_vm_deadline;
+            }
+        } else {
+            timers_state.qemu_icount_bias += warp_delta;
+        }
     }
     vm_clock_warp_start = -1;
     seqlock_write_unlock(&timers_state.vm_clock_seqlock);
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 16bafde..74f3d94 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -755,6 +755,14 @@ int64_t cpu_get_clock(void);
 int64_t cpu_get_clock_offset(void);
 int64_t cpu_icount_to_ns(int64_t icount);
 
+/**
+ * void icount_warp_rt:
+ *
+ * Move icount to the realtime clock or to the next QEMU_VIRTUAL_CLOCK event
+ * when reverse execution is enabled.
+ */
+void icount_warp_rt(void *opaque);
+
 /*******************************************/
 /* host CPU ticks (if available) */
 
diff --git a/include/reverse-execution.h b/include/reverse-execution.h
index 9951549..871d5ee 100644
--- a/include/reverse-execution.h
+++ b/include/reverse-execution.h
@@ -25,6 +25,8 @@
 #ifndef REVERSE_EXECUTION
 #define REVERSE_EXECUTION
 
+#include "qom/cpu.h"
+
 void rexec_setup(void);
 void rexec_step_backward(CPUState *cpu, uint64_t steps);
 void rexec_stop_stepping_back_mode(void);
diff --git a/main-loop.c b/main-loop.c
index 3cc79f8..44ae29a 100644
--- a/main-loop.c
+++ b/main-loop.c
@@ -34,6 +34,8 @@
 
 #include "qemu/compatfd.h"
 
+#include "reverse-execution.h"
+
 /* If we have signalfd, we mask out the signals we want to handle and then
  * use signalfd to listen for them.  We rely on whatever the current signal
  * handler is to dispatch the signals when we receive them.
@@ -489,6 +491,14 @@ int main_loop_wait(int nonblocking)
 
     qemu_clock_run_all_timers();
 
+    /*
+     * Sometimes deadlock can appears because there is no pending event on
+     * virtual clock.
+     */
+    if (rexec_is_enabled()) {
+        icount_warp_rt(NULL);
+    }
+
     return ret;
 }
 
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 5e347d0..aa42bca 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -40,3 +40,4 @@ stub-obj-$(CONFIG_WIN32) += fd-register.o
 stub-obj-y += cpus.o
 stub-obj-y += kvm.o
 stub-obj-y += qmp_pc_dimm_device_list.o
+stub-obj-y += reverse-execution-stub.o
diff --git a/stubs/cpu-get-icount.c b/stubs/cpu-get-icount.c
index 1968de7..d35948f 100644
--- a/stubs/cpu-get-icount.c
+++ b/stubs/cpu-get-icount.c
@@ -7,3 +7,11 @@ int64_t cpu_get_icount(int with_bias)
 {
     abort();
 }
+
+void icount_warp_rt(void *opaque)
+{
+    /*
+     * Should not happen, as rexec_is_enabled() always return false.
+     */
+    abort();
+}
diff --git a/stubs/reverse-execution-stub.c b/stubs/reverse-execution-stub.c
new file mode 100644
index 0000000..e417d31
--- /dev/null
+++ b/stubs/reverse-execution-stub.c
@@ -0,0 +1,32 @@
+/*
+ *  reverse-execution-stub.c
+ *
+ *  Copyright (C) 2014 : GreenSocs Ltd
+ *      http://www.greensocs.com/ , email: info@greensocs.com
+ *
+ *  Developed by :
+ *  Frederic Konrad   <fred.konrad@greensocs.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <stdbool.h>
+
+bool rexec_is_enabled(void);
+
+bool rexec_is_enabled(void)
+{
+    return false;
+}
-- 
1.9.0

[Qemu-devel] [RFC V7 10/10] rexec: allow to enable reverse execution.

[fred.konrad]

From: KONRAD Frederic <fred.konrad@greensocs.com>

This creates QEMU options for reverse execution.

Signed-off-by: KONRAD Frederic <fred.konrad@greensocs.com>
---
 cpus.c          | 15 +++++++++++++++
 qemu-options.hx |  5 ++++-
 vl.c            |  9 +++++++--
 3 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/cpus.c b/cpus.c
index 742fc84..56da39b 100644
--- a/cpus.c
+++ b/cpus.c
@@ -519,6 +519,7 @@ void configure_icount(QemuOpts *opts, Error **errp)
 {
     const char *option;
     char *rem_str = NULL;
+    bool enable_rexec;
 
     seqlock_init(&timers_state.vm_clock_seqlock, NULL);
     vmstate_register(NULL, 0, &vmstate_timers, &timers_state);
@@ -532,6 +533,9 @@ void configure_icount(QemuOpts *opts, Error **errp)
     icount_align_option = qemu_opt_get_bool(opts, "align", false);
     icount_warp_timer = timer_new_ns(QEMU_CLOCK_REALTIME,
                                           icount_warp_rt, NULL);
+
+    enable_rexec = qemu_opt_get_bool(opts, "rexec", false);
+
     if (strcmp(option, "auto") != 0) {
         errno = 0;
         icount_time_shift = strtol(option, &rem_str, 0);
@@ -539,11 +543,22 @@ void configure_icount(QemuOpts *opts, Error **errp)
             error_setg(errp, "icount: Invalid shift value");
         }
         use_icount = 1;
+
+        if (enable_rexec) {
+            rexec_setup();
+        }
         return;
     } else if (icount_align_option) {
         error_setg(errp, "shift=auto and align=on are incompatible");
     }
 
+    /*
+     * We don't allow to run reverse execution with use_icount != 1.
+     */
+    if (enable_rexec) {
+        error_setg(errp, "Reverse execution requires icount in fixed mode.");
+    }
+
     use_icount = 2;
 
     /* 125MIPS seems a reasonable initial guess at the guest speed.
diff --git a/qemu-options.hx b/qemu-options.hx
index 365b56c..2273272 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3026,7 +3026,7 @@ re-inject them.
 ETEXI
 
 DEF("icount", HAS_ARG, QEMU_OPTION_icount, \
-    "-icount [shift=N|auto][,align=on|off]\n" \
+    "-icount [shift=N|auto][,align=on|off][,rexec=on,|off]\n" \
     "                enable virtual instruction counter with 2^N clock ticks per\n" \
     "                instruction and enable aligning the host and virtual clocks\n", QEMU_ARCH_ALL)
 STEXI
@@ -3052,6 +3052,9 @@ Currently this option does not work when @option{shift} is @code{auto}.
 Note: The sync algorithm will work for those shift values for which
 the guest clock runs ahead of the host clock. Typically this happens
 when the shift value is high (how high depends on the host machine).
+@option{rexec=on} will enable reverse execution. A snapshot is taken regularly
+and will allow to replay the execution when gdb reverse-step or reverse-cont
+command are used. This requires shift!=auto.
 ETEXI
 
 DEF("watchdog", HAS_ARG, QEMU_OPTION_watchdog, \
diff --git a/vl.c b/vl.c
index f69d1f1..96588c8 100644
--- a/vl.c
+++ b/vl.c
@@ -556,8 +556,10 @@ static QemuOptsList qemu_icount_opts = {
         }, {
             .name = "align",
             .type = QEMU_OPT_BOOL,
-        },
-        { /* end of list */ }
+        }, {
+            .name = "rexec",
+            .type = QEMU_OPT_BOOL,
+        }, { /* end of list */ }
     },
 };
 
@@ -4582,5 +4584,8 @@ int main(int argc, char **argv, char **envp)
     tpm_cleanup();
 #endif
 
+    if (rexec_is_enabled()) {
+        rexec_cleanup();
+    }
     return 0;
 }
-- 
1.9.0