From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48558) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X83FP-0005B9-8o for qemu-devel@nongnu.org; Fri, 18 Jul 2014 04:10:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X83FG-0004kH-DQ for qemu-devel@nongnu.org; Fri, 18 Jul 2014 04:10:30 -0400 Received: from greensocs.com ([178.33.234.66]:49036) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X83FG-0004jk-40 for qemu-devel@nongnu.org; Fri, 18 Jul 2014 04:10:22 -0400 Message-ID: <53C8D66C.7020009@greensocs.com> Date: Fri, 18 Jul 2014 10:10:20 +0200 From: Frederic Konrad MIME-Version: 1.0 References: <20140717110153.8352.80175.stgit@PASHA-ISP> In-Reply-To: <20140717110153.8352.80175.stgit@PASHA-ISP> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC PATCH v2 00/49] Series short description List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Pavel Dovgalyuk , qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, peter.crosthwaite@xilinx.com, mark.burton@greensocs.com, real@ispras.ru, batuzovk@ispras.ru, pbonzini@redhat.com On 17/07/2014 13:01, Pavel Dovgalyuk wrote: > This set of patches is related to the reverse execution and deterministic > replay of qemu execution Our implementation of deterministic replay can > be used for deterministic and reverse debugging of guest code through gdb > remote interface. > > Execution recording writes non-deterministic events log, which can be later > used for replaying the execution anywhere and for unlimited number of times. > It also supports checkpointing for faster rewinding during reverse debugging. > Execution replaying reads the log and replays all non-deterministic events > including external input, hardware clocks, and interrupts. > > Reverse execution has the following features: > * Deterministically replays whole system execution and all contents of the memory, > state of the hadrware devices, clocks, and screen of the VM. > * Writes execution log into the file for latter replaying for multiple times > on different machines. > * Supports i386, x86_64, and ARM hardware platforms. > * Performs deterministic replay of all operations with keyboard, mouse, network adapters, > audio devices, serial interfaces, and physical USB devices connected to the emulator. > * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue. > * Supports auto-checkpointing for convenient reverse debugging. > > Usage of the record/replay: > * First, record the execution, by adding '-record fname=replay.bin' to the > command line. > * Then you can replay it for the multiple times by using another command > line option: '-replay fname=replay.bin' > * Virtual machine should have at least one virtual disk, which is used to > store checkpoints. If you want to enable automatic checkpointing, simply > add ',period=XX' to record options, where XX is the checkpointing period > in seconds. > * Using of the network adapters in record/replay mode is possible with > the following command-line options: > - '-net user' (or another host adapter) in record mode > - '-net replay' in replay mode. Every host network adapter should be > replaced by 'replay' when replaying the execution. > * Reverse debugging can be used through gdb remote interface. > reverse-stepi and reverse-continue commands are supported. Other reverse > commands should also work, because they reuse these ones. > * Monitor is extended by the following commands: > - replay_info - prints information about replay mode and current step > (number of instructions executed) > - replay_break - sets "breakpoint" at the specified instructions count. > - replay_seek - rewinds (using the checkpoints, if possible) to the > specified step of replay log. > > Paper with short description of deterministic replay implementation: > http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html > > Modifications of qemu include: > * adding missed fields of the virtual devices' states to the vmstate > structures to allow deterministic saving and restoring the VM state > * adding virtual clock-based timers to vmstate structures, because virtual > clock is the part of the virtual machine state > * modification of block layer to support automatic creation of the overlay > files to store the changes and snapshots while recording > * disabling of system reset while loading VM state to avoid generating of > interrupts by reset handlers > * adding warpers for clock and time functions to save their return > values in the log > * saving different asynchronous events (e.g. system shutdown) into the log > * synchronization of the bottom halves execution > * synchronization of the threads from thread pool > * recording/replaying user input (mouse and keyboard), input from virtual > serial ports, incoming network packets, input from connected USB devices > * adding HMP/QMP commands to monitor for controlling replay execution > > v2 changes: > * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov) > * Added QMP versions of replay commands (as suggested by Eric Blake) > * Removed some optional features of replay to make patches cleaner > * Minor changes and code cleanup were made Thanks for reshaping the patches. We are a little busy just now, but I will review that as soon as possible. Thanks, Fred > > --- > > Pavel Dovgalyuk (49): > acpi: accurate overflow check > integratorcp: adding vmstate for save/restore > pcspk: adding vmstate for save/restore > fdc: adding vmstate for save/restore > parallel: adding vmstate for save/restore > serial: fixing vmstate for save/restore > kvmapic: fixing loading vmstate > hpet: fixing saving and loading process > pckbd: adding new fields to vmstate > rtl8139: adding new fields to vmstate > piix: do not raise irq while loading vmstate > mc146818rtc: add missed field to vmstate > pl031: add missed field to vmstate > ide pci: reset status field before loading the vmstate > softmmu: fixing usage of cpu_st/ld* from helpers > target: save cpu state fields > target-i386: update fp status fix > migration: add vmstate for int8 and char arrays > replay: global variables and function stubs > block: add suffix parameter to bdrv_open functions > sysemu: system functions for replay > replay: internal functions for replay log > cpu: invent instruction count for accurate replay > target-arm: instructions counting code for replay > target-i386: instructions counting code for replay > replay: interrupts and exceptions > vga: do not use virtual clock for blinking cursor > replay: asynchronous events infrastructure > replay: recording and replaying clock ticks > replay: recording and replaying different timers > replay: shutdown event > replay: checkpoints > replay: bottom halves > replay: replay aio requests > replay: thread pool > pl031: vmstate in replay mode > replay: initialization and deinitialization > replay: command line options > replay: snapshotting the virtual machine > replay: recording of the user input > tap-win32: destroy the thread at exit > replay: network packets record/replay > replay: audio data record/replay > replay: serial port > replay: USB passthrough > replay: replay_info command > replay: replay_break command > replay: replay_seek_step command > gdbstub: reverse debugging > > > Makefile.target | 1 > arch_init.c | 8 > async.c | 45 ++ > audio/audio.c | 14 + > audio/audio_win_int.h | 3 > audio/winwaveaudio.c | 167 +++++++-- > block.c | 134 ++++++- > block/blkdebug.c | 2 > block/blkverify.c | 4 > block/cow.c | 2 > block/qcow.c | 2 > block/qcow2.c | 10 - > block/qed.c | 2 > block/raw-posix.c | 6 > block/raw-win32.c | 4 > block/sheepdog.c | 4 > block/vmdk.c | 8 > block/vvfat.c | 2 > blockdev.c | 11 - > cpu-exec.c | 34 +- > cpus.c | 81 ++++ > dma-helpers.c | 10 - > exec.c | 12 + > gdbstub.c | 79 +++- > hmp-commands.hx | 41 ++ > hw/acpi/core.c | 7 > hw/arm/integratorcp.c | 38 ++ > hw/audio/pcspk.c | 19 + > hw/block/fdc.c | 11 - > hw/block/virtio-blk.c | 10 - > hw/char/parallel.c | 22 + > hw/char/serial.c | 115 ++++-- > hw/display/vga.c | 4 > hw/i386/kvmvapic.c | 22 + > hw/ide/ahci.c | 4 > hw/ide/atapi.c | 9 > hw/ide/core.c | 18 + > hw/ide/pci.c | 10 + > hw/input/pckbd.c | 3 > hw/intc/apic_common.c | 5 > hw/net/rtl8139.c | 5 > hw/pci-host/piix.c | 22 + > hw/timer/arm_timer.c | 2 > hw/timer/hpet.c | 13 - > hw/timer/mc146818rtc.c | 5 > hw/timer/pl031.c | 35 +- > hw/usb/hcd-uhci.c | 2 > hw/usb/host-libusb.c | 525 ++++++++++++++++++--------- > include/block/aio.h | 17 + > include/block/block.h | 21 + > include/block/thread-pool.h | 4 > include/exec/cpu-defs.h | 1 > include/exec/cpu_ldst_template.h | 28 + > include/exec/exec-all.h | 31 ++ > include/hw/host-libusb.h | 105 +++++ > include/migration/vmstate.h | 13 + > include/qemu-common.h | 3 > include/qemu/main-loop.h | 1 > include/qemu/timer.h | 48 ++ > include/qom/cpu.h | 4 > include/sysemu/char.h | 25 + > include/sysemu/cpus.h | 1 > include/sysemu/sysemu.h | 2 > include/ui/input.h | 2 > main-loop.c | 5 > monitor.c | 49 +++ > net/Makefile.objs | 1 > net/clients.h | 3 > net/dump.c | 6 > net/hub.c | 1 > net/net-replay.c | 66 +++ > net/net.c | 7 > net/slirp.c | 14 + > net/socket.c | 35 ++ > net/tap-win32.c | 25 + > net/tap.c | 23 + > net/vde.c | 14 + > qapi-schema.json | 62 +++ > qemu-char.c | 55 +++ > qemu-img.c | 6 > qemu-io-cmds.c | 2 > qemu-io.c | 4 > qemu-nbd.c | 2 > qemu-options.hx | 28 + > qemu-timer.c | 45 ++ > qmp-commands.hx | 60 +++ > replay/Makefile.objs | 11 + > replay/replay-audio.c | 228 ++++++++++++ > replay/replay-char.c | 99 +++++ > replay/replay-debug.c | 148 ++++++++ > replay/replay-events.c | 381 ++++++++++++++++++++ > replay/replay-input.c | 107 ++++++ > replay/replay-internal.c | 159 ++++++++ > replay/replay-internal.h | 224 ++++++++++++ > replay/replay-net.c | 190 ++++++++++ > replay/replay-qmp.c | 57 +++ > replay/replay-time.c | 181 +++++++++ > replay/replay-usb.c | 188 ++++++++++ > replay/replay.c | 735 ++++++++++++++++++++++++++++++++++++++ > replay/replay.h | 218 +++++++++++ > savevm.c | 32 +- > slirp/slirp.c | 9 > softmmu_template.h | 18 + > stubs/Makefile.objs | 1 > stubs/replay.c | 42 ++ > target-arm/Makefile.objs | 1 > target-arm/helper.h | 3 > target-arm/machine.c | 5 > target-arm/replay_helper.c | 38 ++ > target-arm/translate.c | 62 +++ > target-i386/Makefile.objs | 1 > target-i386/cpu.c | 1 > target-i386/cpu.h | 8 > target-i386/fpu_helper.c | 5 > target-i386/helper.h | 3 > target-i386/machine.c | 7 > target-i386/replay_helper.c | 38 ++ > target-i386/translate.c | 91 ++++- > tests/test-thread-pool.c | 7 > thread-pool.c | 53 ++- > trace-events | 2 > translate-all.c | 15 + > ui/input.c | 79 +++- > util/iov.c | 4 > vl.c | 108 +++++- > vmstate.c | 6 > 126 files changed, 5433 insertions(+), 543 deletions(-) > create mode 100755 include/hw/host-libusb.h > create mode 100755 net/net-replay.c > create mode 100755 replay/Makefile.objs > create mode 100755 replay/replay-audio.c > create mode 100755 replay/replay-char.c > create mode 100755 replay/replay-debug.c > create mode 100755 replay/replay-events.c > create mode 100755 replay/replay-input.c > create mode 100755 replay/replay-internal.c > create mode 100755 replay/replay-internal.h > create mode 100755 replay/replay-net.c > create mode 100755 replay/replay-qmp.c > create mode 100755 replay/replay-time.c > create mode 100755 replay/replay-usb.c > create mode 100755 replay/replay.c > create mode 100755 replay/replay.h > create mode 100755 stubs/replay.c > create mode 100755 target-arm/replay_helper.c > create mode 100755 target-i386/replay_helper.c >