* [PATCH v2 0/4] replay: minor fixes and tests @ 2024-01-25 16:08 Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace Nicholas Piggin ` (3 more replies) 0 siblings, 4 replies; 9+ messages in thread From: Nicholas Piggin @ 2024-01-25 16:08 UTC (permalink / raw) To: qemu-devel Cc: Nicholas Piggin, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal replay-dump.py script still chokes quite easily on a trace of Linux boot, so I revived some patches from last year that fix the script and add some test cases for it. The runstate patch doesn't have a test case (yet), but you can hit the problem by replaying to the machine shutdown and then reverse-stepping. Thanks, Nick Since v1: - Rebase onto script that now has initial v12 support. - avocado tests now also cover replay_kernel.py because replay_linux.py is very heavyweight and flaky. replay_kernel is feasible to run in CI and general hacking. - Rearrange replay-dump.py decoders. Nicholas Piggin (4): replay: allow runstate shutdown->running when replaying trace scripts/replay-dump.py: Update to current rr record format scripts/replay-dump.py: rejig decoders in event number order tests/avocado: excercise scripts/replay-dump.py in replay tests include/sysemu/runstate.h | 1 + replay/replay.c | 2 + system/runstate.c | 19 ++++ scripts/replay-dump.py | 172 +++++++++++++++++++++++---------- tests/avocado/replay_kernel.py | 16 +++ tests/avocado/replay_linux.py | 16 +++ 6 files changed, 173 insertions(+), 53 deletions(-) -- 2.42.0 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace 2024-01-25 16:08 [PATCH v2 0/4] replay: minor fixes and tests Nicholas Piggin @ 2024-01-25 16:08 ` Nicholas Piggin 2024-01-29 5:06 ` Pavel Dovgalyuk 2024-01-25 16:08 ` [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format Nicholas Piggin ` (2 subsequent siblings) 3 siblings, 1 reply; 9+ messages in thread From: Nicholas Piggin @ 2024-01-25 16:08 UTC (permalink / raw) To: qemu-devel Cc: Nicholas Piggin, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal When replaying a trace, it is possible to go from shutdown to running with a reverse-debugging step. This can be useful if the problem being debugged triggers a reset or shutdown. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- include/sysemu/runstate.h | 1 + replay/replay.c | 2 ++ system/runstate.c | 19 +++++++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/include/sysemu/runstate.h b/include/sysemu/runstate.h index 0117d243c4..fe25eed3c0 100644 --- a/include/sysemu/runstate.h +++ b/include/sysemu/runstate.h @@ -9,6 +9,7 @@ void runstate_set(RunState new_state); RunState runstate_get(void); bool runstate_is_running(void); bool runstate_needs_reset(void); +void runstate_replay_enable(void); typedef void VMChangeStateHandler(void *opaque, bool running, RunState state); diff --git a/replay/replay.c b/replay/replay.c index 3fd241a4fc..2951eed3bd 100644 --- a/replay/replay.c +++ b/replay/replay.c @@ -383,6 +383,8 @@ static void replay_enable(const char *fname, int mode) /* go to the beginning */ fseek(replay_file, HEADER_SIZE, SEEK_SET); replay_fetch_data_kind(); + + runstate_replay_enable(); } replay_init_events(); diff --git a/system/runstate.c b/system/runstate.c index d6ab860eca..bd0fed8657 100644 --- a/system/runstate.c +++ b/system/runstate.c @@ -182,6 +182,12 @@ static const RunStateTransition runstate_transitions_def[] = { { RUN_STATE__MAX, RUN_STATE__MAX }, }; +static const RunStateTransition replay_runstate_transitions_def[] = { + { RUN_STATE_SHUTDOWN, RUN_STATE_RUNNING}, + + { RUN_STATE__MAX, RUN_STATE__MAX }, +}; + static bool runstate_valid_transitions[RUN_STATE__MAX][RUN_STATE__MAX]; bool runstate_check(RunState state) @@ -189,6 +195,19 @@ bool runstate_check(RunState state) return current_run_state == state; } +void runstate_replay_enable(void) +{ + const RunStateTransition *p; + + assert(replay_mode == REPLAY_MODE_PLAY); + + for (p = &replay_runstate_transitions_def[0]; p->from != RUN_STATE__MAX; + p++) { + runstate_valid_transitions[p->from][p->to] = true; + } + +} + static void runstate_init(void) { const RunStateTransition *p; -- 2.42.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace 2024-01-25 16:08 ` [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace Nicholas Piggin @ 2024-01-29 5:06 ` Pavel Dovgalyuk 0 siblings, 0 replies; 9+ messages in thread From: Pavel Dovgalyuk @ 2024-01-29 5:06 UTC (permalink / raw) To: Nicholas Piggin, qemu-devel Cc: Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> On 25.01.2024 19:08, Nicholas Piggin wrote: > When replaying a trace, it is possible to go from shutdown to > running with a reverse-debugging step. This can be useful if the > problem being debugged triggers a reset or shutdown. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > include/sysemu/runstate.h | 1 + > replay/replay.c | 2 ++ > system/runstate.c | 19 +++++++++++++++++++ > 3 files changed, 22 insertions(+) > > diff --git a/include/sysemu/runstate.h b/include/sysemu/runstate.h > index 0117d243c4..fe25eed3c0 100644 > --- a/include/sysemu/runstate.h > +++ b/include/sysemu/runstate.h > @@ -9,6 +9,7 @@ void runstate_set(RunState new_state); > RunState runstate_get(void); > bool runstate_is_running(void); > bool runstate_needs_reset(void); > +void runstate_replay_enable(void); > > typedef void VMChangeStateHandler(void *opaque, bool running, RunState state); > > diff --git a/replay/replay.c b/replay/replay.c > index 3fd241a4fc..2951eed3bd 100644 > --- a/replay/replay.c > +++ b/replay/replay.c > @@ -383,6 +383,8 @@ static void replay_enable(const char *fname, int mode) > /* go to the beginning */ > fseek(replay_file, HEADER_SIZE, SEEK_SET); > replay_fetch_data_kind(); > + > + runstate_replay_enable(); > } > > replay_init_events(); > diff --git a/system/runstate.c b/system/runstate.c > index d6ab860eca..bd0fed8657 100644 > --- a/system/runstate.c > +++ b/system/runstate.c > @@ -182,6 +182,12 @@ static const RunStateTransition runstate_transitions_def[] = { > { RUN_STATE__MAX, RUN_STATE__MAX }, > }; > > +static const RunStateTransition replay_runstate_transitions_def[] = { > + { RUN_STATE_SHUTDOWN, RUN_STATE_RUNNING}, > + > + { RUN_STATE__MAX, RUN_STATE__MAX }, > +}; > + > static bool runstate_valid_transitions[RUN_STATE__MAX][RUN_STATE__MAX]; > > bool runstate_check(RunState state) > @@ -189,6 +195,19 @@ bool runstate_check(RunState state) > return current_run_state == state; > } > > +void runstate_replay_enable(void) > +{ > + const RunStateTransition *p; > + > + assert(replay_mode == REPLAY_MODE_PLAY); > + > + for (p = &replay_runstate_transitions_def[0]; p->from != RUN_STATE__MAX; > + p++) { > + runstate_valid_transitions[p->from][p->to] = true; > + } > + > +} > + > static void runstate_init(void) > { > const RunStateTransition *p; ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format 2024-01-25 16:08 [PATCH v2 0/4] replay: minor fixes and tests Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace Nicholas Piggin @ 2024-01-25 16:08 ` Nicholas Piggin 2024-01-30 21:36 ` John Snow 2024-01-25 16:08 ` [PATCH v2 3/4] scripts/replay-dump.py: rejig decoders in event number order Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests Nicholas Piggin 3 siblings, 1 reply; 9+ messages in thread From: Nicholas Piggin @ 2024-01-25 16:08 UTC (permalink / raw) To: qemu-devel Cc: Nicholas Piggin, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal The v12 format support for replay-dump has a few issues still. This fixes async decoding; adds event, shutdown, and end decoding; fixes audio in / out events, fixes checkpoint checking of following async events. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- scripts/replay-dump.py | 132 ++++++++++++++++++++++++++++++----------- 1 file changed, 98 insertions(+), 34 deletions(-) diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py index d668193e79..35732da08f 100755 --- a/scripts/replay-dump.py +++ b/scripts/replay-dump.py @@ -20,6 +20,7 @@ import argparse import struct +import os from collections import namedtuple from os import path @@ -63,6 +64,10 @@ def read_byte(fin): "Read a single byte" return struct.unpack('>B', fin.read(1))[0] +def read_bytes(fin, nr): + "Read a nr bytes" + return fin.read(nr) + def read_event(fin): "Read a single byte event, but save some state" if replay_state.already_read: @@ -134,6 +139,18 @@ def swallow_async_qword(eid, name, dumpfile): print(" %s(%d) @ %d" % (name, eid, step_id)) return True +def swallow_bytes(eid, name, dumpfile, nr): + "Swallow nr bytes of data without looking at it" + dumpfile.seek(nr, os.SEEK_CUR) + return True + +def decode_exception(eid, name, dumpfile): + print_event(eid, name) + return True + +# v12 does away with the additional event byte and encodes it in the main type +# Between v8 and v9, REPLAY_ASYNC_BH_ONESHOT was added, but we don't decode +# those versions so leave it out. async_decode_table = [ Decoder(0, "REPLAY_ASYNC_EVENT_BH", swallow_async_qword), Decoder(1, "REPLAY_ASYNC_INPUT", decode_unimp), Decoder(2, "REPLAY_ASYNC_INPUT_SYNC", decode_unimp), @@ -142,8 +159,8 @@ def swallow_async_qword(eid, name, dumpfile): Decoder(5, "REPLAY_ASYNC_EVENT_NET", decode_unimp), ] # See replay_read_events/replay_read_event -def decode_async(eid, name, dumpfile): - """Decode an ASYNC event""" +def decode_async_old(eid, name, dumpfile): + """Decode an ASYNC event (pre-v8)""" print_event(eid, name) @@ -157,6 +174,35 @@ def decode_async(eid, name, dumpfile): return call_decode(async_decode_table, async_event_kind, dumpfile) +def decode_async_bh(eid, name, dumpfile): + op_id = read_qword(dumpfile) + print_event(eid, name) + return True + +def decode_async_bh_oneshot(eid, name, dumpfile): + op_id = read_qword(dumpfile) + print_event(eid, name) + return True + +def decode_async_char_read(eid, name, dumpfile): + char_id = read_byte(dumpfile) + size = read_dword(dumpfile) + print_event(eid, name, "device:%x chars:%s" % (char_id, read_bytes(dumpfile, size))) + return True + +def decode_async_block(eid, name, dumpfile): + op_id = read_qword(dumpfile) + print_event(eid, name) + return True + +def decode_async_net(eid, name, dumpfile): + net_id = read_byte(dumpfile) + flags = read_dword(dumpfile) + size = read_dword(dumpfile) + swallow_bytes(eid, name, dumpfile, size) + print_event(eid, name, "net:%x flags:%x bytes:%d" % (net_id, flags, size)) + return True + total_insns = 0 def decode_instruction(eid, name, dumpfile): @@ -166,6 +212,10 @@ def decode_instruction(eid, name, dumpfile): print_event(eid, name, "+ %d -> %d" % (ins_diff, total_insns)) return True +def decode_shutdown(eid, name, dumpfile): + print_event(eid, name) + return True + def decode_char_write(eid, name, dumpfile): res = read_dword(dumpfile) offset = read_dword(dumpfile) @@ -177,7 +227,7 @@ def decode_audio_out(eid, name, dumpfile): print_event(eid, name, "%d" % (audio_data)) return True -def decode_checkpoint(eid, name, dumpfile): +def __decode_checkpoint(eid, name, dumpfile, old): """Decode a checkpoint. Checkpoints contain a series of async events with their own specific data. @@ -189,14 +239,20 @@ def decode_checkpoint(eid, name, dumpfile): # if the next event is EVENT_ASYNC there are a bunch of # async events to read, otherwise we are done - if next_event != 3: - print_event(eid, name, "no additional data", event_number) - else: + if (old and next_event == 3) or (not old and next_event >= 3 and next_event <= 9): print_event(eid, name, "more data follows", event_number) + else: + print_event(eid, name, "no additional data", event_number) replay_state.reuse_event(next_event) return True +def decode_checkpoint_old(eid, name, dumpfile): + return __decode_checkpoint(eid, name, dumpfile, False) + +def decode_checkpoint(eid, name, dumpfile): + return __decode_checkpoint(eid, name, dumpfile, True) + def decode_checkpoint_init(eid, name, dumpfile): print_event(eid, name) return True @@ -212,15 +268,23 @@ def decode_clock(eid, name, dumpfile): def decode_random(eid, name, dumpfile): ret = read_dword(dumpfile) - data = read_array(dumpfile) - print_event(eid, "%d bytes of random data" % len(data)) + size = read_dword(dumpfile) + swallow_bytes(eid, name, dumpfile, size) + if (ret): + print_event(eid, name, "%d bytes (getrandom failed)" % (size)) + else: + print_event(eid, name, "%d bytes" % (size)) return True +def decode_end(eid, name, dumpfile): + print_event(eid, name) + return False + # pre-MTTCG merge v5_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), Decoder(1, "EVENT_INTERRUPT", decode_interrupt), Decoder(2, "EVENT_EXCEPTION", decode_plain), - Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(3, "EVENT_ASYNC", decode_async_old), Decoder(4, "EVENT_SHUTDOWN", decode_unimp), Decoder(5, "EVENT_CHAR_WRITE", decode_char_write), Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), @@ -242,7 +306,7 @@ def decode_random(eid, name, dumpfile): v6_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), Decoder(1, "EVENT_INTERRUPT", decode_interrupt), Decoder(2, "EVENT_EXCEPTION", decode_plain), - Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(3, "EVENT_ASYNC", decode_async_old), Decoder(4, "EVENT_SHUTDOWN", decode_unimp), Decoder(5, "EVENT_CHAR_WRITE", decode_char_write), Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), @@ -266,7 +330,7 @@ def decode_random(eid, name, dumpfile): v7_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), Decoder(1, "EVENT_INTERRUPT", decode_interrupt), Decoder(2, "EVENT_EXCEPTION", decode_unimp), - Decoder(3, "EVENT_ASYNC", decode_async), + Decoder(3, "EVENT_ASYNC", decode_async_old), Decoder(4, "EVENT_SHUTDOWN", decode_unimp), Decoder(5, "EVENT_SHUTDOWN_HOST_ERR", decode_unimp), Decoder(6, "EVENT_SHUTDOWN_HOST_QMP", decode_unimp), @@ -296,32 +360,31 @@ def decode_random(eid, name, dumpfile): v12_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), Decoder(1, "EVENT_INTERRUPT", decode_interrupt), - Decoder(2, "EVENT_EXCEPTION", decode_plain), - Decoder(3, "EVENT_ASYNC", decode_async), - Decoder(4, "EVENT_ASYNC", decode_async), - Decoder(5, "EVENT_ASYNC", decode_async), - Decoder(6, "EVENT_ASYNC", decode_async), - Decoder(6, "EVENT_ASYNC", decode_async), - Decoder(8, "EVENT_ASYNC", decode_async), - Decoder(9, "EVENT_ASYNC", decode_async), - Decoder(10, "EVENT_ASYNC", decode_async), - Decoder(11, "EVENT_SHUTDOWN", decode_unimp), - Decoder(12, "EVENT_SHUTDOWN_HOST_ERR", decode_unimp), - Decoder(13, "EVENT_SHUTDOWN_HOST_QMP_QUIT", decode_unimp), - Decoder(14, "EVENT_SHUTDOWN_HOST_QMP_RESET", decode_unimp), - Decoder(14, "EVENT_SHUTDOWN_HOST_SIGNAL", decode_unimp), - Decoder(15, "EVENT_SHUTDOWN_HOST_UI", decode_unimp), - Decoder(16, "EVENT_SHUTDOWN_GUEST_SHUTDOWN", decode_unimp), - Decoder(17, "EVENT_SHUTDOWN_GUEST_RESET", decode_unimp), - Decoder(18, "EVENT_SHUTDOWN_GUEST_PANIC", decode_unimp), - Decoder(19, "EVENT_SHUTDOWN_GUEST_SUBSYSTEM_RESET", decode_unimp), - Decoder(20, "EVENT_SHUTDOWN_GUEST_SNAPSHOT_LOAD", decode_unimp), - Decoder(21, "EVENT_SHUTDOWN___MAX", decode_unimp), + Decoder(2, "EVENT_EXCEPTION", decode_exception), + Decoder(3, "EVENT_ASYNC_BH", decode_async_bh), + Decoder(4, "EVENT_ASYNC_BH_ONESHOT", decode_async_bh_oneshot), + Decoder(5, "EVENT_ASYNC_INPUT", decode_unimp), + Decoder(6, "EVENT_ASYNC_INPUT_SYNC", decode_unimp), + Decoder(7, "EVENT_ASYNC_CHAR_READ", decode_async_char_read), + Decoder(8, "EVENT_ASYNC_BLOCK", decode_async_block), + Decoder(9, "EVENT_ASYNC_NET", decode_async_net), + Decoder(10, "EVENT_SHUTDOWN", decode_shutdown), + Decoder(11, "EVENT_SHUTDOWN_HOST_ERR", decode_shutdown), + Decoder(12, "EVENT_SHUTDOWN_HOST_QMP_QUIT", decode_shutdown), + Decoder(13, "EVENT_SHUTDOWN_HOST_QMP_RESET", decode_shutdown), + Decoder(14, "EVENT_SHUTDOWN_HOST_SIGNAL", decode_shutdown), + Decoder(15, "EVENT_SHUTDOWN_HOST_UI", decode_shutdown), + Decoder(16, "EVENT_SHUTDOWN_GUEST_SHUTDOWN", decode_shutdown), + Decoder(17, "EVENT_SHUTDOWN_GUEST_RESET", decode_shutdown), + Decoder(18, "EVENT_SHUTDOWN_GUEST_PANIC", decode_shutdown), + Decoder(19, "EVENT_SHUTDOWN_SUBSYS_RESET", decode_shutdown), + Decoder(20, "EVENT_SHUTDOWN_SNAPSHOT_LOAD", decode_shutdown), + Decoder(21, "EVENT_SHUTDOWN___MAX", decode_shutdown), Decoder(22, "EVENT_CHAR_WRITE", decode_char_write), Decoder(23, "EVENT_CHAR_READ_ALL", decode_unimp), Decoder(24, "EVENT_CHAR_READ_ALL_ERROR", decode_unimp), - Decoder(25, "EVENT_AUDIO_IN", decode_unimp), - Decoder(26, "EVENT_AUDIO_OUT", decode_audio_out), + Decoder(25, "EVENT_AUDIO_OUT", decode_audio_out), + Decoder(26, "EVENT_AUDIO_IN", decode_unimp), Decoder(27, "EVENT_RANDOM", decode_random), Decoder(28, "EVENT_CLOCK_HOST", decode_clock), Decoder(29, "EVENT_CLOCK_VIRTUAL_RT", decode_clock), @@ -334,6 +397,7 @@ def decode_random(eid, name, dumpfile): Decoder(36, "EVENT_CP_CLOCK_VIRTUAL_RT", decode_checkpoint), Decoder(37, "EVENT_CP_INIT", decode_checkpoint_init), Decoder(38, "EVENT_CP_RESET", decode_checkpoint), + Decoder(39, "EVENT_END", decode_end), ] def parse_arguments(): -- 2.42.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format 2024-01-25 16:08 ` [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format Nicholas Piggin @ 2024-01-30 21:36 ` John Snow 2024-02-26 6:39 ` Nicholas Piggin 0 siblings, 1 reply; 9+ messages in thread From: John Snow @ 2024-01-30 21:36 UTC (permalink / raw) To: Nicholas Piggin Cc: qemu-devel, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal On Thu, Jan 25, 2024 at 11:09 AM Nicholas Piggin <npiggin@gmail.com> wrote: > > The v12 format support for replay-dump has a few issues still. This > fixes async decoding; adds event, shutdown, and end decoding; fixes > audio in / out events, fixes checkpoint checking of following async > events. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > scripts/replay-dump.py | 132 ++++++++++++++++++++++++++++++----------- > 1 file changed, 98 insertions(+), 34 deletions(-) > > diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py > index d668193e79..35732da08f 100755 > --- a/scripts/replay-dump.py > +++ b/scripts/replay-dump.py > @@ -20,6 +20,7 @@ > > import argparse > import struct > +import os > from collections import namedtuple > from os import path > > @@ -63,6 +64,10 @@ def read_byte(fin): > "Read a single byte" > return struct.unpack('>B', fin.read(1))[0] > > +def read_bytes(fin, nr): > + "Read a nr bytes" Existing problem in this file, but please use """triple quotes""" for docstrings. > + return fin.read(nr) > + Does it really save a lot of typing to alias fin.read(1) to read_bytes(fin, 1) ...? > def read_event(fin): > "Read a single byte event, but save some state" > if replay_state.already_read: > @@ -134,6 +139,18 @@ def swallow_async_qword(eid, name, dumpfile): > print(" %s(%d) @ %d" % (name, eid, step_id)) > return True > > +def swallow_bytes(eid, name, dumpfile, nr): > + "Swallow nr bytes of data without looking at it" > + dumpfile.seek(nr, os.SEEK_CUR) > + return True > + Why bother returning a bool if it's not based on any condition? Add an error check or just drop the return value. > +def decode_exception(eid, name, dumpfile): > + print_event(eid, name) > + return True > + I suppose in this case, the return is to fit a common signature. > +# v12 does away with the additional event byte and encodes it in the main type > +# Between v8 and v9, REPLAY_ASYNC_BH_ONESHOT was added, but we don't decode > +# those versions so leave it out. > async_decode_table = [ Decoder(0, "REPLAY_ASYNC_EVENT_BH", swallow_async_qword), > Decoder(1, "REPLAY_ASYNC_INPUT", decode_unimp), > Decoder(2, "REPLAY_ASYNC_INPUT_SYNC", decode_unimp), > @@ -142,8 +159,8 @@ def swallow_async_qword(eid, name, dumpfile): > Decoder(5, "REPLAY_ASYNC_EVENT_NET", decode_unimp), > ] > # See replay_read_events/replay_read_event > -def decode_async(eid, name, dumpfile): > - """Decode an ASYNC event""" > +def decode_async_old(eid, name, dumpfile): > + """Decode an ASYNC event (pre-v8)""" > > print_event(eid, name) > > @@ -157,6 +174,35 @@ def decode_async(eid, name, dumpfile): > > return call_decode(async_decode_table, async_event_kind, dumpfile) > > +def decode_async_bh(eid, name, dumpfile): > + op_id = read_qword(dumpfile) > + print_event(eid, name) > + return True > + > +def decode_async_bh_oneshot(eid, name, dumpfile): > + op_id = read_qword(dumpfile) > + print_event(eid, name) > + return True > + > +def decode_async_char_read(eid, name, dumpfile): > + char_id = read_byte(dumpfile) > + size = read_dword(dumpfile) > + print_event(eid, name, "device:%x chars:%s" % (char_id, read_bytes(dumpfile, size))) > + return True > + > +def decode_async_block(eid, name, dumpfile): > + op_id = read_qword(dumpfile) > + print_event(eid, name) > + return True > + > +def decode_async_net(eid, name, dumpfile): > + net_id = read_byte(dumpfile) > + flags = read_dword(dumpfile) > + size = read_dword(dumpfile) > + swallow_bytes(eid, name, dumpfile, size) > + print_event(eid, name, "net:%x flags:%x bytes:%d" % (net_id, flags, size)) > + return True > + > total_insns = 0 > > def decode_instruction(eid, name, dumpfile): > @@ -166,6 +212,10 @@ def decode_instruction(eid, name, dumpfile): > print_event(eid, name, "+ %d -> %d" % (ins_diff, total_insns)) > return True > > +def decode_shutdown(eid, name, dumpfile): > + print_event(eid, name) > + return True > + > def decode_char_write(eid, name, dumpfile): > res = read_dword(dumpfile) > offset = read_dword(dumpfile) > @@ -177,7 +227,7 @@ def decode_audio_out(eid, name, dumpfile): > print_event(eid, name, "%d" % (audio_data)) > return True > > -def decode_checkpoint(eid, name, dumpfile): > +def __decode_checkpoint(eid, name, dumpfile, old): > """Decode a checkpoint. > > Checkpoints contain a series of async events with their own specific data. > @@ -189,14 +239,20 @@ def decode_checkpoint(eid, name, dumpfile): > > # if the next event is EVENT_ASYNC there are a bunch of > # async events to read, otherwise we are done > - if next_event != 3: > - print_event(eid, name, "no additional data", event_number) > - else: > + if (old and next_event == 3) or (not old and next_event >= 3 and next_event <= 9): > print_event(eid, name, "more data follows", event_number) > + else: > + print_event(eid, name, "no additional data", event_number) > > replay_state.reuse_event(next_event) > return True > > +def decode_checkpoint_old(eid, name, dumpfile): > + return __decode_checkpoint(eid, name, dumpfile, False) > + > +def decode_checkpoint(eid, name, dumpfile): > + return __decode_checkpoint(eid, name, dumpfile, True) > + > def decode_checkpoint_init(eid, name, dumpfile): > print_event(eid, name) > return True > @@ -212,15 +268,23 @@ def decode_clock(eid, name, dumpfile): > > def decode_random(eid, name, dumpfile): > ret = read_dword(dumpfile) > - data = read_array(dumpfile) > - print_event(eid, "%d bytes of random data" % len(data)) > + size = read_dword(dumpfile) > + swallow_bytes(eid, name, dumpfile, size) > + if (ret): > + print_event(eid, name, "%d bytes (getrandom failed)" % (size)) > + else: > + print_event(eid, name, "%d bytes" % (size)) > return True > > +def decode_end(eid, name, dumpfile): > + print_event(eid, name) > + return False > + > # pre-MTTCG merge > v5_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), > Decoder(1, "EVENT_INTERRUPT", decode_interrupt), > Decoder(2, "EVENT_EXCEPTION", decode_plain), > - Decoder(3, "EVENT_ASYNC", decode_async), > + Decoder(3, "EVENT_ASYNC", decode_async_old), > Decoder(4, "EVENT_SHUTDOWN", decode_unimp), > Decoder(5, "EVENT_CHAR_WRITE", decode_char_write), > Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), > @@ -242,7 +306,7 @@ def decode_random(eid, name, dumpfile): > v6_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), > Decoder(1, "EVENT_INTERRUPT", decode_interrupt), > Decoder(2, "EVENT_EXCEPTION", decode_plain), > - Decoder(3, "EVENT_ASYNC", decode_async), > + Decoder(3, "EVENT_ASYNC", decode_async_old), > Decoder(4, "EVENT_SHUTDOWN", decode_unimp), > Decoder(5, "EVENT_CHAR_WRITE", decode_char_write), > Decoder(6, "EVENT_CHAR_READ_ALL", decode_unimp), > @@ -266,7 +330,7 @@ def decode_random(eid, name, dumpfile): > v7_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), > Decoder(1, "EVENT_INTERRUPT", decode_interrupt), > Decoder(2, "EVENT_EXCEPTION", decode_unimp), > - Decoder(3, "EVENT_ASYNC", decode_async), > + Decoder(3, "EVENT_ASYNC", decode_async_old), > Decoder(4, "EVENT_SHUTDOWN", decode_unimp), > Decoder(5, "EVENT_SHUTDOWN_HOST_ERR", decode_unimp), > Decoder(6, "EVENT_SHUTDOWN_HOST_QMP", decode_unimp), > @@ -296,32 +360,31 @@ def decode_random(eid, name, dumpfile): > > v12_event_table = [Decoder(0, "EVENT_INSTRUCTION", decode_instruction), > Decoder(1, "EVENT_INTERRUPT", decode_interrupt), > - Decoder(2, "EVENT_EXCEPTION", decode_plain), > - Decoder(3, "EVENT_ASYNC", decode_async), > - Decoder(4, "EVENT_ASYNC", decode_async), > - Decoder(5, "EVENT_ASYNC", decode_async), > - Decoder(6, "EVENT_ASYNC", decode_async), > - Decoder(6, "EVENT_ASYNC", decode_async), > - Decoder(8, "EVENT_ASYNC", decode_async), > - Decoder(9, "EVENT_ASYNC", decode_async), > - Decoder(10, "EVENT_ASYNC", decode_async), > - Decoder(11, "EVENT_SHUTDOWN", decode_unimp), > - Decoder(12, "EVENT_SHUTDOWN_HOST_ERR", decode_unimp), > - Decoder(13, "EVENT_SHUTDOWN_HOST_QMP_QUIT", decode_unimp), > - Decoder(14, "EVENT_SHUTDOWN_HOST_QMP_RESET", decode_unimp), > - Decoder(14, "EVENT_SHUTDOWN_HOST_SIGNAL", decode_unimp), > - Decoder(15, "EVENT_SHUTDOWN_HOST_UI", decode_unimp), > - Decoder(16, "EVENT_SHUTDOWN_GUEST_SHUTDOWN", decode_unimp), > - Decoder(17, "EVENT_SHUTDOWN_GUEST_RESET", decode_unimp), > - Decoder(18, "EVENT_SHUTDOWN_GUEST_PANIC", decode_unimp), > - Decoder(19, "EVENT_SHUTDOWN_GUEST_SUBSYSTEM_RESET", decode_unimp), > - Decoder(20, "EVENT_SHUTDOWN_GUEST_SNAPSHOT_LOAD", decode_unimp), > - Decoder(21, "EVENT_SHUTDOWN___MAX", decode_unimp), > + Decoder(2, "EVENT_EXCEPTION", decode_exception), > + Decoder(3, "EVENT_ASYNC_BH", decode_async_bh), > + Decoder(4, "EVENT_ASYNC_BH_ONESHOT", decode_async_bh_oneshot), > + Decoder(5, "EVENT_ASYNC_INPUT", decode_unimp), > + Decoder(6, "EVENT_ASYNC_INPUT_SYNC", decode_unimp), > + Decoder(7, "EVENT_ASYNC_CHAR_READ", decode_async_char_read), > + Decoder(8, "EVENT_ASYNC_BLOCK", decode_async_block), > + Decoder(9, "EVENT_ASYNC_NET", decode_async_net), > + Decoder(10, "EVENT_SHUTDOWN", decode_shutdown), > + Decoder(11, "EVENT_SHUTDOWN_HOST_ERR", decode_shutdown), > + Decoder(12, "EVENT_SHUTDOWN_HOST_QMP_QUIT", decode_shutdown), > + Decoder(13, "EVENT_SHUTDOWN_HOST_QMP_RESET", decode_shutdown), > + Decoder(14, "EVENT_SHUTDOWN_HOST_SIGNAL", decode_shutdown), > + Decoder(15, "EVENT_SHUTDOWN_HOST_UI", decode_shutdown), > + Decoder(16, "EVENT_SHUTDOWN_GUEST_SHUTDOWN", decode_shutdown), > + Decoder(17, "EVENT_SHUTDOWN_GUEST_RESET", decode_shutdown), > + Decoder(18, "EVENT_SHUTDOWN_GUEST_PANIC", decode_shutdown), > + Decoder(19, "EVENT_SHUTDOWN_SUBSYS_RESET", decode_shutdown), > + Decoder(20, "EVENT_SHUTDOWN_SNAPSHOT_LOAD", decode_shutdown), > + Decoder(21, "EVENT_SHUTDOWN___MAX", decode_shutdown), > Decoder(22, "EVENT_CHAR_WRITE", decode_char_write), > Decoder(23, "EVENT_CHAR_READ_ALL", decode_unimp), > Decoder(24, "EVENT_CHAR_READ_ALL_ERROR", decode_unimp), > - Decoder(25, "EVENT_AUDIO_IN", decode_unimp), > - Decoder(26, "EVENT_AUDIO_OUT", decode_audio_out), > + Decoder(25, "EVENT_AUDIO_OUT", decode_audio_out), > + Decoder(26, "EVENT_AUDIO_IN", decode_unimp), > Decoder(27, "EVENT_RANDOM", decode_random), > Decoder(28, "EVENT_CLOCK_HOST", decode_clock), > Decoder(29, "EVENT_CLOCK_VIRTUAL_RT", decode_clock), > @@ -334,6 +397,7 @@ def decode_random(eid, name, dumpfile): > Decoder(36, "EVENT_CP_CLOCK_VIRTUAL_RT", decode_checkpoint), > Decoder(37, "EVENT_CP_INIT", decode_checkpoint_init), > Decoder(38, "EVENT_CP_RESET", decode_checkpoint), > + Decoder(39, "EVENT_END", decode_end), > ] > > def parse_arguments(): > -- > 2.42.0 > I'll leave the rest alone for now. --js ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format 2024-01-30 21:36 ` John Snow @ 2024-02-26 6:39 ` Nicholas Piggin 0 siblings, 0 replies; 9+ messages in thread From: Nicholas Piggin @ 2024-02-26 6:39 UTC (permalink / raw) To: John Snow Cc: qemu-devel, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal On Wed Jan 31, 2024 at 7:36 AM AEST, John Snow wrote: > On Thu, Jan 25, 2024 at 11:09 AM Nicholas Piggin <npiggin@gmail.com> wrote: > > > > The v12 format support for replay-dump has a few issues still. This > > fixes async decoding; adds event, shutdown, and end decoding; fixes > > audio in / out events, fixes checkpoint checking of following async > > events. > > > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > > --- > > scripts/replay-dump.py | 132 ++++++++++++++++++++++++++++++----------- > > 1 file changed, 98 insertions(+), 34 deletions(-) > > > > diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py > > index d668193e79..35732da08f 100755 > > --- a/scripts/replay-dump.py > > +++ b/scripts/replay-dump.py > > @@ -20,6 +20,7 @@ > > > > import argparse > > import struct > > +import os > > from collections import namedtuple > > from os import path > > > > @@ -63,6 +64,10 @@ def read_byte(fin): > > "Read a single byte" > > return struct.unpack('>B', fin.read(1))[0] > > > > +def read_bytes(fin, nr): > > + "Read a nr bytes" > > Existing problem in this file, but please use """triple quotes""" for > docstrings. Just coming back to this, sorry, was struggling a bit with ppc merge :/ > > + return fin.read(nr) > > + > > Does it really save a lot of typing to alias fin.read(1) to > read_bytes(fin, 1) ...? Not really, I'll squash it. > > > def read_event(fin): > > "Read a single byte event, but save some state" > > if replay_state.already_read: > > @@ -134,6 +139,18 @@ def swallow_async_qword(eid, name, dumpfile): > > print(" %s(%d) @ %d" % (name, eid, step_id)) > > return True > > > > +def swallow_bytes(eid, name, dumpfile, nr): > > + "Swallow nr bytes of data without looking at it" > > + dumpfile.seek(nr, os.SEEK_CUR) > > + return True > > + > > Why bother returning a bool if it's not based on any condition? Add an > error check or just drop the return value. > > > +def decode_exception(eid, name, dumpfile): > > + print_event(eid, name) > > + return True > > + > > I suppose in this case, the return is to fit a common signature. Yes that's why I did it, but it actually can't fit in the normal decoder pattern because a nr has to be supplied as well. I'll change it as you say. Thanks, Nick ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v2 3/4] scripts/replay-dump.py: rejig decoders in event number order 2024-01-25 16:08 [PATCH v2 0/4] replay: minor fixes and tests Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format Nicholas Piggin @ 2024-01-25 16:08 ` Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests Nicholas Piggin 3 siblings, 0 replies; 9+ messages in thread From: Nicholas Piggin @ 2024-01-25 16:08 UTC (permalink / raw) To: qemu-devel Cc: Nicholas Piggin, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal Sort decoder functions to be ascending in order of event number, same as the decoder tables. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- scripts/replay-dump.py | 56 +++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py index 35732da08f..a1d7ae0364 100755 --- a/scripts/replay-dump.py +++ b/scripts/replay-dump.py @@ -144,6 +144,19 @@ def swallow_bytes(eid, name, dumpfile, nr): dumpfile.seek(nr, os.SEEK_CUR) return True +total_insns = 0 + +def decode_instruction(eid, name, dumpfile): + global total_insns + ins_diff = read_dword(dumpfile) + total_insns += ins_diff + print_event(eid, name, "+ %d -> %d" % (ins_diff, total_insns)) + return True + +def decode_interrupt(eid, name, dumpfile): + print_event(eid, name) + return True + def decode_exception(eid, name, dumpfile): print_event(eid, name) return True @@ -203,15 +216,6 @@ def decode_async_net(eid, name, dumpfile): print_event(eid, name, "net:%x flags:%x bytes:%d" % (net_id, flags, size)) return True -total_insns = 0 - -def decode_instruction(eid, name, dumpfile): - global total_insns - ins_diff = read_dword(dumpfile) - total_insns += ins_diff - print_event(eid, name, "+ %d -> %d" % (ins_diff, total_insns)) - return True - def decode_shutdown(eid, name, dumpfile): print_event(eid, name) return True @@ -227,6 +231,21 @@ def decode_audio_out(eid, name, dumpfile): print_event(eid, name, "%d" % (audio_data)) return True +def decode_random(eid, name, dumpfile): + ret = read_dword(dumpfile) + size = read_dword(dumpfile) + swallow_bytes(eid, name, dumpfile, size) + if (ret): + print_event(eid, name, "%d bytes (getrandom failed)" % (size)) + else: + print_event(eid, name, "%d bytes" % (size)) + return True + +def decode_clock(eid, name, dumpfile): + clock_data = read_qword(dumpfile) + print_event(eid, name, "0x%x" % (clock_data)) + return True + def __decode_checkpoint(eid, name, dumpfile, old): """Decode a checkpoint. @@ -257,25 +276,6 @@ def decode_checkpoint_init(eid, name, dumpfile): print_event(eid, name) return True -def decode_interrupt(eid, name, dumpfile): - print_event(eid, name) - return True - -def decode_clock(eid, name, dumpfile): - clock_data = read_qword(dumpfile) - print_event(eid, name, "0x%x" % (clock_data)) - return True - -def decode_random(eid, name, dumpfile): - ret = read_dword(dumpfile) - size = read_dword(dumpfile) - swallow_bytes(eid, name, dumpfile, size) - if (ret): - print_event(eid, name, "%d bytes (getrandom failed)" % (size)) - else: - print_event(eid, name, "%d bytes" % (size)) - return True - def decode_end(eid, name, dumpfile): print_event(eid, name) return False -- 2.42.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests 2024-01-25 16:08 [PATCH v2 0/4] replay: minor fixes and tests Nicholas Piggin ` (2 preceding siblings ...) 2024-01-25 16:08 ` [PATCH v2 3/4] scripts/replay-dump.py: rejig decoders in event number order Nicholas Piggin @ 2024-01-25 16:08 ` Nicholas Piggin 2024-01-29 5:08 ` Pavel Dovgalyuk 3 siblings, 1 reply; 9+ messages in thread From: Nicholas Piggin @ 2024-01-25 16:08 UTC (permalink / raw) To: qemu-devel Cc: Nicholas Piggin, Pavel Dovgalyuk, Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal This runs replay-dump.py after recording a trace, and fails the test if the script fails. replay-dump.py is modified to exit with non-zero if an error is encountered while parsing, to support this. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- scripts/replay-dump.py | 6 ++++-- tests/avocado/replay_kernel.py | 16 ++++++++++++++++ tests/avocado/replay_linux.py | 16 ++++++++++++++++ 3 files changed, 36 insertions(+), 2 deletions(-) diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py index a1d7ae0364..bfea9c099b 100755 --- a/scripts/replay-dump.py +++ b/scripts/replay-dump.py @@ -21,6 +21,7 @@ import argparse import struct import os +import sys from collections import namedtuple from os import path @@ -104,7 +105,7 @@ def call_decode(table, index, dumpfile): print("Could not decode index: %d" % (index)) print("Entry is: %s" % (decoder)) print("Decode Table is:\n%s" % (table)) - return False + raise(Exception("unknown event")) else: return decoder.fn(decoder.eid, decoder.name, dumpfile) @@ -125,7 +126,7 @@ def print_event(eid, name, string=None, event_count=None): def decode_unimp(eid, name, _unused_dumpfile): "Unimplemented decoder, will trigger exit" print("%s not handled - will now stop" % (name)) - return False + raise(Exception("unhandled event")) def decode_plain(eid, name, _unused_dumpfile): "Plain events without additional data" @@ -439,6 +440,7 @@ def decode_file(filename): dumpfile) except Exception as inst: print(f"error {inst}") + sys.exit(1) finally: print(f"Reached {dumpfile.tell()} of {dumpsize} bytes") diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py index 10d99403a4..9b3ee6726b 100644 --- a/tests/avocado/replay_kernel.py +++ b/tests/avocado/replay_kernel.py @@ -13,6 +13,7 @@ import shutil import logging import time +import subprocess from avocado import skip from avocado import skipUnless @@ -22,6 +23,11 @@ from avocado.utils import process from boot_linux_console import LinuxKernelTest +from pathlib import Path + +self_dir = Path(__file__).parent +src_dir = self_dir.parent.parent + class ReplayKernelBase(LinuxKernelTest): """ Boots a Linux kernel in record mode and checks that the console @@ -63,6 +69,8 @@ def run_vm(self, kernel_path, kernel_command_line, console_pattern, vm.shutdown() logger.info('finished the recording with log size %s bytes' % os.path.getsize(replay_path)) + self.run_replay_dump(replay_path) + logger.info('successfully tested replay-dump.py') else: vm.wait() logger.info('successfully finished the replay') @@ -70,6 +78,14 @@ def run_vm(self, kernel_path, kernel_command_line, console_pattern, logger.info('elapsed time %.2f sec' % elapsed) return elapsed + def run_replay_dump(self, replay_path): + try: + subprocess.check_call(["./scripts/replay-dump.py", + "-f", replay_path], + cwd=src_dir, stdout=subprocess.DEVNULL) + except subprocess.CalledProcessError: + self.fail('replay-dump.py failed') + def run_rr(self, kernel_path, kernel_command_line, console_pattern, shift=7, args=None): replay_path = os.path.join(self.workdir, 'replay.bin') diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py index f3a43dc98c..dd148ff639 100644 --- a/tests/avocado/replay_linux.py +++ b/tests/avocado/replay_linux.py @@ -11,6 +11,7 @@ import os import logging import time +import subprocess from avocado import skipUnless from avocado_qemu import BUILD_DIR @@ -21,6 +22,11 @@ from avocado.utils.path import find_command from avocado_qemu import LinuxTest +from pathlib import Path + +self_dir = Path(__file__).parent +src_dir = self_dir.parent.parent + class ReplayLinux(LinuxTest): """ Boots a Linux system, checking for a successful initialization @@ -94,6 +100,8 @@ def launch_and_wait(self, record, args, shift): vm.shutdown() logger.info('finished the recording with log size %s bytes' % os.path.getsize(replay_path)) + self.run_replay_dump(replay_path) + logger.info('successfully tested replay-dump.py') else: vm.event_wait('SHUTDOWN', self.timeout) vm.wait() @@ -108,6 +116,14 @@ def run_rr(self, args=None, shift=7): logger = logging.getLogger('replay') logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) + def run_replay_dump(self, replay_path): + try: + subprocess.check_call(["./scripts/replay-dump.py", + "-f", replay_path], + cwd=src_dir, stdout=subprocess.DEVNULL) + except subprocess.CalledProcessError: + self.fail('replay-dump.py failed') + @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') class ReplayLinuxX8664(ReplayLinux): """ -- 2.42.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests 2024-01-25 16:08 ` [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests Nicholas Piggin @ 2024-01-29 5:08 ` Pavel Dovgalyuk 0 siblings, 0 replies; 9+ messages in thread From: Pavel Dovgalyuk @ 2024-01-29 5:08 UTC (permalink / raw) To: Nicholas Piggin, qemu-devel Cc: Philippe Mathieu-Daudé, Richard Henderson, Alex Bennée, Paolo Bonzini, John Snow, Cleber Rosa, Wainer dos Santos Moschetta, Beraldo Leal Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> On 25.01.2024 19:08, Nicholas Piggin wrote: > This runs replay-dump.py after recording a trace, and fails the test if > the script fails. > > replay-dump.py is modified to exit with non-zero if an error is > encountered while parsing, to support this. > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > scripts/replay-dump.py | 6 ++++-- > tests/avocado/replay_kernel.py | 16 ++++++++++++++++ > tests/avocado/replay_linux.py | 16 ++++++++++++++++ > 3 files changed, 36 insertions(+), 2 deletions(-) > > diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py > index a1d7ae0364..bfea9c099b 100755 > --- a/scripts/replay-dump.py > +++ b/scripts/replay-dump.py > @@ -21,6 +21,7 @@ > import argparse > import struct > import os > +import sys > from collections import namedtuple > from os import path > > @@ -104,7 +105,7 @@ def call_decode(table, index, dumpfile): > print("Could not decode index: %d" % (index)) > print("Entry is: %s" % (decoder)) > print("Decode Table is:\n%s" % (table)) > - return False > + raise(Exception("unknown event")) > else: > return decoder.fn(decoder.eid, decoder.name, dumpfile) > > @@ -125,7 +126,7 @@ def print_event(eid, name, string=None, event_count=None): > def decode_unimp(eid, name, _unused_dumpfile): > "Unimplemented decoder, will trigger exit" > print("%s not handled - will now stop" % (name)) > - return False > + raise(Exception("unhandled event")) > > def decode_plain(eid, name, _unused_dumpfile): > "Plain events without additional data" > @@ -439,6 +440,7 @@ def decode_file(filename): > dumpfile) > except Exception as inst: > print(f"error {inst}") > + sys.exit(1) > > finally: > print(f"Reached {dumpfile.tell()} of {dumpsize} bytes") > diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py > index 10d99403a4..9b3ee6726b 100644 > --- a/tests/avocado/replay_kernel.py > +++ b/tests/avocado/replay_kernel.py > @@ -13,6 +13,7 @@ > import shutil > import logging > import time > +import subprocess > > from avocado import skip > from avocado import skipUnless > @@ -22,6 +23,11 @@ > from avocado.utils import process > from boot_linux_console import LinuxKernelTest > > +from pathlib import Path > + > +self_dir = Path(__file__).parent > +src_dir = self_dir.parent.parent > + > class ReplayKernelBase(LinuxKernelTest): > """ > Boots a Linux kernel in record mode and checks that the console > @@ -63,6 +69,8 @@ def run_vm(self, kernel_path, kernel_command_line, console_pattern, > vm.shutdown() > logger.info('finished the recording with log size %s bytes' > % os.path.getsize(replay_path)) > + self.run_replay_dump(replay_path) > + logger.info('successfully tested replay-dump.py') > else: > vm.wait() > logger.info('successfully finished the replay') > @@ -70,6 +78,14 @@ def run_vm(self, kernel_path, kernel_command_line, console_pattern, > logger.info('elapsed time %.2f sec' % elapsed) > return elapsed > > + def run_replay_dump(self, replay_path): > + try: > + subprocess.check_call(["./scripts/replay-dump.py", > + "-f", replay_path], > + cwd=src_dir, stdout=subprocess.DEVNULL) > + except subprocess.CalledProcessError: > + self.fail('replay-dump.py failed') > + > def run_rr(self, kernel_path, kernel_command_line, console_pattern, > shift=7, args=None): > replay_path = os.path.join(self.workdir, 'replay.bin') > diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py > index f3a43dc98c..dd148ff639 100644 > --- a/tests/avocado/replay_linux.py > +++ b/tests/avocado/replay_linux.py > @@ -11,6 +11,7 @@ > import os > import logging > import time > +import subprocess > > from avocado import skipUnless > from avocado_qemu import BUILD_DIR > @@ -21,6 +22,11 @@ > from avocado.utils.path import find_command > from avocado_qemu import LinuxTest > > +from pathlib import Path > + > +self_dir = Path(__file__).parent > +src_dir = self_dir.parent.parent > + > class ReplayLinux(LinuxTest): > """ > Boots a Linux system, checking for a successful initialization > @@ -94,6 +100,8 @@ def launch_and_wait(self, record, args, shift): > vm.shutdown() > logger.info('finished the recording with log size %s bytes' > % os.path.getsize(replay_path)) > + self.run_replay_dump(replay_path) > + logger.info('successfully tested replay-dump.py') > else: > vm.event_wait('SHUTDOWN', self.timeout) > vm.wait() > @@ -108,6 +116,14 @@ def run_rr(self, args=None, shift=7): > logger = logging.getLogger('replay') > logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) > > + def run_replay_dump(self, replay_path): > + try: > + subprocess.check_call(["./scripts/replay-dump.py", > + "-f", replay_path], > + cwd=src_dir, stdout=subprocess.DEVNULL) > + except subprocess.CalledProcessError: > + self.fail('replay-dump.py failed') > + > @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') > class ReplayLinuxX8664(ReplayLinux): > """ ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-02-26 6:41 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-01-25 16:08 [PATCH v2 0/4] replay: minor fixes and tests Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 1/4] replay: allow runstate shutdown->running when replaying trace Nicholas Piggin 2024-01-29 5:06 ` Pavel Dovgalyuk 2024-01-25 16:08 ` [PATCH v2 2/4] scripts/replay-dump.py: Update to current rr record format Nicholas Piggin 2024-01-30 21:36 ` John Snow 2024-02-26 6:39 ` Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 3/4] scripts/replay-dump.py: rejig decoders in event number order Nicholas Piggin 2024-01-25 16:08 ` [PATCH v2 4/4] tests/avocado: excercise scripts/replay-dump.py in replay tests Nicholas Piggin 2024-01-29 5:08 ` Pavel Dovgalyuk
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).