Re: [PATCH v3 00/17] tests/qtest: migration-test refactoring

[Fabiano Rosas <farosas@suse.de>]

Fabiano Rosas <farosas@suse.de> writes:

> Fabiano Rosas <farosas@suse.de> writes:
>
>> v3:
>>     - Fixed some more hook names I had missed;
>>     - Move addition of migration-qmp.c in meson to correct patch;
>>     - Moved QEMU_ENV_* variables to framework.c where they are used;
>>     - patch 7: improve commit message;
>>     - s/test-framework/framework/
>>     - meson.build style changes;
>>     - did *not* move migration-test into migration/, meson doesn't
>>       like it. We'd need a separate meson.build for that directory,
>>       which I don't think is worth it;
>>     - Drop the other build changes, this is now just the refactoring
>>       of migration-test.
>>
>> Refactor migration-test.c and migration-helpers.c into multiple
>> smaller files structured like so:
>>
>> qtest/
>>     migration-test.c
>>
>> qtest/migration/
>>     framework.h
>>     framework.c - core tests infrastructure: migrate_start|end,
>>                   *_common, wait_for_*, check_guests_ram, etc.
>>
>>     migration-util.h
>>     migration-util.c - general utilities, equivalent to
>>                        migration-helpers.c
>>
>>     migration-qmp.h
>>     migration-qmp.c - utilities for querying information via QMP
>>
>>     bootfile.h - formerly migration-test.h, the old name was confusing
>>     bootfile.c - the guest code
>>
>>     # test code, without main()
>>     compression-tests.c
>>     cpr-tests.c
>>     file-tests.c
>>     misc-tests.c
>>     postcopy-tests.c
>>     precopy-tests.c
>>     tls-tests.c
>>
>> CI run: https://gitlab.com/farosas/qemu/-/pipelines/1563199058
>>
>> v2:
>> https://lore.kernel.org/r/20241113194630.3385-1-farosas@suse.de
>> v1:
>> https://lore.kernel.org/r/20241105180837.5990-1-farosas@suse.de
>> v0:
>> https://lore.kernel.org/r/20241017143211.17771-1-farosas@suse.de
>>
>> Fabiano Rosas (17):
>>   tests/qtest/migration: Standardize hook names
>>   tests/qtest/migration: Stop calling everything "test"
>>   tests/migration: Disambiguate guestperf vs. a-b
>>   tests/qtest/migration: Move bootfile code to its own file
>>   tests/qtest/migration: Move qmp helpers to a separate file
>>   tests/qtest/migration: Rename migration-helpers.c
>>   tests/qtest/migration: Move ufd_version_check to utils
>>   tests/qtest/migration: Move kvm_dirty_ring_supported to utils
>>   tests/qtest/migration: Isolate test initialization
>>   tests/qtest/migration: Move common test code
>>   tests/qtest/migration: Split TLS tests from migration-test.c
>>   tests/qtest/migration: Split compression tests from migration-test.c
>>   tests/qtest/migration: Split postcopy tests
>>   tests/qtest/migration: Split file tests
>>   tests/qtest/migration: Split precopy tests
>>   tests/qtest/migration: Split CPR tests
>>   tests/qtest/migration: Split validation tests + misc
>>
>>  MAINTAINERS                                   |    5 +-
>>  tests/meson.build                             |    2 +-
>>  .../guestperf-batch.py                        |    0
>>  .../guestperf-plot.py                         |    0
>>  .../guestperf.py                              |    0
>>  .../guestperf/__init__.py                     |    0
>>  .../guestperf/comparison.py                   |    0
>>  .../guestperf/engine.py                       |    0
>>  .../guestperf/hardware.py                     |    0
>>  .../guestperf/plot.py                         |    0
>>  .../guestperf/progress.py                     |    0
>>  .../guestperf/report.py                       |    0
>>  .../guestperf/scenario.py                     |    0
>>  .../guestperf/shell.py                        |    0
>>  .../guestperf/timings.py                      |    0
>>  .../initrd-stress.sh                          |    0
>>  .../meson.build                               |    0
>>  .../{migration => migration-stress}/stress.c  |    0
>>  tests/qtest/meson.build                       |   27 +-
>>  tests/qtest/migration-test.c                  | 4031 +----------------
>>  tests/{ => qtest}/migration/Makefile          |    0
>>  tests/{ => qtest}/migration/aarch64/Makefile  |    0
>>  .../migration/aarch64/a-b-kernel.S            |    0
>>  .../migration/aarch64/a-b-kernel.h            |    0
>>  tests/qtest/migration/bootfile.c              |   70 +
>>  .../migration/bootfile.h}                     |    9 +-
>>  tests/qtest/migration/compression-tests.c     |  239 +
>>  tests/qtest/migration/cpr-tests.c             |   58 +
>>  tests/qtest/migration/file-tests.c            |  338 ++
>>  tests/qtest/migration/framework.c             |  971 ++++
>>  tests/qtest/migration/framework.h             |  230 +
>>  tests/{ => qtest}/migration/i386/Makefile     |    0
>>  .../migration/i386/a-b-bootblock.S            |    0
>>  .../migration/i386/a-b-bootblock.h            |    0
>>  .../migration-qmp.c}                          |  567 ++-
>>  tests/qtest/migration/migration-qmp.h         |   46 +
>>  tests/qtest/migration/migration-util.c        |  362 ++
>>  .../migration-util.h}                         |   27 +-
>>  tests/qtest/migration/misc-tests.c            |  275 ++
>>  tests/qtest/migration/postcopy-tests.c        |  106 +
>>  tests/{ => qtest}/migration/ppc64/Makefile    |    0
>>  .../{ => qtest}/migration/ppc64/a-b-kernel.S  |    0
>>  .../{ => qtest}/migration/ppc64/a-b-kernel.h  |    0
>>  tests/qtest/migration/precopy-tests.c         | 1007 ++++
>>  tests/{ => qtest}/migration/s390x/Makefile    |    0
>>  tests/{ => qtest}/migration/s390x/a-b-bios.c  |    0
>>  tests/{ => qtest}/migration/s390x/a-b-bios.h  |    0
>>  tests/qtest/migration/tls-tests.c             |  791 ++++
>>  tests/qtest/virtio-net-failover.c             |    3 +-
>>  49 files changed, 4806 insertions(+), 4358 deletions(-)
>>  rename tests/{migration => migration-stress}/guestperf-batch.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf-plot.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/__init__.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/comparison.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/engine.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/hardware.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/plot.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/progress.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/report.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/scenario.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/shell.py (100%)
>>  rename tests/{migration => migration-stress}/guestperf/timings.py (100%)
>>  rename tests/{migration => migration-stress}/initrd-stress.sh (100%)
>>  rename tests/{migration => migration-stress}/meson.build (100%)
>>  rename tests/{migration => migration-stress}/stress.c (100%)
>>  rename tests/{ => qtest}/migration/Makefile (100%)
>>  rename tests/{ => qtest}/migration/aarch64/Makefile (100%)
>>  rename tests/{ => qtest}/migration/aarch64/a-b-kernel.S (100%)
>>  rename tests/{ => qtest}/migration/aarch64/a-b-kernel.h (100%)
>>  create mode 100644 tests/qtest/migration/bootfile.c
>>  rename tests/{migration/migration-test.h => qtest/migration/bootfile.h} (85%)
>>  create mode 100644 tests/qtest/migration/compression-tests.c
>>  create mode 100644 tests/qtest/migration/cpr-tests.c
>>  create mode 100644 tests/qtest/migration/file-tests.c
>>  create mode 100644 tests/qtest/migration/framework.c
>>  create mode 100644 tests/qtest/migration/framework.h
>>  rename tests/{ => qtest}/migration/i386/Makefile (100%)
>>  rename tests/{ => qtest}/migration/i386/a-b-bootblock.S (100%)
>>  rename tests/{ => qtest}/migration/i386/a-b-bootblock.h (100%)
>>  rename tests/qtest/{migration-helpers.c => migration/migration-qmp.c} (51%)
>>  create mode 100644 tests/qtest/migration/migration-qmp.h
>>  create mode 100644 tests/qtest/migration/migration-util.c
>>  rename tests/qtest/{migration-helpers.h => migration/migration-util.h} (67%)
>>  create mode 100644 tests/qtest/migration/misc-tests.c
>>  create mode 100644 tests/qtest/migration/postcopy-tests.c
>>  rename tests/{ => qtest}/migration/ppc64/Makefile (100%)
>>  rename tests/{ => qtest}/migration/ppc64/a-b-kernel.S (100%)
>>  rename tests/{ => qtest}/migration/ppc64/a-b-kernel.h (100%)
>>  create mode 100644 tests/qtest/migration/precopy-tests.c
>>  rename tests/{ => qtest}/migration/s390x/Makefile (100%)
>>  rename tests/{ => qtest}/migration/s390x/a-b-bios.c (100%)
>>  rename tests/{ => qtest}/migration/s390x/a-b-bios.h (100%)
>>  create mode 100644 tests/qtest/migration/tls-tests.c
>>
>>
>> base-commit: 7872e5fdf38ac0d8d0083aabb98d67da1f530ef4
>
> I'll have to dequeue this, the handling of QTestMigrationState becomes a
> mess once we move stuff out of migration-test.c.

I looked further into this. The issue I was referring to is having
QTestMigrationState lifecycle not tied to the QTestState. This creates a
confusing usage pattern for code that wants to reset the
QTestMigrationState outside of migrate_start ... which is no one apart
from my ping-pong series. So I'll carry on with merging this series.

I have a pile of changes to move the QTestState pointer into
QTestMigrationState and use the latter all over the place instead. That
way we can access both the state data and also pass the qts around. I'll
post them when I respin the ping-pong and cancel series.